Comparing Performance Between iSCSI, FCoE, and FC
The following is written by Ujjwal Rajbhandari, from Dell Storage Product Marketing Group.
There are number of discussions, blogs, and articles comparing Internet SCSI (iSCSI), Fibre Channel over Ethernet (FCoE), and Fibre Channel (FC). Many of them share a common belief that FCoE and FC are better suited as core data center storage area networks (SANs) and that iSCSI is ideal for Tier 2 storage or for SAN deployments in remote or branch office (ROBO) and small and medium business (SMB) environments. That is because iSCSI is characterized as “low-performing,” “lousy,” and “unpredictable.” In this blog I will tackle the misinformation around iSCSI performance as compared to FC and FCoE. I will also compare effective efficiency of the various SAN protocols since efficiency is an aspect of performance.
Both iSCSI and FCoE share the same 10 Gigabit Ethernet (10GbE) at the transport layer. However, the perception is that the TCP/IP overhead makes iSCSI inefficient compared to FCoE and FC (both having better payload to packet-size ratio), thus leading to lower performance and efficiency. Figure 1 shows protocol efficiency calculation for iSCSI (both 1.5K MTU and 9K MTU), FC, and FCoE (2.5K MTU). It can be seen that when jumbo frames are enabled, iSCSI has the best protocol efficiency.
Figure 1: Protocol efficiency comparisons
Regarding performance, iSCSI having low performance might have been true when 1 Gbps was the maximum throughput available per iSCSI port (whereas FC was delivering 2 Gbps, 4 Gbps, and 8 Gbps per port), but with the availability of 10GbE, the commonly held belief that iSCSI performance is not up to par compared to FCoE or FC is no longer true.
The Office of the CTO at Dell conducted a series of performance tests to compare 10GbE iSCSI, FCoE, and 4 Gb FC. To ensure similar workloads, the application throughput was limited to 4 Gb. The host bus adapters (HBAs) used for the different protocols were as follows: a 10GbE network interface card (NIC) with iSCSI offload for iSCSI traffic; a 10GbE converged network adapter (CNA) for FCoE traffic; and a 4 Gbps FC HBA for Fibre Channel traffic.
The goal of the testing was to capture achieved throughput and CPU utilization for a given SAN protocol.
The protocol efficiency comparisons from Figure 1 might be theoretical in nature; Figure 2 shows results from an I/O workload study comparing throughput of 10GbE iSCSI, FCoE, and 4 Gb FC HBAs. To keep the results easy to visualize, the results show the throughput achieved when the application generated 4 Gb throughput. It can be seen clearly that iSCSI outperforms FCoE and FC regardless of read or write operations for various I/O block sizes.
Figure 2: Throughput performance comparisons (MB/s)
Along with capturing the throughput, let’s examine the host CPU utilization to better assess the performance and efficiency of specific SAN protocols. All the host adapters are comprised of hardware-based offload capability to process the protocol-specific traffic, minimizing use of CPU resources. Figure 3 shows the effective CPU utilization for various workloads. It can be seen from this figure that all the host adapters have similar CPU utilization metrics, again reinforcing the fact that iSCSI is as efficient as FCoE and FC.
Finally, Figure 4 shows throughput efficiency, defined as MBps/%CPU, for the various storage protocols. The chart shows 10GbE iSCSI having the best throughput efficiency across the workload types, clearly outperforming FCoE and FC.
From the test results we can undoubtedly summarize that iSCSI as a SAN protocol is not “lower-performing” or “inefficient” compared to FC or FCoE. On the contrary, iSCSI outperforms both FC and FCoE. Customers who are planning to purchase storage for their data centers can consider an iSCSI SAN as a viable option, knowing iSCSI performance is at par or even better than FCoE and FC. Also, customers considering unifying their data center networks over Ethernet can start doing so now with iSCSI. While FCoE can also deliver storage traffic over Ethernet, it is still under development and is not ready for prime time.
Figure 3: CPU utilization (%) for iSCSI offload, FCoE, and FC 
Figure 4: Overall protocol throughput efficiency (MBps/%CPU) for iSCSI offload, FCoE, and FC