Research Shows 10G Ethernet With TCP Offload Outperforms Infiniband

Chelsio Communications, Inc., the established leader in 10-Gigabit Ethernet protocol acceleration technology, today announced benchmark results published by The Ohio State University and Los Alamos National Laboratory prove 10GbE with TCP Offload Engine (TOE) outperforms InfiniBand (IBA), Myrinet and standard 10G NICs without TOE in high performance computing applications. Dr. D.K. Panda, a recognized leader in high performance computing and professor of Computer Science and Engineering at Ohio State, concluded that 10GbE technology utilizing a TOE delivers performance that is better than niche technologies such as IBA and Myrinet in a system-area network environment to support clusters and that 10GbE is particularly well-suited for sockets-based applications. “Although significant research has been performed on the individual technologies, no work had been done to compare the capabilities of these technologies in a homogenous environment. For high performance computing environments where TCP/IP is dominant (such as legacy scientific applications, grid-based or heterogeneous-computing applications, file and storage systems, and other commercial applications), 10GbE with TOE is the only high performance solution and Chelsio is the only vendor offering this solution today,” said Dr. Panda. “Chelsio's 10GbE TOE adapter is easy to install and use, very stable and flexible, and clearly delivers significant performance benefits for several of the above-mentioned HPC applications.” Dr. Wu Feng, also a recognized leader in high performance computing and team leader of Research & Development in Advanced Network Technologies (RADIANT) at Los Alamos National Laboratory, proved that 10GbE TOE delivers superior performance on all vectors – throughput, CPU utilization and latency – compared with standard 10GbE NICs without TOE. “In our performance characterization of 10GbE adapters, we evaluated the performance of the Chelsio 10GbE adapter, which is powered by TOE, relative to a traditional 10GbE adapter with a non-offloaded TCP stack. What we found was that the Chelsio TOE delivered higher throughput (50% to 100%) and higher transaction rate (e.g., an order of magnitude higher on heavily-loaded web servers) while using 40% less CPU cycles,” said Dr. Feng. “Furthermore, unlike the non-offloaded TCP stack, the TOE was capable of sustaining line-rate throughput using standard Ethernet frame sizes. The fact that this was achieved at low CPU utilization obviates the need for non-standard ‘jumbo’ frames, the disadvantages of which are well-known.” Dr. Panda, Dr. Feng and team members from Ohio State University and Los Alamos National Lab conducted performance evaluations across 10GbE TOE, 10GbE NICs without TOE, IBA, and Myrinet with identical cluster-compute nodes in order to enable a coherent comparison with respect to the sockets interface. The benchmarks include both detailed micro-benchmark evaluations using the sockets interface and the popular Message Passing Interface (MPI) as well as application-level evaluations with sample applications from different HPC domains, including a bio-medical image visualization tool known as the Virtual Microscope, an iso-surface oil reservoir simulator, a cluster file-system known as the Parallel Virtual File-System (PVFS), a popular cluster management tool known as Ganglia, and the Apache web server. “Gigabit Ethernet-based clusters have cornered 42% of the Top500 Supercomputer List, more than any other interconnect technology,” said Kianoosh Naghshineh, CEO of Chelsio. “And with the imminent lower pricing of 10GbE, the ubiquity of Ethernet and TCP/IP, and the adoption of RDMA and iSCSI protocols, 10GbE TOE is poised to become a dominant interconnect technology.” The technical papers from both OSU and LANL are available for download from both Chelsio Communications at www.chelsio.com/technology/wp.htm#perf and Ohio State University at its Web site.