SCIENCE

The Hadoop performance results, demonstrated with the known Terasort benchmark (an I/O intensive benchmark for Hadoop), showed that the total job runtime was cut in half when using Hadoop-Direct and Mellanox RDMA-enabled ConnectX-2 InfiniBand or 10GbE with RoCE-enabled network adapters compared to traditional Ethernet alternatives. Hadoop-Direct is transparent to Hadoop users and applications, and does not require any modification for existing application and usage models.

“With data exponentially increasing in Hadoop clusters and Web 2.0 applications, bottlenecks are emerging in network, storage and compute resources that limit application performance and increase cost,” said Michael Kagan, chief technology officer at Mellanox Technologies. “Mellanox’s innovations in hardware and software target the Web 2.0 applications and data analytics infrastructure for improved user experience and increased infrastructure efficiency.”

“Network bandwidth and usage of compute capacity per node to process network-related functions are key factors that limit efficient scale of Hadoop clusters,” said Dhruba Borthakur, distinguished member of the Hadoop Apache Development Team. “Hadoop-Direct with Mellanox networking solutions help minimize the latency of data access; the use of higher bandwidth enables overlapping communications and computation thus improving Hadoop cluster's performance.”

Memcached services, when running over Mellanox’s networking hardware and acceleration software, deliver two to three times more transactions per seconds with up to 10 times faster response time. The results provide web developers with faster web application services while using fewer servers.

The advanced feature set, combined with the highest RDMA performance of any standard interconnect, makes Mellanox’s ConnectX-2 adapters with Hadoop-Direct the leading performance solution for Hadoop clusters. Together with Mellanox’s world-leading networking switch systems, copper and fiber cables, and comprehensive fabric management and storage acceleration software, Mellanox provides its customers with the richest, most advanced and highest performing end-to-end networking solutions for improving and scaling Hadoop and Memcached infrastructures.

Availability

In addition to software availability, Mellanox is also offering a Hadoop-Direct and Memcached demonstration platform that includes Hadoop-Direct and Memcached acceleration software, ConnectX-2 40Gb/s InfiniBand adapters and an IS5030 36-port 40Gb/s InfiniBand switch. To learn more about the demo kit, please visit www.mellanox.com/content/pages.php?pg=web_2_0 or email Mellanox at info@mellanox.com.