The IBM Blue Gene/P Intrepid at the Argonne Leadership Computing Facility (ALCF), located at the U.S. Department of Energy's (DOE) Argonne National Laboratory, will soon have the data analytics and visualization capability to complement its distinction as the fastest computer in the world for open science and the third fastest overall computer in the world. Argonne awarded GraphStream, Inc., Belmont, Calif., a contract that will help to make data analytics and visualization at this scale possible through the world's largest installation of NVIDIA Quadro Plex S4 external graphics processing units (GPU). This new supercomputer installation, nicknamed Eureka, will allow researchers to explore and visualize the data they produce with Intrepid. The powerful installation will offer 104 dual quad core servers with 208 Quadro FX5600 GPUs in the S4s. "During a massive computation on Intrepid, torrents of data can be unleashed onto the multi-petabyte parallel file system," ALCF acting director Pete Beckman said. "For example, in just a little over a minute, Intrepid can produce the equivalent of 1,000 DVDs of file data. Eureka will be used to peer ever deeper into scientist's data, from simulations of the electrical signals of the human heart to exploding supernova. Aided by Eureka, scientists will plow through the tidal wave of data produced by Intrepid faster than ever before, searching for new insights." Most applications that run on large-scale systems like Intrepid generate huge volumes of data that represent the results of the calculations. An essential tool for understanding those results after the run has completed is to be able to explore rapidly the output data and convert it to a visual representation. To do so at the scale required by Intrepid applications requires a system with Eureka's power. GraphStream, a supplier of scalable computer systems, will use the NVIDIA Quadro Plex (S4) visual computing system as the base graphics building block. "With the addition of Eureka, the GraphStream/NVIDIA collaboration to provide the world's most advanced scalable visual computing systems now extends to the sites of the three most powerful supercomputers in the world," said Craig Dunwoody, CEO of GraphStream. "Using the NVIDIA Quadro Plex S4 visual computing system as the base graphics building block, Eureka will deliver a quantum leap in visual compute density, enabling breakthrough levels of productivity and capability in visualization and data analysis." "Argonne National Laboratory's adoption of NVIDIA Quadro Plex visual computing solutions for their groundbreaking IBM Blue Gene/P supercomputer speaks volumes to the importance placed today on the role of visualization in scientific research and analytics," said Jeff Brown, general manager of the professional solutions business at NVIDIA. "The close partnership between NVIDIA and the Argonne Leadership Computing Facility has been key to this landmark installation. We are extremely excited to see the impact that GPU technologies will have on the quality and pace of the scientific research carried out there." The cost-effective approach being used takes four very high-end graphics cards and places them in a 1U "pizza box." A very dense configuration, this solution handles all the power and cooling issues associated with the graphics cards. An alternative configuration using 4U servers with two cards each would take 10.5 racks to match the same number of graphics cards that the proposed approach provides in just four racks. "This addition to the ALCF's high-end computing platform will provide a vital link between simulation and analysis by allowing scientists to probe and interrogate their data in an interactive manner," said Paul Fischer, a computational scientist at Argonne, who conducts reactor core hydrodynamics research on the Blue Gene/P. The ALCF's Blue Gene/P Intrepid provides resources for the DOE's Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program, which supports computationally intensive projects from industry, scientific researchers and research organizations. The ALCF is a leadership-class computing facility that enables the research and development community to make innovative and high-impact science and engineering breakthroughs. Argonne operates the ALCF for the DOE Office of Science as part of the larger DOE Leadership Computing Facility strategy that is organized by the DOE Office of Advanced Scientific Computing Research. DOE leads the world in providing the most capable civilian supercomputers for science. Technical information The base server building block is the SuperMicro 6015-UR. The S4 attaches to a server on either side of it, forming a "sandwich." To the servers, it appears as if they have two Quadro FX5600 graphics cards inside of them. While there are small system disks in the server, all of the data comes from the large storage system over the network. Economical, low-latency modular switches represent the heart of the data-management system. The nine-switch complex supports up to 2,048 connections, each of which simultaneously exchanges data at roughly 1 billion bytes per second. The storage system offers a bank of more than 10,000 disk drives that will send and receive data from the Blue Gene/P's more than 100,000 processors. Altogether, this system can deliver nearly 80 billion bytes per second to and from the disk—the equivalent of transferring the content of 100 full CDs every second.