Open-source software powers top US academic supercomputer

SPECIAL COVERAGE FROM ISC2006 - The newest edition of the 500 fastest supercomputers in the world, released today, lists Indiana University's supercomputer cluster, Big Red, as the fastest supercomputer among all US academic institutions and ranked 23rd overall in the world—and it runs on open-source software. Big Red consists of IBM’s very latest technology, an e1350 BladeCenter Cluster, that uses new chip technology and high speed internal networks to perform calculations at very fast speeds. Running the SLES 9 operating system, Big Red as of today is the largest IBM e1350 system in the world, with a peak theoretical capability of 20.4 trillion mathematical operations per second. It contains a total of 1024 dual-core IBM PowerPC 970 MP chips. Each chip has two floating point processor elements, one vector processor per chip, and runs at a clock rate of 2.5GHz. Big Red is made up of 512 JS21 Blade servers, each of which contains two IBM dual-core PowerPC 970MP processors and 8GB of RAM. The JS21 Blade servers have Ethernet and Myrinet2000 interconnects. Indiana University will be relying on a suite of open-source software to operate Big Red. Open-source software offers the best opportunity to achieve high levels of performance and to get this very new and innovative system up and running quickly so that it is producing new scientific breakthroughs as rapidly as possible. One of the tremendous challenges in achieving high levels of performance from applications running on Big Red and other large supercomputers will be managing multiple layers of complexity and many processors in the system. In Big Red, each PowerPC 970 chip has two floating point units and a vector unit. Two dual-core PowerPC970 processors share 8GB of RAM on one JS21 Blade server and the 512 Blade servers in Big Red each have two different communication paths with other parts of the system—Myrinet2000 and Ethernet. To achieve the greatest parallel processing efficiency, IU will be using the OpenMPI implementation of the Message Passing Interface specification. OpenMPI was created by an international consortium of several major research labs including the Open Systems Lab, part of Pervasive Technology Labs at Indiana University. OpenMPI provides especially advanced tools for taking advantage of and effectively utilizing a complex supercomputer cluster such as Big Red. In addition, Indiana University will use the performance analysis tool Vampir NG, produced by the Technische Universität Dresden, to study and improve the performance of applications running on this system. Vampir NG uses the open-source Open Trace Format for storing data about application performance on this system. As supercomputers get faster and more complex, open-source software provides the capabilities and nimbleness required to extract the best possible application performance—and thus the most important scientific breakthroughs—from these massive new supercomputers. Big Red will also play a major role in the TeraGrid, the National Science Foundation’s flagship effort to create an advanced national cyberinfrastructure. Cyberinfrastructure refers to supercomputers, massive data storage systems, advanced instruments, and people all connected by high speed networks, enabling new possibilities in scientific research. The National Science Foundation’s goal for the TeraGrid is to make US scientific research more productive and to enhance the international competitiveness of US scientists. Big Red will be connected to the TeraGrid this summer, and will at that time be the fastest supercomputer connected to this innovative national grid computing system. Indiana University is in the process of expanding its activities related to high performance computing, and as a result has leadership openings available. A search is underway for two leaders at the Director level. Consideration of applications for these positions will begin July 10. For additional details, see: RAC Web site Director Web site