Indiana University Attains Teraflop Status with IBM SP

By Steve Fisher, Editor In Chief -- Last week Indiana University (IU) announced that it had acquired the nation's largest university-owned supercomputer. The IBM SP, which has been expanded to triple the university's previous computing capacity, will support IU researchers in areas such as life sciences, archaeology, astronomy, and computational physics. To learn more Supercomputing Online interviewed Craig Stewart, director for research & academic computing, IU’s university IT services. Supercomputing: Please describe IU's new SP for the readers. Architecture, processors, networking, software...the works. STEWART: IU's IBM SP roughly triples the size of our earlier SP installation. The currently installed ten SP frames hold a total of 616 Power3+ processors within 143 SP nodes. The current system is operated as one Loadleveller instance (the IBM job management system, which we use on our SP), spanning two logical SPs. The nodes within each logical SP are interconnected via the IBM SP Switch (a low-latency, 150 megabytes/second a crossbar switch). A SP Switch router provides a high-speed link between the two subcomponents of our SP. Most of the processors in the SP are installed in 4-processor thin nodes, but we have one frame that holds four, 16-processor high nodes (each with 8 GB RAM) for large-memory jobs. The larger of the two logical SPs, aimed specifically at supporting large parallel jobs, comprises 508 Power3+ processors. A bit later this fall we will take delivery of the last piece of the system: a 16-processor Regatta node, equipped with IBM's new Power4 processors. This will also be large-memory node, and we plan to target this node especially at high performance database work. Once this last node is in place, the peak theoretical capacity of the entire system will be 1.005 TFLOPS. So this will be the first University-owned supercomputer in the US to have a peak theoretical capacity of over one TFLOPS. Supercomputing: When do you expect to have the system up and running? STEWART: Other than the Power4-based Regatta node, the system is up and running smoothly. With greater availability of processors, usage is going up. We already have several periods of near-capacity utilization. Supercomputing: If you would, please detail the InGen (Indiana Genomics) initiative and the planned genomics research collaboration with IBM. STEWART: The Indiana Genomics Initiative (InGen) was created by a $105M grant from the Lilly Endowment to Indiana University. The goal is quite simple: to take advantage of the revolution in biomedical science, and leverage the expertise of researchers at Indiana University, in order to make new breakthrough discoveries that will improve the quality of human health. This grant should also have beneficial effects on the economy of Indiana, which is becoming known for its accomplishments in advanced IT. InGen has six main program areas: genomics, bioinformatics, medical informatics, bioethics, education, and training. In addition there are nine supporting cores, one of which is the Information Technology core. InGen builds upon IU's excellence in medical research, and its outstanding accomplishments in IT, to create a new facility that will put IU among the top tier of institutions engaged in genomic science. It will also attract top experts to Indiana University as faculty members. We have had one very significant success in recruitment already. Dr. Eric Meslin has joined Indiana University as Director of the IU Center for Bioethics, and is additionally the Director of InGen's Bioethics program. INGEN is now searching for a Director for its new Bioinformatics program (www.medicine.iu.edu/fac_positions/), and I certainly hope that IU's new supercomputing facilities will help INGEN attract a world-class person for this important position. IBM has a very significant life sciences initiative - and a history of life sciences IT research that goes back to well before this area became fashionable. IU and IBM have a long history of successful researcher - to - researcher collaborations. Our life sciences partnership will take this collaboration to a new level, in which IU and IBM will collaboratively create new supercomputer applications in the life sciences. We're looking at a number of areas for possible collaboration, including intelligent searching and retrieval of information from disparate sources, cellular modelling, biomedical visualization, computational phylogenetics, and life sciences portals designed to make supercomputers more easily accessible to the practicing biomedical researcher. Supercomputing: How's the mood among IU's high performance computing folks? They've got to be thrilled. STEWART: Thrilled indeed. Everyone is terrifically excited. Faculty, staff, and graduate student researchers have now locally available the type of resource that was previously available only at national or state centers. And of course the staff members of University Information Technology Services are energized and excited. The staff members of UITS' Research & Academic Computing Division (a group of the best IT professionals one could ever hope to work with) care about two things first and foremost: enabling important research, and working with and developing tools for the newest high performance computing technology. The expansion we just announced elevates IU to a new level in the HPC arena. A few years ago IU President Myles Brand set as a goal for IU that we be a leader, in absolute terms, in the use of information technology. The Teraflop SP is just the most recent example of this goal being accomplished. Supercomputing: Why did IU select the IBM SP over the competition? STEWART: IU has had an IBM SP since 1996 - one that we've kept constantly upgraded. Prior to the recent expansion our SP had a peak theoretical capacity of about 300 GFLOPS - which is in itself very respectable for a university. The key goal for our supercomputer expansion was to enable IU scientists to analyze massive quantities of data and perform the most advanced analyses possible. Given our existing partnerships with IBM, our happiness with the performance and robustness of our SP, and the opportunity to form a new life sciences partnership with IBM, expansion of our SP to one TFLOPS was the best course for us to take. Supercomputing: Is there anything else you'd like to add? STEWART: We live in times of unprecedented opportunities to advance science and improve quality of life through use of supercomputers. Indiana University is in a position to take a leadership role in capitalizing on these opportunities, and it's tremendously exciting for the University as a whole and myself personally to be a part of that. If people are interested in learning more, check web sites about InGen (www.ingen.iu.edu/), University Information Technology Services (www.indiana.edu/~uits/), or the outreach page for the Research & Academic Computing Division of UITS (www.indiana.edu/~rac/outreach.html ). ---------- Supercomputing Online wishes to thank Craig Stewart for his time and insights. ----------