INTERCONNECTS

The University of Arizona’s two newest high-performance computers are among the world’s elite in not just one, but two categories. The UA's SGI Altix ICE system has just been ranked by The Top500 Supercomputer Sites as the 237th most powerful computer in the world. In addition, The Green500 List, run by Top500, rates the UA system as the 50th greenest in the world in electricity consumption. The rankings, published in June, cover HPC facilities worldwide including corporations and private research centers, as well as universities. Click here to listen to an interview with Michael Bruck, UA research computing. The system is actually comprised of two machines. The first, a 628-core CPU SGI Altix 4700 “shared memory” system, was installed in March 2007. Michael Bruck, assistant director of research computing at the UA, said this machine is the smaller of the two, but “serves a special purpose.” “All the memory on every one of the processors is available for access to all of the other processors. If researchers need an extremely large amount of memory, the shared memory machine allows them to use a small number of processors with a very large memory access,” Bruck said. Such systems are typically three to four times the cost of a simple cluster computer. The rest of the supercomputing budget was used to acquire an SGI Altix ICE, a lower-cost, high-performance cluster, that went online on April 1 of this year. This computer has 1,392 core processors, but can also accommodate additional compute nodes provided by researchers themselves. “Research computing operates two-thirds of the processors for all its clients, with eight research group ‘buy-in partners’ receiving priority status on one-third of the system,” Bruck said. “Campus research groups are able to buy additional processors on the machine, outsourcing their computing maintenance, space needs, expansion potential and consulting to research computing, while maintaining scheduling priority on the processors they purchase.” Together the new high-performance system has increased computing power from 0.6 trillion floating point operations per second, or TeraFLOPS, to 19.4 TeraFLOPS, or about 32 times the capacity of the nearly four-year-old system it replaced. In addition, the new machines take up far less space and require substantially less air conditioning. The old system also required 18 cabinets. The new one uses nine cabinets, some of which still have empty space that will allow both systems to expand. Rather than using traditional air conditioning for cooling, the new system is instead connected to the UA’s chilled water lines. Craig Cook, UITS operations manager, was able to drop the air conditioning requirements for the room by 20 percent, and the air conditioning unit that once cooled the space, now stands as a backup. “There are probably 40-50 other universities with facilities that are about this size,” Bruck said. “This is actually a small, high-performance computing system when compared to those top 500 others (in the rankings). It is reasonably small but compared to other research universities this is a good-sized system for a local campus.” The drive to greener computers is being fueled by manufacturers using lower-power chips. The UA’s SGI system uses fewer power components, power supplies and fans that eat electricity. They still have redundancy, Bruck said, but without duplication of components on every motherboard. Water-cooling systems have recently been reintroduced into the high-performance computing market. Manufacturers offer them as an option, but it is generally cost-effective for centers that have chilled water systems. As systems get extremely large, power consumption eventually becomes more important than the price of the computer. The UA’s system and its nearly 1,400 processors puts it in elite company, but the largest national super computer centers, with 20,000 to 50,000 processors, operate at a level that requires a separate data center and the ability to handle the power and air conditioning requirements. Currently there are 97 research groups spread across eight colleges and 30 departments at the UA using the HPC facility. Nearly all the research intensive groups on campus use it, including science, engineering, medicine and optical sciences. Researchers in social and behavioral sciences, especially psychology and linguistics, also are among the users. “This university has had a high performance computer or supercomputer for about 20 years,” Bruck said. “And it’s typical that every three to five years, the computer is replaced or the technology is refreshed. These two computers we have now replaced a computer system that was about 3.5 years old, and have 32 times the processing power of the previous system. That is the kind of change that happened over the past three years, and assuming we have the same budget, I expect that it will continue to grow at that rate or faster. As manufacturers continue to put more processors onto a single chip, we’re going to get more and more processing power into smaller spaces. "Computing in general has an extremely high strategic value for almost all research. As new academic areas, including social and behavioral science and psychology, move into digital data acquisition, high-performance computing becomes important in virtually every aspect of university research," Bruck said.