Salvaged Components Reconfigured To Build CMU's Narwhal Computer

Parts of a decommissioned Los Alamos National Laboratory (LANL) supercomputer have found a new home at Carnegie Mellon University, where the components are being reconfigured into a new computing cluster for use in education and research related to large-scale computer systems.

The new computing cluster, called Narwhal, is being built at CMU from 448 blade computers salvaged from Cerrillos, the smaller stablemate of what was once the world's fastest computer, Roadrunner. In 2008, Roadrunner became the first computer to break the petaflop barrier, performing more than one million billion calculations per second, or one petaflop.

Roadrunner and Cerrillos, both taken offline last year, shared an innovative hybrid architecture that harnessed two types of computer processors to greatly increase overall processing speed. 

Narwhal will include 1,792 processor cores — a fraction of Cerrillos' 14,400 cores, which in turn was a fraction of Roadrunner's 122,400 cores. But Garth Gibson, professor of computer science and principal investigator for Narwhal, said the new CMU supercomputing cluster will provide a unique and powerful resource for students and researchers.

"Working with Los Alamos National Laboratory over the last decade, CMU research has been able to contribute to innovative fault tolerance and parallel file systems at the largest scale," Gibson said.  "With Narwhal, we open a new front — assistance with large-scale computer systems software education. Roadrunner and Cerrillos may be retired, but even a sliver of these machines' core capabilities is more capable than most educational computing resources."

For five years, the $120 million Roadrunner served as the nuclear weapons lab's workhorse. Its speed enabled it to perform computer simulations that helped the scientists charged with maintaining the U.S. nuclear deterrent to better understand nuclear weapons phenomena. In Cerrillos, the same technology supported non-classified research at LANL.

"Roadrunner had an innovative design and it contributed mightily to our understanding of weapons physics during its years of service," said Gary Grider of the New Mexico lab's High Performance Computing Division. "It is gratifying to know that a portion of this important capability will continue to serve as an educational and research tool for years to come. Roadrunner, even after decommissioning, continues to cover new ground."

Gibson had earlier worked with Grider and other collaborators to establish the Parallel Reconfigurable Observational Environment (PRObE), a National Science Foundation-sponsored computer research center built with open/unclassified computers that had been decommissioned by the Department of Energy (DOE). To create Narwhal, Los Alamos and Carnegie Mellon were able to arrange a loan of DOE equipment to CMU for at least two years under an existing LANL research contract with the university, a partnership called the Institute for Reliable High Performance Information Technology.

The cluster's name, Narwhal, was chosen in honor of PRObE's tradition of selecting names with Alaskan themes. A narwhal is a one- to two-ton Arctic whale famous for its single long ivory tusk, often protruding seven to 10 feet in front of its head.     

Narwhal uses Roadrunner's technology, but the machines differ markedly, both in size and architecture. Roadrunner had a unique hybrid design in which general purpose AMD Opteron processors were linked to special graphics processors from IBM, called Cells. The AMD processors handled basic tasks while the IBM Cells performed computationally intense calculations, contributing to Roadrunner's overall speed.

Gibson said Carnegie Mellon did not request the Cell processors because the academic parallel and distributed computing goals for Narwhal were better met with more blades and fewer Cell co-processors. Researchers and students will be able to experiment with parallel computing applications and infrastructure, controlling and instrumenting all software down to the bare metal, at a scale that's an order of magnitude larger than most university clusters. 

The university's Parallel Data Lab has purchased and will install more than 400 magnetic disks as it sets up the Narwhal cluster this summer. Narwhal will be housed in the Mehrabian Collaborative Innovation Center on the CMU campus.