SCIENCE

Obsidian has announced a collaboration with the NASA Advanced Supercomputing (NAS) Division at NASA's Ames Research Center, Moffett Field, Calif., and the Hyperion Project at the Department of Energy's Lawrence Livermore National Laboratory (LLNL) to assess new software engineered for networks with multiple subnets in complex topologies.

“Multiple subnet architectures provide better scaling, faster initialization, fault isolation and, very importantly - the ability to support very large scale distributed heterogeneous infrastructure. The 'Subnet Manager' – the software responsible for pre-calculating traffic paths within a subnet, – is currently limited to dealing with simple, regular topologies (such as Clos networks or hypercubes). By enabling multiple subnets, BGFC allows larger systems to be constructed by joining subnets using different topologies together into a single system. “For example a multi-subnet supercomputer, storage arrays, visualization systems and many smaller clusters could be combined into a single, optimally routed complex fabric, spanning a campus or even a Wide Area Network, continued David.Obsidian's Chief Visionary Officer, Dr. David Southwell, explains the motivation behind this program: “Today's fastest supercomputers are assembled from many thousands of servers coupled with high-performance InfiniBand interconnect. These systems have been confined to a single subnet architecture – by removing this restriction next-generation systems can expand as required.”

“Without routing, we are limited in how we build and reliably operate large-scale InfiniBand based networks. I designed a simple way to merge the Pleiades supercomputer’s 11-D hypercube with two 2-D tori onto a single subnet, but it required changes to the subnet manager and quite a bit of effort to set up correctly to work around the issue,” said Bob Ciotti, systems lead and chief system architect in the NAS Division at Ames. “We are at the practical limit of those modifications and require a more sophisticated approach not only for routing, but in managing single subnets. It's a hard problem that needs more work. We look forward to collaborating with our partners in this area."

Pleiades achieved 1.09 petaflop/s with a ranking of 7^th on the June Top500 list and is the world's largest InfiniBand network with 63 miles of cables and 80,890 active ports, all programmed and controlled by the OpenSM subnet manager.

Matt Leininger, manager of the Hyperion Cluster built by Appro International at LLNL, shares the sentiment - “The supercomputing program at LLNL is very interested in utilizing large-scale multi-subnet InfiniBand environments in our data centers, and we are especially excited about the possibilities Obsidian is bringing forward to the open-source community with BGFC."

The Hyperion project brings together LLNL and 10 industry leaders to accelerate the development of next-generation Linux clusters. Initiated by DOE’s National Nuclear Security Administration (NNSA), Hyperion serves as a testbed for HPC technologies critical to Livermore’s national security missions and industry’s ability to make petaFLOP/s computing and storage available to U.S. industry.

A public demonstration of BGFC is planned for the Supercomputing 2011 conference in Seattle this November.