Sandia National Laboratories, DataDirect Networks, Cluster File Systems, Inc. and Voltaire combined solutions to deliver 15 Gigabytes per second to Sandia’s high performance RoSE Visualization Cluster. The RoSE clusters are designed to visualize, analyze and manage output from two of the world’s most powerful supercomputers, Red Storm and Thunderbird. These supercomputers enable simulations with extraordinary precision and fidelity, and the size of the output is enormous. Each output can be tens of terabytes in size. “A high-performance file system solution is central to the operation of Sandia’s RoSE clusters, especially for challenging tasks like interactive visualization of large simulation results,” said Milt Clauser, Distinguished Member of the Technical Staff at Sandia National Laboratories. “Working with DataDirect Networks, Cluster File Systems and Voltaire enabled us to achieve our goals.” In August, RoSE’s high performance parallel file system solution underwent a series of tests and achieved record-setting performance exceeding 15 Gigabytes/second. Since then, the file system has been delivering solid performance in a production computing environment on the Red RoSE cluster. The tests used 12 Silicon Storage Appliances (S2A 8500) from DataDirect Networks to provide high performance parallel access to 360 terabytes of SATA storage. The Lustre file system, developed by Cluster File Systems, integrated with the S2A’s high speed capabilities delivered the performance record to the RoSE cluster. An InfiniBand network from Voltaire interconnected the nodes of the RoSE cluster achieving high performance by providing high bandwidth and low latency with little load on the cluster CPUs. A new InfiniBand network abstraction layer (NAL) enabled the Lustre file system to take full advantage of the high performance network. The NAL was developed by Cluster File Systems with assistance from Voltaire and funding from the PathForward program of the Department of Energy's Office of Advanced Simulation and Computing.