EARTH SCIENCES

DataDirect Networks' S2A9550 was the HPC storage solution of choice by the National Center for Data Mining (NCDM) at UIC, the winner of this year's prestigious Bandwidth Challenge at SuperComputing '06 held in Tampa, Florida. The S2A9550 also powered the 2nd and 3rd place winners, Cal Tech and Indiana University. A team of experts from the University of Illinois at Chicago's NCDM, Northwestern University and Johns Hopkins University won the 7th annual Bandwidth Challenge by transporting the 1.3 TB Sloan Digital Sky Survey Data (SDSS) from the University of Illinois at Chicago to the SC'06 floor at Tampa with a sustained data transfer rate of 8Gb/s over a 10GbE link, and a peak rate of 9.18 Gb/s. The data set was the BESTDR5 catalog data set from the Sloan Digital Sky Survey and when compressed consisted of 60 files of about 23 GB each and totaling 1.3 TB. The technology that made this easy was an open source peer-to-peer storage solution, powered by DataDirect's high performance S2A9550, called SECTOR that NCDM. SECTOR is built using UDT, an open source high performance network transport protocol designed to distribute large e-science data sets such as the Sloan Digital Sky Survey. This was a major new milestone that demonstrated that it is now practical for working scientists to transfer large data sets from disk-to-disk over long distances using a 10GbE network versus traditional physical mail carriers. Additionally, DataDirect's S2A9550 high performance storage system also enabled the 2nd and 3rd place winners of the Bandwidth Challenge. The 2nd place team consisted of CalTech, CERN, University of Florida and the University of Michigan for their project "High Speed Data Gathering, Distribution and Analysis for Physics Discoveries at the Large Hadron Collider." The 3rd place team consisted of Indiana University, Pittsburgh SuperComputing Center and the Oak Ridge National Laboratory for their project "All in a Day's Work: Advancing Data Intensive Research with the Data Capacitor."