Archival Capacity Grows by 43% to 36 Petabytes to Support Extreme Data-Intensive Needs

The San Diego Supercomputer Center (SDSC) at the University of California, San Diego, has completed a comprehensive upgrade to significantly expand its tape-based archival storage capacity to a total 36 petabytes (PB), or 36 thousand trillion bytes of information. The upgrade maintains the university’s standing as having the largest archival storage capacity of any educational institution in the world.

SDSC recently entered production with 32 x IBM EU6 (TS1130) tape drives to support its archival storage systems (HPSS and SAM-QFS). The new IBM System Storage TS1130 Tape Drive provides higher densities on the tape cartridges, giving SDSC the ability to store up to one terabyte (TB) of uncompressed data per cartridge, up from 700 gigabytes (GB). With a native data rate of 160 megabytes per second (MB/s), storage backups can now be completed up to 54 percent faster than the previous- generation drive E05 (TS1120).

 

With these changes, SDSC’s archival tape storage capacity was increased by 43 percent, to 36 petabytes from 25 petabytes.  To put that in perspective, 36 PB is approximately 1,400 times the digital plain-text equivalent of the entire printed collection residing in the Library of Congress. In addition, SDSC has an additional 3 PB of online disk storage.

According to the latest annual report from IDC, a global provider of information technology information based in Framingham, Mass., the amount of digital information is expanding so rapidly that in 2007, for the first time ever, the amount of information created, captured, or replicated exceeded available storage capacity. Moreover, the report said that the digital universe grew to an estimated 281 exabytes (281 quintillion bytes) last year – 10 percent more than previously forecast. With that figure expected to grow almost 10-fold in just three years, by 2011 almost half the digital information generated will not have a permanent home.
 
“Having adequate amounts of storage capacity for this deluge of digital information is becoming a major issue for every one of us, whether we are in academia, government, or commerce,” said Richard Moore, Deputy Director of SDSC.

SDSC hosts an allocated storage infrastructure for community digital collections for a diverse range of disciplines. Nearly 100 collections are housed at SDSC, including digital data from the Library of Congress, tomographic images of the human brain, astronomical observations from the 2-Micron All Sky Survey, digital visualizations of earthquake simulations, and tsunami data. The collections also include digital videos of bee behavior, Chinese text from the Pacific Rim Library Alliance, and digital images of Japanese art.

SDSC also hosts the Protein Data Bank, a global resource for protein information that to-date has more than 55,000 protein structures catalogued in its online database. SDSC also is partnering with UCSD Libraries, the National Center for Atmospheric Research, and the University of Maryland in a data preservation project called Chronopolis, a geographically distributed data grid that supports the long-term management, stewardship, and access to digital collections. The system incorporates “trust” and reliability through replication, service level agreements, monitoring, and rule-based systems.

The storage upgrades at SDSC follow the opening of a new 80,000 square-foot building expansion last fall, doubling the overall size of the supercomputer center on the UC San Diego campus. The new building contains a new energy-efficient data center, giving SDSC about 18,000 square feet of overall machine room space.