ACADEMIA

To achieve higher resolution weather models that produce extremely precise forecasts for United States military operations all over the world, the U.S. Navy's Fleet Numerical Meteorology and Oceanography Center (FNMOC) has again turned to Silicon Graphics. FNMOC has added to its arsenal of SGI Origin family supercomputers a recently purchased SGI Origin 3900 supercomputer, additional service and support, and on-site SGI Professional Services. The total contract is valued at $3.6 million. Located in Monterey, Calif., FNMOC is one of the world's leading numerical weather prediction (NWP) centers and employs sophisticated meteorological and oceanographic models to forecast all aspects of the air-sea environment. FNMOC's customers include all branches of the Department of Defense (DoD), other government organizations such as the National Oceanic and Atmospheric Administration (NOAA) and the National Weather Service, a number of colleges and universities, and the general public. FNMOC processes more than 6 million observations per day, resulting in about 500,000 daily oceanographic and atmospheric charts, analyses, forecasts, and related data sets projected out to a week and beyond. FNMOC produces an integrated air-sea weather forecast for the entire world every 12 hours, 365 days a year, as well as innumerable higher-resolution regional weather and ocean forecasts world-wide. Since 2001, FNMOC has relied on the robust operations of the SGI(R) Origin(R) 3000 family of high-performance compute (HPC) servers and supercomputers, storage networks, and services to fulfill its mission. All systems are integrally linked by SGI(R) InfiniteStorage Shared Filesystem CXFS(TM) global shared filesystem that allows sharing of data across all operating systems, with no copying (or data/file redundancy) necessary. FNMOC employs two primary models, the Navy Operational Global Atmospheric Prediction System (NOGAPS) and the regional Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS(R)), along with a wide variety of specialized ones. High resolution regional weather prediction modeling and storage requirements have risen dramatically and the new 256-processor SGI Origin 3900 supercomputer was added to the SGI(R) InfiniteStorage SAN to run these and future models at optimal speeds as well as to support some special Navy needs. "At FNMOC the demand for higher resolution products is always increasing and as resolution increases, computer requirements rise exponentially," said Mike Clancy, Chief Scientist and Deputy Technical Director, FNMOC. "At the same time, the models coming out of the R&D effort -- mainly by FNMOC partner, the Naval Research Laboratory (NRL) -- are getting more sophisticated. These sophisticated models generally require more computer time to produce a better answer. FNMOC's road map looks ahead five or six years and demand for supercomputing cycles and terabytes of storage soars, year after year. SGI's been a wonderful partner and we've been very pleased with the relationship. They've worked really well with us, and it's been a really good evolution." For instance, in 2001, FNMOC targeted 100 gigaflops sustained as the performance level required during the changeover from Cray to SGI(R) Origin(R) HPC systems. Because of the continuous nature of FNMOC operations, an architecture that could scale in place to meet future performance goals (200 GFLOPS in 2003 and 400 GFLOPS in 2005) was considered critical, leading to the choice of the scalable SGI(R) NUMAflex(TM) computer architecture. With the September purchase of SGI Origin 3900, FNMOC surpassed the 2003 requirement, reaching a level of 240 gigaflops (GFLOPS) sustained. "It's important to emphasize 240 gflops sustained," noted Clancy. "Many centers talk about computer power in terms of peak performance, and typically sustained performance on the SGI Origin is about 10% of peak. FNMOC's peak performance is actually about 2,400 GFLOPS on the floor, which translates to 2.4 Teraflops peak. But what really counts in the business of 365x24 weather forecasting is sustained performance -- the average rate of the calculations based on the workload." FNMOC's workload is about to expand with a significant national effort to field the next-generation regional weather prediction model, the Weather Research and Forecast (WRF, pronounced "wharf") model. WRF will be a modeling framework that will allow several different regional model configurations to be implemented operationally to meet specific customer requirements. It will also facilitate multi-model ensemble weather prediction at the regional scale, in which multiple forecasts with different supporting data and model configurations are made for the same weather event to quantify forecast uncertainty. The Navy has joined the WRF program along with Air Force, NOAA and academia, and COAMPS science and technology is currently folding into the WRF development effort. In the next couple of years, FNMOC will transition from running COAMPS to running the Navy implementation of WRF. The compute power of the new SGI Origin 3900 and the robust, scalable architecture of SGI InfiniteStorage solutions are expected to play a vital role in ensuring the WRF model's requirements are met. "WRF is ultimately going to be a better model, mainly because it will leverage a broad national effort in R&D related to weather prediction," continued Clancy. "WRF will drive the requirements for supercomputer cycles ever higher. As models become more sophisticated they require more and more computer power, and we will want to apply WRF in higher and higher resolutions, leading demands on supercomputing and storage capacity to escalate dramatically." A High-Performance, Shared-Access Storage Architecture The storage architecture at FNMOC is as critical to the success of ongoing operations as the high-performance computing systems. FNMOC currently requires storage capable of supporting throughput of up to 2TB every 12 hours. The storage architecture was designed by SGI with no single point of failure to support FNMOC's availability goals. To provide the necessary storage and throughput to meet the needs of its SGI Origin 3000 series systems, FNMOC has deployed a high-speed Fibre Channel storage area network (SAN) with 13TB of high-performance disk storage in SGI(R) InfiniteStorage TP9400 RAID 5 arrays. Redundant Brocade(R) SilkWorm(R) switches and multiple Fibre Channel connections to each system and storage array provide the necessary bandwidth and availability. FNMOC's 13TB of online storage is backed by 100TB of nearline storage in a StorageTek PowderHorn(R) 9310 automated tape silo with eight high-speed StorageTek 9840 Fibre Channel tape drives. The combination of online and nearline storage provides an easily managed virtual storage pool of nearly unlimited capacity. The servers are joined in an SGI InfiniteStorage Shared Filesystem CXFS cluster, providing simultaneous shared access to stored data. CXFS enables multiple systems to access the same files while maintaining strict data integrity. Shared access is essential at FNMOC, because all models rely on the same database of weather and ocean observations. "The nice thing about SGI CXFS, it allows all machines in the cluster to basically share the disk storage and that's really important because we like to have a lot of flexibility," Clancy said. "Between the four large multiprocessor SGI Origin systems, we may decide to move jobs around from one to the other based on the complexity of things we need to do. By having this shared filesystem, CXFS, it gives us a lot of flexibility and it's very redundant, too, in terms of no single points of failure. Plus, we're migrating to an Intel-based Linux solution on the product distribution side. There's a security bridge we developed that allows us to easily move data over from the CXFS SGI side." In addition, FNMOC uses SGI Origin 3000 series systems to provide hierarchical storage management (HSM) services using SGI(R) Data Migration Facility (DMF), the most advanced HSM platform available. As with CXFS, FNMOC has the capability to failover DMF from one server to the other to ensure availability. DMF automatically migrates data from online storage to tape-based storage according to administrator-defined criteria-most commonly based on file size and age. Files are automatically recalled to online storage as they are accessed without user or system administrator intervention. "From the Persian Gulf to the Sea of Japan, every day U.S. forces around the world depend on FNMOC to deliver accurate weather forecasts to plan and carry out important missions. FNMOC uses SGI Origin 3000 systems because they are the world's most scalable microprocessor-based supercomputers, expandable to over 1,000 processors in a shared-memory environment," said Dr. Jill Matzke, manager of environment marketing at SGI. "Because of the close cooperation, collaboration and partnership of FNMOC and SGI personnel, FNMOC succeeds daily in its stated mission: To combine innovative technology with the best available science in order to provide the best weather and oceanographic products, data, and services to the operating and support forces of the DoD anywhere, anytime. SGI is very proud of our long association with FNMOC and proud that our products deliver the robustness and reliability the situation demands now and into the future."