Global supercomputer leader Cray Inc. today announced that overall customer-reported scores for the Cray X1 supercomputer system are the best for any high-performance computing (HPC) system on the new HPC Challenge benchmark tests. The new set of tests, co-sponsored by the DARPA HPCS (High Productivity Computing Systems) program, the U.S. Department of Energy and the National Science Foundation, was introduced at the SC2003 conference in November 2003. Assembled by Jack Dongarra and Piotr Luszczek of the University of Tennessee, with collaborators from the U.S. and Europe, the HPC Challenge benchmark suite is intended to test multiple attributes that can contribute substantially to the real-world performance of HPC systems. The suite of tests includes High Performance Linpack, the single test primarily of processor performance that is the basis for the semi-annual TOP500 supercomputer ranking, and substantially augments this with six additional tests. More tests may be added over time, Dongarra said. "Linpack is useful, but no single test can accurately reflect the overall performance of HPC systems," Dongarra said. "The HPC Challenge benchmark test suite stresses not only the processors, but the memory system and the interconnect. It is a better indicator of how an HPC system will perform across a spectrum of real-world applications." Cray X1 Supercomputer Has Highest Reported Scores Based on test results reported by customers including the Army High Performance Computing Research Center (AHPCRC), Oak Ridge National Laboratory (ORNL) and the U.S. Army Engineer Research and Development Center (ERDC), a Cray X1 supercomputer had the highest overall test scores of any system listed as of June 10, 2004. Comparing a 252-processor Cray X1 system with a 256-processor IBM p690 system, for example, the Cray X1 system was about 4 times faster on Linpack, 115 times faster on PTRANS, 18 times faster per processor on STREAM Triad, and 14 times faster per processor on the Random Access MPI test. Comparing a 124-processor Cray X1 system with a 128-processor HP Integrity zx6000 system, the Cray X1 system was approximately 4 times faster on Linpack, 9 times faster on PTRANS, 11 times faster per processor on STREAM Triad, and 8 times faster per processor on Random Access MPI. Results for all listed systems are available at http://icl.cs.utk.edu/hpcc/hpcc_results.cgi. User Organizations Call For Widespread Adoption of HPCC Benchmark Suite "Because the new HPC Challenge benchmark tests are multi-faceted, they provide a more comprehensive insight into the performance of today's high performance computing systems and are a major improvement over traditional single point-benchmarks in predicting how HPC systems will perform on many real-world applications," said AHPCRC Support Infrastructure Director Paul Muzio. "I believe that the new HPC Challenge benchmark test, by incorporating a number of measures in addition to Linpack, should be widely adopted as it is a better indicator of performance." "The HPC Challenge benchmark suite is an important advance that will help us better understand which applications perform best on which HPC systems," according to Thomas Zacharia, ORNL's associate director for computing and computational sciences. "The results available today are already useful. We look forward to widespread adoption of this new measurement tool by the global HPC community." "No single test can accurately compare the performance of HPC systems. Now that the more comprehensive, informative HPC Challenge benchmark suite is available, it should be used in preference to comparisons and rankings based on single tests," said Cray Chairman and CEO Jim Rottsolk. Rottsolk applauded the Department of Energy, the DARPA HPCS program, the NSF, Jack Dongarra and Piotr Luszczek for providing the new tests. "Cray is proud to be the leading performer today on these new, more comprehensive tests that provide a stronger basis for measuring HPC systems. We expect to increase our lead as larger versions of our current Cray X1 product are installed, and as results are submitted for the Red Storm system and the Cray XD1(TM) product we obtained through the OctigaBay acquisition."