GOVERNMENT
SPEC releases CPU2006 benchmarks
Update of standard for computer performance evaluation improves applications and testing for new generation of systems -- The Standard Performance Evaluation Corp. (SPEC) today released SPEC CPU2006, the latest version of the non-profit group’s software for evaluating compute-intensive performance across a wide range of hardware systems. SPEC’s CPU benchmarks have been the worldwide standard for measuring compute-intensive performance since their introduction in 1989. 
SPEC CPU2006 was developed by SPEC's Open Systems Group (OSG). It measures component- and system-level performance for a wide variety of operating systems and hardware that ranges from desktop systems to workstations to large-scale servers. SPEC CPU2006 replaces SPEC CPU2000, which will be phased out over the next six months. Performance results from SPEC CPU2006 cannot be compared to those from CPU2000, since new benchmarks have been added and existing ones changed. “Like bread, benchmarks need to stay fresh,” says Walter Bays, SPEC president. “SPEC CPU2000 was a remarkably successful performance measurement tool, but it’s time for a new standard that better reflects changes brought on by the latest computer technology.” Real-world performance testing SPEC CPU2006 is a useful tool for anyone interested in how hardware systems will perform under compute-intensive workloads based on real applications. This includes computer users, buyers evaluating system options, hardware system vendors, researchers, and application developers. Those who do not own a SPEC CPU2006 license can track performance results on SPEC's web site. Actual applications provided as source code are the basis for SPEC CPU2006 benchmarks, which exercise a computer’s processor, memory architecture and compilers. Improvements in SPEC CPU2006 include new and expanded benchmarks that represent a wider range of application areas; larger applications with greater complexity that better represent what today's users are running; and larger and longer-running workloads that take into account today's system capabilities and provide headroom for future performance growth. SPEC CPU2006 benchmarks were selected from applications and source codes running across multiple platforms. Some were derived from an industry-wide search that paid up to $5,000 for application code and datasets. Each benchmark was tested on different platforms to improve portability and to determine if it was relevant and suitable for the final SPEC CPU2006 suite. What the benchmark measures SPEC CPU2006 includes two benchmark suites: CINT2006 for measuring compute-intensive integer performance, and CFP2006 for compute-intensive floating point performance. The CINT2006 suite includes 12 application-based benchmarks written in C and C++ languages. CFP2006 includes 17 CPU-intensive benchmarks written in C, C++, Fortran, and a mixture of C and Fortran. Performance metrics within SPEC CPU2006 measure system speed and throughput. The speed metrics, SPECint2006 and SPECfp2006, compare the ability of a computer to complete single tasks; in this case, each benchmark within SPEC CPU2006. The throughput metrics, SPECint_rate2006 and SPECfp_rate2006, measure the rate at which a computer can complete a number of tasks in a given amount of time. SPEC CPU2006 run and reporting rules permit users to compile the application source code in two different ways that reflect common user preferences. Base rules provide stricter compilation guidelines for those who prefer a relatively simple build process. Peak rules have less strict requirements, and are designed for those willing to invest more time and effort in developing build procedures to maximize performance. A supporting toolset in SPEC CPU2006 improves benchmarking consistency for different users and testing environments. New features include automated reporting of compiler flags, a config file preprocessor, and expanded documentation. A Sun Ultra Enterprise 2 workstation with a 296-MHz UltraSPARC II processor is the reference machine for SPEC CPU2006. All benchmark results are computed as ratios against the reference machine, so bigger scores are better. An initial set of 70 SPEC CPU2006 results based on testing from nine different SPEC member companies are available at the SPEC web site. Now shipping SPEC CPU2006 (CINT2006 and CFP2006) is available for immediate delivery on DVD from SPEC. The cost is $800 for new customers, $400 for upgrades from SPEC CPU2000, and $200 for qualified educational institutions. General requirements for running the benchmarks include a Windows or UNIX operating system, a DVD drive, 1 to 2 GB of memory, 8 GB of unused disk space, and a set of compilers. Additional product and ordering information is available on the SPEC web site at its Web site.
SPEC CPU2006 was developed by SPEC's Open Systems Group (OSG). It measures component- and system-level performance for a wide variety of operating systems and hardware that ranges from desktop systems to workstations to large-scale servers. SPEC CPU2006 replaces SPEC CPU2000, which will be phased out over the next six months. Performance results from SPEC CPU2006 cannot be compared to those from CPU2000, since new benchmarks have been added and existing ones changed. “Like bread, benchmarks need to stay fresh,” says Walter Bays, SPEC president. “SPEC CPU2000 was a remarkably successful performance measurement tool, but it’s time for a new standard that better reflects changes brought on by the latest computer technology.” Real-world performance testing SPEC CPU2006 is a useful tool for anyone interested in how hardware systems will perform under compute-intensive workloads based on real applications. This includes computer users, buyers evaluating system options, hardware system vendors, researchers, and application developers. Those who do not own a SPEC CPU2006 license can track performance results on SPEC's web site. Actual applications provided as source code are the basis for SPEC CPU2006 benchmarks, which exercise a computer’s processor, memory architecture and compilers. Improvements in SPEC CPU2006 include new and expanded benchmarks that represent a wider range of application areas; larger applications with greater complexity that better represent what today's users are running; and larger and longer-running workloads that take into account today's system capabilities and provide headroom for future performance growth. SPEC CPU2006 benchmarks were selected from applications and source codes running across multiple platforms. Some were derived from an industry-wide search that paid up to $5,000 for application code and datasets. Each benchmark was tested on different platforms to improve portability and to determine if it was relevant and suitable for the final SPEC CPU2006 suite. What the benchmark measures SPEC CPU2006 includes two benchmark suites: CINT2006 for measuring compute-intensive integer performance, and CFP2006 for compute-intensive floating point performance. The CINT2006 suite includes 12 application-based benchmarks written in C and C++ languages. CFP2006 includes 17 CPU-intensive benchmarks written in C, C++, Fortran, and a mixture of C and Fortran. Performance metrics within SPEC CPU2006 measure system speed and throughput. The speed metrics, SPECint2006 and SPECfp2006, compare the ability of a computer to complete single tasks; in this case, each benchmark within SPEC CPU2006. The throughput metrics, SPECint_rate2006 and SPECfp_rate2006, measure the rate at which a computer can complete a number of tasks in a given amount of time. SPEC CPU2006 run and reporting rules permit users to compile the application source code in two different ways that reflect common user preferences. Base rules provide stricter compilation guidelines for those who prefer a relatively simple build process. Peak rules have less strict requirements, and are designed for those willing to invest more time and effort in developing build procedures to maximize performance. A supporting toolset in SPEC CPU2006 improves benchmarking consistency for different users and testing environments. New features include automated reporting of compiler flags, a config file preprocessor, and expanded documentation. A Sun Ultra Enterprise 2 workstation with a 296-MHz UltraSPARC II processor is the reference machine for SPEC CPU2006. All benchmark results are computed as ratios against the reference machine, so bigger scores are better. An initial set of 70 SPEC CPU2006 results based on testing from nine different SPEC member companies are available at the SPEC web site. Now shipping SPEC CPU2006 (CINT2006 and CFP2006) is available for immediate delivery on DVD from SPEC. The cost is $800 for new customers, $400 for upgrades from SPEC CPU2000, and $200 for qualified educational institutions. General requirements for running the benchmarks include a Windows or UNIX operating system, a DVD drive, 1 to 2 GB of memory, 8 GB of unused disk space, and a set of compilers. Additional product and ordering information is available on the SPEC web site at its Web site.