ENGINEERING

Intel will announce the details of the first release of its Many Integrated Core (MIC) coprocessors, the Intel Xeon Phi, at the international Supercomputing 2012 (SC12) conference in mid-November, at which point many in industry and academia will begin using this innovative technology. The Texas Advanced Computing Center (TACC) at The University of Texas at Austin was selected as the first installation to deploy the new MIC technology at large scale in the forthcoming Stampede supercomputer, funded by the National Science Foundation, and is launching extensive training and technical information activities to help users take advantage of this imminent, innovative technology.

 

"Intel Xeon Phi is going to be an exceedingly popular and powerful mainstream technology, and we are excited to help the world figure out how to use it most effectively — as soon as possible," said Jay Boisseau, TACC Director.

 

Stampede is scheduled to go into full operation on January 7, 2013, with 6400 Dell C8220X nodes comprising 12,800 Intel Xeon E5 ("Sandy Bridge") processors plus several thousand 'special edition' Intel Xeon Phi (MIC) coprocessors. The Xeon Phi coprocessors alone will provide over 7 petaflops of performance, making Stampede a tremendously powerful computational resource for the open science community.

 

The Intel Xeon Phi coprocessor employs a new design that promises greater performance while using less energy per operation. The coprocessors have many (more than 50) small cores and massive parallel threading capabilities, allowing many more computations to be performed simultaneously than on a traditional CPU. However, unlike other many-core accelerators such as GPUs, the Xeon Phi is based on the x86 architecture — the dominant architecture in enterprise, consumer, and research computing. The Xeon Phi is designed to be coded using common programming languages like C/C++ and Fortran, and common parallel programming tools like OpenMP and MPI. This out-of-the-box functionality was one of the characteristics that attracted TACC to the new technology and that convinced them to include the Xeon Phi as the "innovative capability" in their successful proposal to the National Science Foundation for the Stampede system.

 

The hands-on testing of MIC technology on behalf of the computational community began in early 2011, when TACC and other institutions started working with a prototype software development platform. TACC has been testing pre-release versions of the Intel Xeon Phi coprocessor and associated software stacks over the last few months in support of their ongoing Stampede deployment. Leading experts like Karl W. Schulz, TACC Director of Scientific Applications, were able to get large codes to compile, run, and scale on the MIC technology in hours.

 

"We are at an exciting evolutionary crossroads at TACC in the continued pursuit of increased performance per watt necessary to ultimately achieve exascale computing," Schulz said. "Stampede provides not only a boost in the scale and performance of traditional x86/InfiniBand compute infrastructure, but it also provides the academic community with a convenient mechanism to incrementally leverage coprocessors to exploit data and algorithmic parallelism using familiar coding constructs."

 

TACC's expertise lies in optimizing the performance and tuning of its HPC systems and software stack so that scientists can achieve maximum research productivity. The center employs 37 PhDs, more than a third of them focused on HPC performance. Together, they are developing a set of tools and techniques to boost the experience of users on the heterogeneous system, and will begin supporting users worldwide with new training workshops, documentation, and hands-on code optimization.

 

"MIC's support of standard programming languages and tools allow almost any code to be compiled for MIC and natively executed on MIC," explained TACC research staff member Lars Koesterke. "In fact, since the Xeon Phi development environment supports native C/C++ and Fortran cross-compilation and direct login access to the coprocessor, the porting process is generally very straightforward." Koesterke notes, however, that optimization efforts should still be considered after initial porting to maximize vectorization and parallel efficiencies on this new architecture. These are focus areas of early TACC training and documentation.

 

"While the process of porting code to MIC can begin with a simple recompile," Koesterke continued, "this is usually so easy that some users may be fooled into expecting that this is all the effort needed, if they are not informed of the optimization techniques for using MIC effectively in advance. Such users may perceive MICs either as tedious to program for, or as useless, but this false perception is a result of how easy the initial porting effort is — which is in reality a good thing, since optimization effort can begin that much sooner. But users must still optimize to achieve the full performance potential of the MICs, as with any new processor technology."

 

Community experiences and training on the principles and programming paradigms relevant to the MIC architecture began at the two-day "TACC-Intel Highly Parallel Computing Symposium" in April and continued this week with training sessions at Cornell University, a partner on the Stampede project, on November 6. On Monday, November 11 at the Peery Hotel in Salt Lake City, TACC staff will lead a special tutorial on how to use the Xeon Phi coprocessors on Stampede. The "Sneak Preview" during SC12 will include an overview of the Stampede architecture and Intel Xeon Phi coprocessor, discussions about programming models and performance tuning on Stampede, and hands-on exercises on the system. Additionally, TACC director Jay Boisseau, as well as the other primary investigators on Stampede, will give short presentations in the TACC booth on both Tuesday and Wednesday of SC12. [For a full list of Stampede-related talks at SC12, visit the "TACC@SC12" website.]

 

In December and January, leading up to the launch of Stampede on January 7, both TACC and Cornell will offer training, first to early users and then to anyone interested in learning about the new technology. Information from the training will be made available online, helping to prepare the research community for Stampede.

 

In the spring, TACC staff will teach a new class to graduates and undergraduates at The University of Texas at Austin on hybrid programming for heterogeneous systems like Stampede. Taught in the Division of Statistics and Scientific Computing, the class will introduce students to the programming models most relevant to systems that use more than one type of processor architecture, which are becoming the mainstay of the largest HPC systems internationally.

 

To assist in the transition to a new system, TACC develops comprehensive user guides for each of its resources, anticipating questions, explaining new capabilities or aspects of the system, and guiding users through the process of accessing and running jobs on the system. In early December, TACC will release the Stampede user guide, including documentation on the full configuration of the system. Shortly after, a friendly user period will commence and some of the most experienced users from the HPC community will be invited to test, evaluate and provide feedback on the operation of the system.

 

"My group can't wait to get its hands on Stampede," said Omar Ghattas, director of the Center for Computational Geosciences in the Institute for Computational Engineering and Sciences at The University of Texas at Austin and a leader in the computational science community. "This system will be a great boon for our work on solving inverse problems for large-scale models, where observational data are used to infer uncertain parameters in the models, and hence characterize uncertainties in their predictions."

 

The rise of multi-petaflop and eventually exascale systems comes with intense power needs, and a need to take advantage of parallelism on every level. Systems like Stampede that use very fast, power efficient many-core coprocessors like the Intel Xeon Phi represent an evolutionary point on the way forward.

 

"Stampede's order of magnitude leap in performance relative to what had been available to NSF researchers comes at the right time," Ghattas said. "This system's impact will be felt across a spectrum of fields including astronomy, astrophysics, biology, climate, energy, earth sciences, infrastructure, manufacturing, materials, medicine, and social sciences, to name a few."