Texas Supercomputing

TACC, UT System partnership garners exciting scientific results
High-performance computing (HPC) systems are research accelerators, enabling investigations that can only be achieved with the help of powerful, parallel-processing supercomputers, speeding the path to discovery throughout science and engineering.

The Texas Advanced Computing Center (TACC) at The University of Texas at Austin and the broader UT System (of which UT Austin is one institution) are in the third year of an important partnership. Since 2006, TACC has been providing access to high-end computing resources and services to UT System researchers helping to expand HPC throughout the state and to expand research potential.

Researchers from the UT System’s nine universities and six health institutions have used approximately 60 million CPU hours on TACC’s Lonestar supercomputer over the last three years, and this special access has allowed researchers to explore and address a wide range of scientific and societal problems that would not have been possible without HPC, including weather prediction, oil and water resource modeling and management, and new drug design. Early on in the partnership, UT System made a $3 million investment that contributed to the upgrade of Lonestar from six teraflops (a teraflop is one trillion floating point operations per second) to 62 teraflops of theoretical peak performance.

Over the course of the last three years, TACC and UT System’s HPC users have grown together. As TACC emerged as one of the top centers for advanced computing in the world, scientists throughout Texas have advanced their research, increased their use and proficiency with HPC resources and applications, and, in some cases, progressed to TACC’s Ranger supercomputer, a system with significantly more power.

Below are seven case studies — from aerospace engineering to genomics — that illustrate how UT System researchers have put their partnership with TACC to great use, applying both Lonestar and Ranger to diverse research problems. Together, they prove the value of this system-wide collaboration and the impact that investments in high-performance computing have on scientific and engineering research.

******************************************************************************************************************************

Computational Chemistry & Biophysics
Development of computationally driven hit-to-lead optimization strategies for BRCT inhibitors (and other projects)

Claudio Cavasotto
University of Texas Health Science Center at Houston




Fig. 1 Androgen Receptor Ligand Binding Domain in complex with DHT. Wild type in blue. Colored structure is F764L mutant. Both structures overlayed by protein Calpha atoms.

Claudio Cavasotto, assistant professor in the School of Health Information Sciences at The University of Texas Health Science Center, uses the Ranger supercomputer at the Texas Advanced Computing Center to perform extensive in silico simulations of large biomolecular systems, in order to study their molecular interactions and dynamics. He uses this knowledge to help in the design of novel therapeutic treatments. His high-resolution protein models and molecular dynamics simulations have helped clarify the peptide-BRCT interaction, study the diffusion of glucose in bulk and through nanochannels at the atomic level, and explore the impact of nuclear receptors’ mutations in disease.

With multiple large allocations on Ranger, Cavasotto’s research is at once theoretically-based and highly-applicable, helping to discover the underlying mechanisms of disease, and aiding in the development of future treatment methods and new and improved drugs.

“We are working on several distinct and relevant projects, which share one thing in common: their need of a large amount of CPU time. TACC is the solution to our computational needs. Using their resources has an impact not only on our research, but on the prospect of funding as well.”

Selected publications:

6-Methoxy-N-alkyl isatin acylhydrazone derivatives as a novel series of potent selective cannabinoid receptor 2 inverse agonists: design, synthesis, and binding mode prediction.
Diaz P, Phatak SS, Xu J, Astruc-Diaz F, Cavasotto CN, Naguib M.
J Med Chem. 2009;52:433-44.

The Binding Mode of Petrosaspongiolide M to the Human Group IIA Phospholipase A(2): Exploring the Role of Covalent and Noncovalent Interactions in the Inhibition Process.
Monti MC, Casapullo A, Cavasotto CN, Tosco A, Dal Piaz F, Ziemys A, Margarucci L, Riccio R.
Chemistry. 2009;15:1155-1163.

******************************************************************************************************************************

Atmospheric Sciences
Three-dimensional modeling of Titan's Upper Atmosphere

J. H Waite and Jared M. Bell
The University of Texas at San Antonio

J. H. Waite, adjoint professor in the physics and astronomy department at The University of Texas at San Antonio and scientist at the Southwest Research Institute, and Jared M. Bell, a post-doctoral researcher working with Dr. Waite at the Southwest Research Institute, are utilizing a cutting edge theoretical/computational model to describe the atmosphere of Titan, the largest moon of Saturn and the only object other than Earth where clear evidence of surface liquid has been found.

Their work on the Lonestar supercomputer at TACC is simulating the chemistry, dynamics, and energetics of the Titan atmosphere above 500 km. The model will be compared to ongoing observations by the Cassini Ion Neutral Mass Spectrometer during its extended mission to the Saturn system, and will help direct observations and clarify our understanding of this important celestial body.

Waite’s studies will initially focus on the escape of hydrogen, methane, and nitrogen from Titan's atmosphere and the consequences of isotopic separation. Once this effort is successfully completed, he will turn his focus to seasonal dynamics on Titan and the formation of the northern polar vortex. Waite is the leader of Cassini’s Ion Neutral Mass Spectrometer (INMS) team, and was a recent recipient of a SwRI internal research grant for the development of Advanced Mass Spectrometry Techniques for Future Missions to Titan.

The Texas Advanced Computing Center (TACC) has provided an unprecedented amount of computational resources that are critical in the theoretical modeling of Titan’s Upper Atmosphere. Using Computational Fluid Dynamics (CFD) techniques in a parallel processing environment represents perhaps the most efficient and complete method for exploring planetary atmospheres. In short, the many complexities of Titan’s upper atmosphere could not be investigated fully without the use of the significant computer resources at the TACC.

Selected publications:

“The Process of Tholin Formation in Titan's Upper Atmosphere,” J. H. Waite, Jr.,1* D. T. Young,1 T. E. Cravens,2 A. J. Coates,3 F. J. Crary,1 B. Magee,1* J. Westlake4, Science 11 May 2007:
Vol. 316.

“Simulating The Global Mean Structures of Titan’s Upper Atmosphre Using the Global Ionosphere-Thermosphere Model,” J. M. Bell, S. W. Bougher, J. H. Waite, A. J. Ridley, B. Magee, A. Bar-Nun, G. Toth, and V. De La Haye, Planetary and Space Science, Submitted for Publication in a Titan Special Issue.

******************************************************************************************************************************

Computational Fluid Dynamics
High-Order Large Eddy Simulations for Shock-Blunt Body Interaction – Numerical, Experimental and Analytical Study

Chaoqun Liu and Frank Lu
The University of Texas at Arlington




Flow separation control by active micro vortex generators over a NACA0012 airfoil. Animation shows the countours of instantaneous spanwise vorticity on the mid-plane.

Rapid improvements in computational resources and advances in direct numerical simulation techniques have led to improved flow modeling, allowing researchers to investigate ever more complex flow interactions. Chaoqun Lui, Director of the Center for Numerical Simulation and Modeling and Frank K. Lu, Professor and Director of the Aerodynamics Research Center, both at the University of Texas at Arlington, have been collaborating on a project to explore complex shock interactions at supersonic speeds.

Using the Ranger supercomputer at the Texas Advanced Computing Center, they have been simulating how micro vortex generators (MVGs) — small vanes installed on the leading edge of an aircraft wing to maintain steady airflow over the wing surface — modify the inner structure of a boundary layer to make the layer more resistant to separation, such as when a strong shock impinges on it. A practical advantage of MVGs is that their small size results in less drag than their conventional counterparts; however, no experimental or computational results have been obtained to support this suggestion.

With a large allocation (300,000 SUs) on the world’s most powerful academic supercomputer for open scientific research, the researchers are able to simulate rapidly evolving, three-dimensional shock/boundary-layer interactions. Explaining these interactions will have important ramifications in the design and production of future aircraft and space-shuttles, making them more aerodynamic and less likely to stall.

“TACC has become our major computing resourse and it provides us with first class service,” Chaoqun Liu said. “Without the assistance from TACC, we would not be able to complete our task for the Air Force Office of Scientific Research. “

Selected publications:

Numerical study of passive and active flow separation control over a NACA0012 airfoil, to appear Computers and Fluids. with Shan, Jiang, Love and Maines.

Braun EM, Lu FK, Wilson DR. Experimental research in aerodynamic control with electric and electromagnetic fields. In press: Progress in Aerospace Sciences, doi 10.1016/j.paerosci.2008.10.003

******************************************************************************************************************************

Physics
Modeling of Syngas to Ethanol Synthesis Mechanisms on MoS2 and TM metal surfaces

Min Huang
The University of Texas, Dallas

Ethanol has become an important fuel, used in millions of flexible fuel vehicles built by many manufacturers. A mixture of H2 and CO can be used to synthesize ethanol (CH3CH2OH) using transition metal catalysts such as Molybdenum Disulfide (MoS2) and Rhodium (a member of the platinum family) supported on oxides. These two catalysts can produce alcohols from H2 and CO; however, they have problems of selectivity (unwanted methanol production from MoS2 catalysts) and supply limitation (in the case of Rh). It is therefore highly desirable to develop new catalyst systems that can perform similarly to Rh without using precious metals.

Min Huang, a research fellow in Physics at The University of Texas, Dallas, uses the Lonestar supercomputer at the Texas Advanced Computing Center to glean a better understanding of the structures and properties of the active sites of these industrially relevant catalysts, in order to meet the increasing need for ethanol and the development of new hydrotreating catalysts. Her systematic modeling study of the fundamental structure-properties of Molybdenum sulfide surfaces and TM metal surfaces will provide the necessary understanding of their reaction mechanisms to facilitate the development of new ethanol synthesis catalyst systems, and aid in the transition from a fossil-fuel based world.

Selected publications:

Min Huang and Kyeongjae Cho, “Density Functional Theory Study of CO Hydrogenation on MoS2 Surface”, accepted by Journal of Physical Chemistry C, 2009.

******************************************************************************************************************************




Example of COMPASS profile-profile alignment confirmed by a recently solved 3D structure: The header includes brief information about the hit, followed by COMPASS score and E-value. In this example, the top and consensus sequences for compared profiles are displayed. Position matches with positive scores are marked by '+', identical residues in the two consensus sequences are marked by the residue symbol. Below: A recently solved structure for a member of DUF185 family (PDB ID 1zkd) confirms our prediction. Side chains of the invariant glutamate residues are shown in red, glycine-rich loop is circled.

Biophysics
Precise statistical models of protein sequence comparison;
Large-scale multiple alignments of protein domain families; and,
Measuring accurate evolutionary distances between proteins

Nick Grishin
The University of Texas Southwest Medical Center

With the current surge of genomic data, the task of cataloging genomic sequences is dramatically ahead of our understanding of their roles in processes of life. Experimental characterization of a given biological molecule can be tedious, long, and sometimes very challenging, and consequently the 3D structure is known for a small fraction of protein molecules. Thus, one of the biggest challenges for computational biology is to use the vast pool of genomic information to infer the functional role of proteins, and to facilitate laboratory experiments aimed at understanding molecular machinery in the living cell.

The most effective way to infer structure and function of an uncharacterized protein is to find similar, evolutionarily related proteins that have been studied experimentally. Nick Grishin, associate professor of Biochemistry at the University of Texas Southwestern Medical Center at Dallas and a Howard Hughes Medical Institute Investigator, has developed powerful methods for comparison and classification of related proteins. Methods for detecting extremely remote sequence similarity (COMPASS and ProCAIN) and for accurate alignment of multiple protein sequences (PROMALS and PROMALS3D) compare families of related proteins and find intricate sequence patterns shared by them. To reveal even more distant protein relations, Grishin combined both sequence and 3D structure information and developed a support vector machine (SVM)-based classifier that discriminates between similarities caused by shared ancestry and by evolutionary convergence of unrelated proteins.

Using the parallel processing power of TACC’s Ranger supercomputer, Grishin expects to find many more new connections between distant protein families, which will help uncover the structure and function of previously uncharacterized proteins, and enable researchers to move towards a comprehensive picture of the evolution of the protein world.

“Large-scale computation is essential in computational biology of proteins, where we deal with vast universe of complex and diverse macromolecules produced by billions years of evolution," Grishin said. "TACC resources proved indispensable for our work aimed at protein classification, structure and function prediction.”

Selected Publications:

Sadreyev RI, Grishin NV.
Accurate statistical model of comparison between multiple sequence alignments.
Nucleic Acids Res. 2008 Apr;36(7):2240

Cheng H, Kim BH, Grishin NV.
Discrimination between distant homologs and structural analogs: lessons from manually constructed, reliable data sets.
J Mol Biol. 2008 Apr 4;377(4):1265

******************************************************************************************************************************

Physics
A Density Functional Study of the Surface Electronic Behavior of Actinide Metals

Asok K Ray
The University of Texas at Arlington




The elements from actinium (element 89) to lawrencium (element 103) form a distinct group — the actinides — within the periodic table and are characterized by strong relativistic effects.

A research group led by Dr. Asok Ray, Professor of Physics at The University of Texas at Arlington, has been performing computational electronic structure studies of plutonium (Pu), americium (Am), and curium (Cm) surfaces, and molecular adsorptions on Pu and Am surfaces, using the Lonestar system at TACC. These materials, known as actinides, represent the class of radioactive elements at the bottom of the periodic table and are characterized by the increasing prominence of relativistic effects.

By simulating the electronic structures of actinides, the opportunity exists for Professor Ray and his colleagues to understand how electron correlations affect bonding and lattice structures and how it may be possible to progress beyond one-electron band theories. This has important implications for the physics and chemistry of all elements in the periodic table and may pave the way for a unified electron theory of solids.

Moreover, according to the U. S. Government, about fifty tons of surplus plutonium exists nationwide and must be dealt with appropriately. The study of these materials will guide the U.S. towards a safe disposition of actinide elements like plutonium when necessary. Without the supercomputing facilities at the Texas Advanced Computing Center (TACC), it would not be possible to carry out this research.

Selected publications:

"Does Hybrid Density Functional Theory Predict a Non-Magnetic Ground State for δ – Pu?,” (with R. Atta-Fynn), Europhysics Letters, in press.

“A Relativistic DFT Study of Water Adsorption on δ – Pu (111) Surface,” Chemical Physics Letters, in press.

******************************************************************************************************************************

Condensed Matter Physics
Electronic and Transport Properties of Graphene Nanoribbons

Geun Sik Lee
The University of Texas at Dallas

The future of semiconductors may lie in the use of graphene, an exotic, one-atom-thick honeycomb of carbon atoms with high electron mobility and useful quantum properties, which many believe will replace silicon as the basis for computer chips. Much remains to be understood about this promising material, however its microscopic size and complex manufacturing process make it difficult to test experimentally. Therefore computational simulations, together with laboratory research and theoretical predictions, are necessary to explore graphene’s physical and electronic characteristics.

Geunsik Lee, research associate in the department of physics at The University of Texas at Dallas, uses the Lonestar supercomputer at TACC to perform first-principle simulations of graphene nanoribbon (GNR) systems. Focusing on how the structural and chemical modifications at the edges of GNRs affect their electronic and transport properties, he considers three kinds of graphene edges: zigzag, armchair, and their composite. His simulations will obtain electronic and transport data relating to GNRs with differing edge structures and chemistries, which has not yet been reported in the literature. This, is turn, will help guide experiments, leading to future semiconductor designs and the continued miniaturization of electronic devices.

******************************************************************************************************************************

Additional reading:

More Than A Magnifying Glass
TACC, UTSW pair electron microscopes with Ranger and Spur to open door to biological insights

TACC supercomputer performs laser cancer surgery on canine
Remarkable collaboration with UT's Institute for Computational Engineering and Sciences and M.D. Anderson Cancer Center enables Lonestar to carry out real-time, data-driven treatment

Aaron Dubrow
Texas Advanced Computing Center
Science and Technology Writer