GOVERNMENT

TACC, UTSW pair electron microscopes with Ranger and Spur to open door to biological insights

In 2006, Chris Gilpin found himself limited by the capabilities of interpreting 3D electron microscopy.

For centuries, biologists had used light microscopes to explore molecules and cells through slices. However, slices showed only part of an organism, producing often-unreliable two-dimensional perspectives.

“When using an electron microscope, a single two-dimensional slice through an object can be incredibly misleading about what the three-dimensional shape of that object is,” Gilpin, Assistant professor of Cell Biology and Director of the Molecular and Cellular Imaging Facility at UT Southwestern Medical Center at Dallas, said. “We didn’t want to look at one thin piece of the structure; we wanted to look at all of it.”

Biology experts were beginning to realize that the slices weren’t simply two-dimensional, either. The thickness could be combined with new techniques and tools to create three-dimensional representations of cellular structures, making it possible to see the inner reaches of the cell more clearly, and helping to understand how life operated.

Gilpin wanted to use this extra-dimensional data to create fuller and more telling images of biological structures, but the data created by these technologies was unwieldy and difficult to analyze. Data sets ballooned beyond the point where they could not be interpreted on a workstation or even a campus-wide grid. The three-dimensional possibilities that Gilpin hoped to tap were unrealized.

But when Kelly Gaither came to campus, everything changed.

Gaither, associate director of the Data and Information Analysis group at the Texas Advanced Computing Center (TACC), was also interested in interpreting three-dimensional images and finding ways for computers to intelligently detect structures in complex data.

Gaither told the UTSW audience about TACC’s massive computational capacity and specialized visualization services, which help researchers make sense of large data sets by transforming numbers into interpretable images.

A natural connection was made, and the two began working together.

“It’s the ideal type of collaboration,” Gilpin said. “TACC has the tools and expertise, and I have data sets that could benefit from them. They’re interested in pushing the capabilities on their side, and I have biological questions that need to be answered, and that are helped enormously by having access to TACC’s staff and their skills.”

Over the past few years, Gilpin and Gaither, as well as other TACC visualization experts, have tackled increasingly challenging biomedical imaging questions, each an important component in the ongoing struggle to understand biology and fight disease. From autism to tumor detection, their collaborative research is expanding the capabilities of microscopes and supercomputers in important ways.

Exploring the Causes of Autism

Autism, a brain development disorder characterized by impaired social interaction, has been increasingly diagnosed in the last two decades. However, scientists are genuinely perplexed about the disease and what causes it.

One hypothesis that Gilpin and other researchers at UTSW are exploring proposes that the cause of autism may lie in the structure of synapses, or nerve junctions, in the brains. Believing that a complete picture of the synaptic structure might provide some useful insights, Gilpin started collecting data, together with Dr. Xinran Liu from the department of Neuroscience, to develop a 3-D representation of a synapse.

Synapses contain vesicles that carry neurotransmitters, chemical substances released by the end of a nerve fiber that are thought to play a role in autism. “We want to identify the synaptic vesicles, to know where they are located in a synapse in relation to a particular structure, how tightly packed they are, and how far they are from the docking membrane,” Gilpin explained.

However, electron microscopes, for all their power, only provide researchers a grey-scale value with an x-y coordinate. “That’s all we’re given to try to understand where a structure is and where its boundaries are,” he said. “That’s where the algorithms and the math comes in — to understand how we can get a computer to see what we can see and do the segmentation.”

Data segmentation, teaching computers to find the edges of objects, happens to be Gaither’s specialty. So working with TACC, Gilpin and Gaither have been able to explore the synaptic topography and glean a baseline understanding of how the vesicles and the synapses relate. Though still in its early stages, their collaboration advances the ways microscopes and supercomputers can be used together to propel medicine forward.

“One of the most important things that comes out of this type of research is how little we know about the normal function of cells and tissues,” Gilpin said. “Until we can understand how things work, we can’t understand how they break and how to fix them, and that applies to almost any disease that you can think of.”

Tagging Tumors — Beyond Simple Visualization

A second project that highlights the potential of microscope/HPC collaborations involves a new method of detecting tumors in the body.

Dr. Jinming Gao at UTSW has been trying to identify early-stage tumors by introducing an external marker into the body that sticks to a specific tumor type. The markers are composed of nano-test tubes containing a paramagnetic iron oxide that is highly visible in an MRI scan.

“If we can coat those test tubes with a biologically active molecule that will specifically stick to a particular type of tumor, then we can apply those to a patient, take an MRI and see if they have a tumor,” Gilpin explained. “It’s a screening and diagnostic tool.”

However, the nano-test tubes are so small that researchers need an electron microscope just to see them. At that scale, how could they know if they were successful in filling the test tubes with the even smaller iron oxide molecules?

“We had a 2-D view of the tube, and that didn’t tell us anything about how full the tube was,” Gilpin said. “It could look full and not be full. So we did the 3-D reconstruction and got some images showing that the particles stuck to the walls of the tube and the center of the tube was empty.”

Their combined abilities in imaging, data segmentation, and visualization allowed Gilpin and Gaither to tell the researchers that they needed to try a new approach if they wanted their nano-tubes filled, which has inspired a new round of questioning and experimentation.

Visualization + HPC = The Whole Story

As cutting-edge as Gilpin’s three-dimensional visualizations on TACC’s Maverick system were, they often still fell short of a full accounting of the biological terrain. Visualization frequently needs to be accompanied by complex computation and analysis to mine data, measure, and determine the nature of cell structures, in order to provide a full picture.

The size of the three-dimensional data sets Gilpin hoped to work with, a petabyte for a single amoeba, exceeded the capacity of specialized visualization clusters and often required large migrations of data.

“It was clear that once a machine like Ranger came online, people would be able to generate data sets that were so significant in size that the previous model of shipping the data off somewhere and then post-processing it at a single vis workstation was going to breakdown,” Gaither said.

Recognizing this challenge, in October 2008, TACC added a first-of-its-kind large-scale visualization system, named Spur, directly into the Infiniband fabric of Ranger, their petascale HPC system. “Adding graphics directly into the interconnect of a computational resource of this scale is unprecedented,” Gaither said.

Composed of 32 NVIDIA GPU processors, Spur lets researchers visualize enormous data sets while simultaneously performing calculations to facilitate quantitative aspects of the research.

“It’s very inefficient to move data sets to another platform for measuring,” Gilpin said. “So wouldn’t it be nice if we had a hardware platform that could do all the analysis without shuffling data around? That’s what Spur will do.”

Together, these capacities significantly expand the reach of Gilpin’s research. “We’ll be able to do both visualization and the computational side of segmentation and measuring all on the same platform, and be able to analyze larger data sets with it.”

Growing Science

Gilpin is not only a researcher; he is an enabler of science. At UTSW, he runs a microscopy lab that lets hundreds of scientists dig more deeply into their research. And through his collaborations with TACC, he has helped instigate new tools, new techniques, and a new vision of microscopy-HPC synergy.

“He’s a scientist that sees beyond the boundaries of what he can do today,” Gaither extolled. “He’s energized by technology and he’s not looking at what he could do even five years or ten years from now. He’s thinking about how to get from where he is now to what is really needed to cure cancer.”

To that end, Gilpin is involved in discussions with the TeraGrid to develop a web-based portal, so researchers with all types of three-dimensional data can visualize their structures remotely, capitalizing on the knowledge he has developed at TACC.

With Ranger and Spur now at his disposal, Gilpin can ramp up his specific research, as well the development of methods that will ultimately help the medical profession as a whole.

“Not every campus is going to have a Ranger. Yet through TACC and the TeraGrid, I have access to Ranger and can do valuable research,” Gilpin said. “I think that’s really the great community service that TACC and the TeraGrid are providing.”

******************************************************************************************************************************

The Ranger supercomputer is funded through the National Science Foundation (NSF) Office of Cyberinfrastructure “Path to Petascale” program. The system is a collaboration among the Texas Advanced Computing Center (TACC), The University of Texas at Austin’s Institute for Computational Engineering and Science (ICES), Sun Microsystems, Advanced Micro Devices, Arizona State University, and Cornell University.

Ranger is a key resource of the NSF TeraGrid (www teragrid.org), a nationwide network of academic HPC centers, sponsored by the NSF Office of Cyberinfrastructure, which provides scientists and researchers access to large-scale computing power and resources. Teragrid is a partnership of people, resources and services that enables discovery in U.S. science and engineering by providing researchers with access to large-scale computing, networking, data-analysis and visualization resources and expertise.

For more information visit the website of the Molecular and Cellular Imaging Facility at UT Southwestern Medical Center at Dallas.

Aaron Dubrow
Texas Advanced Computing Center
Science and Technology Writer