Grid-enabled tools continue to facilitate global collaboration. Learn more at the 2012 EGI Technical Forum, 17-21 September in Prague

This summer will go down in history for its transformative scientific achievements. While 2012 gets all of the glory, the discovery of the Higgs boson particle, for example, would not have been possible without 48 years of dedicated effort by the global High Energy Physics (HEP) community which drove much of the technical innovation that enables multinational collaboration in all research domains today.

Their quest began in 1964 when theoretical physicist Peter Higgs, and others, described the mechanism that would explain the origin of mass. Of course it took many years for the theory to be accepted by the community. Then, useful technology was developed, on many fronts, which accelerated the process of discovery. Arguably, the introduction of grid-enabled technologies was key to their success. 

It all began in 1995 in San Diego, California at the Supercomputing conference where a team led by Ian Foster (Argonne National Laboratory and University of Chicago, US) demonstrated the successful distributed execution of a number of applications at the 17 geographically distributed sites participating in the I-Way experiment, using a middleware called I-Soft (that would later, in collaboration with Carl Kesselman and his colleagues at the University of Southern California, US, become Globus Toolkit).

In the US, Globus Toolkit continues to provide homogeneity, with XSEDE, Open Science Grid (OSG), and many other projects depending on it. In Europe, several countries and domains embraced the concept, and new middleware varieties were funded and developed for specific applications since 1996. With four or five-year funding, not all have survived. With global collaborations, navigating disparate middleware presents challenges, and federated e-Infrastructures have found that variety is difficult to sustain in terms of development and funding. This is probably why the number of options in Europe dropped from five common in 2007, to four dominant in 2011 (gLite, ARC, Globus Toolkit, and UNICORE)—with two of these besides Globus Toolkit (gLite and ARC) including Globus components. Only Globus Toolkit and UNICORE are common to PRACE and EGI and have the ability to bridge between the e-Infrastructures by offering a common interface to the user. In the US, OSG continues to depend heavily on both Globus Toolkit and Condor Project software as well as community-developed software for handling its massive amounts of data and jobs.

In late 2002, the HEP community formed a coordinated effort known as the Large Hadron Collider (LHC) Computing Grid, or LCG, which leveraged LCG-2 middleware. This would become their high-throughput highway to the LHC at CERN (European Organization for Nuclear Research) near Geneva, between Switzerland and France. LCG involved high-throughput distributed resources from the OSG in the US and Europe’s Enabling Grids for E-sciencE (EGEE, which became European Grid Infrastructure (EGI) in 2010). There were four major experiments at CERN, but the ATLAS and CMS (Compact Muon Solenoid) projects were launched to cross-check and verify Higgs boson findings. ATLAS and CMS each represent a vast multinational collaboration of more than 3,000 physicists from 41 countries and 179 institutes, with some overlap. They built upon research by many projects which leveraged the Large Electron Positron (predated LHC at CERN); the US Department of Energy’s Tevatron Collider at Fermilab; and the Stanford (University-US) Linear Accelerator Center (SLAC). In 2010, high energy capability was introduced to the LHC (first operational in 2008). That’s when the HEP community finally had what they needed to prove Higgs’ theory on the 4th of July, 2012. EGI Deputy Director Catherine Gater chronicled the five years leading up to the discovery in an 11 July, 2012 International Science Grid This Week (iSGTW) feature.

While the global HEP community was first to embrace grid technologies to this extreme, today research teams from all arenas span the globe in pursuit of life-transforming discoveries. Their workflows include a variety of resources and leverage advanced networks to engage the high-throughput systems represented by EGI and OSG, plus high-performance supercomputers (HPC), storage, visualization resources, and expertise offered by the Partnership for Advanced Computing in Europe (PRACE) and the eXtreme Science and Engineering Discovery Environment (XSEDE) in the US. To facilitate this diversity, XSEDE includes access to OSG as a supported resource allocation request. There is also a joint process that allows EU-US collaborative teams to submit unified requests for allocations of PRACE and XSEDE resources (the 2012 deadline is 15 September, 2012).

Last spring’s EGI Community Forum in Munich, Germany, was co-located with the Initiative for Globus in Europe’s (IGE) annual user conference, the European Globus Community Forum (EGCF). During the conference, IGE signed a memorandum of understanding with the European Middleware Initiative (EMI), a close collaboration of Europe’s major middleware providers. IGE and EMI deliver middleware components for deployment by European e-Infrastructure providers that facilitate multinational collaboration. Through IGE and EMI’s relationship with EGI, a quality assurance process was established to specify requirements, test, solicit feedback, and apply lessons learned in an effort to continuously improve EGI’s offerings.

EMI is a three-year project that engages European users and global infrastructure providers to assess specific needs, identify redundancies, and develop a collection of consolidated and harmonious software components. Deliverables include three major releases and subsequent minor revisions, as necessary. Each set is designed to comply with open-source guidelines and to integrate with Europe’s mainstream operating systems. Major releases include Kebnekaise (EMI-1, 12 May, 2011); Matterhorn (EMI-2, 21 May, 2012); and Monte Bianco (EMI-3, 28 February, 2013).

Although many consider the Globus Toolkit to be US software, it is open source and its developer and user communities include many Europeans who recognize its value. On 25 October 2010, IGE’s roadmap was presented by Steve Crouch (UK-University of Southampton) and Helmut Heller (Germany-LRZ) at the first EGI Technical Forum in Amsterdam. At that time, EGI’s Unified Middleware Distribution (UMD) officially recognized IGE as a technology provider. Their plan included timelines for the integration of resources by European e-Infrastructure providers, including EGI, PRACE, and EU-IndiaGrid2.

Globus Toolkit has been widely used in Germany since their D-Grid initiative began in 2005. The Leibniz Supercomputing Centre (LRZ) in Munich installed it on its supercomputers in 2002. Europe’s fastest computer, the SuperMUC, became operational at the LRZ in August this year. SuperMUC and LRZ are committed to serve IGE-supported middleware and will most likely be driving forces for future development and use of Globus Toolkit by Europe’s scientific community.

Globus Online Software-as-a-service

At the GlobusWORLD 2012 conference in Chicago last April, Foster (Globus Project co-founder) quoted the late Steve Jobs (Apple) who said “Start with the customer experience and work back toward the technology—not the other way around.” Applying this philosophy and a commitment to continuous improvement, Foster and the Globus team recently launched a new effort that leverages cloud technologies to develop the Globus Online software-as-a-service (SaaS) offering. With hosted, professionally operated services, and intuitive Web 2.0 interfaces, Globus Online increases usability and functionality dramatically relative to past grid software. The SaaS model makes it easy to deliver new features and rapidly refine the service’s capabilities based on early user feedback. When EU countries add the Globus Toolkit (in particular, GridFTP and MyProxy servers) to their middleware stack, they can take advantage of Globus Online services without requiring additional software.

At the March IGE meeting, the University of Chicago’s Steve Tuecke, Globus Online co-founder, presented its capabilities and anticipated future development with European interoperability in mind. Globus Online features for high performance, secure file transfer were recently integrated with the ATLAS PanDA workload management system and it is in the testing phase. An upcoming Globus service that simplifies big-data storage and sharing could substantially enhance how the HEP community manages the massive amounts of data generated by the LHC and the new subatomic field of physics research launched by the Higgs boson discovery. Future development will target additional services to offer a comprehensive research data management solution delivered using SaaS approaches.

Of course, the biggest challenge faced by multinational collaborations is satisfying the security and privacy policies of every institution, government, and network along the way. Globus Online incorporates Globus Nexus, a service that manages user identities, including profiles, groups, and information about resources connected to the Globus research cloud. Like all Globus services, Globus Nexus features may be accessed via a Web browser, command line, and a REST-ful programming interface that enables organizations to better integrate Globus services into their infrastructure.

For more information about EGI, IGE, and Globus Online, visit their web sites.

EGI Technical Forum 2012, 17-21 September, Prague, Czech Republic (tf2012.egi.eu)

This year’s event will take place at the Clarion Congress Hotel in Prague, Czech Republic. GlobusEUROPE 2012 is co-located and scheduled for Monday 17 September. The event is hosted by EGI.eu in partnership with CESNET, the consortium of Czech universities and the Czech Academy of Sciences that represents the country in the EGI Council.