Commentary: The next high concept--Grid computing

By Lou Agosta, director, Forrester Research -- Grid computing, also known as utility computing, is a high concept, a computing grand challenge and the "next big thing." What GC is not is a killer application that end-user enterprises need to go out and get. In fact, an information technology department couldn't buy or build one if it tried. It is the latest hot thing to hype, and like moths to a flame, vendors, professional services firms and even some industry analysts are drawn inexorably to it. Caution and selective debunking are clearly appropriate. Yet the high concept of the computing (and data) grids will succeed in driving technology innovations, leveraging Web services and, ultimately, redefining the limits of the possible. Unlike "on demand," "unbreakable" and "demand more," the computing grid is something to which the average consumer can relate. After the power failures in the United States and Canada and separately in Italy, everyone knows what a grid is--a metaphor for shared resources. Of course, the average person will think of the electrical grid. The computing grid is an extension of the metaphor in the direction of shared computational services, and it suffers from some of the same problems--too many moving parts, too hard to monitor, not enough standards and sometimes too many incompatible standards. Call it monumentally unfortunate timing, but no end-user enterprise can be expected to have much patience for a marketing campaign that promises a grid or a computer system that behaves like one and fails unexpectedly, given that we already have an electric power system that does. Further explanation of what works and what doesn't (and why) is needed. The example that best represents the grand challenge in grid computing is the vision of an application that shares the resources of multiple computers. But think about it. The application was written to execute on Windows, and the grid includes Unix, Linux and zSeries operating systems. Where's the abstraction layer to enable those to be shared? Similar considerations apply to different data formats: How are they to be reconciled? How is security handled? If a resource petitions to join the grid, how can the grid administrator verify that the applicant does not innocently bring a virus infection with it? Initially, the grid was conceived to share the vast unused resources of idle processors. But what if the application is mission-critical? Where is the software to negotiate and implement the quality of service to be provided? Standards such as the Globus toolkit are coming online, but uncertainties outweigh what is known. As with most new concepts and new technologies, the terminology is in flux, and different collaborators use the language in diverse ways to emphasize the importance of their own contributions. Since "grid" is basically a metaphor, it is hard to say that any given extension or spin is wrong in itself. However, some uses will become more mainstream than others. The original problem in requiring the construction of grids in the scientific community was driven by the need for collaboration across organizations; the opportunity to pool computing resources to exploit unused processor cycles; and data volumes and computational requirements that exceeded the capacity of existing supercomputer resources. The resulting basic definition of a computing grid: coordinated resource sharing and problem solving in the dynamic context of multi-institutional virtual organizations. This significantly narrowed definition may well redefine other "grid" examples as protogrids or quasigrids. Thus, one can call a clustered computer a grid, but it will not be multi-institutional. One can call peer-to-peer file sharing a grid, and it will be multi-institutional, but in P2P, the participants are not working on solving the same problem or application. Likewise, computing resources (file and processor) distributed across a campus or enterprise may be called a grid, but they are not multi-institutional nor are the resources usually attacking the same problem. Examples of collaboration in the commercial market that can exploit grid computing include vendor-managed inventory, where large volumes of point-of-sale data require aggregation into a data warehouse, possibly across multiple sites, that must be accessed by multiple suppliers. This is fundamentally different than using a Web browser and Secure Sockets Layer (SSL) to look through someone's firewall at what is basically a central data warehouse. Service bureaus and ASPs are also strong candidates to benefit from shared distributed resources that are provisioned across organizational boundaries. Another possibility: Financial services present requirements for fund and information transfers between different organizations across highly secure networks. Recommendations Grid computing is a research project, not a requirement for end-user organizations. Absent a specific high-performance application that requires collaboration across organizations--think of aerospace, automotive engineering or genetic design--commercial enterprises should monitor developments and research but take no further action. Grid computing vendors such as IBM, Oracle, Microsoft and infrastructure vendors such as Sun Microsystems, EMC, Hewlett-Packard will find opportunities for technology innovation and marketing hype within the high concept of the computational grid. Investors will find opportunities to roll the dice on technology innovation and marketing hype. Hackers will find a new source of mischief. The grid is a vision of shared computational, storage and application resources that spans the virtual organization in a highly heterogeneous environment. It is now being tested at academic, scientific and distributed data centers. IBM has the most realistic concept of the grid and is telling the truth about the complexity of the challenge of getting different technology stacks to collaborate over the grid "on demand"; the need for integration of heterogeneous data, processing and applications computing resources; and the requirement for work on standards such as the Globus toolkit and a next generation of Web services. Be skeptical of tools and technology that are repositioned as "grid offerings" in order to make them sound attractive. But be prepared to be amazed once again by technology within five years, as the bounds of the possible are expanded by innovations that are spun off from the initiative, even all of the promises can't be kept.