By Steve Fisher, Editor -- Last week, Entropia, Inc., a leading provider of PC grid computing solutions, announced the availability of DCGrid 5.0, a PC grid computing platform that provides high performance computing capabilities by aggregating the unused processing cycles of networks of existing Windows-based PCs. To delve a little deeper into DCGrid 5.0 and its capabilities, Supercomputing Online interviewed Entropia’s Director of Product Management, Dave Johnson. SCO: Please provide the readers with a highly technical description of DCGrid 5.0. JOHNSON: The DCGrid™ platform provides an open, secure and enterprise-scalable PC grid computing environment that manages, schedules, deploys, and executes compute intensive applications. DCGrid’s patented isolation technology prevents disruption to both the user PC environment as well as the distributed application that runs unobtrusively on the desktop. Unused PC resources are harvested based on user and organizational policies, with settings centrally monitored and managed with a web-based grid management interface. Work is scheduled to PCs based on application resource requirements, and is monitored and rescheduled as necessary if there are system disruptions or resource unavailability. DCGrid 5.0 addresses critical requirements across three key areas of distributed computing: client management, resource scheduling, and job administration. Client Management The Client Management layer provides communication, management, and security for the physical client machines as well as application control for unobtrusiveness. • Communication Services – allow the system and applications to communicate securely, reliably, and conveniently, despite intermittent network connectivity, dynamic IP address assignment, and network features such as firewalls. • Management Services – capture static and dynamic information about each physical client (e.g., physical memory, CPU speed, disk size, available space, client version, data cached, etc.), reporting it to the centralized system server. In addition, the physical client management layer ensures that clients can be recovered from runaway applications and/or poor application behavior. • Security Services – employ a range of encryption and binary virtual machine technologies to protect both distributed computing applications and the underlying physical client. Application communications and data are protected with cryptographic techniques. The virtual machine manages the operations and resources that are visible to distributed applications on the physical clients, controlling access to protect the underlying machine. • Application Control – the virtual machine controls the usage of resources by the distributed computing application. This ensures that the application does not interfere with the primary users of the system – it is unobtrusive without requiring a rewrite of the application specifically for a distributed computing environment. Resource Scheduling A PC grid system managed by DCGrid consists of resources with a wide variety of configurations and capabilities. The resource scheduling layer is responsible for matching individual work units to qualified client machines and then scheduling those work units to the identified client resources. The resource scheduling layer adapts to changes in client resource status, unsuccessful execution of individual work units, and ensures reliable and robust execution of each work unit despite these problems. This layer also provides useful abstractions for IT administrators, that automate the majority of administration tasks with reasonable defaults, but allow detailed control if desired. Job Administration A distributed computing application that performs large amounts of computation (hundreds to thousands of CPU hours) is often designed to operate as a single, self-contained job. Such jobs often arise from statistical studies, exhaustive parameter sweep, or database search applications. The Job Administration layer provides a higher level of management of the units of work that are executing on the various client machines as well as the ability to automate the pre-processing and post-processing requirements of the jobs as they decompose large data sets and recompose large volumes of result data returned by the individual work units. The DCGrid 5.0 release packages this functionality in two modules: DCGrid Scheduler and DCGrid Manager. The core module of the DCGrid™ product family is DCGrid Scheduler. DCGrid Scheduler is a PC grid platform that schedules, deploys, manages and safely executes distributed applications to networked PCs within an enterprise. DCGrid Scheduler: • Safely executes any native Win/32 binary. No source code is required. • Operates in a PC’s spare capacity, harnesses only unused CPU, memory, and disk resources. Pauses work when insufficient client resources exist. • Prevents compromise of PC configuration, programs, and data integrity. • Prevents malicious or accidental misuse of network, operating system, or disk resources. • Matches application requirements to machine resources. Monitors ongoing work and automatically reschedules due to system disruption and resource unavailability. • Easily integrates into existing infrastructure, with use of an available API. DCGrid™ Manager provides a web-based, centralized management console for managing PC grid work and resources. The product’s configurable job wizards simplify the submission of work to the grid. Job workflow capabilities automate the splitting, submission, and gathering of distributed job results. Web-based usage and reporting features provide real-time status of in-flight work as well as the ability to manage priorities, permissions, and other usage policies. Key capabilities include: • Wizard driven interfaces that automate and simplify common tasks. • Central prioritization and control of PC grid usage by user, resource, and application. • Real-time visibility to job status and usage. • Web-based interface that allows job submission and results gathering to be conducted anywhere – even by non-technical people. • Customizable interfaces that allow DCGrid to be extended to meet any application requirement. SCO: How does DCGrid deliver “fast, easy, application integration” across a broad range of computing applications? JOHNSON: DCGrid includes a complete Application Enabling Toolkit that allows any native Win32 executable (and its associated environment, DLLs, etc.) to be prepared for execution within the DCGrid Client Virtual Machine. This toolkit is unique in that it works at the binary level without requiring any recompilation, relinking, or other access to application source modules. There are entire classes of applications (parameter sweep, ab initio, Monte Carlo, etc.) for which this simple preparation step is all that is required to make an application ready to execute on DCGrid – the process literally takes only a few minutes in most cases. Other classes of applications that make use of large input datasets (such as Molecular Docking, genetic sequence comparison, etc.) require the one-time execution of two additional steps: describing to DCGrid how to partition the large dataset into N smaller datasets so that the individual work units can be created; and describing to DCGrid how to combine the results from the individual work units into a single result set that corresponds to the original large input dataset. The Application Enabling Toolkit provides numerous examples of these additional steps; in many cases, one of the existing methods can be used without further modification. SCO: I noticed that Entropia’s patent-pending isolation technology is mentioned several times in your recent press release. Please tell us something about that technology. JOHNSON: The DCGrid Client (the Entropia software running on each of the PCs on the grid) includes a Virtual Machine (VM) in which the guest work unit executes. This VM presents the application with an environment that it cannot distinguish from a native Windows execution environment – yet it can monitor and control every facet of the guest application’s interaction with the host PC. The VM provides a mechanism for absolute control over memory utilization, CPU utilization, thread/process creation, disk consumption, inadvertent use of GUI features, intranet / internet utilization, registry access, etc. so that the presence of the guest application is virtually undetectable by the host PC user. The VM also operates in an obfuscated environment that is only meaningful to the DCGrid Client software – this ensures that even a curious user cannot gain any knowledge of the guest application or its data by examining the contents of the Virtual Machine environment. Further, any attempt to alter the VM environment results in a data integrity violation that is detected by the DCGrid Client. The work unit will be deleted from the current PC and resubmitted to another PC by the DCGrid Server. SCO: Please provide some information on DCGrid’s encryption and tamper protection features that are involved in protecting data and applications on a grid. JOHNSON: Well, the Virtual Machine technology we’ve already discussed provides a lot of these features inside the client machine. Everything (including executables, data, files, and directories) inside the VM is protected with Triple-DES – the data is only meaningful to the DCGrid Client and the guest application running inside the VM. A checksum-like facility that confirms that the encrypted information is unchanged is also provided within the VM – if it detects any change to any file or directory structure, it will not allow the guest application to continue and will cause the work unit to be reloaded on a different client machine for tamper-free execution. SCO: What do you consider to be one or two barriers to enterprises adopting DCGrid and related technologies? How do you plan to overcome those barriers? JOHNSON: There are two primary issues that we are encountering in the marketplace: • Much of the early “hype” and positioning from companies such as IBM and Sun have depicted PC Grid architectures as only existing as part of a vast, far-reaching vision that is not yet achievable. This has created a perception that the technology has not yet developed to a point where it can be productively used. • Historically, PCs have not been viewed as a viable hardware platform on which to deliver high performance computing – the notion of a “Personal” computer does not convey the image of a resource that can be effectively and securely shared to provide a mission-critical computational resource. We overcome this these two barriers by demonstrating that the DCGrid solution is scalable, configurable, and unobtrusive – we do this in the customer’s environment, on the customer’s equipment, with the actual applications requested by the customer. This allows our customers to realize that they can derive value today from the DCGrid products while the larger details of interoperation within and among grid components are worked out. SCO: This is a real stretch, but let's say I’m the head IT honcho at a large company running an even larger grid running the DCGrid platform. On a scale of one to ten (one being easiest), how difficult is it for me to watch over and generally manage everything, and why? JOHNSON: The incremental effort to provide day-to-day administrative oversight for a production DCGrid implementation is surprisingly low. We have implemented a browser-based approach that allows a network administrator (with appropriate permissions for both the network and the DCGrid system) to perform all grid administration functions from a central location. This includes client functions (stop/start/reset, alter operational parameters), user functions (add/change/delete, alter permissions), and job functions (stop/start/pause, alter priority or execution characteristics). Overall, I’d have to give us a 3 on your 1 to 10 scale. The DCGrid 5.0 release goes a long way toward simplifying the overall system management tasks associated with a PC Grid. Of course, there’s still room for improvement – we’re working on enhancements in our upcoming releases that should turn our ‘3’ into a ‘1’. SCO: There are others out there delivering PC grid computing technology. Who do you view as “the competition?” What's Entropia’s edge? How will you prevail? JOHNSON: Entropia believes that any distributed computing system utilizing PC grid technology must include a certain set of core capabilities to be useful in the enterprise environment: • Execute work units on desktop machines • Robust execution of work units even though machines may be turned off unexpectedly • Simple, unattended, automatic installation of new PC clients • Centralized management and monitoring of clients and work units • Unobtrusive behavior of well-behaved guest applications on host machines • Support for heterogeneous PC configurations (hardware capabilities and operating system version(s)) found in most enterprise environments We have two primary competitors: United Devices offers a product that directly competes with DCGrid; Platform Computing offers a component within their product suite that competes indirectly with DCGrid – Platform does not focus on PC Grid technology. Both of these competitive offerings appear to address some of these six points. What differentiates DCGrid is its superior fulfillment of the core requirements while delivering additional key features and benefits: • Three-Dimensional Protection • Protects integrity of desktop machines: DCGrid protects the desktop configuration, programs, and data from corruption by grid application errors as well as the privacy of desktop users from snooping. The grid application cannot accidentally or intentionally access or modify the PC configuration or data files. Unlike other error-prone approaches, DCGrid presents a cleanly isolated, corruption-free environment. • Protects integrity of grid applications and proprietary data: DCGrid shields applications, proprietary data and resources distributed to the desktop PCs by using encryption and tamper detection. Proprietary data and research sent out to hundreds of PCs in an enterprise will be protected from desktop user inspection or malicious corruption. • Protects PC responsiveness and maintains unobtrusiveness: DCGrid automatically monitors and limits grid work so it does not intrude on the PC user. Unlike other solutions, DCGrid remains invisible at all times, never demanding inputs or responses from the desktop user, and never impacting the user’s performance. • Rapid, Safe Application Enabling for any native Win32 application – the combination of ease of installation and rapid enabling yields a quicker time-to-benefit for the DCGrid solution • Job-level Management and Administration • Scalability and Configurability • Focus on desktop environment – scheduling technology to achieve maximum throughput in this environment DCGrid 5.0 delivers on all five of these important points – every organization considering PC grid computing should be concerned about these issues as well. We have found that the best way to appreciate these key differentiators is to experience the DCGrid system in person – seeing an application generate the “blue screen of death” in stand-alone mode on a PC and then seeing that same application execute within the DCGrid Virtual Machine without any adverse effect on that same PC is a compelling demonstration. SCO: Is there anything you'd like to add? JOHNSON: We believe that most organizations in search of additional high-performance computing resources can benefit from accessing the untapped power in their desktop PCs – even a moderately-sized organization with several hundred employees can turn their existing desktops into a computational resource that can outperform a dedicated cluster solution that would cost hundreds of thousands of dollars to acquire and maintain. This compelling cost/benefit equation means that the desktop PC should always be considered as a complementary source of additional high-performance computing resource for most companies.