A Year is a Long Time on the Grid

Dave Newbold is a lecturer in the High Energy Physics Group at the University of Bristol. He was Chair of the GridPP User Board for the year until September. Here, Dave looks back at the high and lows of his time as Chair, how things have changed and what remains to be done.
September 2005, when I took over as Chair of the User Board, now seems a very long time ago. Like all particle physicists, I keep an involuntary historical record in the shape of overfull old email folders. Looking back in time, GridPP members were concerned with topics such as: "How can we encourage experiments to use the Grid?" "What if we build it, and nobody comes?" "Hey - we bought all this disk, and nobody is using it..." A year on, it's hard to recognise this picture. Today, the LHC experiments are running thousands of jobs per day on the LCG fabric, and the UK centres are amongst the most heavily used. Data moves around the network at tens of Terabytes per day, and disks are never empty! Most importantly, as we approach the startup of LHC, we can start to see a real system taking shape that will support physics discoveries in the UK, and the ambitions of the community are starting to challenge the resources we have at our computing centres. Despite all this progress, and in some cases because of it, the last year has been a major challenge to GridPP. The ramp-up of resources at computing centres has suffered unforeseen problems - when you are doubling the scale each year, any delay has a major impact on resource available to users. Middleware scaling issues are once again becoming manifest. We have also begun to challenge the scalability of our organisational structure, with a clear need for closer contacts between experimental physicists and those deploying and operating the systems. However, at the end of 2006, we now have a real feel for the shape and size of the system that will record the first LHC data. We've proved that we can operate the system at the level required for 2007 running - though there is much work to do yet in improving reliability and interfaces. Crucially, "real users" are now confronting the Grid and finding interesting ways to use and abuse it that we didn't foresee a year ago. Less publicly, but just as importantly, we've laid the ground for a continuation of the GridPP project into the LHC running era. I passed the User Board baton to Glenn Patrick just in time to 'get my hands dirty' again in the CMS CSA06 data challenge. That challenge is over and has been a clear success (the champagne has flowed almost as readily as the data files), due in large part to specialists from all areas of GridPP working together in a highly coherent way, focussing on delivery of results for physics. Glenn will face the challenge of making this our clear strategy for the year of LHC start-up, and bringing the 'users' and 'providers' ever closer together as we face a brand new set of technical and political challenges. I wish him the best of luck! We have good reason to look forward to another year's excellent progress - this one marked by the first processing of real LHC data at all the UK centres! Credit: GridPP