Whamcloud and EMC Collaborate on PLFS and Lustre Integration

Whamcloud announced it is extending its working relationship with EMC Corporation (NYSE: EMC). The relationship between the two companies began over a year ago and promotes the open source availability of Lustre. Whamcloud and EMC, a fellow member of the OpenSFS consortium, are extending their collaboration for an additional year.

Whamcloud and EMC will continue working together to provide deeper integration between the Parallel Log-structured File System (PLFS) and Lustre. As part of their joint efforts, Whamcloud and EMC will continue augmenting Lustre's IO functionality, including the enhancement of small file IO and metadata performance. The two companies will look for multiple ways to contribute to the future feature development of Lustre.

PLFS is a parallel IO abstraction layer that rearranges unstructured, concurrent writes by many clients into sequential writes to unique files (N-1 into N-N) to improve the efficiency of the underlying parallel filesystem. PLFS can reduce checkpoint time by up to several orders of magnitude. Lustre is an open source massively parallel file system, generally used for large scale cluster computing. It is found in over 60% of the TOP100 supercomputing sites.

"This is a great example of the growing Lustre ecosystem. World-class companies like EMC are committing serious resources to expand Lustre into new areas," said Brent Gorda, CEO of Whamcloud. "Fundamentally, we see the Whamcloud - EMC relationship creating greater technology choices for EMC's customers. We think this is a very positive step forward for Lustre."

"Lustre already has a wide spread following in the technology and HPC communities. It helps address many scalability and performance issues that our HPC customers demand," said Percy Tzelnic, Senior Vice President Fast Data Group and EMC Fellow at EMC. "Lustre continues to grow in importance, and we are pleased to be working with Whamcloud to move the technology into new areas."

As part of the collaboration, Whamcloud and EMC will be working closely to modify some portions of the Lustre client and core code, with the goal of getting the Lustre client included in the upstream Linux kernel.