SCIENCE

The Stork team at Louisiana State University announced that a new version of Stork Data Scheduler (Stork 2.0) is now available on the Stork Project Web page.

Stork 2.0 comes with a novel "Throughput Estimation and Optimization Service." This new service implements a state-of-the-art model developed by Dr. Tevfik Kosar and his team to predict the optimal number of parallel streams to achieve the best data transfer throughput. The model can generate very accurate predictions with minimal sampling overhead. In Stork 2.0, this model is implemented for GridFTP transfers and the team is planning to implement it for other data transfer protocols in future releases.

This new service can be used for both data throughput estimation and optimization purposes. In case of "estimation," the users can easily predict how long a data transfer will take using a single stream, what would be the optimal number of parallel streams to use for this transfer, and how long it would take using the optimal number of streams etc. In case of "optimization," Stork scheduler will automatically determine the optimal number of parallel streams for the data transfer job submitted to it and will use this setting to perform the transfer.

In addition to this new estimation and optimization service, Stork 2.0 release comes with support for more protocols and more platforms, remote logging capability, checkpointing, security enhancements, and many bug fixes.

The NSF-funded Stork Data Scheduler makes a distinctive contribution to distributed computing community because it focuses on planning, scheduling, monitoring and management of data movement tasks and data resources. Unlike existing approaches, Stork treats data resources and the tasks related to data access and movement as first class entities just like computational resources and compute tasks, and not simply the side effect of computation. Development of Stork is funded through NSF STCI program, and the related research efforts are funded through NSF CAREER program.