ENGINEERING

A team of software engineering students, in conjunction with cloud data giant EMC, have teamed up to tackle a new world record for a computational math and chess challenge and in the process produced a new framework for distributed computing.

John S. Dvorak, Tim Sze, and Ruan Pethiyagoda attend Hack Reactor, a San Francisco-based immersion school for students of computer programming. Sze and Pethiyagoda entered the program in March as novices, while Dvorak entered as an experienced engineer. At the school, they dove head-first into practical projects, guided by former industry programmers from Google, Twitter, and Adobe.

The trio first encountered the algorithmic challenge—known as the N-Queens problem—while working through Hack Reactor’s curriculum, and Pethiyagoda kicked off further experimentation by writing a tweet-sized solution to the algorithm. “It was just over 100 characters,” said Pethiyagoda.

The trio then rewrote their algorithm to run on Hadoop, a distributed computing framework, and worked in conjunction with EMC’s Pivotal Initiative to deploy their code to the organization’s 1,000-computer cluster.

“As of Sunday night, the [Hadoop] cluster was on the verge of calculating the total number of potential solutions for N=27—which would break the world record,” reported Wired magazine. Pivotal’s Hadoop cluster, Analytics Workbench, which processes enormous amounts of data for NASA, Intel, and other well-known entities, was pushed to its limits by the team’s algorithm.

In preparation for the attempt, the team also built a browser-based supercomputing framework named Smidge that distributes work to browsers that visit their website. The effort is similar to prior distributed computing efforts such as SETI@Home, with the exception that no software need be installed by contributors, and that the system can be even be used by mobile devices.

“We were able to scale [Smidge] across every device in the building, including everyone’s laptop, iPhone, Android phone. Even my BlackBerry ran it, which surprised me,” said Pethiyagoda.

Hack Reactor is currently accepting applications for upcoming classes from driven and intelligent would-be software engineers, with or without prior experience.