ACADEMIA

Parabon Computation announced today the latest release of Parabon Crush, a massively scalable, statistical data mining and predictive modeling application that draws its power from Parabon’s Frontier Grid Platform. Now enhanced with a search algorithm developed for NASA, Crush uses the idle capacity of thousands of computers on a Frontier Grid to discover optimal statistical models within high-dimensional datasets that are too computationally challenging for traditional approaches.

Crush statistical models can be used to identify hidden correlations, explain variability and predict future phenomena in practically any domain. Dr. William Petros, Pharm. D., of the Mary Babb Randolph Cancer Center, has used Crush to identify genomic markers that indicate whether a patient will respond to particular chemotherapies. He stated, “Crush revealed significant explanatory models that previous conventional approaches used by experienced, Ph.D. statisticians failed to find.” In industry, Crush has been put to such diverse uses as forecasting stock performance and generating psychometric models of consumer preferences. Government customers have used it to forecast contract overruns and determine the causal factors for the spread of West Nile Virus.

Economists see Crush as a welcome analytical complement to the deluge of information being made available under the transparency initiatives of the Obama administration, such as Data.gov – an online catalog of U.S. Federal datasets. Dr. Antony Davies, Associate Professor of Economics and Statistics at Duquesne University, stated, “Crush allows us to develop statistical models over large datasets that are practically impossible to analyze otherwise. Having all the data in the world is of very little value without analytical tools like Crush to extract value from it.” Davies, a researcher in the fields of econometric and psychometric modeling, went on to say, “The last mile in transparency is analysis. Transparent data can be a double-edged sword because analysts don’t always know what data is valuable and what data is irrelevant. Crush makes transparent data truly valuable by delivering the combination of computing power and analytic tools necessary for separating the valuable from the irrelevant.”

Earlier versions of Crush performed exhaustive searches, examining every possible candidate statistical model, but many datasets have so many combinations of factors that they can never be exhaustively analyzed. For example, it would take 4M years to exhaust a dataset with just 100 factors, even using 10B computers, each evaluating 1M models per second. The latest version of Crush employs a novel evolutionary algorithm that searches arbitrarily large model spaces deeply and effectively without needing to exhaust them. The so-called Opportunistic Evolution (OE) algorithm, which is part of Parabon’s Origin Evolutionary Software Development Kit, is a combination of genetic algorithms specifically designed to maximally leverage grid-scale capacity, enabling Crush to discover important statistical relationships that are practically impossible to find otherwise.

The new release of Crush can be installed as a Microsoft Excel Add-in, enabling modeling runs entirely from within Excel. Crush can also be called from the command line, facilitating its incorporation into data mining workflows. It supports a variety of regression models and has a pluggable architecture that makes it easy to add others. Another example of the growing number of applications powered by Frontier, Crush can be used online across the Parabon Computation Grid or within the enterprise running atop Frontier Enterprise.

“With the addition of evolutionary search, Crush has become the super-powered statistical modeling application that I dreamed of having, nearly fifteen years ago, while working in the investment field,” stated Dr. Steven Armentrout, Founder and CEO of Parabon. “In fact, it was my investigation into how to power such an application, and the realization that no general-purpose grid software platform existed to address problems of this scale, that ultimately led to the formation of Parabon.” He added, “A lot has gone into this release and we’re eager to see what great results the new version of Crush produces for our customers.”