The US Naval Research Laboratory (NRL) has integrated Obsidian's Longbow devices into a next-generation network being developed under the Pentagon-funded “Large Data” Joint Capabilities Technology Demonstration (LD-JCTD) program. The Large Data Program The Large Data mission is to architect, construct and prove-out a next-generation distributed enterprise data storage concept capable of handling very large data volumes and large (streaming) data files. Department of Defense agencies within the US government recognize the infrastructure challenges associated with the quantity of data that will need to be moved, stored and processed in future systems. The Large Data network currently comprises five nodes - four in the mainland US and one overseas. Data centers are located at each node, deploying large-scale data processing and storage. During the trials, this infrastructure has been used to archive, transport and manipulate very high resolution imagery. It is a goal of this program to demonstrate the construction of large-scale data center infrastructure with an emphasis on open standards and free open-source software where possible. This maximizes the potential for the widespread adoption of the architecture across many application domains. The approximately $50M program started in May 2006 and has been extended into four years to perform operational testing and evaluation. At the end of the demonstration phases, transition teams work to migrate the new capabilities into military operational deployments. Obsidian's Longbow Technologies The Large Data architecture's main data flows are built entirely around the InfiniBand protocol due to performance, scalability, cost and technology road map considerations. NRL initially contracted Obsidian's engineering team in 2004 to develop a prototype device capable of carrying InfiniBand over global distances. Such capability is critical to the Large Data architecture because without it, InfiniBand would be limited to just meters of range. Subsequently, Obsidian was formed to pursue the development of the prototypes into carrier-grade products in 2005. NRL supplied product specifications to Obsidian to ensure the Longbows met the needs of Large Data-like networks. Longbow XR devices are used to connect some of the 10 Gigabits per second server and storage network links in the Large Data network, paired with military-grade encryptors. A slower overseas link, at 2.5 Gigabits per second, is handled using a more suitable second source device. Longbow Campus devices are used in Large Data to transport data over a few kilometers within the physically secured facilities of several of the larger sites involved in the program, using direct fiber optic cable connections. Additionally, Longbows are employed in an unclassified testbed as delay generators providing system support by emulating the operational network environment. James Hofmann, the Large Data JCTD Technical Manager working at the Naval Research Laboratory, noted that: “During the first three years, two important milestones were achieved by the program: * A limited military utility assessment featuring real military analysts utilizing the large data architecture and reporting significant time savings and increased production and availability of data * An independent assessment which found the Large Data architecture to be highly cost effective and would improve performance greatly over current solutions in a distributed data storage environment. "The program is on track to achieve its primary demonstration goals due in no small part to the use of the Obsidian Longbow XR and the Longbow Campus devices." On future developments, James Hofmann added: “Currently, the Large Data JCTD is supporting an intensive year of operational evaluation and “transition” testing within DoD agencies. The operational evaluation will provide a full scope evaluation of transport technologies and an initial concept of operations for military use. NRL is conducting a series of performance tests and initial designs to integrate the technologies into current architectures and influence emerging architectures. The output and vision that the Large Data JCTD supports is very much aligned with the vision that the Director of National Intelligence has published in its Information Sharing Strategy (its Web site) where he declares that the DoD and the Intelligence Community are moving from an “agency-centric” to an “enterprise-centric” architecture. This Single Information Environment being advocated by the DNI will require unprecedented information sharing between agencies and a logical outcome of this is a distributed, high performance federated architecture such as what is being demonstrated within the Large Data JCTD.” “Built upon high-performance open standards-based hardware and free open source software, the Large Data architecture is an exciting first taste of next-generation super-scale data centers,” said Dr. David Southwell, President and Co-CEO of Obsidian Strategics. “InfiniBand is a key component of NRL's visionary Large Data work, which has pioneered large-scale global Data Center architecture based on this increasingly popular high-performance technology. I am very pleased that Obsidian has been tapped by the Large Data team for delivering wide-area InfiniBand connectivity for this effort,” said Dr. Southwell, “As world-wide data storage and compute capacity continues geometric growth, I believe that the Large Data-JCTD network will serve as an exemplar for a new class of future high-performance distributed data centers in mainstream commercial and government domains.”