U.S. Defense Agency Feeds Python

The U.S. Defense Advanced Research Projects Agency (DARPA) has given 3 million dollars to Texas-based software provider Continuum Analytics with a view to helping fund the improvement of the Python language's data processing and visualization power for big data tasks.

DARPA intends for this donation to fund investigation into new methods for data analysis and explore the possibility of new techniques for visually portraying multi-dimensional large data sets.

This investment will extend the work already underway with the the NumPy and SciPy Python libraries. NumPy is the package for scientific computing with Python containing an N-dimensional array object and tools for integrating C/C++ and Fortran code. SciPy (pronounced "Sigh Pie") is an instance of open-source software for mathematics.

Continuum Analytics president Peter Wang reminds us that Python is an easy language to learn for non-programmers and says that this is important, because most big data analysis will probably not be carried out by programmers. "If they can learn an easy language, they won't have to rely on an external software development group to complete their analysis."

This whole project is part of DARPA's XData research program, a multi-million dollar initiative designed to provide the U.S. Defense Department (and other government agencies) with new tools to work with and manage what amounts to huge quantities of "sensor data" and other forms of big data.

"With big data systems, you find new things you want to look at every week. You can't wait for that process any more," said Wang.

Continuum Analytics has traditionally focused on large-scale distributed computing, array-oriented programming frameworks, scientific computing, and algorithm development, plus graphics and interactive data visualization. The firm will now work to produce add-on products and services so that developers can more easily use Python for big data analytics.