Channels ▼

Jonathan Erickson

Dr. Dobb's Bloggers

The SPADE Programming Language: Dig It?

May 14, 2009

It's always interesting to follow technologies as they make their way from the lab to the real world. An example is Intel's Ct, a programming language that lets you abstract data-parallel programming away from the hardware while supporting deterministic behavior to avoid races and deadlocks. Intel recently announced Ct will be incorporated in real products by year's end.

Another example is the SPADE programming language out of  IBM's T.J. Watson Research Center. In fact, SPADE (short for "Stream Processing Application Declarative Engine") has been around at least since 2007 when IBM rolled out its System S stream computing solution, a system with 800 or so processors. The System S software lets application software split up tasks, then reassemble the data into something that makes sense. In other words, stream computing leverages SIMD (Single Instruction, Multiple Data), letting applications achieve some degree of parallel processing. System S is designed to analyze hundreds or thousands of simultaneous data streams -- stock prices, retail sales, weather reports, etc. -- and deliver nearly instantaneous analysis to those who need to make split-second decisions. You don't need me to tell you that the amount of data is skyrocketing -- and not just structured information found in databases, but also unstructured, incompatible data captured from electronic sensors, web pages, email, blogs and video.  By 2010, the amount of digital information is expected to reach 988 exabytes. According to IBM that's roughly the equivalent of a stack of books from the Sun to Pluto and back.

So what is SPADE? Again, it is a programming language and a compilation infrastructure, specifically built for streaming systems. It is designed to facilitate the programming of large streaming applications, as well as their efficient and effective mapping to a wide variety of target architectures, including clusters, multicore architectures, and processors such as the Cell/B.E. processor. The SPADE programming language lets  stream processing applications be written with the finest granularity of operators that is meaningful to the application, and the SPADE compiler appropriately fuses operators and generates a stream processing graph to be run on the System S Stream Processing Core. Details of the language are available in the SPADE Language Specification.

(Not that it is specifically related to SPADE, but I found the following interesting: As part of the System S development at the T.J. Watson Research Center, IBM researchers used fuzzy logic to develope a risk adaptive access control model described in this paper titled Fuzzy MLS: An Experiment on Quantified Risk--Adaptive Access Contro l.)

Of course, IBM isn't the only player in the stream processing game. RapidMind bases its platform on the SPMD (Single Program, Multiple Data) Stream Programming model while also mapping efficiently onto multicore CPUs from AMD and Intel. Coincidentally, both the RapidMind platform and IBM's System S are based on the Cell/B.E. multicore processor. Likewise, vendors developing GPUs (graphics processors) are also deep into stream processing. AMD and NVIDIA come to mind. The reason why is that GPUs are designed to accomodate a data-centric model (like images, video, and DSP) -- like that GPUs are originally designed for.

As for SPADE, it is an interesting language that's designed to solve a massive problem. 

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.