Channels ▼


Building Distributed Processing With Hadoop

VMware has announced Spring Hadoop, an integration of the Spring Framework and the Apache Hadoop platform designed to give developers an option to build distributed processing solutions with Apache Hadoop.

VMware has a habit of talking about the "new wave of data-driven applications" as if previous application breeds were in some way largely bereft of data-centricity. But if we take the firm's focus on so-called Big Data issues at face value, there is interest in VMware delivering a streamlined programming model that could make Spring a natural way to integrate Hadoop systems into the enterprise app landscape. "Spring Hadoop brings the benefits of Spring — simplicity, ease-of-use — to Hadoop by providing a comprehensive, lightweight framework that will allow developers to easily build solutions around the Hadoop platform," said the company.

The situation here is that (as you know) "data volumes" have undeniably grown. Further, "data access" choices in enterprise applications have grown exponentially (i.e., there is widespread secure access to the corporate data center via smartphones, tablets, laptops, and dedicated mobile devices of all kinds) — and this in many senses sums up the challenge brought about by Big Data as we know it today.

VMware reminds us that in answer to these new data challenges, Spring continues to focus on enabling enterprise Java developers to incorporate new data access patterns into their applications through the Spring Data projects.

Key aspects of Spring Hadoop include:

  • Support for configuration, creation, and execution of MapReduce, Streaming, Hive, Pig, and Cascading jobs via the Spring container
  • Comprehensive HDFS data access support through JVM scripting languages (Groovy, JRuby, Jython, Rhino, etc.)
  • Declarative configuration support for HBase
  • Dedicated Spring Batch support for developing powerful workflow solutions incorporating HDFS operations and all types of Hadoop jobs
  • Declarative and programmatic support for Hadoop Tools, including FsShell and DistCp

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.