Channels ▼


Dr. Dobb's Agile Update 11/09

In This Issue

  • Crossing the Developer-Data Divide
  • Participate in the "State of the IT Union Survey" and potentially win a copy of "Leading Lean Software Development"
  • Hot Links

Crossing the Developer-Data Divide

It isn't news that modern applications include a combination of functionality and information, nor is it news that developers and data professionals need to work together effectively to successfully build modern applications. Unfortunately this doesn't seem to be happening as often as it should, with many organizations suffering from a "developer-data" divide. My experience is that this is primarily the result of each group not understanding, nor fully appreciating, what the other group is trying to achieve. The cause of this appears to be inadequate support for cross training developers and data professionals in each other's skills.

Dr. Dobb's State of the IT Union Survey (September 2009) included questions which explored the developer-data divide. The survey was promoted in this space and in Jonathan Erickson's blog during the month of September. There were 763 respondents, 49% of whom were developers and 21% were in management. Furthermore, 83% had 10 or more years in IT and 26% worked in orgs of 500 or more IT people. From a geographical point of view, 74% of respondents were North American (typical for Dr. Dobb's surveys), 16% were European, and 7% were from Asia Pacific countries.

One question explored strategies which organizations apply so that developers can learn data skills. For most people the best way to gain data skills is to learn them in a hands-on environment, and according to the survey such strategies were reasonably popular. Of the respondents, 22% work in organizations where developers are mentored by data professionals in data skills and 21% work in organization where developers pair with data professionals to gain data skills (12 % offered just mentoring, 10% just pairing, and 10% offered both). Training in data skills is also a good option, particularly when complemented by mentoring and/or pairing. 18% of survey respondents claimed that they work in organizations which offer some form of data training to developers, with 13% indicating that their organization offers data modeling training to developers and 10% offer data administration training to developers (5% offered both).

Now for the bad news. The majority of respondents, 61%, indicated that they work in organizations that expect developers to pick up data skills on their own. 45% indicated that developers pair with other developers to gain data skills, potentially a case of the blind leading the blind although better than doing nothing at all. Luckily only 10% indicated that they work in organizations which believe that developers don't need data skills at all, and indication that Frederick Taylor's grip on our organizational culture isn't as pervasive as I'd feared.

Another question explored the strategies are organizations applying so that data professionals can learn development skills. The news is a bit better in this regard. 26% of respondents work in organizations where data professionals pair with developers to gain development skills and 21% work in organizations where data professionals are mentored by developers in development skills (15% offered just mentoring, 10% offered just pairing, and 11% offered both). Furthermore, 17% of respondents work in organizations which provide development training to data professionals. On the downside, 40% work in organizations where data professionals are expected to pick up development skills on their own (which is better than the support given developers). Only 23% work in organizations where data professionals pair with other data professionals to gain development skills, once again better than support given to developers. Disturbingly, 20% indicated that they work in organizations where which believe that data professionals do not need development skills at all, impairing their ability to interact effectively with the development teams which they are expected to support.

So why is this important? The July 2009 State of the Union Survey found that in organizations which had 100 or more IT people, and which had a data group (smaller organizations had a tendency not to be large enough to warrant a separate data group), 10% of development teams avoid working with their organization's data group whenever possible and 29% preferred not to work with them but were forced to anyway. On the other hand, 50% of development teams found their organization's data group to be effective. Although some development teams and data groups work together well, there is clearly a problem within a large percentage of organizations. This may be having an impact on data quality within those organizations. The 2009 Agile Project Initiation Survey found that only 45% of agile development teams were working with legacy data, compared with 57% working with legacy systems, indicating that the majority of agile development teams were creating new, silo databases for their applications. While I have no doubt that this makes sense for a small percentage of teams, there is no way that 55% of development teams should be doing this. Creating yet another data source when existing data sources are available decreases overall data quality within an organization. We need to do better.

One thing that surprised me was that encapsulation strategies such as object/relational (O/R) mapping frameworks didn't seem to motivate a widening of the developer-data divide. Only 4% of respondents indicated that developers don't need data skills because database access in encapsulated and only 3% indicated that data professionals don't need development skills for that reason. I had expected a much higher rate seeing as this excuse is often bandied about in internet discussion forums as a potential cause of the developer-data divide.

It is possible to overcome the developer-data divide. The first step is to understand that it exists and the second step is to recognize the harm that it is causing your organization. Your third step is to address the divide through training, mentoring, and increased collaboration via non-solo strategies such as pairing. Many organizations are already on that path but many more need to begin the journey.

Survey: State of the IT Union

We invite you to participant in the November 2009 edition of "Dr. Dobb's State of the IT Union Survey. The goal of this ongoing survey series is to find out what IT professionals are actually doing in practice. The survey should take you less than 5 minutes to complete, and your privacy will be completely protected.

At the end of the survey you will be given the chance to be entered into a draw for one of 10 copies of Leading Lean Software Development, by Mary and Tom Poppendieck (Addison Wesley, November 2009. Additionally, Dr. Dobb's again is giving you the opportunity to be entered in a draw for one of 5 official Dr. Dobb's hats.

The results of this survey will be summarized in a forthcoming article by Scott Ambler. Furthermore, this is an open survey, so the source data (without identifying information to protect your privacy), a summary slide deck, and the original source questions will be posted at so that others may analyze the data for their own purposes. Data from previous surveys have been used by university students and professors for their research papers, and hopefully the same will be true of the data from this survey. The results from several other surveys are already posted there, so please feel free to take advantage of this resource.

Hot Links

Crossing the Object-Data Divide (February 2000) described in detail the challenges which data professionals and object programmers faced in working together effectively.

The Cultural Impedance Mismatch Between Data Professionals and Application Developers explores the divide from the points of view of developers in general, not just object-oriented programmers, as well as data professionals.

The Ambysoft 2009 Agile Project Initiation Survey investigated what agile development teams do at the beginning of a project.

The Skillset of an Agile DBA summarizes the skills which database administrators need to be effective members of agile teams.

Data Skills for Agile Developers overviews the data skills which agile developers should have.

The Surveys Exploring the Current State of Information Technology Practices page links to the results of all the Dr. Dobb's surveys I've run over the years.

My Agility@Scale blog discusses strategies for adopting and applying agile strategies in the complex environments.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.