Quality Concerns
It should be no surprise that 95.7 percent of organizations considered data to be a corporate asset (although it is surprising that 4.3 percent don't). If data is a corporate asset, doesn't it make sense that you have a test suite in place to validate it? Apparently not, because only 40.3 percent of respondents indicated that they do. Worse yet, of those organizations, only 63.3 percent let developers run this test suite whenever they needed to, hampering their ability to detect whether their development efforts would inject defects into the database. Of the organizations that didn't have a test suite, only 31.6 percent had discussed putting one in place, implying that 40.8 percent (59.7 percent * 68.4 percent) of organizations are seriously challenged with respect to ensuring data quality.
The problem just gets worse. 63.7 percent of respondents indicated that their organizations implement mission-critical functionality in the database, yet only 46 percent of those had a regression test suite in place. If functionality is mission critical, or even if it isn't for that matter, shouldn't you test it? Similar to data quality testing, only 66.3 percent of respondents work in organizations where developers can run this test suite whenever they need to, and in organizations without such a test suite, only 38.6 percent had discussed putting one in place.
Although these numbers sound bad, and they are, I suspect that they're optimistic. The survey didn't distinguish between traditional regression testing where the majority of testing is done late in the lifecycle and the more agile test-driven development (TDD) approaches where testing is done throughout development on a continuous basis. A survey being run in September addresses this issue, and more, and will be summarized in early 2007.
Have We Given Up?
61.9 percent of respondents indicate that their organizations have problems with their existing production data. Although this number is arguably low, very few data sources are perfect, and we can often live with minor data problems. However, considering that most organizations consider data to be a corporate asset, shouldn't we be doing something to fix it? As Figure 2 reveals, many organizations seem to be struggling with addressing legacy data problems.
Of the respondents working in organizations with data problems, 18 percent report that there is no strategy in place to address the problems and 33 percent have strategies not to make things worse. In my opinion, these two strategies will both eventually lead to failure: With developers commonly going around data groups and often doing a questionable job of database design as a result, and with business users using existing applications to do new things that weren't considered in the original data design, things are bound to get worse. 8 percent of organizations indicate that they intend to rewrite everything at once, a strategy that I suppose could work for smaller organizations. The good news is that 33 percent indicated that their organizations are taking an evolutionary approach to fixing data sources, which in my opinion is the most viable approach.
