Channels ▼

James Reinders

Dr. Dobb's Bloggers

Removing bias from survey answers about the future

March 18, 2009

I recently had the pleasure of seeing Guy Currier, of Ziff-Davis Enterprise, give a talk about their research on spending.  His approach seemed more like an engineering analysis than what I've come to expect from a survey-based researcher. And that is a GOOD thing.

The breath of fresh air in his presentation was simple: he had "confidence factors" or "proposed adjustments" for survey results based on historical data that points our bias toward what I refer to as "out with the old, and in with the new." 

I've grown to hate research which is simply a list of questions asked and answers given. We all answer questions with a bias - in my observation, generally because we are all positive and hopeful about a better tomorrow.

This generally means we believe in "out with the old, and in with the new." So, Fortran, COBOL, mainframes and old fashioned servers would seem on the way out. Hence, surveys predict their demise more quickly than what their action actually cause. All of these are far to useful to dump as quickly as we might think as we answer surveys.

As for "in with the new category" - surveys tell us Java is growing quickly, VOIP can't happen fast enough, the future is all about services oriented architectures (SOA) and software as a service (Saas), we are shifting a lot to business-process outsourcing and RFID are all in our immediate future.  Guy showed us data that told a different story. Each of these will get its share of spending - but "in with the new" happens at a different rate than we tend to optimistically predict.

I don't mind the optimism we show in our survey answers - I think it is healthy. But if you are planning a business, it seems to me that before you believe this year's spending surveys - you should do two things:

(1) look at how accurate prior surveys are

(2) put yourself in the decision-makers shoes, and consider what they will really do

For (1), Guy took an addition step and actually saw consistency in the biases year-after-year and has proposed adjustment which seem pretty stable.  As I watched him explain item by item, I realized that this was simply what I call "out with the old, and in with the new" bias.

For (2), Guy suggested in his articles that items which can show very short term savings will be funded much more easily in this climate than those with long term benefits. 

With optimism, we all bias our survey answers toward  "out with the old, and in with the new" but when reality hits we go with the proven and trusted solutions more than we'd like to admit, and our management approves spending most easily on that which shows a quick return.

While that seems obvious - I still found it refreshing to have a research stand-up and try to quantify it - and give us details we could trust more than just the survey results from this year.

Guy's article which is closest to the talk that I saw is at along with information on how to reach him.






Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.