Don't Prevent Disasters
January 21, 2010
A while ago I was working with a very fine biologist who wanted me to
write a program that ran a certain calculation involving sequences of
DNA. I pointed out to him that a detail he specified didn't match the
specification for the standard we were using. (Does a space in a
header line indicate that the following characters are just comments?
The spec says no.)
But he wanted it that way! Very badly!
"There are all these other programs and your program won't interact
with them correctly. If you use somebody else's data, you might miss
what you're looking for!"
(This happens all the time. Usually it goes unnoticed and some other
lab makes the discovery that our lab could have. But we'll never
know.)
Well, we sorta growled about it and I said we had to talk to my
boss. I couldn't make that decision without his backing.
We ended up in a big meeting, 10 of us. I made a formal proposal on
dealing with the issues. They were hotly debated. VERY HOTLY! A
conclusion was arrived at that I felt reasonable. I implemented it and
it worked fantastically.
And while I was at it, I improved performance by 100x and extended its
range from about 1 million sequences to 10 trillion. (We were close to
the 1 million mark at that time.)
I was very pleased with the result and went out of my way to thank
people for their hard work.
Unfortunately, this view was not widely shared.
As I look at this in retrospect, I realize that I disturbed
everybody's quiet life of working on what they really wanted to work
on. I forced them to deal with an issue they had avoided for five
years and were happy to avoid for another five.
More ironically, even the significant improvements I made to the
program will go unnoticed. Because it works OK today and it will work
OK tomorrow. If their jobs now run in 9 hours instead of 10 hours,
they won't notice. It's still an overnight job.
Had I merely hacked the first program to do the "wrong" thing, my
collegue would have been satisfied. My peers would eventually run into
the 1 million sequence limit and a hew and cry would have gone out. A
hero would have stepped forward and saved the day.
Instead, I'm the goat. :-(
-Bil