Testing Software, Coffee, Whatever...

Finally, here's a job I'm qualified for -- testing coffee. Actually, you'd think that the issues involved in testing, say, coffee ought to apply to software or other testable things.

No, I didn't come up with this idea. That honor belongs to Dr. Laura Brandan Briones, a researcher at the University of Twente in The Netherlands.

What Brandan Briones has done in her research, which was undertaken as part of the NWO project Systematic Testing of Realtime Embedded Software Systems (STRESS), is develop metrics for establishing the reliability of tests. She does so by determining which part of the system has been tested: the degree of coverage. An equally large problem during the testing of software is the gigantic number of scenarios that a system offers. This is more than the number of elementary particles in the universe. It is therefore impossible to test all of these. The degree of coverage is a measure of the reliability of the system. For instance, if you have a coffee machine with tea and coffee and you test all of the possible options that supply coffee then you have a degree of coverage of 50 percent.

According to Brandan Briones, the old method for determining this degree of coverage assumed all possible processes in a system to make, for example, a cup of black coffee. However the system can deliver the same cup of coffee in various ways. Using the method of Brandan Briones all of the different ways are assigned the same value. Therefore the degree of coverage no longer varies.

The method Brandan Briones has developed also takes into account the severity of possible errors. This is a major advantage for companies that want to make a risk estimate. Taking into account the importance of the errors provides a more accurate and realistic measure of reliability.

Okay, if "reliability" translates to a rich cup of mocha java everytime, that's fine with me.

Posted by Jon Erickson at 12:45 PM  Permalink