Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼


SEI, Army Team Up on Software Testing

The Software Engineering Institute (SEI) and the U.S. Army Aviation and Missile Research Development and Engineering Center (AMRDEC) -- the organization that provides research, development, and engineering technology and services for aviation and missile platforms across the program lifecycle -- have launched a one-year project to develop a comprehensive approach to overcome deficiencies with the testing now being done to validate software and system reliability. This comprehensive approach will include the development of a roadmap for the research and application of technologies such as model-based engineering, assurance cases, analytical tools, and industry standards, such as the Architecture Analysis and Design Language (AADL) in the assessment and validation of complex system reliability. Participants say they hope the framework they build can become an industry practice for analytical architectural validation.

"Avionics systems are going to more digitally oriented systems -- digital cockpits," says Willie J. Fitzpatrick, AMRDEC Aviation Division chief. "Assuring that those digital cockpits are safe has become a bigger problem. And the cost for assuring us they're safe, reliable, and dependable has been going up. Our customers, who are the project managers for the different aviation platforms, have limited budgets for delivering a system to a soldier in the field. They would rather invest in getting the soldier as much functionality as they can, but they can't trade off system safety."

"System safety, reliability, and performance are increasingly affected by system-level problems due to the time-sensitive nature of software interaction on a set of networked computers despite best design and test practices," added the SEI's Peter Feiler. "We bring to the table an architecture-centric model-based approach that allows us to discover these problems earlier in the life cycle and make the validation process, which is currently very heavily test based, more cost effective and efficient. Similarly, we will consider assurance cases, which have been used successfully in the UK to reason about what is sufficient evidence to certify safety-critical public transportation systems."

In the context of this project, architecture models with precise semantics drive quantitative and predictive analysis. Bruce Lewis, an SEI resident affiliate from AMRDEC, explains, "We're actually doing quantitative analysis of the system architecture -- software and hardware -- through virtual integration to determine if the system can perform in a way that is correct and meets the requirements for safety and other non-functional qualities before building it. This approach gives us deeper insight into the complexities of the hardware or software system and also integrates through the framework and roadmap deeper reasoning about evidence for assurance required and the correctness of requirements."

"This comprehensive approach and the roadmap will give us things that testing cannot do," says Fitzpatrick. "I'm hoping for a change in best practices. The approach in the past has always been 'Let's build it then test the heck out of it.' We will still rely on some testing. But even if we were given all the money required for testing, we wouldn't have enough time."

Lewis agrees, "Testing says given these test parameters, does the system work? An analytical approach gives you a much broader ability to say this will or won't work, and then, of course, you have to validate your assumptions and validate that the implementation actually does what you modeled."

"We are not looking for a specialized solution that works only for a particular program or just the Army," says Feiler. "We expect the framework will accommodate a range of technologies from the SEI and the research community. A roadmap will outline a strategy for its evolution and its incorporation into an industry-wide software-reliant system practice."

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.