Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼

Management Metrics

Management Metrics

In her comprehensive SD West presentation, “Effective Metrics for Pragmatic Project Managers,” project-management maven Johanna Rothman got to the meat of the matter in the first few minutes: Confronting her audience with the question, “What things are important about your project?” she proceeded to tackle all the enumerated ducks in a row, provoking a lively interchange with her inquisitive audience.

Noting the comment attributed to Hewlett Packard’s Stephan Leschka, that “Without metrics, you’re just another person with a different opinion,” Rothman stated, “The more measurements we take at the beginning, the more likely we are to have a fabulous result at the end.”

The best chance for success, Rothman maintained, depends on a collaborative data-gathering method that quantifies project risks and risk-handling methods. Successful project management requires far more than simple task analysis, Rothman warned. PM courses tend to focus on task lists, she explained, but the critical paths in software projects go beyond tasks to people and sometimes machines.

Focusing on the human element, Rothman stressed the necessity of first determining who’s playing schedule chicken. “The first fellow says he’s going to make Friday, so the next one says Friday, and the next one, too, but what they don’t know is that everyone’s sweating, knowing that Friday’s an impossible dream,” she explained, to the audience’s knowing laughter.

The way to avoid schedule strangulation? Throughout the class, Rothman reiterated one concept: Early intervention. “The farther along you go in the project, the more likely you are to play chicken, and the less likely you are to make helpful changes.”

Rothman then discussed the necessity of developers employing uncertainty to their advantage. “In answer to the dreaded question, 'When will it be done,’ you don’t want to say 'I don’t know.’ But there are times in the project when you really don’t know the entire state of the project.”

Rothman described essential project management awareness as a pair of parallels: What’s done and what’s left to do; what’s working and what’s not working. To elucidate this, she stressed the necessity of identifying defects and unearthing obstacles at the earliest possible time. Breaking bad news early, she advised, tends to soften the blow.

Major Players
Next enumerating measurements for project completion, Rothman ticked off the major players in the PM equation:

  • Earned value: How many person days you need to do a project. Instead of focusing on how much time has elapsed, you should look at a particular deliverable, measuring what’s been accomplished and comparing the ratio of accomplishments made to the time used.
  • Release Criteria: Those few critical criteria that tell you when the project is complete.
  • Estimates versus actuals for major and appropriate minor milestones: Rothman recommended starting estimates at what she called the “fuzzy front end” of the project so as not to get caught limping toward the finish line burdened with unwieldy—and unrealistic—expectations.

    Earned value, she explained, is especially difficult to determine in software projects, where parts are interdependent. “If you can’t tell when a part is complete,” she explained, “you can’t measure earned value.” Thus, the necessity for breaking up a project into usable chunks at the very beginning, not halfway through, when the entire feature set is “smushed together.” If a customer can use the first chunk, she explained, project managers can calculate an earned value for that deliverable.

    “Ask yourself,” she exhorted the audience, “what’s the earliest possible date you can get something done. Your best bet is to make the smallest chunks possible and then measure earned value: ‘How much did it cost me to get to this chunk?’” For Extreme Programming projects, she stated, iterative development can play into the chunking technique, because at the end of each iteration, developers are providing something of value, and need ask only if the estimate met their actuals.

    EQF Explained
    To evaluate to estimation metrics, Rothman recommended efficiency expert Tom DeMarco’s Estimation Quality Factor process, which provides a histogram of the most significant duality in project management: the date of estimate versus estimated end date.

    EQF is useful in three ways, Rothman stated: First, as an early warning sign to see if events outside your project are consuming people when they should be focused on your project; second, as a check against the initial estimations on your next project; and third, as a means of determining if you have a chance of completing the current project.

    Conspicuous Consumption and Corporate Overload
    When the project starts to slip, Rothman recommended, ask team members to log their time for a week and let you know—but don’t show—the results. One team member, she recalled, was a coffee addict, logging in six to eight time-consuming coffee breaks per day. She had no need to admonish him—“I’m going to tell another adult he can’t drink coffee?” she laughed—self-monitoring kick-started him into a caffeine diet.

    Rothman then attacked corporate overload dictating multiple parallel projects, noting that most developers just can’t handle context shifting among several projects at different levels in a single day. This problem can best be dealt with by negotiation to arrive at a reasonable portfolio management. “Go to the other project manager and say, ‘We can’t do this; give me Sally for a week,’” she suggested.

    Changing Course Midstream
    Rothman then recommended measuring the number of requirements early on in the project, as well as enumerating the number of major and minor changes per week over the course of the project, warning about the disaster potential of midstream major requirements changes.

    For midstream project trends, defect find and close rates rise to the fore. “I track my own defects on everything,” she explained, “when we review requirements specs and in project plans, all on the same chart, so the developers are much more likely to track defects in the products they create, too.”

    Tracking defect trends can help project managers keep control of the process, enumerating how long it takes to fix defects, as well as their costs. In what seemed like a conundrum, she used example charts from projects completed to detail the truism that the more proactive you are, the higher your costs will be later in the project. “The fewer defects you find later,” she stated, “the higher the cost of each defect, but the lower the overall total cost.”

    Rothman then explained the Fault Feedback Ratio, examining the total picture of productivity, asking the Big Questions “What do the creators create? How much is good stuff, and how much is redo?” FFR, she stated, tells you the ratio of problem solving to wheel spinning.

    To fix a high FFR, Rothman advised, allow nothing to go into the configuration management system until it’s been reviewed by more than one person. “Pair programming works great for this,” she recommended.

    Better Late than Never
    As the project winds down, it’s not too late to save your bacon: Defect trend analysis can still be useful if you ask the right questions. Determining the number of defects that remain open and making sure developers aren’t getting stuck fixing the easiest problems first can go a long way in salvaging a project that seems headed for the barrel room.

    For diagnostic techniques, Rothman advised a mixed bag. “Maybe I need to use different techniques for the substrate or a piece of the project than for the rest of it,” she explained, again recommending pair programming and increased communication with developers.

    Gathering the Goods
    For all this data collection, Rothman claimed that simple tools are usually sufficient. “You need a defect tracking system, a configuration management system and a project scheduling tool you can extract info from,” she stated. “I set up scripts, the release engineers help me with the defect tracking system and config management system, and I check how many open and closed defects I can find.” In the Unix environment where she often works, she said that ChronJobs does the trick, but stressed that many tools are equal to the job, depending on the nature of your project.

    Finally, Rothman admitted, beyond numbers, project management is as much art and craft as it is science. No matter what metrics you use, you’re best served by navigating the middle course between unbridled optimism and paralyzing pessimism.


  • Related Reading

    More Insights

    Currently we allow the following HTML tags in comments:

    Single tags

    These tags can be used alone and don't need an ending tag.

    <br> Defines a single line break

    <hr> Defines a horizontal line

    Matching tags

    These require an ending tag - e.g. <i>italic text</i>

    <a> Defines an anchor

    <b> Defines bold text

    <big> Defines big text

    <blockquote> Defines a long quotation

    <caption> Defines a table caption

    <cite> Defines a citation

    <code> Defines computer code text

    <em> Defines emphasized text

    <fieldset> Defines a border around elements in a form

    <h1> This is heading 1

    <h2> This is heading 2

    <h3> This is heading 3

    <h4> This is heading 4

    <h5> This is heading 5

    <h6> This is heading 6

    <i> Defines italic text

    <p> Defines a paragraph

    <pre> Defines preformatted text

    <q> Defines a short quotation

    <samp> Defines sample computer code text

    <small> Defines small text

    <span> Defines a section in a document

    <s> Defines strikethrough text

    <strike> Defines strikethrough text

    <strong> Defines strong text

    <sub> Defines subscripted text

    <sup> Defines superscripted text

    <u> Defines underlined text

    Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

    Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.