A 2-day class; a 60-minute multiple-choice test; a grade that many teachers would think of as D work.

Is that valuable?

Evidently so, at least if you look at all the HR departments that stipulate some sort of agile certification in their job ads and at the importance that many people put on them.

I'll grant that learning is valuable, and a certificate can get you in the door of a company that's clueless enough to think that they mean something. By all means, take the class if you think you'll learn something new. I'm certainly not opposed to education.

But to my mind, Agile certification, as it's currently practiced, is actively destructive.

Let me explain.

When I got my private-pilot certificate, I had to study several books for a couple months, take a class, and pass a three-hour written test that covered a broad area of expertise — everything from meteorology, to the physics of flight, to navigation and planning. That test was just the beginning. I then had to actually fly an airplane for long enough to get pretty good at it (more months). Like everybody else, I studied one-on-one with someone who was very experienced (an apprenticeship, if you will). Once I satisfied my instructor's scrutiny, I was passed on to the FAA for the real test: three hours actually flying an airplane, executing a set of prescribed tasks that are intended to surface your weaknesses. That test was administered one-on-one by a very experienced pilot, who also quizzed me verbally on all the material from the written test and more. He drilled deeply into my weakest subject areas (using the written-test results and a conversation to determine that). There was only one passing grade: 100%.

Most importantly, when I was handed the certificate, it was made clear to me that I wasn't a master of anything. "This is a license to learn," they said.

None of the test criteria for the FAA tests are secret, so you know exactly what you're studying and practicing for, and more importantly, somebody who's evaluating you based on the test result knows exactly what you know. (The FAA actually publishes 100% of the questions and uses a random subset for the actual exam. Interestingly, that doesn't skew the results at all. The same number of people fail as would if nothing were published.) There are also many ways you can learn the material (books, classes, videos, online classes, individual tutoring), so you can learn in the way that works best for you.

Most importantly, the testing is done by a neutral third party (the FAA), not by the people who teach the classes, so passing isn't contingent on knowing some secret handshake that you can get only by taking a $1500 seminar.

Now let's compare a pilot's certificate to a Certified Scrum Master (CSM) certificate:

• It takes anywhere from a couple hours to 2 days to learn the 16-page Scrum Guide.
• 35 simple multiple-choice questions on a single framework;
• Get 24 right (That's 68% — a D or F in most elementary schools.)
• Voilà you're a master!

So much for rigor.

The CSM-test questions are usually secret, so somebody who's using the cert for evaluation doesn't know what you know, and you often have to take that $1500 seminar, too. (Credit to scrum.org for at least providing a stripped-down sample test with questions drawn from the real one.)

But is it even possible to assess competence in agile development with a test?

In the case of the CSM, there is no industry-wide governing board that sets agile-certification standards, simply because there can't be. There are no such things as "best practices" in agile development. There are pretty good practices that worked for me and might work for you, but there's no agreement about "best." Moreover, the notions of "best" are constantly changing, vary widely between methodologies, and are applicable only in certain contexts. Different organizations need to approach agility in different ways. There is no one-size-fits-all agile, and knowing a single framework (e.g. Scrum) is far from knowing "agile."

To make matters worse, the organizations in the business of selling certificates use different curricula, standards, and issuance rules. The companies often push a particular individual's agenda or methodology.

Since no HR department looks at any of that, what, exactly, are they filtering for? Certainly not specific knowledge.

Finally, many people really believe that these things are valuable. I hear: "The only people who disparage certificates are the people that don't have them" (usually said by people who have invested a lot of money into getting a certificate). The certificate boosters imply that the critics don't get certified because that would uncover the depth of their ignorance. Let's leave aside the pesky issue that the people who came up with the certificates didn't, themselves, have those certificates, so they must be incompetent by inference. Let's also leave aside the fact that somebody who really knows agile also knows that the test covers only a small corner of the discipline, so it doesn't tell you much of value when it comes time to actually do something. Finally, let's leave aside the fact that you can get a cert based on an afternoon of reading, and that certification is more about feeding the certification industry than assessing real competence.

I don't know about you, but I'd rather hire somebody with the sense to recognize a waste of time when they see it than somebody who's waving a certificate around. Many of the original signatories to the Agile Manifesto hold no certificates.

The ultimate proof is in the pudding. Does a CSM certificate make you more competent than anybody else who's read a book or two on Scrum? Well, no. I can give you many examples of incompetent certificate holders who I've encountered in the wild. There are, of course, competent SMs who happen to hold certificates, but their competence doesn't arise from the certification process.

Does holding a Certified Scrum Trainer (CST) certificate make someone a better trainer than a good agile generalist? Well, again, no. I've spent considerable time cleaning up the aftermath left in the trail of the CSTs. Too often, the CSTs teach an inflexible party line with no ability to adapt to the needs of the individual organization. They often know a lot about Scrum, but a lot less about agility. They often have little to no practical experience. A few real experts are also CSTs, of course, but again, they aren't experts because of the certificate.

So, let's get back to my original claim: Certificates are actively destructive to the agile community.

Problem One: Certification, as done today, creates only the illusion of competence. The certificated person often thinks that they are competent (because the certifying organization tells them they are!). More to the point, the certificate-holder's coworkers or employers might think the same thing. CSM certification doesn't usually center on actual practice. A CSM may have no idea how to put together a "story," for example, or actually be able to groom a backlog.

If you assume the certificate holder is an expert, the only conclusion that you can draw from a failure is that agile techniques and processes are themselves to blame. "Agile doesn't work."

Problem Two: When you hire people based on certification, you don't actually know what they know. Solidly agile organizations don't use certificates as a hiring criterion because they understand the limitations. An organization in transition will pay attention to certificates precisely because they can't assess competence themselves. They are often not getting the experts they expect, however, and when the people the do hire fail, they blame "agile."

Problem Three: The certificates all focus on a single framework. Agility is a much larger and complex subject than can be covered by a single framework. People who know only one framework won't be able to solve problems that lie outside their narrow specialty. They are like car mechanics who understand only transmissions. They're in deep trouble when the engine fails.

The single-framework folks also tend to promote "big bang" all-or-nothing transitions that are almost guaranteed to fail. They tend to be zealots, which leads people to reject agile techniques out of hand. You hear: "Those agile guys are idiots." and "That won't work in the 'real' world."

Problem Three: Transitioning organizations often confuse agility with a single practice framework (the Scrum==Agile fallacy). When people see that someone is certified in Scrum, they tend to think that the person is certified in "Agile." Scrum will fail in many situations where real agility would work just fine. Consequently, people say "Agile doesn't work" when they really mean "Scrum doesn't work" or "We weren't ready to implement the full process."

Problem Four: Inexperienced, but certified, trainers — I'm I'm talking about people whose only claim to competence is the certificate, not competent trainers who happen to have a certificate because it gets them work — can do active damage by perpetuating the myth that agile practice can implemented in a bubble. Agility impacts everything from governance to company structure to customer support. It's not limited to the Engineering Department.

You hear "we brought in the certified trainers, but that stuff they taught us didn't work (was crazy, was impractical, and so on). Agile is just nuts!"

Put this all together, and the certificates we have now are doing real damage when it comes to infusing agility into an organization.

Is There a Solution?

Yes, but it's not an easy one.

Ideally, I'd do away with certification entirely and replace it with an apprentice/journeyman/master system. Lacking that:

• Simplistic multiple-choice tests should be banned.
• The certificate criteria must be completely transparent. At minimum, there must be at least one complete test freely available, along with answers. The SAT "practice" tests are an example. I prefer the FAA model where all possible questions are published and the test draws from the large question pool to assemble a smaller test.
• Tests must be vetted thoroughly by people other than the certificate issuers. The tests I've seen often have errors in the phrasing that make a correct answer impossible, for example.
• The certificate issuers would not be permitted to sell training.
• It must always be possible to earn the certificate through self study. The granting organization must publish a complete study guide, and people who know that material (and only that material) must be able to pass the exam.
• The test must be a learning vehicle. You must always get, not just the results, but also explanations why every incorrect answer is incorrect. A single retest should be inexpensive or free.
• The certificate must demonstrate real mastery of the subject. Some vendors (e.g. Scrum.org) have a "level-2" certificate that claim to be this demanding, but since they don't make the test public, there's no way for either you or a potential employer to assess its quality, which brings us back to the transparency issue.
• The word "Agile" should never be used to describe a certificate that assesses knowledge in only a single methodology or practice framework. The people who teach the certificate must emphasize that they're certifying you in a specific framework, not in "agile."

Curling up with a 16-page pamphlet and taking a simple multiple-choice test just doesn't do it.