Scott is Dr Dobb's contributing editor and Chief Methodologist for Agile and Lean at IBM Rational.
Over the past several years, certification has been a raging debate within the agile community and the issue isn’t black and white, regardless of what some of the debaters may want you to believe. Certification has, unfortunately, been muddied by the profit motive, although this problem isn’t unique to the agile community. It has also been harmed by a lack of separation between those defining the certification, those delivering certification-oriented services, and those running validation activities (in the few cases where such exist). In this article, I'll discuss the pros and cons of certification in general, share results from a recent survey exploring the Certified ScrumMaster (CSM) scheme (I’m being exceptionally polite using the term "scheme"), discuss CSM as a concept, and suggest potential remedies for both the “certified” and the certifiers.
On the benefit side, effective certification programs motivate people to improve their knowledge base, to improve their skill sets, and to continue maintaining both. Respectable certifications, or more accurately a collection of respectable certifications that address a wide range of topics within IT, could even provide a common baseline upon which people’s CVs could be compared. Smart organizations do far more than consider people’s certifications when considering them for employment, but the reality remains that many organizations look for certain certifications to filter candidates.
There are many challenges associated with certifications. First, they are completely inaccurate at rating people. Sally can have certification X and be absolutely brilliant at certain X-related things; whereas John can have the same certification and be completely inept at those things. Second, certification can be a form of discrimination. In the late 1990s, James Bach and Luke Hohmann wrote a brilliant article on this topic, pointing out several ways certifications support discrimination. For example, if you don’t have a certain level of certification an organization won’t consider hiring you, regardless of your actual skill level. They are also concerned about the discrimination aspects of how the certification body decides who is in the club and who isn’t. For example, my understanding is that to become a Certified Scrum Trainer (CST), you need to have five other existing CSTs support you in your certification efforts. The positive spin is that it helps to ensure the level of quality amongst the CST community, but it’s clearly a way to keep unwanted people out of the club.
The point is that there is a range of opinions regarding certification, I’ve just covered the tip of the iceberg here, and in the hot links section at the end of this article, I’ve included some interesting references. Although I respect both sides of this debate, I personally lean towards certification as long as it’s done “right.” For me, “right” means that people must have to invest significant effort to earn the certification, they must have a minimum of related experience, they must show a level of knowledge and/or expertise that has been validated in a defined manner by an independent body, and they must continue investing effort to remain certified.
To be clear, I must point out that for years I have been a vocal and outspoken critic of the CSM certification scheme, having written about it for Dr. Dobb’ and in my blog on IBM developerWorks. I have also criticized it in various talks at international conferences, in particular in my “Agile by the Numbers” and “Reality over Rhetoric” presentations, where I summarize findings from my various surveys over the years. Instead of presenting yet another diatribe against the CSM scheme, I thought instead would find a way for the agile community itself to weigh in on this issue. So naturally, I ran a new survey.
The 2010 Scrum Certification survey ran during the last half of October and first week of November, 2010. I spent several weeks designing it, with feedback from several researchers with backgrounds in IT and in survey design. I had run a previous survey on this topic during the summer of 2010, and unfortunately, had allowed some bias to creep into its design, so I was highly motivated not to let that happen again. The survey was announced on my Twitter feed, on the Agile Alliance discussion list on LinkedIn, and on my personal mailing list. Of the 325 respondents, 171 respondents had the CSM but did not have the CST (most of the CSTs are also CSMs), and henceforth, shall simply be referred to as CSM respondents. 13 respondents were CSTs, which I believe is about 10 percent of the overall CST community; 38 had some other form of non-CSM Scrum certification (many of the CSMs had more than just the CSM); and 123 respondents had no Scrum certification at all.
As I said, yet another diatribe against the CSM scheme would be of little value. However, seeing what CSMs think of the CSM scheme would be interesting. To be clear, to “earn” the CSM designation you must attend a two- or three-day course taught by a CST and then take an online test. For several years, there was no test involved to become a CSM; then for a few months you had to take and pass the test in order to become a CSM; now my understanding is that you need to attend the course and take the test, but the mark you receive doesn’t affect whether you are allowed to call yourself a CSM. As Jon Kern has blogged (see Hot Links section), the CSM designation is little more than a “participation trophy” in practice.
One of the survey questions explored why someone would take the CSM course. In particular, 60 percent of CSM respondents indicated that they took course to learn about agile and 46 percent said the considered getting the CSM designation as simply a bonus of attending the course. Furthermore, 27 percent believed it would improve their "hireability" and 15 percent said it was required for their job. Interestingly, the Scrum Alliance, the organization responsible for the CSM scheme, doesn’t publish this sort of information.
But do the people who have “earned” the CSM designation respect it? Apparently, many don’t. Another question explored whether people with the CSM designation actually promoted the fact, either by including it on their email signature, their business card, through word of mouth or other mechanisms. According to the survey, 27 percent of the CSM respondents indicated that they didn’t promote the fact they were a CSM at all, 37 percent indicated that they seldom did so, and 18 percent indicated that they sometimes did so. This to me indicates that there’s a general level of embarrassment amongst people with the CSM designation regarding it, and rightfully so seeing as it’s so easy to “earn.”
The survey also included a free-form text question, which asked people what they thought of the CSM scheme. The comments from CSM respondents varied, as they did for other types of respondents; but in general, they were more negative than positive. Some representative comments are:
- Certification tells me someone knows the language. Calling them a Master is wrong — it should be something like Scrum Foundation Certificate, not Master. I've seen a number of newly-minted CSMs who seem to think that they truly are mastered, and become dogmatic about their approach.
- I don't put much stock in it. It was easy enough to get if you have experience doing Scrum for a good period of time. I did it mostly for resume/hiring purposes as many job postings say "CSM is a plus" or "required."
- I find the CSM to be almost completely devoid of value. I don't think it proves competency, much less basic comprehension. As a result, I think the designation has done significant harm to the advancement of certifications for a valuable purpose in our industry.
- I hire Scrum Masters, User Story Specialists, and Agile Coaches. I would not hire anyone who was not a CSM or CSPO as a minimum because it tells me that the candidate values Scrum enough to invest in it or to convince their employer to invest in them. I know exactly what the qualification really is, but like passing a driving test it means that the candidate will then start to learn by doing, having achieved an entry level qualification.
- I think the current CSM is not a real certification. All that is required for certification is to attend a two-day course. You should be required to take a test to validate your competency.
For the sake of transparency, all of these comments are posted online anonymously (see Hot Links section), as are the survey results in general. I invite you to look at the details and decide for yourself what CSMs think about the CSM scheme. As an aside, at the Agile 2007 conference in Washington D.C., I made a casual comment at a networking event (that’s an agile conference euphemism for drinking session) that my dog Luke could become a CSM if it wasn’t for the fact that I’m too cheap to send him on the course. This observation went viral, and over the next few months, several CSTs privately informed me that an unwritten rule was put in place that house pets couldn’t become CSMs. A few years later, the test/quiz was put in place, which to Luke Hohmann’s and James Bach’s point, would have the side effect of discriminating against all dogs around the world (smile).