Developer's Reading List
, April 17, 2012 C++ concurrency, web crawlers, Google testing, and more: This month's reading list is packed with great books on interesting topics.
How Google Tests Software
by James Whittaker, Jason Arbon, and Jeff Carollo
This book contains a detailed description of the hierarchy of testing personnel at Google, and the responsibilities of the testers and test managers at the different levels. As such, it focuses little on the tools (although they are at times discussed in more than superficial detail, their use is rarely the point), but mostly on the interactions between the testers, their managers, the developers, and, of course, with the system under test.
The book seems to want to be several different things without deciding which: a presentation and explanation of how Google tests various products (which is what the title would imply), or a friendly set of interviews with staff about testing (what it seems to really want to be), or a hodge podge of both with various reference documents and diagrams sprinkled throughout (what it is).
I found the interviews to be poorly chosen in general. Few of the test professionals have anything very interesting to say, and each interview loses a page or two up front asking the interviewees about their personal lives. ("Why did you decide do join Google?") The technical value of the interviews shows up in the occasional fact or anecdote. But frequently, it seems that the authors don't seize on it, nor do they sufficiently excise unnecessary material.
The overall perception you get is that testing at Google is not significantly different from at other large companies. The notable difference is that Google is more likely to write its own testing tools. Only in a few areas does the company depend on commercial, third-party products. Its internal processes reveal not so much a "view" of how testing should be done or even a great commitment to rigor, but an evolution of various techniques that have worked in the past and been adopted then adapted for current projects. Occasionally, these practices are insightful. For example, Google classifies all tests as small, medium, large, or enormous. The criterion that defines the categories is the duration of the test: There are expected averages and maxima in place. In many ways, I like this approach. It does away with the concepts of unit tests, integration tests, system tests, and UATs, and instead classifies tests in ways that make it possible to determine easily which regression run can be run at any given moment.
Anyhow, if you want to meander down Google's testing halls and absorb bits of knowledge and advice from the denizens there, this book is for you. If you want a more formal explanation of the how, when, and why of Google testing, this book might well be disappointing.
— Andrew Binstock

