Channels ▼

Mike Riley

Dr. Dobb's Bloggers

The Art of Performance Testing Book Review

February 27, 2009

Don't be fooled by the deceptively small page count of this book.  Author Ian Molyneaux has distilled a wealth of hard-earned lessons into a set of checklists and bullet points that crystalize the majority of application performance testing principles in real world practice today.  Read on for a full review.

Coming in at just over 150 pages, The Art of Performance Testing is nevertheless a valuable read for any developer and/or tester bestowed with the responsibility of ensuring an optimal application execution experience.  As is always the case in theoretically successful application construction and testing, performance expectations and realistic limits should be clearly defined before the first line of code is written.  What distinguishes Ian's book apart from other, more verbose books on the subject is his advocacy for the performance testing business case and, in the book's third chapter, two case studies that show the lifecycle of such real-world testing outcomes.

The book opens with a chapter on explaining the historical rationale behind performance testing, including the establishment of performance measures and outcomes, and why leaving performance testing to the last leg of an application development cycle is a really bad idea.  This is followed by the largest chapter in the book and covers the fundamentals of application performance testing, the selection of performance measurement tools, constructing the test environment and setting realistic performance targets via business process owner consensus, key performance targets (such as availability/uptime, concurrency/throughput and response time), making sure an application is stable enough to test (obtaining a frozen code build, specifying the transaction list and replay validation) and ensuring high-quality, accurate and valid performance design and data capture.

Chapter 3 reviews best practice model approaches to performance testing via Proof of Concept (POC) checklists and form requirements via a six step process.  This chapter also reviews two real-world case studies, one for an online banking analysis and the other for a call center.  Both of these are invaluable to see how the principles and techniques promoted by the author are put into practice.  

The next chapter examines the recommended statistical and analytical procedures a tester should consider when interpreting the real-time and post-captured results.  Linear, knee-spikes, standard distribution curves are some of the more frequent data views expected when monitoring Key Performance Indicators (KPIs).  The chapter closes with helpful pre through post task checklists sure to make performance evaluations more meaningful and relevant.

The final chapter in the book presents a rapid overview of some of today's newest application frameworks and recommended ways for dealing with measuring their performance.  Rich Internet Application (RIA) technologies like AJAX, Windows Presentation Foundation (WPF), Adobe's Flash/Flex and enterprise systems like SAP, Oracle and SOA are very briefly touched upon.  The book closes with 5 appendixes, spanning from transaction examples to KPI templates showing three screen grabs of performance trees, one which erroneously displays a duplicate of the Citrix MetaFrame tree instead of the intended Microsoft IIS tree. As of this review post, no replacement image has yet been posted on O'Reilly errata web page for the book.

In summary, while The Art of Application Performance Testing is rather expensive for its short notebook on status, there are a couple of gold nuggets of hard-earned experience that can be uncovered in the book and practiced in your own application development lifecycles immediately.  More cost-concious readers may find plenty of free material with nearly as good checklists and recommendations by Googling the various performance testing blogs and even several articles on Dr Dobbs main website, but for those looking for a less time-consuming and more accurate, trustworthy source of practice and experience, this book delivers the goods in considerably less time and with greater impact.

Title: The Art of Application Performance Testing
Author: Ian Molyneaux
Publisher: O'Reilly Media
ISBN: 978-0-596-52066-3
Page Count: 158
Cost: $34.99 US


Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.