Repeat after me: The release iteration is not a testing phase. The release iteration is not a testing phase. The release iteration is not a testing phase. The majority of testing occurs throughout the project in the form of confirmatory testing performed by the development team itself, or investigative testing performed by independent testers to catch any pernicious bugs that slip through the cracks (again see "Agile Testing Strategies," January 2007). Having said that, as indicated in Figure 2, you will still do some testing during the release iteration.
- First, the independent investigative testing efforts to verify the output of the last construction iteration still need to occur. Some of the defect stories will be addressed during the release iteration, although many will be put on the work items stack for "Release N+1" of the system.
- Second, some release-specific testing efforts, in particular pilot/beta testing where you deploy your system to a subset of your end-user community, may occur. You may also need to do true acceptance testing where authorized project stakeholders work with the system and decide whether it truly meets their needs.
Although "agile acceptance testing," also known as "customer testing," is a part of the confirmatory testing efforts that occur throughout a project, you may still need to perform a final release acceptance testing effort where people outside of the team are involved. Testing of your installation/deployment scripts is particularly important during the release iteration because you don't want any surprises when you actually try to deploy.
Don't get me wrong, the activities that I've described as release testing in Figure 2 can and should be done much earlier in the lifecycle. My point is that these forms of testing are particularly pertinent during the release iteration to assure your stakeholders that your system is truly ready for deployment. Many organizations require that a system be put through some sort of formal sign-off process before deployment regardless of the development paradigm. The good news is that the increased focus on quality and testing by agile teams streamlines this effort.
More Than You Think to Releasing a System
The common rhetoric within the agile community is that our software is always "production ready" because we deliver tested, working software at the end of each construction iteration. This is a good thing, and for simple systems, you might be able to simply copy the software onto a production server and declare success. In more complicated situations, you may need to: