To meet the challenge of finessing Dell's already efficient online buying process, Frog made a bold move and began testing with live pages. "We've put up alternate designs on the site," says Rolston. "We put them up in real time and watched thousands of people try out two different, competing methods to find out very specifically which one works best. We got hard core testing results."
To test new features, the group researches and develops a model that predicts how users will respond to a particular interaction or task. "In the real world, though, there is rarely one best answer, so we try and reduce it to the two or three best ideas," says Rolston. The ideas are added to Dell's proprietary FLEX content management system (which pairs XML and XSL to define and build pages). Each idea is turned into an independent, full-scale site. Then, Dell randomly switches between these options with visiting customers over a test phase that lasts between one and two weeks. The results are tallied via sales numbers, click-through data, and questionnaire responses. After determining which site works best, Dell shuts off the other sites.
Live testing can be dangerous if a feature fails or prohibits a customer from making a purchase. But that's also what makes live testing so powerful. The features that work will increase purchases or decrease shopping time. "This was the most satisfying way to test the designs, because there is no lie," says Rolston. "The real customers are putting their credit card numbers down." Although Rolston wouldn't go into specifics about testing tools, he notes that they are all custom built. Dell measures traffic and purchasing activity with its own measurement system, and the data is exported to Microsoft Excel format where it can be viewed. To be sure of the efficacy of the testing, Frog's and Dell's usability team worked together with a third party usability firm to create the testing conditions. More traditional usability tests covered the home and business segments, with separate paths for each segment. Twenty-four users were tested over four days in one-on-one think-aloud usability tests.
Gray Space
With both SAP and Dell, the Digital Media Group at Frog Design went beyond traditional design practices. Rather than focus on getting the right color or pushing the right pixel, it created a process that closely involved the client. With the correct language, all sides worked together to build some of the world's most notable Web applications.
One of the reasons for Frog's success in a difficult market is its willingness to borrow and build on the languages, processes, and theories of other disciplines, as it did from the software development industry. This borrowing and building was instilled in the company by founder Hartmut Esslinger. Esslinger liked to modify the well known design dictate "form follows function" by saying that "form follows emotion," meaning that there is more to design than pure functionality.
Jennie is a freelance technology writer and former Web developer. She welcomes your feedback at [email protected].
