Step Up Your Game
Follow the Leaders
Marketers typically use two methods for measuring the performance of their e-mail programs. The most commonly used method is A/B testing, which compares the effectiveness of two or more e-mails across one independent variable such as subject line or layout. Less common is multivariate testing, a method that tests various independent variables within each e-mail to determine the optimal combination. An example might be testing your subject line, offer and image using different combinations of each of these three components.
Multivariate testing can seem overwhelming considering all the possibilities and factors involved—from creative elements such as layout, images, copy and colors, to details on your offer such as pricing, to your landing page and online forms. But the good news is small changes derived from multivariate testing can lead to good results. Changing your subject line or swapping out one image for another can make a big difference. And positive results can be realized by testing a limited number of e-mails at a time.
Leading marketers know A/B testing and multivariate testing are not exclusive and can be used together as part of an effective e-mail testing approach. By conducting an A/B test first, you can determine which factors influence your results. You then can employ multivariate testing to fine-tune your approach.
As depicted in the testing strategy diagram on page 35, a relatively straightforward but comprehensive testing strategy begins with an A/B test of a small part of the total available audience. This test, for example, can be used to determine the winning e-mail layout and should include vastly different versions. Once a winner is determined, this layout can be rolled out to the rest of the audience.
The winning layout then undergoes a multivariate test to determine which elements affect conversion and the best possible combination of those elements. To confirm the tests, the winner of the initial A/B test and the winner of the multivariate test should be tested against one another. The learnings from this strategy can be used in other channels or campaigns. Testing then begins again at the A/B stage with the testing of another major component of the e-mail campaign.