Step Up Your Game

Multivariate testing can help post major e-mail gains

November 2007 By Brett Charney

multipage

It isn’t difficult to make a case for e-mail testing. After all, when performed correctly, testing enables marketers to continuously develop and improve their e-mail programs and realize positive results. For instance, it’s not uncommon to see a 20 percent increase in your open rate by making small changes to an e-mail’s subject line. What is there not to like about that?

Even though it is a critical component of e-mail, many marketers are uncertain about how to test properly. While most marketers perform some general types of tests, some make critical missteps in setting up their tests and evaluating results. Others have yet to take the next step in their marketing programs by performing more advanced—yet very possible—measurement techniques.

With some advanced planning, it is possible to take steps now to improve the performance of your e-mail marketing program.

Keep Your Eye on the Goal

As you begin planning your next campaign, it is critical to establish specific goals up front. Some marketers forego the advance work, put together an e-mail and test it. However, isolated, “one-off” tests that are not part of a planned approach are not likely to help you see sustained improvement in your results.

Your goals and the metrics you are testing will determine the methods you use to test. For example, if your goal is to lift your open rate, you can test multiple subject lines.

Perhaps the No. 1 problem marketers face is they think they know what the results will be before a campaign runs. While it may be tempting to base your current strategy on past performance or on knowledge gained from other industries, try not to enter into testing with any preconceived bias or a strong “gut feeling” about what your results will reveal. Use your past experiences to frame your current tests, but unless you test, it shouldn’t count.

Follow the Leaders

Marketers typically use two methods for measuring the performance of their e-mail programs. The most commonly used method is A/B testing, which compares the effectiveness of two or more e-mails across one independent variable such as subject line or layout. Less common is multivariate testing, a method that tests various independent variables within each e-mail to determine the optimal combination. An example might be testing your subject line, offer and image using different combinations of each of these three components.