Step Up Your Game
• List type/source (co-registration, in-house list);
• Users who have not responded to a campaign in the past X months; and
• Any other metrics that may be pertinent or available from your data.
To make things easy, your segments should be relatively straightforward. For example, SMA and SFA could be used for “Subject Line A—Male Recipient” and “Subject Line A—Female Recipient,” respectively. The key is to track each segment individually.
Be careful not to cut the segments too small. The smaller the segments, the longer it will take to accumulate the number of conversions it takes to become a statistically valid test. The chart at left shows how many messages you need to send to achieve a statistically valid sample.
You ‘ve done a good deal of work to get ready to test. Now that you are set up properly, you can track your e-mails by segment, clickthrough and conversion rates—and realize your goals.
Now is the time to go back to the metrics you set up in the beginning and examine these same elements at the end of the campaign.
Never assume because one e-mail outperformed another it is superior in every way. Instead, look at your data more closely. The top-performing e-mail may have resulted in more clickthroughs, but did it result in fewer sales or margin? Likewise, if you are conducting a multivariate test that tests image and sales copy, it does not mean the image from the winning combination is the best on its own.
Look at all metrics, and act on the important ones. If you set up a test with a focus on increasing clickthrough rates, at the end of the test be sure to see if the changes you made ultimately changed Web site or call-in behavior. Remember, simple changes can have profound effects, and you don’t want to increase one metric at the sake of another.