Investing in Testing? These Traps Can Cost You Dearly
4. Mistaken identity: Email marketing isn't done in a vacuum; other variables are bound to influence your test. The best way to make sure your results are caused by your testing variable is to limit your variables as much as possible. In addition, be conscious of those factors you can't influence. Some of the major ones you can guard against (unless of course they're what you're testing for) are delivery day and time (remember to account for different time zones), device or email client, and engagement (have some segments been inactive for months? Purchased recently?).
In addition to those variables, factors outside the email landscape will affect your campaign's performance. Direct mail campaigns, in-store promotions and other offers can influence your tests. Make sure you're analyzing your results with these factors in mind.
5. Hidden costs: A hidden cost is a problem you don't discover until it's too late. For email testing, this refers to a test being deemed a winner based on one aspect of evaluation (e.g., read rate) when in fact it's a loser in other aspects (e.g., complaints) and ultimately ends up hurting your email program. When performing a test, track all of your metrics to prevent this kind of collateral damage. You may think a test is successful if your campaign's read rate is higher than average, but if your complaints soar and threaten your inbox placement, your campaign wasn't the winner you thought it to be. Keep a close eye on metrics like your delete unread rate to make sure you aren't driving your customers away.
6. Stale findings: There's a reason we have channels like CNN that broadcast the news 24/7: The world is constantly changing. Similarly, your email program is constantly evolving. Over time you'll gain (and lose) subscribers, generate new content, and develop new offerings. With all these changes, should you assume your tests from a year ago are still valid? As you gain new people and offerings, you should retest to stay relevant to your current subscriber base. Just because your audience responded to a particular tactic a couple of years ago doesn't mean it works today.