Investing in Testing? These Traps Can Cost You Dearly
While others debate about what to send, marketers who test already have the upper hand on creating content that kills. Previously I discussed the places you should test in your email program, but before you act on your results, make sure you're analyzing them correctly. You could be passing over the winning result if you fall into these six testing traps:
1. Poor testing conditions: As any scientist would tell you, you need to keep your lab clean. For a scientist this means clean beakers, test tubes, slides, etc. For a marketer it means your email data. Performing tests in an unclean environment can invalidate the results. Until you're sure your emails are getting to the inboxes of active users, hold off on performing tests, otherwise you may be calling a winning campaign a dud, when the real dud is your deliverability.
2. Jumping to conclusions: Remember taking tests in school? How you would race against the clock to answer all you could (and maybe guess on a couple multiple choices)? How would you feel if your teacher decided to stop the test early? Would it be an accurate representation of your knowledge? Similarly, ending an email test before it has enough time to collect statistically valid results will give you an inaccurate view of your subscribers’ preferences. At minimum, give a test 48 hours to 72 hours to increase the amount of data points as well as its accuracy.
3. Poor representation: While four out of five dentists might agree on gum, you should still ask more than five if you want reliable guidance. While an email test should be performed on as small a sample of your subscriber list as possible, you need to make sure the test group is large enough to be statistically significant. Before you begin a test, calculate the sample size needed to produce results that are representative of your subscribers. Online calculators are a great, free resource here.