5 Important Answers to Your Email Testing Questions
If you don't have that capability, I'd recommend very strict Google Analytics (or whatever analytics platform you use) campaign codes on the links in each variant. For example:
Email A: http://www.litmus.com?utm_campaign=newsletter+version+A
Email B: http://www.litmus.com?utm_campaign=newsletter+version+B
That way, when you are viewing your analytics data, you can tie actions back to specific test variants.
5. If you change multiple things in one email, how do you know which component is the thing that's pulling better?
To identify the thing, you should absolutely change only one specific variable per test variant. That's the textbook answer for sure.
To be candid, though, and speak as an email marketer with strained time and resources, sometimes it can't be that academic in real life. In the value proposition test I mention in the webinar, for example, we were more broadly testing two concepts for value proposition statements (product-focused or solution-focused), so I was less concerned with which individual tagline was going to make the difference, but rather which theme.
Honestly, sometimes you might just need to know that "this group of things did better than that group of things" and try to distill a reason as to why.
Matt Byrd is the senior email marketing manager at Cambridge, Mass.-based Litmus. Reach him at email@example.com.