6 Mistakes to Avoid in Email Testing
Testing is unwanted in both direct mail and email marketing. Everyone says that testing is essential, but most marketers postpone tests until some later date when they're not so busy and not under pressure to make revenue goals.
Let’s face it, testing takes time away from getting out tomorrow’s email or next week’s direct mail campaign. It requires setting up control groups that get something different from what everyone else gets. If your control group does worse than your primary mailing, you lost needed revenue.
Email marketing makes testing very simple and accurate. Best of all, results come back in 24 hours. But there are several mistakes that many email marketers make when testing. Here are six of them:
1. Making too many changes at once. When you change several items in one version of an email and test it against your control, you won’t learn much. Suppose that two versions of an email are completely different in terms of layout, promotions, copy and even products. Any one change might have helped or hurt response, but by putting them all together you can’t tell what was doing what. One change might have helped the open rate increase 10 percent, while another change in the same email might have reduced clicks 10 percent. The rule is to test only one element at a time.
2. Looking only at conversions. There's a logical sequence for email recipients: they're attracted by the subject line so they open the message; they like the way the salutation greets them when they open; the copy is interesting and the offer stimulates them; and they click on a couple of links and finally order a product. A weak subject line, bad salutation, uninspiring offer, faulty link, poor product or pricing, or confusing order form can make your email fail at any time. Which of these elements was responsible for the low conversion rate? You may never know if you look at the conversion rate by itself.
Test each element separately. First, select a subject line that works better than others. Then get your best salutation. Now you're ready to try different versions of copy. You also need to test pricing and your order form. Nobody ever said testing was easy, particularly if you neglect testing elements separately.
3. Offering too many choices. Time after time, direct marketers have tested direct mail pieces that offer readers two or more choices against a mail piece that offers only one possible answer. One choice always beats two or more choices. For example, let's say a company selling tours tried two approaches: “Only $69 for a family vacation weekend. Chose any of the following six weekends and the vacation is yours” or “We have only one available family weekend left for $69. Call right away.” The second message will always beat the first by a large margin.
Don’t think you can’t learn from direct mail folks just because you're an email marketer. There are a lot of fundamental truths about marketing that have already been learned. Choice killing response is a fundamental truth. Don’t make the mistake of giving readers a choice.
4. Not knowing the territory. You'll get better results for your email campaigns if you know who you're sending your emails to. Are they affluent seniors or college students? Are they married women or single men? Do they live in big cities or out in the country? Are they previous buyers or subscribers who have never bought anything?
The elements you test may attract some people but alienate others. If you know nothing about your audience, stick with big, significant changes in your testing — e.g., a dramatic ease of navigation or a promotion that's of general interest to all.
If you do know the territory, show different content to different segments. Run different tests for each segment. You’ll get clearer answers on your tests if the population who sees the test is more consistent.
5. Testing by committee. The worst group to create an email campaign is a committee representing different departments in your company. Some people like one idea and others like something else. You compromise while good ideas are edited out of the process as being bad ideas even before they can be tested. The result is a test that's really an A/A test — i.e., there may be a little tweak here or there, but the two versions are essentially the same.
A better approach is to let your creative people loose to test new ideas. Are you in business to create peace in a committee or to build customer loyalty and sales? After all, it’s only an email. Your whole company’s reputation doesn’t hang on every word. Cut loose. Try something new. Use what you learn.
6. Using personalization in the subject line. Since personalization in the body of the email improves clicks and conversions, why not use it in the subject line? At one time that was a good idea, but spammers have ruined it. By now millions of people have received emails from unknown people who seem to know their first and last name. They picked them up from a phishing expedition. If you use a subscriber’s name in the subject line, you may be deleted as spam.
Arthur Middleton Hughes is vice president of The Database Marketing Institute. This article is an excerpt from his new book from McGraw-Hill, "Strategic Database Marketing, 4th Edition." Arthur can be reached at firstname.lastname@example.org.
Related story: The Danger in Mailing Too Often