Direct Selling: Testing 101

Six common testing mistakes made by direct marketers

November 2006 By Jack Schmid

multipage

Mistake #2: Not Testing the Big Things That Make a Difference

Smart marketers prioritize their testing to zero in on those things that will improve response rates and average order value, and ultimately that can change sales, profitability and even the business model. Here are the big things that should top your testing priority list:
• offers;
• new formats;
• mix of customer contacts, e.g., catalogs, direct mail, “push” e-mails;
• the optimal number and timing of customer contacts by all media; and
• new products and new product categories.

Putting on your statistical hat, remember that to be pure a test needs to be mailed:
• to exactly the same audience segment, e.g., last 12 month, two-time, $150-plus lifetime sales buyers, or to prospects from the same list;
• at exactly the same time as the control;
• with no other variables except the test to impact the results; and
• testing one variable at a time.

Think about what will make the biggest difference in your next promotion and find ways to test the variables.

Mistake #3: Not Testing Creative

Smart direct marketers use personalization, new covers, wraps and even dot whacks to test creative variations. Here are two ways you can use these techniques to test new creative.

1. To test an offer, highlight the special deals on the homepage or catalog front cover. This type of test requires only a change to your order form, which is easy to make. It’s even economical to test two, three or four offer versions.

2. Reactivate inactive buyers or inquiries through special e-mail campaigns, catalog cover changes or special wraps that bear a message like, “We miss you,” or “Don’t let this be your last mailing.” The key to testing such a reactivation message, however, is to test a special copy versus no copy, and measure the difference. You might even test offering a premium for reactivation.

Mistake #4: Not Testing Catalog Front Covers

It’s amazing how often I hear, “I’m too small to worry about cover testing. That’s something reserved for the big catalogers.” Would you consider testing covers if you saw a 10 percent difference in overall response? How about 20 percent? I’ve seen cover testing that has produced as much as a 35 percent difference in sales. On a side note, many larger multichannel companies also produce a separate prospecting catalog that has fewer pages and concentrates on winning items. The customer version is larger and offers considerably more product, especially new items.

Mistake #5: Not Testing Alternative Media To Grow Your Customer List

While list rentals historically have played a huge part in building the direct marketing industry, today’s successful marketers are looking at a wide variety of alternatives to help build customer lists. The Internet has created a complete paradigm shift in new customer acquisition as well as in finding new direct buyers who heretofore have not been attracted by direct mail or cataloging. List rental response rates are falling, and this historic media is becoming less and less productive. The cost of mailing is up again this year with postal and paper increases. But there is help. It’s in testing new or alternative media that doesn’t require postage or at least shares it. Consider the following list of alternative ways to build your buyer list without renting names:
• search engine marketing, both paid and free
• viral marketing campaigns
• affinity marketing efforts
• newspaper and magazine space ads (test a direct sale of a winning product in the ad versus generating an inquiry)
• best customer referrals
• insert media (package inserts, statement stuffers, ride-alongs, etc.)
• co-op mailings
• trade shows (especially for B-to-B mailers)
• broadcast media (television and radio)

Mistake #6: Testing Quantities That Are Too Small

To determine the correct quantity for test mailings, consult your statistical textbook and determine the correct level of confidence to assure statistical significance. A quick basic rule to help determine test quantities is the Rule of 100: For each test cell, you should have a minimum of 100 responses for any level of statistical reliability and validity. It’s the percent response that determines how many to test. For example: If you expect a 2 percent response, you need 5,000 names to get 100 responses. If you expect a 1 percent response, you need 10,000 names to get 100 responses.

You see the logic. Business mailers who traditionally get small responses like .3 percent have a problem when it comes to testing. Ideally, to end up with 100 responses each test cell would need to be 33,333, which can be prohibitively expensive. Many times, there aren’t enough names available either from housefile or outside, rented lists. In this case, test quantities that give you a result with which you’re comfortable.

Testing can benefit direct marketers, especially if they prioritize testing those things that will produce the greatest dividends. Offers, lists and formats are biggies that can change results. Good testing.

Jack Schmid is president of J. Schmid & Associates Inc., Mission, Kan. He can be reached at (913) 236-8988 or jacks@jschmid.com.