Direct Selling: Testing 101
Mistake #6: Testing Quantities That Are Too Small
To determine the correct quantity for test mailings, consult your statistical textbook and determine the correct level of confidence to assure statistical significance. A quick basic rule to help determine test quantities is the Rule of 100: For each test cell, you should have a minimum of 100 responses for any level of statistical reliability and validity. It’s the percent response that determines how many to test. For example: If you expect a 2 percent response, you need 5,000 names to get 100 responses. If you expect a 1 percent response, you need 10,000 names to get 100 responses.
You see the logic. Business mailers who traditionally get small responses like .3 percent have a problem when it comes to testing. Ideally, to end up with 100 responses each test cell would need to be 33,333, which can be prohibitively expensive. Many times, there aren’t enough names available either from housefile or outside, rented lists. In this case, test quantities that give you a result with which you’re comfortable.
Testing can benefit direct marketers, especially if they prioritize testing those things that will produce the greatest dividends. Offers, lists and formats are biggies that can change results. Good testing.
Jack Schmid is president of J. Schmid & Associates Inc., Mission, Kan. He can be reached at (913) 236-8988 or firstname.lastname@example.org.