B-to-B Insights: Test Your Way to Success
• Media research has shown that we need to test five list sources. Five thousand records are available for testing from each source. Four sources are controlled circulation subscriber files from trade publications in which ACME has advertised, and the other is a file compiled from D&B that ACME can use at half the cost per thousand of controlled circulation files.
• A 2 percent or greater response is required for any cell to be within the acceptable range for cost per inquiry.
Assuming all of the above, the testing grid would look like the Test Matrix shown below.
Although each of these 30 cells contains 833 names, it doesn’t mean the result from each cell is, by itself, statistically valid. For each cell in the grid to be statistically valid, the test would have to involve roughly 150,000 pieces, or twice ACME’s 75,000-piece mailing universe.
However, much can be deduced about market desire from the results of a single cell—when looked at in the aggregate. When conducting this type of multivariate testing, you are looking to identify strong trends and hot spots that will tell you what and what not to do next.
Let’s assume the test generated the response rates shown in the Response Rate Matrix (see below). In this example, it is safe to conclude the following:
• Nothing worked with List One.
• Package Y is a clear winner over Package X.
• Offer C worked best with Lists Two, Three and Four.
• Offer B worked best with List Four.
Step Two: Confirm Your Findings, Expand Your Testing
This is the point at which you conduct a confirmation test of the results from step one. This time, however, you must use only statistically valid sample sizes within test cells to confirm your results with confidence. Here’s a good rule to follow: You need 50 observations from a given cell, drawn from a sample size of 5,000 to read your results at a 95 percent confidence level.