B-to-B Insights: Test Your Way to Success

How to quickly find your control

June 2006 By Russell Kern

multipage

As direct specialists, we are a lucky group of marketers. Unlike the bulk of our brand-building brethren, we actually can measure what works and what doesn’t. Our ability to do this, however, is a mixed blessing. For, as frequently as it makes us feel like heroes, it also makes us feel like crawling under the desk.

Working with B-to-B and B-to-C direct marketers both large and small for 25 years, it has become increasingly clear that B-to-B marketers aren’t doing anywhere near enough testing.

The B-to-B Conundrum
B-to-B marketers are in a quandary: Should they adhere to the traditional guidelines of testing, or yield to their emotional desire to generate better results, right away, to boost their job performance ratings?

There are three traditional rules that hamstring B-to-B marketers:
• You can test only one thing at a time.
• You must always test head-to-head.
• You must always use statistically valid samples.

These rules are quite valid when we can mail or blast out in high volumes and can afford to allocate 10 percent to 20 percent of a campaign to testing. More specifically, traditional guidelines apply when we can afford to create sample sizes of 5,000 to 10,000 contacts for a given test cell. For example, a mailer dropping 1 million packages at a time can conduct 40 different tests of 5,000 each, and still mail the control package to 80 percent of the file.

But what is a B-to-B marketer to do when the suspect universe is constrained by targeting criteria—geography, industry, revenue size, etc.—to 100,000 or 50,000 names, or even perhaps as few as 25,000 names?

When only 50,000 names can be reached, conventional testing methodology limits the number of cells to between five and 10—in effect, turning the entire file into one giant test. Often, that’s just not enough testing to optimize success, particularly when there is no control or offer in place and little or no time to find one.

In the past decade, many of our clients have come to us in similar predicaments. They have to get out in the market quickly, and they either have no control package or the results of their current control have been deteriorating. They must hit their sales numbers, and need direct marketing to feed the sales pipeline. If we can help them find a winner quickly, they tell us, they’ve got resources set aside for a consistent rollout.

To help these clients solve their problems, The Kern Organization has developed its own methodology for multivariate testing. Called ControlFinder™, we employ this methodology to help clients test multiple offers, multiple creative approaches and multiple media sources, all at once. How do you know when you need this type of multivariate testing program? When there is no control. When there has been no testing, prior testing has failed, or when there is no meaningful performance data for lists, packages or offers.

• MORE ON
• B-TO-B