Direct-Mail Testing Upended With Bayesian Analytics
Direct-mail marketers have relied on either A/B testing or multivariate testing to evaluate winning campaigns for generations. Those evaluations, unfortunately, weren’t always based on statistics, but often on educated guesses or office surveys. But a confluence of technology and something called Bayesian Analytics now enables direct mailers to pre-test and predict responses accurately before mailing.
Bayesian Analytics may well upend how we test to identify the highest profit-producing control more quickly and at a fraction of the cost of traditional testing methods. Bayesian Analytics is already being used in astrophysics, weather forecasting, insurance risk management and health care policy. And now, a few cutting-edge mailers have successfully used this analytics approach, too.
Usually, direct-mail marketers test four categories of variables, such as price, headlines, imagery and formats.
Within each of those variables, direct marketers often want to test even more options. For example, you might want to test the relative effectiveness of discounts of $5 off, $10 off, 10 percent off or 15 percent off. And you want to test multiple headlines, images and formats.
The following matrix illustrates the complexity of testing multiple variables. Let’s say you want to test four different pricing offers, four headlines, four imagery graphics and four direct mail formats. Multiplying 4 x 4 x 4 x 4, you find there are a possible 256 test combinations.
It’s impractical and costly to test 256 combinations. Even if your response rate dictated you only needed to mail 5,000 items per test for statistical reliability, you’d still have to mail over 1.2 million pieces of mail. If each piece costs $0.50, the total testing cost is $600,000.
Bayesian Analysis works with a fraction of the data required to power today’s machine learning and predictive analytics approaches. It delivers the same or better results in a fraction of the time. By applying Bayesian Analysis methodologies, direct mailers can make significant and statistically reliable conclusions from less data.
“Bayesian inference remained extremely difficult to implement until the late 1980s and early 1990s when powerful computers became widely accessible and new computational methods were developed. The subsequent explosion of interest in Bayesian statistics has led not only to extensive research in Bayesian methodology but also to the use of Bayesian methods to address pressing questions in diverse application areas such as astrophysics, weather forecasting, health care policy, and criminal justice.”
Bayesian Analysis frequently produces results that are in stark contrast to our intuitive assumptions. How many times have you used your intuition to test a specific combination of variables thinking it would result in a successful direct-mail test, only to be disappointed in the results?
Bayesian Analytics methodology takes the guess-work out of what to test in a live-mailing scenario. Instead of testing and guessing (as the late Herschell Gordon Lewis wrote in his recent column, Rather Test or Guess?) you can now pre-test those 256 combinations of variables before the expense of a live mail test. The pre-test reveals which combination of variables will produce the highest response rate in the live test, resulting in substantial test savings.
But wait, there’s another benefit: You can learn what mix of variables will produce the best results for any tested demographic or psychographic group. It’s possible to learn that a certain set of variables work more successfully for people who are, for example, aged 60+, versus those aged 40-59. This means you may be able to open up new prospecting list selections that previously didn’t work for you.
Again, a handful of mailers have already pre-tested this new Bayesian Analysis methodology — it has accurately predicted the results in live testing at a 95 percent level of confidence. Now that beta testing has been completed and the methodology is proven to be reliable, look to hear more about it in the future.
There’s more about this methodology than can be shared in a single blog post. To learn more, download my report.
My new book, “Crack the Customer Mind Code” is available at the DirectMarketingIQ bookstore. Or download my free seven-step guide to help you align your messaging with how the primitive mind thinks. It’s titled “When You Need More Customers, This Is What You Do.”
Reinventing Direct is for the direct marketer seeking guidance in the evolving world of online marketing. Gary Hennerberg is a mind code marketing strategist, based on the template from his new book, "Crack the Customer Mind Code." He is recognized as a leading direct marketing consultant and copywriter. He weaves in how to identify a unique selling proposition to position, or reposition, products and services using online and offline marketing approaches, and copywriting sales techniques. He is sought-after for his integration of direct mail, catalogs, email marketing, websites, content marketing, search marketing, retargeting and more. His identification of USPs and copywriting for clients has resulted in sales increases of 15 percent, 35 percent, and even as high as 60 percent. Today he integrates both online and offline media strategies, and proven copywriting techniques, to get clients results. Email him or follow Gary on LinkedIn. Co-authoring this blog is Perry Alexander of ACM Initiatives. Follow Perry on LinkedIn.