Brian Kurtz on DM Testing in Hard Times
A solid control. A perfect pitch. A piece that’s pleasing prospects. Got it? Congrats. Now beat it.
Not as in “get out of town,” but instead getting back to work at making that effort even better. That should be the job of every direct marketer using the mail, especially in these hardscrabble, competitive times—but many go the opposite direction and pause their testing operations, or they never tested much in the first place.
One prominent marketer who believes that testing is essential for direct mail success is Brian Kurtz, executive vice president at Stamford, Conn.–based Boardroom Inc. It publishes Bottom Line/Personal, one of the largest consumer newsletters in the country, as well as books and five other newsletters.
Ethan Boldt: Why is testing such a good idea, even in an economic slump?
Brian Kurtz: Testing is critical. I grew up under the tutelage of Dick Benson, the well-known direct mail consultant. One of his basic tenets was that mailers don’t test enough, and another corollary to that is [from] Gordon Grossman, who is convinced that mailers often test the wrong things. He has an expression, “Don’t make tiny tests.” It’s even more important today to understand that because the cost of a test panel is so much more expensive after this last postal increase.
But it is absolutely essential to test all the time. A couple of things that we probably use as sort of a motivator for testing is an expression that the control is your enemy. Marty Edelston, our founder, always says, “The King is dead! Long live the King!” So the day we get a new control is the day we need to start figuring out how we’re going to beat it.
I don’t think the economic times should affect what you do. I think the best package is the best package. I think with the tough times, you have more pressure to get breakthroughs and do dynamic testing.
EB: Are big tests or small tests better during downtimes?
BK: To test something that’s too small doesn’t give you the ability for a breakthrough or at least a 20 percent or 30 percent lift, which is kind of a rule of thumb we use. If we don’t think the test can produce [that kind of] lift, it’s probably not worth doing. Whereas, in years past, if you got a 5, 10, 15 percent lift, it was pretty good. But under these very difficult circumstances in terms of high postage costs, high paper costs, etc., to plan a test based on that is a mistake.
EB: What’s too small a test?
BK: Changing the color of paper, the kind of stock, a border. You’ll see over the years that those things give you a lift, but those are the types of things I don’t have patience for anymore—those are tiny tests. I will say, however, that little things still mean a lot, such as anywhere near the order device or order card.
EB: Is it a good time to violate some of the rules, such as the rule that states you should test only one variable at a time?
BK: That single variable is still the rule of thumb; it’s what makes direct mail so fantastic as a testing medium. We’ll make a list of all these little things that we’d like to do to the package: change this to red, move this to page 4 from page 2, etc. So we’ve violated that rule a little bit over the years and created what we call the “killer package.” It’s how we look for that 30 percent lift. It [also] appeases a lot of people internally to get some things that they’d love to see in the package changed.
EB: If you want to test a new package with a new copywriter, do you have him or her start from scratch?
BK: Yes. Having a new writer writing against the control is your best chance of getting a winner, as opposed to just tweaking the existing package. This is something that we’ve seen over the past 20 to 25 years: The best way to unseat a control is to go to a brand-new copywriter and have them start from scratch. Ninety percent of the time we’ve had a big lift, up to 30, 40 and 50 percent, is because we did a brand-new package by a new copywriter.
EB: How intricately involved should the designer be in a new test?
BK: Very involved. Designers are very underappreciated and undervalued in this context. We are in an era of magazine people using voucher packages and well-known brands using very straightforward #10 packages with very few inserts and four-page letters. But once you get into more extravagant packages, whether it’s magalogs or bookalogs or a multicomponent package, I think that the design must really link up with the copy.
I’m not saying you have to hire the team at the time, but the best copywriters in the country also know the best designers. Getting them to work together is critical, especially in the kind of stuff that we do. Magalogs, tabloids … 16- to 20-page highly designed packages. You could have the best copy in the world, but if you don’t have the right designer, you’ve got a loser. The designer can make or break the package.
We’ve even taken existing controls, given them to a top-notch designer and had that designer redesign the whole package from scratch—but keeping the content the same—and had some good results.