The standard rule in direct mail testing is to roll out your control to the majority of your list and send a smaller panel of test packages to compare results. That's the safe bet, putting most of your eggs in your previously top-performing direct mail basket.
Well, instead of making the safe bet, Garden & Gun magazine decided to gamble a little bit in its late September 2008 acquisition mailing. The Southern lifestyle publication rolled the dice by actually sending out more packages of one of its test panels-which featured covers that recently had performed stronger than the control's covers on the newsstand-than its control. "I probably broke some rules, but because the covers were really stronger newsstand covers, I actually mailed more of that than I did my control," admits Gary Michelson, circulation director at South Norwalk, Conn.-based magazine circulation management company Circulation Specialists, Garden & Gun's circulation vendor. "I almost made it kind of my control, if you will, and back-tested the older one."
Michelson elaborates that a little more than better newsstand results came into play in his decision. Garden & Gun, which launched in April 2007, had sent out initial mailings prior to the publication's launch with a certain concept of what the magazine would entail. However, when the magazine actually came to be, it was a little different than was previously thought, meaning the original direct mail acquisition package had to be updated to fit the magazine. As a result, "we developed a whole new package," explains Michelson. "It was our control by default, but it did pretty well. We had some really good response from some of our lists, so we used it again."
Last September, Michelson mailed nearly 227,000 total acquisitions in three panels: the control and two tests. The control, which had been Garden & Gun's top performer since April 2008, was a 6" x 10-1/2" mini-magalog with the headline, "Come enjoy the best the South has to offer" and a cover with a dog on the front, along with a cover of a man fishing on the back. Inside, the magalog features magazine spreads highlighting the vast array of subjects the magazine tackles. One test consists of the exact same copy throughout, including an identical headline, that simply replaced the covers featured with covers that had performed stronger on the newsstand. The dog on the front was replaced with golfer Davis Love's cover, and on the back, a young woman sitting on a boat usurped the man fishing. The second test has the same covers as the control but tweaked the copy slightly. The headline was changed to, "Come enjoy the very best of the new South," and reader testimonials were added inside (Archive code #202-717547-0901A).