Testing, The Dirty Dozen (1,848 words)
Mistake #10: Shelving a Close Runner Up
Just because a test didn't beat the control doesn't mean it should be abandoned. Sharp marketers will take almost-winners, and look for ways to viably engage them.
For instance, take a creative package that outperforms the control, but because of higher in-the-mail costs does not produce a return on investment that meets your company's hurdle rate. Perhaps the package would be effective if it were only mailed to those lists that typically return higher margins. Another technique is to take an "extractive" approach and lift an element from a test to incorporate into the control.
Mistake #11: Forgetting Old Controls
Old controls were great until they were ushered out the back door by some new, flashy upstarts. Oh well, they were losers anyway—right? Wrong! Old controls, especially in creative, often can be dusted off, re-tested and sometimes successfully reintroduced.
All sorts of factors can contribute to the demise of a control position, from overuse resulting in package, envelope or offer fatigue, to the morphing nature of the core audience's demographics or psychographics. The reality of direct marketing is that many past controls have been re-engaged.
Here, too, extractive techniques can work. By taking each element individually and testing accordingly, marketers are sometimes able to raise the efficiency of current control positions.
Mistake #12: Inadequate Reporting and Measurement of Results
Inadequate reporting and measurement can ruin your testing program. So starting out knowing how and with what criteria winning positions will be determined is important.
Make sure a direct response-oriented recording and reporting discipline is firmly established. Know what your evaluative criteria for determining successful tests are before you test. Is return on investment the ultimate goal or is it top line revenue? Understand how you'll measure and how you'll factor. Make sure you test in accordance with statistically valid rule sets.