|
Testing is the foundation on which we build a direct marketing business. Therefore, without proper knowledge of test planning and analysis, we are not in the best position possible to help our company grow. It’s that simple!
There are many secrets of test design and analysis guaranteed to maximize the effectiveness of your testing programs and ensure you stay within budget. Let me share six of them with you now.
Secret 1: When rolling out with a new promotional strategy for the first time, you must always back-test or re-test the old package at least one time. The reason is that if this new package does not hold up in rollout, you may very well be misled into thinking the new package was a loser. In reality, it may have performed badly due to a name selection error, list fatigue or seasonality. Without a back test you will not be able to determine the true reason the new package did poorly and, as a result, you might erroneously revert back to the old package. Be careful.
Secret 2: When planning new creative or price tests, you must first conduct a break-even analysis to determine the appropriate sample sizes. Many marketers just set up their test sample sizes to be able to read a 10 percent difference as significant. In most cases this is not tight enough. Every test is different, and some will require a tighter read than others. You must conduct a break-even analysis first to help you properly assess the maximum amount of error you can tolerate in your testing results. For example, if you are conducting a new and more expensive format test and the test needs five additional orders to breakeven versus the control, make sure you set your sample sizes to read a minimum increase of .005 percent or .5 percent over your control response rate as significant.
Secret 3: Only make one change at a time to your test panels. If you make multiple changes to your test packages, you will not be getting the most out of your testing program.
For example, if your test package loses against your control, it may very well be a result of only one of the changes you made to the package and not all. You will never know which changes were working and which were not. As a result, you may overlook potentially winning elements. This simply is not smart testing.
There are many secrets of test design and analysis guaranteed to maximize the effectiveness of your testing programs and ensure you stay within budget. Let me share six of them with you now.
Secret 1: When rolling out with a new promotional strategy for the first time, you must always back-test or re-test the old package at least one time. The reason is that if this new package does not hold up in rollout, you may very well be misled into thinking the new package was a loser. In reality, it may have performed badly due to a name selection error, list fatigue or seasonality. Without a back test you will not be able to determine the true reason the new package did poorly and, as a result, you might erroneously revert back to the old package. Be careful.
Secret 2: When planning new creative or price tests, you must first conduct a break-even analysis to determine the appropriate sample sizes. Many marketers just set up their test sample sizes to be able to read a 10 percent difference as significant. In most cases this is not tight enough. Every test is different, and some will require a tighter read than others. You must conduct a break-even analysis first to help you properly assess the maximum amount of error you can tolerate in your testing results. For example, if you are conducting a new and more expensive format test and the test needs five additional orders to breakeven versus the control, make sure you set your sample sizes to read a minimum increase of .005 percent or .5 percent over your control response rate as significant.
Secret 3: Only make one change at a time to your test panels. If you make multiple changes to your test packages, you will not be getting the most out of your testing program.
For example, if your test package loses against your control, it may very well be a result of only one of the changes you made to the package and not all. You will never know which changes were working and which were not. As a result, you may overlook potentially winning elements. This simply is not smart testing.
|
Secrets of Direct Marketing Testing
PURLs for Profit