Database: Model Behavior
Far more accurate and revealing is looking at both cross-correlation and autocorrelation, which compares results to past (usually fairly recent) data series or with recent results. If you're analyzing total sales, for instance, cross-correlation determines how the recent history of various media activity affects current sales, while autocorrelation determines the relationship with prior sales.
The complex relationship of a series of data to other series of data is the most important consideration in building an accurate predictive model.
And Your Final Question
The acid test for any modeler is predictive accuracy in real time. If your model predicted that 25,000 consumers would respond to your call center this week, how many actually responded?
We counsel our clients to strive for the highest possible predictive accuracy so they can make, for example, realistic staffing decisions for their call centers. The larger and more complex your marketing program, the more important additional accuracy becomes to your ROI.
A good indicator of the ultimate quality of analytic models is the quality of the modelers themselves. Ask about the education and experience of the analysts who build the models. That should include advanced degrees in mathematics and statistics, as well as a significant number of years of experience in decision support modeling. Ask about the types of clients they've built and run models for. They should be clients as big and as sophisticated as your company.
What's Your Payoff?
Analytics is a frightening topic to many in marketing. Why is that? It can't be because it's complex. Marketing, especially in this age of splintered media, is equivalent to rocket science (where brand and product is the payload). But increasingly, analytics is the telemetry (literally "remote measurement") of marketing, the feedback data that allows us to change course when necessary. At its core is statistical mathematics that are daunting to all but the most highly trained.