How to Deal With Missing Data: Databases, Zeroes and Imputation
In any case, database managers should constantly be aware of fill rate of each variable, and such figures must be compared with the ones from the previous updates. Often, model shelf life is greatly affected by fluctuations in missing rate. Conversely, it is prudent to check the missing percentage of each model variable when sudden changes in model group distribution is observed.
These few guidelines regarding the missing data may add more flavors to statistical models and, in turn, may also prolong the predictive power of models. After all, missing data may be very meaningful when treated properly.
Stephen H. Yu is vice president of data strategies at Omaha, Neb.-based marketing product and services provider Infogroup. Reach him at Stephen.Yu@infogroup.com.
- People:
- Stephen H. Yu
Stephen H. Yu is a world-class database marketer. He has a proven track record in comprehensive strategic planning and tactical execution, effectively bridging the gap between the marketing and technology world with a balanced view obtained from more than 30 years of experience in best practices of database marketing. Currently, Yu is president and chief consultant at Willow Data Strategy. Previously, he was the head of analytics and insights at eClerx, and VP, Data Strategy & Analytics at Infogroup. Prior to that, Yu was the founding CTO of I-Behavior Inc., which pioneered the use of SKU-level behavioral data. “As a long-time data player with plenty of battle experiences, I would like to share my thoughts and knowledge that I obtained from being a bridge person between the marketing world and the technology world. In the end, data and analytics are just tools for decision-makers; let’s think about what we should be (or shouldn’t be) doing with them first. And the tools must be wielded properly to meet the goals, so let me share some useful tricks in database design, data refinement process and analytics.” Reach him at stephen.yu@willowdatastrategy.com.
