Catch Bad Data Before It Wrecks Your Business
During the past two decades, conventional wisdom surrounding data quality has drastically changed. As decision support systems, data warehousing and business intelligence have triggered greater scrutiny of the data used to measure and monitor corporate performance, a changing attitude has gradually altered our perception of what is meant by “quality information.”
Instead of focusing on specific uses of data in the context of how data sets support operation of transactional systems, we have started to consider data reuse and repurposing, noting the data’s inherent value, which goes beyond its ability to make functional applications work. And, as opposed to a knee-jerk reaction to data errors, the industry now focuses on evaluating conformance to business rules that are indicative of a data set’s fitness for its (potentially numerous and varied) purposes.
Even so, the fundamental aspects of data quality improvement have generally remained the same and center on a virtuous cycle:
- Evaluate data to identify any critical errors or issues that are impacting the business
- Assess the severity of the errors and prioritize their remediation
- Develop and deploy mitigation strategies
- Measure improvement to the business
- Go back to step one
This cycle of excellent data quality requires that businesses rely on effective tools and techniques for each step. Tools help uncover the existence of data errors, evaluate the severity of the problem, eliminate the root causes and correct the data, and further inspect and monitor ongoing data quality activities. The real challenge lies in understanding: when data values are or are not valid and correct; how data values can be made correct; and how data cleansing services can be integrated into the environment. By focusing on five key aspects of data quality management—including data cleansing, address data quality, address standardization, data enhancement, and record linkage and matching—businesses achieve a practical and proactive approach to data quality management.