Database : Get the Dupes Out

Modern tactics for ridding databases of duplicate customer records

Older de-duplication processes use character-based logic and look-up tables that are easily deceived by minor variations in the name and address elements, such as married names versus maiden names, nicknames, typos and mis-keys.

Fortunately, marketers are starting to tap newer technology that uses referential transaction databases. Overall, these databases make it possible to provide current addresses for the majority of movers for which no information exists. This often happens when the mover does not complete the NCOA process or when an NCOA card or entry did not match the existing name and address on file.

These databases serve as a knowledge base of millions or billions of data elements culled from purchase transaction information, the marketer's own data and outside data sources. Including transactional data helps create a highly accurate view of consumers at any given address or point in time.

How It Works
Newer technology can automate processes that once required human intervention. These newer solutions employ advanced customer recognition technology and "fuzzy logic" to address the identity recognition issues caused by the dynamic nature of customers and the multiple methods of data collection in use today. These methods leverage newer and more sophisticated data matching algorithms and are applied to customer recognition challenges. Such solutions can now identify and "collapse" customers with multiple identities, regardless of name and address permutations, omissions or mis-keys.

Today's advanced customer recognition solutions incorporate several distinct types of advanced matching algorithms, each designed to group records into temporal data sets for the purpose of bringing visibility to distinct patterns of repetitious error.

Once patterns of error are identified, the records can be referenced to external data sources, such as a separate transactional database, to correct the errors and uncover the true identity of the customer. Comparing your data to an external data source provides transactional evidence and validates that a person exists at an address at a specific point in time (see an examplle, in the mediaplayer to the right, of a typical duplicates situation and the resulting consolidation).

Getting Started
Until recently, advanced de-duping tools were out of reach financially for the average marketer.

Fortunately, as the adoption of newer approaches increase, it is bringing the cost of these solutions within the realm of possibility for many marketers. Plus, more marketers have realized success in their multichannel programs, and this is driving overall interest in technological solutions that can bring about cost savings. Improvements in technology have, historically, been embraced by direct marketers, who realize that, by decreasing the number of undeliverables, the technology will help pay for itself.

Ideally, marketers should turn to tools that can readily integrate with their existing processes and can perform the following functions:

• Repair and perfect incomplete addresses.
• Accurately identify undeliverable addresses.
• Locate movers who were not found using NCOA-Link alone.

It is only a matter of time before automated "Data 2.0" solutions are widely adopted in the marketplace. After all, the results are significant. You will reduce your mail costs, and you will truly have the correct, consolidated identity and location of every customer for all of your direct marketing campaigns. You will then be able to determine each customer's individual value to your company. Plus, you will help increase customer loyalty and boost your image, because you will no longer be sending irrelevant or duplicate mailings to your top customers.

Rod Ford is president and CEO of CognitiveDATA (a Merkle Company), a marketing technology company serving multichannel merchants. He can be reached at (866) 243-7883 or rford@cognitivedata.com.

• MORE ON
• DATABASE & CRM