Where the Data Analyst's Job Ends and the Algorithms Begin
Today, data-driven marketing has a new partner. Algorithms are not only changing the role of data, they’re also changing the lives of the analysts who spend their time working with that data. If you were to pull back the curtain at Google, Facebook, Amazon, the stock market and election forecasts, you would find that data is driving the mighty world of algorithms.
Let’s explore three common scenarios to define where the analyst’s job ends and the algorithm begins, or vice versa.
1. Day-to-Day Reporting
Data is often exported out of various databases to be reported on a recurring basis, be it daily, weekly, or monthly. For example, sales data has to be taken from multiple sources and delivered to management in a dashboard format. Sales transaction data is stored in POS, web, order management and even general accounting databases.
Regardless of how the sale was captured, each transaction includes multiple data points. In order for this transactional data to be organized and summarized in a dashboard, analysts must have a way to move the data from one field into another. In cases where all the orders end up in a single customer data platform (CDP) of some type, the platform usually generates the report while analysts drive the reporting process. This report generator uses simple algorithms to query and generate reporting dashboards.
In cases where a single customer data platform is not available, an analyst must query the data and then manually copy and paste it into a reporting dashboard. Algorithms play a crucial role in these repetitive reporting processes. Procedures to query, copy and paste elements can be programmed algorithmically, reducing the overall work by as much as 90%. While an analyst must still QC the reporting and handle anything that falls outside the norm, algorithms are invaluable in reducing workload. However, it remains that the analysts are ultimately in charge of the deliverables.
Attribution is a process where transactional data is combined with marketing source data to determine where the credit for each sale should go. Algorithms often are used to attribute online and offline sales in a matchback environment.
Transactional data can be analyzed by algorithms to look for historical consistency within the data. For example, historical data might tell us that 90% of email clicks that later convert to orders, do so within 48 hours of the email launch. In this case, an algorithm can be written to assign all orders that happened within 48 hours of an email click-through be attributed to email channel.
Similarly, algorithms can be used to look for and act on rules that are based on behavioral patterns in search, social media and other online clickthroughs. Each medium gets credit for their orders based on rules put in place by the analysts. What is left over after this attribution are the overlapping and unknown transactions. These must be handled by analysts to complete the attribution process.
In the cases of attribution, while algorithms can provide insights, analyst are intimately in charge of the process and resulting allocation of sales.
3. Artificial Reality
Where artificial reality is concerned, the analyst may NOT be in complete control of the eventual deliverable.
Consider a “Deal of the Day” delivered as a pop-up on a website. While this process is most often done manually by an analyst, algorithms can be written to rotate a series of discounts for each day of the week to populate a popup template. The algorithm pulls from a library of “files” to be shown under certain conditions. The analyst controls the algorithm to determine which deals are shown, what behavior triggers them, and how often a deal is made available.
A more advanced level of algorithms can allow this communication to be delivered by voice, through Alexa, Facebook Messenger, or a similar technology. This is where the risk of the algorithm taking control becomes much higher. The same “file” of available deals is present but the deal is delivered by a voice and most importantly, a third party device. This third party device operates according to its own data sets, algorithms and even artificial intelligence. This is where the fun starts.
The voice command to access the “Deal of the Day” may have more than one brand to choose from, and ultimately serve up another brand’s “Deal of the Day.” This is likely to happen if the voice command does not specify which deal to serve up properly, or uses different words than the device is programmed to understand. The resulting answer may wander from the searcher’s intended choice and serve up something totally irrelevant. This happens often with Siri on my iPhone, and can happen on any device where the algorithms for the third-party device are written by a different analyst then the one who writes the “files” for each brand.
In these cases, the device ends up in charge of the artificial reality it offers, not the analysts who write the algorithms.
Where the algorithms end, the analysts’ challenges begin. There’s no question that algorithms are here to stay, the fun is in meeting those challenges!