One of my favorite quotes about data analytics, by Duke economist Dan Ariely, compares it to teenage sex: “everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.” 86% of executives surveyed by McKinsey say their organizations have been at best only somewhat effective at meeting their primary objective of their data and analytics programs, including more than one quarter who say they’ve been ineffective. Most of your peers have no clue what they’re doing either, and large swathes of those who do, aren’t doing it very well. They’re gathering all of this data into expensive warehouses, full of servers without a clue as to what to do with all of it. To be completely honest, chances are that you are, too.
Let’s put the spotlight on you, for a moment. You’ve been reading about all the ways these big companies take data and learn incredible things about their customer base. You’ve seen headlines about Netflix’s data science teams using an algorithm to drive 5% of all internet traffic towards their content, and now you’re wondering how you can do the same. Here’s the thing- you can’t compare yourself to these other companies. Know why? They aren’t facing the same problems that you are. Drawing success from data doesn’t start with the fancy artificial intelligence and machine learning algorithms. It starts with a question. Your question. Take a moment and think- say it out loud if you have to- what do I want to know? How many of your customers are switching over to eCommerce from in-person shopping? Are certain products being bought more online than in-person? When should I restock my inventory to make sure my customers are always able to buy the products they want? What products do they want? If you can’t come up with a question that your company needs answered, I want you to close this tab right now and go figure that out.
Once you have your question and you know that using data will add value, not just consume resources, then you have your first building block. It’s not uncommon that “data analytics” is just used as a buzzword and all that’s being done is mindless data mining, wasting time and money that a company could otherwise use for something more productive. I’m sure you’ve been in meetings listening to a presentation and these esoteric terms like “Machine Learning”, “Artificial Intelligence”, and “Predictive Analytics” were thrown out in an effort to impress the audience, only to never be explained as to how they actually contribute to the value of the project, or if they are even needed. Ultimately, your goal is to make decisions and your approach to data should be exactly the same as approaching a hammer- only pick it up if you plan to hit a nail.
It’s important to note that not only is data analytics a tool that should be used with purpose, but it is an iterative approach, as well. There are five main layers, creating this progression towards a complete analytics program. Depending on the problems you are facing you may elect to stop at step two or three, but the framework is there to guide its implementation, beginning with descriptive analytics. This layer is your hindsight- producing reports as to what happened during a specific event or time period and answering specific questions such as “what are my most popular products?”. The second layer is diagnostic analytics- your oversight. What are you seeing in your data, real-time? How are your customers behaving, and what can this tell you about what is happening? Diagnostic analytics builds on descriptive analytics by enabling you to gaze not just into the past, but into the present moment and see things in motion.
George E. P. Box, a renowned British statistician who passed in 2013, had made a particularly accurate assessment that is very relevant to data analytics, but more specifically predictive analytics, the third layer in this framework. He said, “All models are wrong. Some are useful”. Too often people reach this layer and assume that algorithms and AI and all these fancy toys that they hear about are going to solve all of their problems, and make all the difficult decisions for them. The point of predictive analytics is to help identify patterns, not to know exactly what’s going to happen in the future. Here’s an example- you can’t as a human being look at all historical weather data over the entire world for the past century and then predict when a hurricane is going to hit the eastern coast of the United States. To be honest, it’s pretty difficult to create a model that will do the same, let alone be correct in their predictions. You can, however, compare the timeline of the model’s predictions with the context of your model. You can see that the model predicts hurricanes within a certain part of the year, under certain conditions, and you can see from drawing feature importance from your model that your algorithm has placed heavy emphasis on tracking, let’s say, pressure differences in a certain part of the globe. You can then adjust your approach to warning citizens of an impending disaster, saving lives and money. Let’s think of another example- worker’s compensation. It was found that a worker has a slower recovery from their injury when their commute is longer, compared with peers with similar injuries, jobs, and everything else. The shorter the commute, with all other factors being relatively the same, the faster the recovery. Do you know how this was figured out? Feature importance. The model may have predicted the recovery time but didn’t tell you why until you asked it. Remember, at the end of the day predictive analytics isn’t here to solve your problems for you, it’s to here make you better at solving them yourself.
These final two layers are a little more recherché, but are nonetheless the height of maturity for any data analytics program. Prescriptive and cognitive analytics are focused more on the human aspect of analytics. Think of prescriptive analytics as insight into your data. How can you optimize what happens? What’s the next best action to take on what you’ve discovered? Cognitive analytics can be thought of as having the “right” sight (have you caught on to the theme, here?)- what is the right decision to be making in this moment for the context that you find yourself in? Remember that question I asked you to have before diving into all of this? It’s here that you go from having an answer to having the best answer. At this point you should not only have the answer you need to move your business forward, but also you should be asking new questions from the answer(s) you’ve gathered up to this point.
Data analytics is a process that leads to more analytics- if, that is, you’re doing it right. It’s a loop that reinforces itself, and reinforces those who participate in it. Once you’re in, it takes care of itself, leading you to a better and better understanding of the context you find yourself in. It’s a mindset and a business process, a way of thinking and a lifestyle that helps you keep up with a constantly changing landscape of new technology, new customers, and new economics. It’s been said that there’s not just an exponentially increasing number of data analytics products and possibilities, but a combinatorically increasing number- all possible interconnections, linkages, and interactions between you, your customers, your products, and future products. Combining new models, datasets, APIs, applications, cloud services, automation, Artificial Intelligence, and so much more, data analytics is a future that you simply cannot afford to miss.