Businesses worldwide are in the midst of a data feast and statistical Renaissance. Data scientists are being hired at a rapid clip, the likes of which haven’t been seen since the launch of the Internet and the frenzied search to hire anyone who could do HTML programming and build web sites. Based on reports from the front lines of business analysis, it would appear safe to conclude that the promise of Big Data is being realized daily. But is it?
We certainly have more and more data on where people are at any moment with their cell phones, and how they browse and shop online. Google almost singlehandedly has launched the most recent transformation of the advertising industry toward ever greater data-based decision making. For companies deep in the throes of large-scale transformation of how they interact with customers and learn about their buying patterns, it can feel right now like you are living in the eye of a hurricane: things may be relatively calm at this very instance (meaning you aren’t losing a ton of business to new and emerging business models), but the prospects for calm weather and an easier time making money are a long ways off.
My take on all of this: don’t get swept up in the hype. Yes, there are real insights to be gained from all the new data that’s pouring in, and any leader who ignores the potential of that data is a fool. But even more important is not catching new data fever, and losing sight of how you actually make money. Once the dust settled after the Internet’s commercial launch and web sites became a standard part of business practice, the competitive advantages of the vast majority of companies – how you uniquely make money and protect yourself against competitors – had not changed. Today everyone has a good, well-functioning website that is an integral part of how you interact with your customers – and that web site typically is not a source of competitive advantage.
The future of Big Data and the role it plays in business strategy will be not much different than the arc of the Internet’s first phase of development: some businesses and industries will be radically transformed, others will gain important insights that are helpful but not transformative, and many, many others will emerge on the other end looking pretty similar to the way they do today.
Where Big Data and the new science of business are providing real insights. It is absolutely true that big insights yielding big returns are coming out of the Big Data revolution. The ability to monitor consumers’ movements via their cell phones is rapidly advancing our understanding of when and where people make the decision to buy. Same thing for the ever-more-precise measurement and monitoring of people’s online browsing and purchasing decisions. The analysis requires high powered computers and data scientists who know how to sift through massive quantities of data to glean the insights that matter.
Big Data is helping us to much, much better understand who the people are who are browsing our web sites and making buying decisions. Getting better data on when and where they make purchasing decisions helps with targeting advertisements and deploying sales people and customer support personnel within our existing business models. Optimizing how those resources are used increases efficiency and decreases the cost of sales and customer service, all of which flow directly to the bottom line.
But the fundamental way we make money from selling to those customers hasn’t been changed by Big Data – and won’t be for the vast majority of businesses. Big Data at its simplest is statistical correlation analysis. We see a bunch of things happening at the same time, and the trick is sorting out what variables matter more than others – which correlations to pay attention to, and which to ignore. The huge gap that continues to exist, and which Big Data cannot solve, is understanding what drives purchasing behavior.
Where Big Data falls short. Let’s be clear about where the real insights are coming from. They do NOT come from random analysis of shopping and browsing patterns. In order to make sense of all the reams of data that come in by the minute, the best data scientists apply models of human behavior, testing and validating them with the new data: what leads someone to buy milk versus alcohol or an electric car versus gas powered; how people choose to save and invest their money; why some people own pets and others never will; etc. But the insights on why the purchasing decisions are made come from what we already know about demographics and consumer behavior: who they are, where they live, how much money they make, the personal characteristics that drive their decisions (do they have kids, a spouse, recreational interests, professional affiliations, who they socialize with, etc.). Big Data is giving us more precise measurements on those variables, but not fundamentally changing how we view and interpret those data.
The biggest returns to business investment, and the riskiest, are in discovering what data on past behaviors can’t tell us: how people will respond to new products. This is why, for all its supposed insights, Big Data hasn’t fundamentally changed the process of research and new product development. At its core, Big Data is about looking at what people did in the past to better optimize the products and services we already know how to sell. Big Data didn’t predict that the Uber and Airbnb business models would be successful. Yet once those business models were launched, Big Data is now helping them to better understand who their customers are to optimize what they offer and how to price it. Data scientists can help you better understand how to optimize what you’re already doing, but they aren’t the source of the core knowledge on what creates your competitive advantage.
Statistics don’t ensure insights. The basic challenge is that statistical analysis of consumer and shopping data does not guarantee that you will get meaningful insights from that data. The real insights come from understanding the fundamentals of motivation and behavior: what leads people to do what they do, and make their purchasing decisions; why they react enthusiastically to some new products while others are duds; and so on. Uber, Airbnb and all other successful new businesses have succeeded because they identified a gap in the market that they could fill with products and services at price points that the other companies weren’t adequately addressing. There was no analysis that could reliably predict their new business models would be successful. Instead, their leaders looked at gaps in performance that the incumbent providers weren’t filling, and targeted those gaps.
Big data and statistical analysis can be great for fine tuning what you are already doing, but the insights they provide are backward looking, based on past patterns of consumer behavior. The biggest contributions to your competitive advantage are going to come from closing the gaps in your strategy execution. Data scientists can help paint part of the picture of what’s going on in your business, but that is not enough to create breakthrough performance most of the time.
