The headlines blare it loud and clear: “Big Data, Big Problems” was the title run several months ago by the renowned business daily The Wall Street Journal, a newspaper not known for being hostile to innovation. Shortly thereafter, the Swiss business magazine Bilanz led with an even more explicit headline: “The Big Data Lie.” The journalists were further supported by the iconic newsletter published by CB Insights. The market research institute had analyzed how often startups used the terms “big data” and “artificial intelligence” during teleconferences with investors. It found that “artificial intelligence” dethroned “big data” as the dominant term in mid-2016 and has now become the conversation topic three times as often.
The euphoria over big data has definitely faded since the days around a decade ago when the magazine Wired proclaimed that this technology would render conventional research superfluous. Wired, the bible of Silicon Valley geeks, wrote that theories and hypotheses would henceforth no longer be necessary; computers would now discover correlations entirely on their own. In 2011, McKinsey & Company predicted that big data would enable the public sector in Europe to save EUR 250 billion annually, an amount greater than Greece’s gross domestic product. Five years later, the consultancy firm reviewed the state of play and acknowledged that only 10 to 20% of that cost-saving potential, at the most, had been realized.
So what’s the truth? Is big data making the world a better, more efficient, and more knowledgeable place? Or is it the new technology and not conventional research, contrary to the prophecy, that has turned out to be superfluous?
Misconception #1: Big Data Means Oodles of Data
The difficulties start with the definition: The description of what big data really is remains awfully vague. Dan Ariely, an acclaimed psychology professor who specializes in the study of irrational behavior, drew parallels between big data and the intimate love lives of adolescents in a widely publicized tweet: “Big data is like teenage sex: Everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.”
The common definition of “big data” encompasses four terms that all start with the letter V: “Volume” means that enormous quantities of data are involved. “Velocity” refers to the fast speed with which data accrues and is processed. “Variety” expresses the fact that data can be of very different natures ranging from simple tweets to complex traffic data, and “veracity” means that the data must be truthful.
Misconception #2: Computers Are Intelligent
Computers are perfectly suited to performing a vast array of different tasks, but are simply too underdeveloped to handle other, more sophisticated jobs. Computers are still “remarkably dumb,” says John Giannandrea, the former head of artificial intelligence at Google and the new AI chief at Apple. He compares computers’ current stage of development to that of a “four-year-old child.” Users of smart speakers from Apple, Amazon, or Google know just what he means. A study revealed that smart speakers understand almost every question asked of them, but answer correctly only around 75% of the time.