Data analysis has been exploding with lots of investment being made in analytical tools, universities bursting with class offerings and students eager to learn. Seminars and webinars are held daily across the country and indeed across the world. The power of statistics, mathematics, computer technology, big data analytics, etc., are greatly enhancing the insights that can be gleaned from data. The problem is that not all insights are golden.
People tracking weather have noticed that they can gain a lot of insight about the location of storms from the location of twitter data. Imagine the surprise of people in Manhattan when they were told that Superstorm Sandy was centered on Manhattan, more than one hundred miles north of where it came ashore. The reason for the incorrect information? Twitter hits were higher in Manhattan (more cell phones) and early wind damage took out a lot of cell service further south where the storm was hitting so they were under reporting.
Should you hire people for leadership positions based on their shoe size?Aheadlinereported that a larger shoe size was correlated to a greater chance of being an executive. The actual study had noted that more men were executives and thus increased the percentage of people who were taller than the average person (male and female). Since height was positively correlated with shoe size… No, buying oversized shoes will not help you get a promotion (unless you work in a circus).
“Garbage In, Garbage Out”was used in a November 10, 1957 newspaper article about mathematicians and their work with early computers. It is more relevant today. Bad data can lead to very erroneous conclusions. In 2008, Google started predicting flu trends successfully based on search results. However, when their success was touted, the search for things like “flu prediction success”, “Google flu predictions”, “predicting flu”, etc., led to a 50% overestimation of flu cases. The new data was not related to flu cases per se making the data flawed.
Sometimes an algorithm can be successful for the wrong reason. A company built its own recommendation engine or at least tried to. When the team couldn’t solve the technical problems, they substituted a fixed recommendation regardless of what else was purchased to meet a project deadline. It worked… for a few weeks. The site recommended bed sheets to all customers and sales increased until the demand for bed sheets was met.
Sometimes the data and the analysis are correct, but the decision makers don’t believe it because it clashes with their preconceived notions or biases. In one case, the management of a travel logistics company had their own idea of how customers navigated their site even though the analysts told them that they did it in a different way. Their results were rejected until an A/B test proved conclusively that they were correct leading to some management changes.
Data analytics is complicated. Tools – computer power, sophisticated applications, algorithms, intelligence (human and artificial) – are necessary but not sufficient to assure success. Analysis needs the help of those who are experienced with the processes, those who are experienced with the tools, and those who are experienced finding the hidden gems in the mounds of data. Open minds are essential because data can be easily misused to draw wrong conclusions. The result of your efforts will lead to a significant competitive advantage well worth the effort.