Big Data: the seer who predicts the future

Big Data: the seer who predicts the future

Leopoldo Mitre – March 23, 2015

Use properly Big Data may be the equivalent to having the Ark of the Covenant , to have at hand the power to win every battle. But Big Data is tricky: the information have to be reliable. If the information we analyse is untrustworthy, the accumulation of data will be just rubbish.


big-data-the-seer-who-predicts-the-future

In any business -as in life itself- to have the ability of prediction is the equivalent of owning the Ark of the Covenant. Be able to predict what will happen give us the power to win every battle. Unfortunately, like all conquerors of ancient times, who tirelessly searched for that contraption, CEO’s have not received a prediction method bulletproof; so they had always been at the expense of their talent to “guess” what the market wants. At least until the arrival of Big Data.

Big Data concept refers to the massive accumulation of data, in Information Technology & Communication Industry. It focuses on the management of the processes of: capture, storage, search, sharing, analysis and visualization of all the data that we want to manipulate. To put it in simple words: the phenomenon born because of the need to observe patterns or trends through the amount of information collected by an entity over time. Banks, Telecommunications Companies and Internet services -like Netflix– have been putting over the table the usefulness of these tools.

But why we begin to hear of this computer panacea until now? Because Big Data is the evolution of Cloud services, just as Cloud has been to the Internet. It has been a trend that comes in the wake of another. And like any trend it has had to be tested and has to prove its effectiveness before more users take the decision to adopt it. The bad news for many businesses is that Big Data marks a new step on the way to the top of business success. And if you have been of those late adopters of Cloud Services who continue resisting to use them, it is only a matter of time before your competitors show how deadly it may have been to taken so much time in using this new weapons in the Market Battle. It’s like when the Internet emerged: those who decide to use it later had been take by assault for those early adopters who assimilate its benefits before and start with Cloud Services just when the others have not even hear about it.

For example Netflix, the company that started as video club with online and traditional mail video delivery services, but whose streaming video platform catapulted it as the most successful company in history. Netflix can be a good example of how Big Data pays more than the investment they make to build it. Do you remember the popular series ‘House of Cards’? Yes, the one that is directed by David Fincher and in which Kevin Spacey plays the role of Frank Underwood, the dark politician who use all kinds of tricks to become President of the United States. Well, this serie was acquired by Netflix with a contract of two seasons for 100 million USD… Without even seeing a pilot! Yes, they trusted 100% in the statistics they had. Numbers told them that the popularity of a political thriller starring Spacey and directed by Fincher would be on top. No bets based on hunches, only cold data. In the end the audience gave them the reason and ‘House of Cards’ became a success.

With 27 million users only the US and 33 million overseas, Netflix has to disposal a great amount of information. So, with the appropriate methods of analysis they have now the perfect formula to measure the perception of the audience in real time. Up to know Netflix may not know exactly why you pause a program, but when enough people pause, advance or rewind the same period of time during the same program Big Data analysts can begin to make some inferences. Not only that, when they cross data about the user (by region, country, age, gender and even stored trends) algorithms for data analysis become more interesting. And this is the right moment where the magic happens.

This can only be achieved with many many data. Exactly how much data? According to GigaOm, Netflix observed about 30 million “plays” per day -including when you pause, rewind and forward-, also four million subscribers scores, three million searches and also they analice exact daytime at which the content is being consumed and when from which devices. But no matter how impressive it could sound, the amount of information is not as important as the criteria used to find patterns. In here is important to say that even small businesses, which already collect customer data, can predict trends with the correct algorithms.

However, like all magic, Big Data is tricky: the information has to be reliable. If data (the simplest form of information) is untrustworthy, the accumulation over time will be just rubbish. And it is not a trivial issue, if we are going to bet the future of our business on data analysis is better that it reflect reality. If the system and processes defined to help us in this task do not guarantee that the information collected and processed is reliable, it would be better to continue using the intuition. Remember the old saying, “be careful with what you measure because it is exactly what you ‘re gonna get”.

It is also important to say that Big Data is not for everyone, but its principles and techniques can be applied even to small businesses. However is vital to approach to an expert if we want to implement Big data strategies, since the amount of information that would be stored is exponential. The complexity and experience necessary to ensure that data are reliable increases as the information we want to process increases. Think of it this way: it may be very easy to make a gourmet dish for one person in an afternoon, but will it continue be just as easy for that person to prepare that dish for 100 people in a juts a few minutes? Probably not, and that is why an experienced Chef is a valuable asset in a good restaurant.

Now, returning to the famous series ‘House of Cards ‘ after the release last month of its third season. Would you like to see a fourth season just to know if Frank Underwood will win the presidency of the USA? You don’t have to say it, Netflix (with its Big Data) already knows the answer.