After Brexit stumbling predictions, do we need better big data driven forecasting?

The data driven forecasting such as opinion polling and prediction market came in for a major shock when later in the month of June, Great Britain opted to vote against staying in the EU, much contradictory to the overwhelming cry till date that UK would ‘Remain’. Amidst the mix reaction of surprise, fury and relief, the question that definitely becomes inevitable is ‘where did the market prediction go wrong?’ Do we need a better data driven forecasting? Before we set our foot on the debate let’s find out what went wrong and why has data driven Brexit come as a wakeup call for analysts? Forecasting results when it comes to large amount of data such as in elections have been fairly common. And given a scenario in any country across the globe, the know-hows keep coming up with the current support rate for a candidate and their chances to win. The call for Brexit, in the same way fetched a charged up campaign and public polling to know what the citizens favoured. They were called in for voting, with a few opting for telephone and others via online voting. The votes were divided into “Remain” and “Leave”. Given the scenario we would have hope either a clear winner or a tie. But the results were all the opposite.   The overall predictions itself came out faulty which could be witnessed through various poll trackers. The poll tracker for Brexit powered by Bloomberg showed that Remain was maintaining a lead with 2% while Brexit had a bare minimum chances of 25% win. The same kind of results were shown by YouGov where the ratio of Remain vs. Leave was 52:48. With 90% of the market claiming to be in favour of ‘Remain’ and results coming out in favour of Brexit, there have been questions and concerns raised over a need for better data driven forecasting. With a serious lag curbing up between predictions and the results, it has raised critical questions about the reliability of these forecasts in the coming future. It definitely indicates that the long practised method of collecting thoughts and opinions from people strolling up the street or simply by calling them up is a passé and we need something more extra-ordinary than that. Apart from polling, social media platforms and web search which were considered as a basis to form out predictions went haywire. Given the ambiguity of mere discussions rather than standing out as strong support, made it difficult to declare a clear emerging winner. It again leaves us to the point of having a strongly knit data forecasting tool. When it comes to real world forecasting, it could be quite difficult to get the right results. Though there have been a few online polls, social media and Google search volume timeline closer to the actual results which were in favour of Brexit, improvements in “big data” measurements to come out with better and robust predictions is the need of the hour.