Organizations Continue To Face Challenges With Big Data: Lets Deep Dive

Since early 2014, we have seen a steep increase in the number of companies adopting big data and analytics. Data is proliferating in every part of businesses and organisations, and they are more focused on reaping the benefits of big data. While big data has played a major role in overall growth of organisations, it is not the only factor that can drive growth. It comes with its own side effects. Leveraging everything on big data will serve only a limited purpose if an organisation as a whole does not integrate with its goals completely. The organisational mindset should look at a broader horizon apart from relying solely on big data insights. It should also be accepted by all employees at various levels.

Challenges From A Management Perspective

According to Randy Bean, CEO of NewVantage Partners, the challenges in implementing big data lies in the approach followed by the management of a company. He says, “The vast majority of the challenges companies struggle as they operationalise big data related to people, not technology — issues like organisational alignment, business process and adoption, and change management.” He emphasises on bringing a rigid cultural change in any organisation along with providing opportunities in innovation with this domain. This is where Indian tech companies have to concentrate. Adapting to a new technology will happen gradually but how well it is infused in the corporate culture is a key factor in its growth. Another challenge is the technology disparity present in companies. Although, these days professionals in India grasp latest tech happenings, the prospect of replacing conventional tech is still a matter to brood. Big data alone is not just sufficient to tackle this issue. Bridging the skills and technology gap is a daunting task.

The Human Factor

Big data and related technologies won’t help much if human analysis is totally left in the dark. Intelligent machines and automation may provide quicker and accurate results but may not reveal details in hindsight. For example, a family doctor’s diagnosis of patients may actually give out more details to assess health on a long run. This cannot be possible if machines perform diagnosis because they don’t have knowledge of previous data. Big data will be more insightful if human intuition along with social sciences are considered. This is due to the fact there are other factors in account such as human behavior and diverse cultural spread.

Privacy Concerns

The contrasting side of big data in a company is the possibility of intrusion from other entities to data which are present in large amounts and are passed around the company’s network. Even though data protection offers a certain level of security to counter these bad influences, there may be chances of mismanagement of personal and customer data inside or outside a company. In a paper titled Big Data Challenges by Alexandru Adrian of Romanian-American University, the author highlights the core problem associated with data privacy. He explains, “For companies the privacy issue is more related to the sensitive data that they work with. Whether it is financial data, clients list, perspective projects, all represents valuable data that may or may not be disclosed.” This shows that critical data in large volumes can be detrimental to the organisation, if it falls into the wrong hands. Now, for an Indian ecosystem which spans a massive population of 1.2 billion, the data generated will be humongous. In addition, social media presence is very strong which leads to more personal data in the online space. Big data may deal with all these information efficiently, but privacy will be an illusion.

Aberration To Technological Innovation

Big data when viewed from an innovation perspective might not necessarily be always useful for companies. The insights gained may not predict a causal relationship between big data and innovation. In a review paper by Sen Chai and Willy Shih, a wonderful supposition is discussed: “Although we acknowledge that data mining has enabled tremendous advances in business intelligence and in the understanding of consumer behavior — think of how Amazon figures out what you might want to buy or how content recommendation engines such as those used by Netflix Inc. work — it’s important for executives who oversee technical disciplines to be thoughtful about how they apply this approach to their areas.” This is where applying only big data does not have a strong impact. Research and innovation go hand in hand, but without depending solely on technology.

Conclusion

As the article discussed, several pitfalls of big data, where it is made to be the main focus of companies or organisations can be avoided. These problems can be alleviated if the management in an organisation concentrate on all areas equally, without pushing only big data and analytics solutions. This means, all technological and managerial aspects should be aligned with the organisation’s interest.