With the advent of powerful computing systems and big data, technology has really enabled businesses to leverage the power of frameworks like Hadoop and Spark to get insights out of their vast data treasure trove using analytics and machine learning. But the question still remains, “Is analytics still a big hype or have we been able to break the hype surrounding analytics and utilize it for something tangible and useful?”
I’m sure you will see hundreds of online courses and training programs related to Big Data, Hadoop and Data Science. Companies are also investing dedicated infrastructure and teams just to get breakthroughs in analytics which might help them in growing their business, reducing costs and increasing efficiency. However, there is a major gap between the current state of analytics and the end results which are communicated to the customers and stakeholders. Some of the major challenges faced in this domain include the following.
- Developing proper use-cases and problems which can be solved by analytics.
- Showcasing the actual business value which analytics based solutions provide.
- Lack of proper skilled workforce.
A combination of these challenges leads to failure more than often when trying to tackle tough problems and people end up having a negative opinion about analytics. Hence they conclude it to be over-hyped. We will look at each of the challenges briefly and how we can tackle them.
Gartner depicts four major types of analytics capabilities which start with “descriptive analytics” indicating “What has already happened?” and then moves on to “diagnostic analytics” which tries to answer the question “Why did it happen?”. The next stage of analytics is “predictive analytics” which tries to say “What will happen in the future?” and finally we have “prescriptive analytics” which kind of prescribes or tells us “What we should do?” Based on the current state of the industry, most companies focus on descriptive analytics and diagnostic analytics based on data summarization, pattern mining and trend analysis using historical data sources.
The area where we still need to make a breakthrough is in predictive and prescriptive analytical capabilities. We are making great progress considering where we were a decade back, but still there is a long way to go. The challenge here is often customers are not happy just seeing flashy reports and dashboards showing what they already know. What they want are more actionable insights which would enable them to make data-driven decisions like predicting when a stock value would plummet or rise.
Another example would be predicting what items to stock in a retail store in different seasons based on purchase patterns. The scope of this is really endless but businesses need to formulate the problem they are trying to solve with the help of analytics clearly before trying to use any specific analytical tool or framework. Domain and business knowledge is as important as technical capabilities to develop complex analytical systems with real business value.
If you are working in the field of analytics or have been a part of any process involving analytics, I’m sure you will be aware that the effort involved in communicating results obtained from analytical capabilities and showcasing their business value is not a piece of cake. Building ad-hoc reports and dashboards are easy and they give you an overall analysis into your data but remember that reporting is not analytics! Measuring business value and benefits obtained from analytical capabilities is not very straightforward and extremely difficult at times which makes people skeptical about analytics altogether.
The best way to deal with this problem would be to have clear expectations set on either side about the end results and to keep in mind that analytics is an iterative process and not a magical black box which will magically produce results out of thin air. Often insights obtained from analytics might be ambiguous or the stakeholders might not be so sure of the exact results they want. Working together in a proper collaborative environment will enable everyone to gradually evolve their process based on continuous feedback and mutually benefit from analytics and see the value. It is not always about the algorithms or tools but also about what the customer wants and their valuable feedback.
Finally, the lack of skilled people in the fields of analytics and data science is a well-known fact. You must have already seen or heard news articles portraying the dearth of capable data scientists, analysts and engineers. This has its benefits and shortcomings. The benefits include great pay packages, benefits and perks if you can build your skill-set in this domain. The shortcomings which have risen recently include the huge number of people who claim to be experts in data science and analytics.
However, if you observe them closely, you might find out that most of them might be taking a couple of courses and trainings in this domain and start selling themselves as expert data scientists and analysts. This results in companies hiring them hastily to satisfy their requirements and ultimately leads to increase in attrition rates when they fail to add value to the organization. One must remember that investing in this domain and building a good team for analytics needs a lot of patience, finding the right people for the job and making sure they have ample knowledge about math, statistics, algebra and computer science. Willingness to pick up new technology and apply it is an added bonus since technology stacks keep evolving over time. The most important skill however is being able to communicate the end results effectively which showcases the real value of analytics.
If we keep these things in mind while we go about our daily jobs trying to solve tough analytics problems, this would surely help us in breaking the hype surrounding analytics.
The views and opinions expressed in the article are solely those of the author and do not necessarily reflect the views held by Intel or its employees.
