Analytics and Data Science industry has seen a sharp increase in terms of demand for insights and has opened jobs for highly-skilled professionals. Be it Fortune 500 name or startups — everyone is using analytics to garner insights from data. However, the industry is still riddled with a lot of challenges in terms of talent, reaching the right consumers and gathering data, among others.
This month Analytics India Magazine decided to find out what are the challenges faced by this sector. Be it organisational challenges, educational questions or operational quandaries — we dug deep to uncover what’s stopping India from becoming a superpower in this the analytics and data science sectors. We asked C-suite leaders, data scientists, students and mid-level management an array of questions, as comprehensive as possible, to get a complete understanding of the problems faced in this sector.
About The Study:
For this study, we asked respondents to tell us more about the challenges they faced — at an organisational level — in the analytics and data science sector. We took opinions from all those who practice data science — from professionals with less than two years of work experience to CXOs — to get a thorough idea of the issues they faced in this swiftly-developing sector.
Our survey was met with a lot of enthusiasm and we got great insights from it. Some of them were expected, and many of them were real eye-openers. So here are the key problems and challenges faced by the analytics and data science industry as a whole.
Respondents Profile
Work experience and education
City and sector
Key Highlights
- The biggest challenge faced by the data science and analytics sector is the lack of understanding of this sector by clients/stakeholders. Over 28% of the respondents said that this was their main challenge — and this lack of understanding has grown by 6% from 2018. This clearly indicates that there is a lack of education or communication between the analytics service providers or their teams and the clients or the upper-level management
- Almost 46% of the respondents said that the best way to increase the talent pool in analytics is by regular upskilling, which is either sponsored by the companies or employees themselves. In 2018, this number was at a slightly higher 48%
- When asked about the best way to create or increase awareness about analytics at a management level, almost 46% of the respondents said that the management needs to be educated about the benefits of analytics through roadshows, events, etc. This number has seen a significant increase from last year’s 32%
What Do You Think Is The Biggest Challenge Faced By Analytics And Data Science Sector In India?
This key question was aimed at the organisational cadre:
- Clearly, understanding of analytics and data science sector by clients/stakeholders/management was the chief problem faced by 28% of the respondents
- Interestingly enough, shortage in talent was also an important problem faced by 16% of the organisation of all magnitude. This came as a surprise to us, seeing how the number of institutes, MOOCs and online resources that offer education in analytics and data science is increasing steadily
The lack of standardisation of processes and techniques and inflated expectations from stakeholders was another key problem faced by 14% of all the companies and data science and analytics practitioners.
What Do You Think Is The Best Way To Increase The Talent Pool In Analytics?
- The best way to increase the talent pool in analytics, according to 46% of our respondents, is regular upskilling. It can either be sponsored by companies or employees themselves. In an era where so many resources are available at the fingertips of the users, the answer to this challenge seems obvious
- However, our respondents have also voiced their opinion on the quality of education in India. 32% of our readers have said that the best way to increase the talent pool in analytics is to improve the quality of education — especially the courses, the degree programmes and the teachers and instructors
Only 4% of the respondents thought that the talent pool was not an issue they faced in their respective companies or careers. This number has dropped significantly from 10% last year, which is a worrying trend.
How Can We Create/Increase Awareness About Analytics At A Management Level?
This question gave us a clear insight into the attitude of the corporates working or beginning to work with analytics.
- 46% of respondents thought that educating the management about the benefits of analytics through roadshows, events and on other platforms would help them increase awareness about this industry
- On the other hand, 18.5% of respondents felt that making C-level management aware of how analytics can help optimise ROI is one of the keys to smooth adoption and upgradation in the company
28% of respondents thought that rather than educating or teaching their peers, it would be easier if they could demonstrate the benefits of analytics with the help of effective use cases.
What Data Problems Do You Face The Most?
- As data is the fuel for this industry, it came as no surprise when 49% of our respondents said that available, but poorly stored or fragmented data was a key hurdle
- 27% of respondents said that one of the key challenges in the analytics and data science sector was the fact that data sources are many times too complex or siloed.
- Only 9% of the respondents said that the got no data to work with
How Can Standardisation Be Brought To The Analytics Ecosystem?
This was one of the questions where we got a very interesting (if fragmented) answer by our respondents.
- When asked about the level of standardisation in the sector, 35% of our respondents said that analytics and data science sector will work smoothly only if a central authority was created to craft policy and standards
- 35% of the respondents said that standardised tools for everyone’s use would help in streamlining the sector
- Only 18.5% of the respondents were quite pessimistic about the process — they admitted to the fact that they did not think that standardisation was possible at all in analytics and data science fields.
Do You Think Analytics Has Left Many Open-ended Definitions? How Can This Be Fixed?
- One of the qualms about this sector is that it has many open-ended definitions and jargons. Interestingly, 39.5% of our respondents said that if academics pitch in for creating more solid definitions, the problems would be solved
- In line with the question above, 25% of the respondents said that if a central body were to create definitions that everyone in the industry adhered to, the problem would be wiped out entirely
- Interestingly, 15% of our readers felt that open-ended definitions and jargons were not a problem at all in the analytics and data science industry
How Can We Deal With The Problem Of Inflated Expectations From Stakeholders?
Analytics and Data Science is such a sector where most managers think that the system is in place, it will work wonders. Many times it backfires and works against the sector itself. It, therefore, comes as no surprise that 42% of our respondents said that educating stakeholders about the working of the analytics department — workflow, timeframe, results, etc — would help in streamlining the process.
22% of the respondents said that showcasing workflow through successful use cases, even from other companies or projects, would help with the problem of inflated expectations from stakeholders.
20% of our readers said that regular demonstration of the development of products and services would help understand the stakeholders better.
What ROI Problem Do You Face In Your Analytics Firm?
As mentioned earlier, telling the time, date and quantity of the return on investment in any analytics or data science department can be tricky.
- That is why 37% of our respondents said that quantifying ROI was the basic problem they faced
- 30% of the respondents said that they found it difficult to quantify ROI to their customers or the management
- 16% of the respondents felt that the stakeholders or managers did not wait for the tuning period to see the fruits of the labour from the analytics and data science department
