COVID-19 has led to shifting priorities, and companies are re-assessing strategies across their business as resources are constrained. This has led to companies coming to terms with the reality of business value with data science. 

The mass layoffs in the technology industry, including many data scientists, have many saying the talent bubble has finally popped. The pandemic gave a good reason for organisations to decrease the compensations for data scientists, which was earlier soaring high due to the rising demand. It is true that the number of candidates had been increasing exponentially in data science over the past five years.

It is expected that companies will now run for cost-cutting, business process optimisation and automation due to the pandemic, and this will ultimately decrease demand for data scientists in the coming future.

“The last couple of years, the economy had been doing quite well, and since every company wanted to join the AI race, they started pulling up these data science teams. But, they didn’t do the due diligence in hiring. They didn’t have a clear vision in mind as in how their AI strategy is actually going to help,” says Dipanjan Sarkar, Data Science Lead at Applied Materials.

Swapnil Jadhav, Principal Scientist (Applied Research) at DailyHunt also wrote on Medium saying, “It is hype most of the time. AI is a buzzing word, and many companies and startups mislead investors and consumers by saying they are using AI and deep learning.” 

Organisations around the world are now aligning their objectives on the core roles which are associated with the essential business functions. Many say data science, therefore, isn’t an essential function, and that is where a lot of data scientists have been some of the first ones to get laid off.

“Companies may think that they’re not getting any tangible value from large data science teams. This can trigger a move to cut down the staff, which may be non-essential,” Dipanjan added.

Sugarcoating Reality

“I can sugarcoat and tell you how crucial data science is, but for most businesses, 90% of the work and intelligence can be done with simple engineering tricks and manual decision trees. And data science is there to improve that 90% to 95–99%. But many times that may not be even significant. Data Science is essential only where it generates money or saves money or generates intellectual property, which is of future monetary value,” according to Swapnil.

Building scalable distributed systems for deploying ML models needs core computer science skills. For a good engineer, ML models are just one of many things to productionalize. Data science teams are just one small aspect of the complete business unit. Also, data scientists may face resistance as they want to modify or improve the prevalent business processes, which may be why many data science projects remain Proof of Concepts (PoCs) for a long time, say experts.

Where The Money Goes

“Upper-level management probably might have a lack of immediately realizing the value of data science to core business functions and revenue. The gap has always been there between the businesses and the technical subset of people. The key focus remains on solving high-level business problems and being able to showcase the value,” according to Dipanjan.  

If the value is not visible, businesses are probably going to cherry-pick the talent and layoff the rest, as we have seen. With the availability of so much talent in comparison to tangible business value, it is going to affect how businesses look at data science. 

“People need to understand that data science is nothing special than any other IT related field. Furthermore, it is a non-essential work. I firmly believe that data science people will get fired first than engineers in any company’s worst situation (like Covid-19 pandemic,” according to Swapnil Jadhav.

So, what to do about this situation? Many say that data scientists as a standalone skill set might struggle from here without having engineering skills, especially when most of the companies are moving towards the cloud, where data science models could be automated.

Apart from building models to predict something, there’s a wealth of opportunity in other areas. NLP innovation in automated document classification, for instance, is a tangible value add. But, most organisations still don’t know how to hire, develop, or execute data science well because of the widening bridge between business management and technical aspects of data science. 

Experts say data science does not scale like a core software engineering position during tough times, but there will always be a demand for experienced and proven professionals who can bring valuable business insights from data. Maybe the hype will die down. There will not be as many wannabe data scientists, but the job isn’t going away anytime soon.