Data Scientists are having a moment with their jobs being one of the most sought after and highly paid across industries. And the demand for roles that require data science and analytics competencies continues to soar. According to an AIM report, there were almost one lakh data science-related positions available in India in 2019 alone.

This demand is the result of the proliferation of data and the potential it holds in the world of business. This has created a pool of professionals eager to pick up data science skills for a possibly lucrative career. But there is a steep cost involved: high stress.

Whether you are lured by the prospect of financial benefits or the quest to solve real-world problems with data, it is important to prepare for a future that is likely to be riddled with stress. Six data scientists weigh-in:

Problem-Solving & Troubleshooting

By having a broad range of roles that involve working on large data sets, you are setting yourself up for periodic disruptions that happen on a consistent basis. And oftentimes, it takes a long time before you find a discernible source to the issue or get it resolved and this may happen in a continuous loop.

According to data scientist Usha Rengaraju, comparable problems could be a regular fixture in banking. “The models which perform very well locally may not perform the same way during pilot or when scaling up. What is more, in such situations, there is always a lot of pressure to quickly come up with the fixes,” she says.

Likewise, you might have an algorithm that does not quite work the way you see it, or you might be sitting with a conclusion that is not consistent across the board like it should be. 

Data Collection

Before you can dive into the most interesting aspects of the job, you need to collect data, and this step can be particularly stressful. “To even access the data, I have to coordinate with multiple product teams and get the relevant approvals,” says Surya Prakash Manpur, who works as a Data Scientist at RealPage.

Concurs Rengaraju: “Working with huge enterprises in gathering data from several departments across the client company can be rather stressful.”

However, care needs to be taken at this stage because your analysis is only going to be as accurate as the quality of the data you gather. If you pour yourself into perfecting a report whose data itself is unreliable, your report will not be effective.

This could also become a little complicated since you have very little control over the raw data of the company and thus, testing its veracity becomes an issue.

“The next step is to work with developers to extract the quality dataset and understand the business implication of the dataset, which can be stressful processes too” adds Manpur.

System-Wide Collaboration

There has been unanimous agreement from most data scientists who participated in this story regarding the stress that collaborating with stakeholders can create.

“Data Scientists are expected to understand many diverse perspectives from many stakeholders and to communicate the impact of the analysis to them can be challenging,” says Navin Manaswi, Data Science Specialist at WoWExp. “Balancing this act could be a stressful affair, especially for people who are more introverted by nature,” he adds.

According to Rengaraju, negotiation, communication and time management are as important as programming and analytical skills for a data scientist. Moreover, the high expectations placed on them by clients can create a lot of stress for them.

“Understanding business needs and communicating a metric or a KPI to them can be quite stressful,” says Manpur. “The ability to say no is another issue. There can be situations wherein the expected story from the business partner is contradicting with the information extracted from the analysis performed on the dataset,” he adds.

Paradoxically, despite the high level of collaboration the work of a data scientist entails, the truth is that you will be spending a lot of time working alone, especially in large organisations.

“To offset this, our business and data science teams work in tandem in an agile fashion,” says Head of Analytics at Standard Chartered Global Business Services, Vidhya Veeraraghavan. “This also helps the data science team collaborate and support each other to cope with the day’s stresses,” she adds.

Lofty Expectations Of Data Scientists

High-performance expectations because of the profound nature of some of the problems that you solve makes this role challenging and stressful. Data scientists typically work on data for an entire company, which means scouring through thousands of transactions all at once. 

Your analysis has the potential to influence key decisions taken by a business, and that by itself can be a lot to process. But Sray Agarwal, AI and ML specialist at Publicis Sapient, has learned to accept the nature of the job.

“Data science is more exciting and adventurous than stressful,” he says. “It is only stressful when you are working to pay bills, and not to solve real-world problems,” he adds.

Increased Competition Among Aspiring Data Scientists

Data scientists need to put in long hours frequently, especially when working to solve a big problem. But the field has become very competitive in the last few years, and the sheer level of competition can be stressful. 

This necessitates staying on top of the competition and upskilling further. 

“I find picking up domain knowledge very stressful especially in niche areas like genomics, geology and semiconductor, etc.,” says Rengaraju. Adds Tribhuvan Tewari, Data Scientist at Stratbeans, “Keeping up with new technologies and corresponding market requirements is also a constant stress.”