Why Online Collaboration Is So Difficult For Data Scientists

Data science skills are one of the critical sources that can provide businesses with a competitive advantage in the market. It is not only being used as a key tool to generate informative insights for making business decisions but also to optimise their operations using those insights. And therefore, effective collaboration among data science teams is the key for businesses to solve complex business problems. In fact, in an article, co-founder and CEO of data science start-up Dataiku, Florian Douetteau stated that one of the most important factors of successful data science is teamwork and collaboration. He said, “No data scientist is an island, nor could they be.” 

Usually, a data science team would involve data scientists, analysts, and data engineers, and it’s their collaborative efforts that work towards extracting insights from the massive amount of data available. For a data science team to successfully work, they must collaborate while analysing the data, building the model, and sharing insights. As a matter of fact, businesses must encourage their data scientists to collaborate well with engineers to come out with better workflows, better products, and better results.

It is also critical for businesses to have enough collaboration among data scientists so that the shared context reduces the dependency on a handful of employees, and data can be democratised for employees of other departments. Experts believe data scientists sharing their ideas as well as their insights to other team members also acts as a robust feedback process.

Having said that, with this pandemic in place, businesses have mandated remote working for their employees, including their data professionals, and that has given rise to online collaboration with the help of the cloud. Although online collaboration can bring in several advantages for employees, it definitely creates few challenges for data scientists who are working on solving complex business problems with critical data. In this article, we are going to share a few reasons as to why online collaboration isn’t for data scientists.

Diminished productivity and reduced efficiency among data scientists 

One crucial aspect that is lost with online collaboration is the personal contact and in-person communication, which can sometimes lead to confusion, especially for those who are handling a massive amount of data on a daily basis. With critical data in hand, it is usually required to personally communicate with other departments, which is lost with remote working and online collaboration. Online collaboration is a ripple effect of the mandated remote working, which forces employees to be responsible for managing their tasks, turning work on deadline as well as keeping accountability of the projects, which is a lot to handle for data scientists and could hamper their productivity.

Majority of data scientists work with a massive amount of training data and complex data engineering processes, and collaborating with other teams is necessary for them to scale their projects. Although online collaboration has been created so that employees can be connected 24/7, many a time, these isolated data scientists go for days without communicating or collaborating with other employees and teams, which again can have a negative impact on their mental health. 

In fact, in a blog post on Quora, Ben Hamner, the co-founder and CTO of Kaggle has stated, “Most data scientists, and data science teams, have terrible practices for collaboration. The current default workflows have grown organically and are bad. You need to be really intentional to do a lot better, and this yields large gains in productivity and reducing painful frictions.”

Virtual collaboration also brings in a lack of clarity and direction in data science workflows. Also, with continuously working with data, data scientists usually require human interaction to keep up their morale high amid this isolation. In-person collaboration not only brings cohesiveness but also allows data professionals to keep up the loop of communication with other teams. 

Also Read: How Laying Off Data Scientists Amid COVID-19 Pandemic Can Hamper Business’s Future Success

Inability to ask the right questions

Asking the right questions is critical for data scientists to define problem statements and create solutions to solve them. Majority of data science work goes into solving business problems and to get comprehensive details about the problem at every step, it is vital that data scientists ask the right questions. It not only helps data scientists to define their goals and objectives of their working but also helps them in curating better data, building better analysis and making an accurate prediction. However, with virtual collaboration, data scientists face the inability to ask the right question at the right time. And with ineffective communication via virtual collaboration, data scientists could work on wrong data due to lack of understanding and can result in making false assumptions.

In fact, a Singapore-based data scientist, Admond Lee Kin Lim, wrote in his Medium blog post that, “I believe asking right questions and defining problem statements are some of the challenges faced by many beginners in data science.” And for this, constant communication is imperative among data scientists and business managers. But with online collaboration, data scientists lose their storytelling skill which is critical to communicate statistics and data to stakeholders. 

Also Read: What To Expect When You Start A Job As A Junior Data Scientist?

Impedes their learning process

Along with continuous feedback, the in-person collaboration also brings in constant learning among data scientists in the team. However, with mandated lockdown, data scientists rely on online collaboration for their daily communication, which is not only unreliable but also creates isolation. In-person collaboration acts as an active engagement amid colleagues of the same profession, but isolated learning via online collaboration can create disengagement and hamper the overall cause of collaboration. Alongside, no collaboration tools can compete with the real-time nature of in-person collaboration, and it also comes with its own technical issues like network glitches and reduced bandwidth, to name a few.

In fact, Pascal Bugnion, Data Engineer at Faculty — an AI company — has stated in his recent blog post that, “Data science teams are no longer composed of a lone unicorn who works furiously on his laptop to eventually present a few plots or a web application. We now expect that data science is done by cross-functional teams that bring together subject-matter experts, modellers, data visualisation experts, machine learning engineers, product managers and designers.”

Also Read: What Are The Markers Of A Genuine Data Scientist?

Distrust and suspicion 

Lastly, as data scientists and other data professionals work with critical data and sensitive company information, they have always been scrutinised over the privacy and security issues that they can bring in for companies. With online collaboration, the majority of the data science functions work on cloud-based software which always creates a security concern for business leaders. Virtual collaboration can not only create distrust among the team members and their employers but can also create a lag in immediate response as work for home gives them the ability to work on their own time. 

In recent news, a video conferencing tool Zoom has undergone criticism on their privacy issues and security vulnerabilities. Such security faults in cloud collaboration tools will, in turn, create a conservative mindset among business owners to keep their sensitive data in the cloud. And that’s why now, the majority of these cloud providers are enhancing their security credentials for their customers. 

In fact, in a recent report, it has been noted that this shift to remote working has brought in new security risks for companies and has left nearly 50% of those employees worried about impending cyber threats. To which, the senior analyst at the Global Cyber Center of NYC, William Altman has recently stated to the media that, how all organisations are currently witnessing a surge in email-based threats and endpoint-security gaps due to this sudden transition to fully remote working. And that’s why, “it’s now more important than ever to consider both the security practitioner as well as ethical-hacker perspectives in order to stay secure, that’s what this is all about,” said Altman.

Alongside, online collaboration diminishes the ability for data scientists to constantly keep their team members aware of their progress of work, which again creates questions and doubts. To build this belief, business leaders need to constantly update their feedback for the workflow. However, with online collaboration, it creates a gap between the leaders and their employees.