In 2018, foundational models trained to process large-scale data and perform multiple tasks witnessed a growth spurt with Google’s BERT and OpenAI’s GPT-3 and CLIP.  Cut to 2023, the disruptive ChatGPT and LLMs are redefining job roles across sectors. The familiar “ChatGPT can save you hours” posts that talk about using the chatbot with the right prompts to achieve any task, including coding, is pushing the envelope on the use of foundational models in the best way possible. Thriving foundational models also mean that the role of data scientists will see a transformation. 

Source: Financial Times

Today, various AI-backed tools are being rolled out by companies to improve productivity and simplify work. Tasks that traditionally involve data scientists, too, are being increasingly supported via AI tools. Microsoft, for instance, has been expanding its collaboration in enterprise space and releasing workspace tools, such as Loop, to facilitate ease of work. Likewise, BloombergGPT– a first-of-its-kind AI model in the finance sector and an LLM trained with over 50 billion parameters – will be integrated for Bloomberg users. It not only facilitates easy access and interpretation of large datasets but also renders the data scientist, who would have ideally been working on them, redundant. 

The big question: So, what happens to data scientists?

Future of Data Science

In conversation with AIM,  Usha Rengaraju, chief of data science research at Exa Protocol, and world’s first woman triple Kaggle Grandmaster (platform for data scientists to compete globally), believes that though LLMs can give rise to interesting inventions in the future of data science, such as in code generation and data analysis, the current impact on data science jobs is minimal owing to the limitations and capabilities of such foundational models. “LLMs can automate small areas of work, but the majority of skills like critical thinking, problem solving, and programming will remain relevant even in the future,” said Usha. 

“It’s difficult to predict the distant future, with the pace at which technological changes are happening. In the next two years, there will be a reduction in the hallucinations, and incremental improvements to reliability, and interpretability of LLMs will occur,” she added. 

With new trends emerging, Usha does foresee a scenario in 2030 where data science jobs may become obsolete. However, a strong foundation in subjects like mathematics, programming, and critical capabilities like problem-solving will help them land other jobs, even if data science goes out of scope. 

Talking to AIM, CTO and chief scientist at CleverInsight, Bastin Robins shared a rather interesting view. He said that AI may not replace people but people knowing how to use AI to their benefit will surely replace the ones who don’t. He believes that data science and foundational models can go hand-in-hand. “Data science roles are getting more challenging with heterogeneous data, and with the help of LLM, more explainability is drawn from such data.” 

There will also be a shift in the way data science will shape up. Bastin considers that in the coming years, data science can evolve into a system where an increased “business sense” will be incorporated. “The ability to understand data and business ROI is a very useful outcome,” he said, and added that “Generative AI will open up a new arena of SaaS businesses.”

Skill Improvement

Data science will look different with the rise in low-code and no-code platforms. Being a supportive function, users with existing knowledge of data science can benefit from no-code platforms. As per a prediction, an added expertise in cybersecurity tools and techniques can come in handy for data scientists. With these tools, they will be able to help companies protect data. 

It’s true that tools like CopilotX make the task of a data scientist almost redundant, however,  the conundrum of whether foundational models will entirely replace data scientists cannot be ascertained at this time. Though the models can automate a number of tasks, data scientists will still be required to design and develop these models. In the meantime, the role of data scientists might evolve and transcend into other fields like quantum data science with the knowledge of quantum algorithms and computation. Now, that might be a possibility.