The year 2019 was great in terms of analytics adoption as the domestic analytics industry witnessed a significant growth this year. There has been a visible shift towards intelligent automation, AI and machine learning that is changing the face of all major sectors — right from new policies by the Indian Government, to micro-adoption by startups and SMEs.
While customer acquisition, investment in enterprise-grade data infrastructure, personalised products were some of the trends from 2018, this year our industry interaction suggested that democratisation of AI, AI push into hardware and software are much talked about.
Our annual data science and AI trends report for the upcoming year 2020 aims at exploring the key strategic shifts that enterprises are most likely to make in the coming year to stay relevant and intelligent. This year we collaborated with AnalytixLabs, a leading Applied AI & Data Science training institute to bring out the key trends.
Some of the key areas that have witnessed remarkable developments are deep learning, RPA and neural networks which, in turn, is affecting all the major industries such as marketing, sales, banking and finance, and others. Some of the most popular trends, according to our respondents, were the rise in robotic process automation or hyper-automation that has begun to use machine learning tools to work effectively.
The rise in explainable AI is another exciting development that the industry is likely to see in the popularity charts in the coming year, along with the importance of saving data lakes and the rise of hyperscale data centres, among others. Some of the other trends like advancements in conversational AI and augmented analytics, are here to stay.
Semantic AI, enterprise knowledge graphs, hybrid clouds, self-service analytics, real-time analytics and multilingual text processing were some of the other popular trends mentioned by the respondents which are likely to be on the rise in 2020.
01. The Rise Of Hyper-Automation
“2019 has seen rising adoption of Robotic Process Automation (RPA) across various industries. Intelligence infused in automation through data science and analytics is leading to an era of hyper-automation that enables optimization and modernisation. It is cost-effective too but may have risks. 2020 will see enterprises evaluating risks and control mechanisms associated with hyper-automation.”
Anjani Kommisetti, Country Manager, India & SAARC, Raritan & Servertech
“Hyper-automation uses a combination of various ML, automation tools and packaged software to work simultaneously and in perfect sync. These include RPA, intelligent business management software and AI, to take the automation of human roles and organizational processes to the next level. Hyper automation requires a mix of devices to support this process to recreate exactly where the human employee is involved with a project, after which it can carry out the decision-making process independently.”
Suhale Kapoor, Executive VP & Co-founder, Absolutdata
“Automation is going to increase in multitudes. Over 30% of data-based tasks will become automated. This will result in higher rates of productivity and analysts will have broader access to data. Automation will additionally assist decision makers to take better decisions for their customers with the help of correct analytics.”
Vishal Shah, Head of Data Sciences, Digit Insurance
02. Humanized Artificial Intelligence Products
“We will see AI getting deeper into Homes and lifestyle and Human Interaction would begin to increase in the coming year. This means a reliable AI Engine. We have already seen some voice based technology making a comfortable place in homes. Now, with Jio Fiber coming home and Jio disrupting telecom sector it will be interesting to see how the data can be leveraged to improve/ develop devices that are more human than products.”
Tanuja Pradhan, Head- Special Projects, Consumer Insights & New Commerce Analytics, Jio
“Rise of AI has been sensationalised in the media as a battle between man and machine and there are numerous numbers flying around on impact on job loss for millions of workers globally. However, only less than 10% of roles will really get automated in the near future. Most of the impact is rather on non-value-added tasks which will free-up time for humans to invest in more meaningful activities. We are seeing more and more companies releasing this now and investing in reskilling workforce to co-exist with and take advantage of technology.”
Abhinav Singhal, Director, tk Innovations, Thyssenkrupp
“The effects of data analysis on vast amounts of data have now reached a tipping point, bringing us landmark achievements. We all know Shazam, the famous musical service where you can record sound and get info about the identified song. More recently, this has been expanded to more use cases, such as clothes where you shop simply by analyzing a photo, and identifying plants or animals. In 2020, we’ll see more use-cases for “shazaming” data in the enterprise, e.g. pointing to a data-source and getting telemetry such as where it comes from, who is using it, what the data quality is, and how much of the data has changed today. Algorithms will help analytic systems fingerprint data, find anomalies and insights, and suggest new data that should be analyzed with it. This will make data and analytics leaner and enable us to consume the right data at the right time.”
Dan Sommer, Market Intelligence Lead, Qlik
03. Advancements in Natural Language Processing & Conversational AI
“Data Scientists form backbone of organisation’s success and employers have set the bar high while hiring these unicorns. With voice search and voice assistants becoming the next paradigm shift in AI, organisations are now possessing a massive amount of audio data, which means those with NLP skills have an edge over others. While this has always been a part of data science, it has gained more steam than ever due to the advancements in voice searches and text analysis for finding relevant information from documents.”
Sourabh Tiwari, CIO, Meril Group of Companies
“NLP is becoming a necessary element for companies looking to improve their data analytics capabilities by enhancing visualized dashboards and reports within their BI systems. In several cases, it is facilitating interactions via Q&A/chat mediums to get real-time answers and useful visualizations in response to data-specific questions. It is predicted that natural-language generation and artificial intelligence will be standard features of 90% of advanced business intelligence platforms including those which are backed by cloud platforms. Its increasing use across the market indicates that, by bringing in improved efficiency and insights, NLP will be instrumental in optimizing data exploration in the years to come.”
Suhale Kapoor, Executive VP & Co-founder, Absolutdata
“The advent of transformers for solving sequence-to-sequence tasks has revamped natural language processing and understanding use-cases, dramatically. For instance, BERT framework built using transformers is widely being tapped onto, for development of natural language applications like Bolo, demonstrating the applicability of AI for education. AI in education is here to stay.”
Deepika Sandeep, Practice Head, AI & ML, Bharat Light & Power
“2019 was undeniably the year of Personal assistants. Though Google assistant and Siri have seen many winters since their launch but 2019 saw Amazon Alexa and Google home making way into our personal space and in some cases have already become an integral part of some households. Ongoing research in the area of computational linguistics will definitely changes the way we communicate with machines in the coming years.”
Ritesh Mohan Srivastava, Advanced Analytics Leader, Novartis
04. Explainable Artificial Intelligence (XAI)
“Decisions and predictions made by artificial intelligence are becoming complex and critical especially in areas of fraud detection, preventive medical science and national security. Trusting a neural network has become increasingly difficult owing to the complexity of work. Data scientists train and test a model for accuracy and positive predictive values. However, they hesitate to use it in areas of fraud detection, security and medicine. Models inherently lack transparency and explanation on what is made or why something can go wrong. Artificial intelligence can no longer be a black box and data scientists need to understand the impact, application and decision the algorithm is making. XAI will be an exciting new trend in 2020. Its model agnostic nature allows it to be applied to answer some critical questions in data science.”
Pramod Singh, Chief Analytics Officer & VP, Envestnet|Yodlee
“Another area that is taking shape in the last few years is Explainable AI. While the data science community is divided on how much explainability should be built into ML models, top level decision makers are extremely keen to get as much of an insight as possible into the so-called AI mind. As the business need for explainability increases people will build methods to peep into the AI models to get a better sense of their decision making abilities. Companies will also consider surfacing some such explanations to their users in an effort to build more user confidence and trust in the company’s models. Look out for this area in the next 5 years.”
Bhavik Gandhi, Sr. Director, Data Science & Analytics, Shaadi.com
05. Augmented Analytics & Artificial Intelligence
“Augmented Analytics is the merger of statistical and linguistic technology. It is connected to the ability to work with Big Data and transform them into smaller usable subsets that are more informative. It makes use of Machine Learning and Natural Language Processing algorithms to extract insights. Data Scientists spend 80% of their time in Data Collection and Data Preparation. The final goal of augmented analytics is to completely replace this standard process with AI, taking care of the entire analysis process from data collection to business recommendations to decision makers.”
Kavita D. Chiplunkar, Head, Data Science, Infinite-Sum Modelling Inc.
“Augmented Assistance to exploit human-algorithm synergy will be a big trend in the coming years. While the decision support systems have been around for a long time, we believe that advancements in Conversation systems will propel the digital workers in a totally different realm. We witnessed early progress in ChatOps in 2019 but 2020 should see development of similar technology for diverse personas like Database Admin, Data Steward and Governance Officers.”
Sameep Mehta, Senior Manager, Data & AI Research, IBM Research India
“With the proliferation of AI-based solutions comes the need to show how they deliver value. This is giving rise to the evolution of explainable “white box” algorithms and the development of frameworks that allow for the encoding of domain expertise and a strong emphasis on data storytelling.”
Zabi Ulla S, Sr. Director Advanced Analytics, Course5 Intelligence
“The increasing amount of big data that enterprises have to deal with today – from collection to analysis to interpretation – makes it nearly impossible to cover every conceivable permutation and combination manually. Augmented analytics is stepping in to ensure crucial insights aren’t missed, while also unearthing hidden patterns and removing human bias. Its widespread implementation will allow valuable data to be more widely accessible not just for data and analytics experts, but for key decision-makers across business functions.”
Suhale Kapoor, Executive Vice President & Co-founder, Absolutdata
06. Innovations In Data Storage Technologies
“Data explosion increases every year and 2019 was no different. But to manage this ever-increasing data SDS saw an exponential rise, not just to attain agility but also make data more secure, that again has been a boon to SMEs. 2020 will see SME/ SMB sectors rising in the wave of intelligent transformation to make intelligent choices and reducing the total cost of ownership.”
Vivek Sharma, MD India, Lenovo DCG
“Hyperscale data centre construction has dominated the data centre industry in 2019 and provided enterprises with an opportunity to adopt Data Centre Infrastructure Management (DCIM) solutions, befitting their modern business and environment. With the help of DCIM solutions, 2020 will see enterprises designing smart data centres enabling operators to integrate proactive sustainability and efficiency measures.”
Anjani Kommisetti, Country Manager, India & SAARC, Raritan & Servertech
“There is a rise of new innovations in data collection and storage technologies that will directly impact how we do store, process and do data science. These graphical database systems will greatly expedite data science model building, scale analytics at rapid speed and provides greater flexibility, allowing users to insert new data into a graph without changing the structure of the overall functionality of a graph.”
Zabi Ulla S, Sr. Director, Advanced Analytics, Course5 Intelligence
“Data Science and Data Engineering are working more closely than ever. And T-shaped data scientists are very popular! With an increasing need for data scientists to deploy their algorithms and models, they need to work closely with engineering teams to ensure the right computation power, storage, RAM, streaming abilities etc are made available. A lot of organisations have created multi-disciplinary teams to achieve this objective.”
Abhishek Kothari, Co founder, Flexi Loans
07. Data Privacy Getting Mainstream
“Consumers have finally matured to the need for robust data privacy as well as data protection in the products they use, and app developers cannot ignore that expectation anymore. In 2020, we can expect much more investment towards this facet of the business as well as find entirely new companies coming up to cater to this requirement alone.”
Shantanu Bhattacharya, Data Scientist, Locus
“Data security will be the biggest challenging trend. Most AI-driven businesses are in nascent stages and have grown too fast. Businesses will have to relook at data security and build safer and robust infrastructure. Data and Analytics industry will face this biggest challenge in 2020 due to lack of orientation of data security in India. Focus has been on growth and 2020 will get the focus on sustaining this growth by securing data and building sustainability.”
Dr Mohit Batra, Founder & CEO, Marketmojo.com
“As governments start to dive deeper into data & technology, more & more sensitive information will be unearthed. More importantly, we see an increasing trend in collaboration between governments and private sector, for design & delivery of public goods. To make the most of this phase of innovation, it will be critical for governments at all levels to not only articulate how it sees the contours of data sharing and usage (in India, we currently have a draft Personal Data Protection Bill) but also how these nitty-gritties are embedded in the day to day working of the governments and decision makers.”
Poornima Dore, Head, Data-driven Governance, Tata Trusts
08. Increasing Awareness On Ethical Use Of Artificial Intelligence
“The analytics community is starting to awaken to the profound ways our algorithms will impact society, and are now attempting to develop guidelines on ethics for our increasingly automated world. The EU has developed principles for ethical AI, as has the IEEE, Google, Microsoft, and other countries and corporations including OECD. We don’t have the perfect answers yet for concerns around privacy, biases or its criminal misuse, but it’s good to see at least an attempt in the right direction.”
Abhinav Singhal, Director, tk Innovations, Thyssenkrupp
“Artificial Intelligence comes with great challenges, such as AI bias, accelerated hacking, and AI terrorism. The success of using AI for good depends upon trust, and that trust can only be built over time with the utmost adherence to ethical principles and practices. As we plough ahead into the 2020s, the only way we can realistically see AI and automation take the world of business by storm is if it is smartly regulated. This begins with incentivising further advancements and innovation to the tech, which means regulating applications rather than the tech itself. Whilst there is a great deal of unwarranted fear around AI and the potential consequences it may have, we should be optimistic about a future where AI is ethical and useful.”
Asheesh Mehra, Co-founder and Group CEO of AntWorks
09. Quantum Computing & Data Science
“Quantum computers perform calculations based on the probability of the state of an object before it is measured- rather than just microseconds- which means that they have the potential to process more data exponentially compared to conventional computers. In a quantum system, the qubits or quantum bits store much more data and can run complex computations within seconds. Quantum computing in data science can allow companies to test and refine enormous data for various business use cases. Quantum computers can quickly detect, analyze, integrate and diagnose patterns from large scattered datasets.”
Vivek Zakarde, Segment Head- Technology (Head BI & DWH), Reliance General Insurance Company Ltd.
“While still in the very nascent stages quantum computing holds a promise that no one can ignore. The ability to do 10000 years of computations in 200 seconds coupled with the exabytes of data that we generate daily can allow data scientists to train massive super complex models that can accomplish complex tasks with human or superhuman levels of accuracy. 8-10 years down the line we would be seeing models being trained on quantum computers and for that we need AI that works on quantum computers and this area will grow a lot in the coming years.”
Bhavik Gandhi, Sr Director, Data Science & Analytics, Shaadi.com
10. Saving The Data Lakes
“While Data Lakes may have solved the problem of data centralization, they in turn become an unmanaged dump yard of data. As the veracity of data becomes a suspect, analytics development has slowed down. Pseudo-anonymization to check the quality of incoming data, strong governance and lineage processes to ensure integrity and a marketplace approach to consumption would emerge as the next frontier for enterprises in their journey of being data-driven. Further, smart data discovery will enable uncovering of patterns and trends to maximize organizations’ ROI by breaking information silos.”
Saurav Chakravorty, Principal Data Scientist, Brillio
“Data Lake will become more mainstream as the technology starts maturing and getting consolidated. External data will become as one of the main data sources and Data Lake will be the de-facto choice in forming a base for a single customer view. It will help in improving the customer journey thereby increasing efficiency.”
Vishal Shah, Head of Data Science, Digit Insurance
