Anvita Bajpai, master principal cloud architect at Innovation Centre, Centre of Excellence, Cloud Engineering for the Asia Pacific (Apac) and Japan at Oracle, spoke about using analytics, AI/ML, and cloud technologies for extracting hidden insights in large volumes of data at the Data Engineering Summit 2022. The insights could be used for data-driven decisions by deploying end-to-end business solutions. Her talk also covered the deployment architecture of AI/ML systems and a 360-degree view of the industry across market segments.
Access Data Engineering Summit here.
It all starts with data
“In many scenarios, we might have to consider regulation or respect the data privacy. In certain scenarios, we need to understand the flow of data,” said Bajpai.
She also drew an analogy to the industrial revolution. “The transformation from Industry 1.0 (1784) to 2.0 (1870) took eighty-six years, while the transformation from industry 2.0 to 3.0 ( 1969) took close to 100 years. And today, we are on the cusp of industry 5.0. The pace of change is fast,” she said.
Data is key for the digital economy. “Today we have internet, cloud technologies, 5G, faster compute and electricity reaching remote villages. So all these infrastructure-related changes enable us to think of solutions and create them,” she added.
Referring to the IDC report, Bajpai said it is a very good competitive advantage. “In short, we can say real-world applications using innovative technologies require real-time data-driven decisions and actionable insights on a large volume of data. Now, where are the opportunities? And my answer is that opportunities are everywhere,” she said.
New digital markets like YouTube and Netflix didn’t exist decades ago. Given the advancement of technology, fintech firms are scaling new heights. Tech enables healthcare workers to provide better services to senior citizens in rural areas. “Similarly when we talk about traditional markets, businesses which have been in existence, and already matured businesses, how are they going through digital transformation? Fintech or the banking industry is a very good example of ongoing digital transformation,” she said.
Enablers
The first enabler is data which can come in multiple formats ranging from numeric and textual to audio and images. It can also come from sensors. “When it comes to processing the data or getting meaningful insight from it, AI and ML come into the picture. While data processing and visualisation are the second enablers, infrastructure is the third,” Bajpai said.
She said, depending on the type of data, it will be stored in an autonomous database or in an object storage. The data can be used by a data scientist to create a custom AI or ML model depending on the business needs. Further, data stored in the autonomous database can be utilised to schedule ML training jobs. “In certain business use cases, pre-trained AI models which are available to you in the form of AI services can be utilised. That service can be exposed to the external world using API data.
“A layer of infrastructure supports any application created using AI data science or ML services. And also the visualisation because you want to visually understand what data is trying to convey at the end,” Bajpai said.
Data processing
Now, business use cases define how the data should be looked at. And based on the business use case, different algorithms can be selected for the same data. “For example, if you are running an e-commerce portal, you may be interested in understanding how to improve your product recommendation engine or as a retailer, you may be interested in understanding what the right price for maximising profitability is.
“So, depending on the equation or objective or key business requirement that you have, you may be interested in finding the answer to a particular question or solving a particular problem. Now, depending on the problem in hand, we need to understand what data is to be processed. And, if you do not have data, how to secure the data and then what should be the right AI and ML algorithm that can understand some patterns in data. Data can often be very simple, and certain rules can be applied. However, there can often be hidden complexities or patterns inside the data, which AI and machine learning can capture. And depending on the insights you get, you can define your business objectives or utilise the insight for the benefit of your business,” she concluded.
