At their Data Intelligence Day, Databricks recently announced its partnership with Indian customers such as UPL, Air India, Aditya Birla Fashion, Freshworks, InMobi, Meesho, Myntra, Parle, and many others. This will help leverage the Databricks Data Intelligence Platform to drive business innovation, optimise operations, and enhance decision-making.

One among these is UPL Limited, a multinational company headquartered in Mumbai, which provides sustainable agriculture products and solutions in more than 138 countries.

The company’s UPL Agri-Tech Platform, built in collaboration with Microsoft, uses AI and machine learning to provide crop management, pest control, and nutrient application recommendations. 

With a market cap of $4.37 billion as of April 2024, the company’s product portfolio includes crop protection, seed treatment, post-harvest solutions, and other agricultural inputs, with a presence in markets such as India, the United States, Brazil, and Europe. 

Having experienced massive growth in recent years, the partnership between UPL and Databricks becomes critical for the former. 

“In 2022, we were at the end of a massive growth period. We grew 2x in about three to four years,” Vijay Balakrishnan, chief data & analytics officer at UPL Group, told AIM on the sidelines of the event in Bengaluru on March 22.

This growth triggered the need for UPL to leverage data more effectively to make informed decisions and drive further growth. This is when the company found solace in Databricks.

UPL, the largest manufacturer and distributor of agrochemicals in India with a market share of around 20%, has actively started using Databricks’ Unity Catalog, which democratises data across the organisation. 

Using Databricks to Scale AI Solutions Across 50 Countries 

By leveraging Databricks, UPL also aims to enable AI solutions across 20 countries, with the potential to scale up to 50 countries in the future. 

“We aim to democratise access to AI, enabling citizen data scientists and proficient digital business leaders to leverage and operate the AI solutions we’ve established, whether for commercial endeavours, supply chain optimisation, or other applications,” elaborated Vijay.

For instance, they conduct extensive farmer outreach programmes, providing training and advisory services to help farmers adopt best agricultural practices.

The company has also found Databricks’ serverless SQL engine to be efficient, especially in instances where the speed of data pooling is critical. “We have seen it to be quite efficient today because there are a lot of instances where the speed of the data pool is very critical for us,” noted Vijay.

As UPL continues to explore and leverage Databricks’ generative AI and lakehouse capabilities, the partnership between the two companies is poised to drive innovation and transform the agriculture industry through data-driven solutions. 

UPL has started exploring Databricks’ Genie Data Room, which provides a drag-and-drop interface for generative AI. “The Genie Data Room actually makes it almost drag and drop. All we did was create a data room with the data that we’ve already pulled into Databricks and started working,” 

According to Vijay, UPL is also focusing on prompt engineering and even considering using AI for prompt engineering itself. He also emphasised the importance of governance, particularly as metadata becomes complex. 

Leveraging Databricks for Demand Forecasting 

One of the key areas where UPL is leveraging Databricks’ capabilities is demand forecasting. “Demand forecasting has its own challenges in the agriculture sector because many external variables play a role,” Vijay said. 

“For instance, the last 10-12 months have been an El Nino drought situation in most of the Northern Hemisphere. But that has a reverse effect in countries in the Southern Hemisphere,” he added.

These external factors, along with the constantly evolving product portfolio, make demand forecasting a complex task. Timing is also crucial in the agriculture industry, with many products requiring application within a specific three-day window. 

“A lot of our products have to be timed in such a way that you’ve got a three-day window within which a certain product has to be applied. And if you don’t predict that three-day window and apply, you don’t have a play,” explained Vijay.

For instance, if UPL gauges demand incorrectly, they run the risk of spoiled products which may be stuck in transportation – out of cold storage or in transit for extended periods.

To address these challenges, UPL has developed a demand forecasting model that considers various factors. Databricks’ platform provides UPL with a unified and governed data management and analytics environment, enabling more accurate demand forecasts.

Challenges to Solve Going Forward

Vijay also highlighted the challenges faced by the company in data ingestion and orchestration. 

“The way we look at Databricks is we would love for them to be all in analytics for us. Now, there are two areas where it is a little challenging today. One is on the, let’s say, the data collection part or ingestion, but more ingestion and collection part, especially in complex source ecosystems like SAP,” said Vijay.

SAP poses its own set of challenges due to contractual aspects that need to be considered before pulling data, whether in a CDC (Change Data Capture) fashion or a batch mode. 

While Databricks has a good partner ecosystem, Vijay noted that there is still a gap in this area. “That part, while there are good partners, an ecosystem with Databricks, there’s a little bit of a gap, at least from my perspective. And I brought it up on a couple of forums earlier,” he said.

Bhasin acknowledged these challenges and emphasised that while Databricks wants to be everything to everybody, it is an evolving platform, and it’s a matter of different priorities at different points. 

“We take pride in our commitment to identifying common pain points and developing the necessary capabilities to address them. This approach is integral to the evolution of our platform. By actively listening to our customers and discerning prevalent pain points, we prioritise investments that yield meaningful solutions,” he explained.

Databricks has been investing disproportionately in R&D to stay at the cutting edge of technology. Bhasin acknowledged the feedback from UPL and assured that Databricks is working on addressing these challenges. 

Conclusively, as UPL and Databricks continue to collaborate and co-innovate, they aim to address the data ingestion and orchestration challenges in the agriculture industry, unlocking new opportunities for growth and efficiency.