Lately, Zomato has been taking the internet by storm with intriguing posts like the independence day ‘not accepting orders anymore‘ or super friendly notifications reminding you to order food from its platform like never before. That is just one side of the coin. On the backdrop, Zomato has been experimenting with various machine learning models to provide personalised experiences to its customers, driver-partners and restaurants.
Today, many consumer-focused brands are trying to understand customer preferences in real-time and offer personalised experiences. It requires surfing through the data lake, creating useful machine learning models, and deploying them in production. For instance, Swiggy had created a heatmap feature (showcasing demand areas) for its delivery partners, which helped them know where to go next after delivering the food or grocery orders.
Zomato uses ML to predict a lot of challenges in real-time, including last-mile delivery of food orders, allocation of delivery partners, estimation of time for restaurants to prepare food, grooming of delivery partners, assessing fake reviews, etc. “Solving these unknowns helps us provide better customer experience and improve business metrics by either reducing costs or increasing the revenue,” says Ram Singla, VP of Technology at Zomato.
Founded in 2008, Zomato offers services like restaurant discovery and reviews, home delivery, online table reservation, digital payments when dining out, etc. It also works with restaurant partners to provide tools to engage and acquire more customers while empowering them with last-mile delivery services, alongside a one-stop procurement solution, Hyperpure, for ingredients and kitchen supplies.
The company currently has about two lakh+ merchants or restaurant partners, one lakh+ delivery partners and has completed close to 10 crore+ deliveries. It is present in over 23 countries. Recently, the Zomato IPO hit the primary market, and in two days, it witnessed an oversubscription of 4.18 times by retail investors. In FY21, the company made 86 per cent of its operating revenue from food delivery services which stood at INR 1,715 crore.
Here’s how Zomato uses ML
In its latest blog post, the food delivery giant Zomato unveiled its secret ingredients of machine learning. It said it uses a simple equation of three variables — input, brain, and output, via ‘model training’ and ‘model prediction’ techniques. Meaning, if the output and input are known, the process of figuring out the brain is called model training. If the input and brain are known, the method of computing the output is called model prediction.
(Source: Zomato)
Once the team deploys the model to production, it creates a ‘model server’ to make predictions via remote APIs (application programme interface). Then, using these APIs, they make ‘model predictions.’
“As you deploy more ‘model servers,’ you evolve your ‘ML runtime,'” says Singla. He says this formalises the model training and deployment process, thereby increasing the team’s cadence, leading to faster turnaround time, more experimentation, and better models.
Secret sauce
Zomato’s ‘ML runtime’ consists of four essential components: feature compute engine, feature store, model store, and model serving API gateway.
In addition to this, it supports two types of features based on frequency — real-time features and batched features.
At Zomato, real-time features are computed using event streams published on Apache Kafka and are processed in real-time by stream processing engine Apache Flink. These are later stored in an online feature store powered by Redis Cluster. On the other hand, batched features are computed using Apache Spark and are stored in their offline feature store DynamoDB with new features getting cached in Redis Cluster.
Experiments
Early this year, Zomato’s machine learning team load-tested its ‘feature store’ to a maximum throughput of around 18 million requests per minute, both in terms of performance and latency. Surprisingly, this led to a 3x improvement in its ‘feature store’ service compared to last year.
Zomato converts all its production models into a standard format using MLFlow. The team believes that this helps to decouple and makes it possible to write tools that work with models from any ML library without integrating each tool with each library. The ML libraries can be TensorFlow, PyTorch, LightGBM. or sci-kit-learn, etc.
In terms of model deployment, Zomato leverages cloud with Kubernetes. Most of Zomato’s models are tuned for inference on CPU rather than GPU, and to further optimise the cost, the team uses spot instances in its Kubernetes cluster. “This setup of ML on Kubernetes has helped us adapt and scale the model, serving across multiple production models, with ease,” says Singla.
Zomato’s ML API gateway is written in Golang, primarily used to remove coupling at the client-side so that the specific logic sits within the gateway. “Generally, when we re-deploy a retrained model or a new model to the same problem, our API requests to the ML API Gateway do not change,” explains Singla.
That has helped the team deploy and experiment models frequently, with minimal effort. ML API Gateway executes a directed acyclic graph of tasks. It has native support with its ‘feature store,’ where the gateway is responsible for fetching features for the model based on the request from the ‘feature store’ as specified in the model plan. That has reduced its time to deploy a model to production to less than 24 hours.
Conclusion
With the help of a small team of data scientists, ML engineers and data engineers, and ‘ML runtime’ in place, Zomato has built several machine learning models to solve multiple challenges in real-time. Few of them include menu digitisation, personalised homepage restaurant listings, increase in GMV and AOV, predicting food preparation time (FPT), enhancing road detection, active driver-partner dispatch, driver-partner grooming audit and compliance, and more.
