Training data for effective AI deployment

Nearly 60 to 70% of the model development time is spent on data-related processes.

As AI products and services are increasingly being deployed into the real world, ML Data OPs have had to rapidly iterate to meet the challenges of handling data for model training and continuous testing. Sudeep George, VP of Engineering at iMerit, in his talk, ‘Training data for effective AI deployment’ at the Data Engineering Summit 2022, has shared practical insights around handling training data for complex AI systems operating at scale.

Sudeep defined data annotation as “a process of structuring and classifying data so that an ML model can distinguish between the human and the background, the people and vehicles, and understand the actions and intents by analysing its cause.” The process is important because it provides highly accurate ground truth data and helps machines to parse and understand the text.

The annotation journey

Nearly 60 to 70% of the model development time is spent on data-related processes like gathering, cleaning and structuring. The data should be maintained at a very high quality to optimise the model’s performance. To build training data, companies go through three types of vendors. Initially, companies opt for tool providers who can help them annotate complex data objects. Later, the company reaches a point where tons of data needs to be classified, and they bring in workforce providers who offer trained annotators. Soon, the companies reach another inflection point and start working with solution providers.

Model deployment has several stages. First, the model is evaluated using the test dataset. Later, it is tested against fresh production data and, post-deployment, continuously monitored to ensure performance. The model has to be retrained when the performance dips.

Data-centric AI

The traditional approach to building AI systems has been to train a model to reach a baseline performance and tweak and tune it to improve performance. Unfortunately, these models usually fail after deployment. The data-centric approach brings the model to baseline performance and trains it against datasets that represent the environment within which the model has to operate.

For models, it is important to have

  • Data at scale
  • Trustworthy data

Augmentation is a popular way to scale data. Here, two key approaches are used: deep learning and data domain manipulation. The manipulation techniques are specific to the data modality. For instance, for computer vision, image filters or transformation tools (such as image geometry transformation or image colour space transformation) are used to transform the image for training. On the other hand, deep learning techniques like GANs generate synthetic images based on original data. This ensures the dataset is evenly distributed among all classes and minimises the bias. 

Dataset management is important to ensure the collected data is used across various models in a manner that the data is reusable. It also helps understand data distribution and databases.

Edge cases affect the performance of the model. Some unique classes only have limited data available, and when the model encounters these in real-life scenarios, it leads to errors and failures. Depending on the ML life cycle part, an edge case can have different implications. In the collection phase, it is about insufficient data. In the labelling phase, it is about incomplete attributes being classified. In training, it is about incomplete testing against scenarios. Lastly, in the deployment phase, it is about the actual performance of the ML model when working on real-world data. Sudeep discussed some of the common challenges companies face, including guidelines, traceable feedback loop, scenario replication, and end-end system validation. 

Lastly, he discussed an example of a customer who wanted to train a model to distinguish between a cloudy and sunny outside environment. The iMerit team used the shadows cast by cards and other objects to act as a feature point for the ML training process, and the model’s performance improved drastically.


Download our Mobile App

Avi Gopani
Avi Gopani is a technology journalist that seeks to analyse industry trends and developments from an interdisciplinary perspective at Analytics India Magazine. Her articles chronicle cultural, political and social stories that are curated with a focus on the evolving technologies of artificial intelligence and data analytics.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Bangalore

Future Ready | Lead the AI Era Summit

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

20th June | Bangalore

Women in Data Science (WiDS) by Intuit India

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox