Meet Meerkat, A New Data Library To Wrangle Complex ML Datasets

Recently, researchers at Stanford University launched a new data library called Meerkat for working with complex machine learning datasets. The source code of the project is available on GitHub

Data is the oxygen for machine learning. From training and validation data to future predictions, embeddings and metadata, it drives all parts of the machine learning development process. However, organising and managing data is challenging.

To that end, Stanford researchers have proposed a new Python library to help researchers and ML practitioners wrangle data. Data wrangling is a process of cleaning and unifying messy and complex datasets for easy access and analysis. 

How does Meerkat work? 

In a Notion Press blog, ‘Meerkat: Datapanels for machine learning,’ Stanford researchers Sabri Eyuboglu, Arjun Desai and Karan Goel talked about a few areas where Meerkat could solve the data complexity in the machine learning lifecycle.

  • Dataset manipulation techniques like slicing, shaping and transforming datasets have become an increasingly important part of the development process. As the quality of machine learning models and evaluations are primarily products of the data, more time goes into tuning datasets than tuning models. 
  • Model evaluation is emerging as a new bottleneck when building high-performing ML systems. For instance, models have been commoditised to the extent that resources like HuggingFace’s Model Hub can give you a model for text, speech or vision in seconds. But, they are hard to get right, and their failure modes can be opaque. 
  • Multi-modal datasets that combine multiple, complex data types are becoming more prevalent. For example, OpenAI’s CLIP combines natural language with images. 

Meerkat provides the DataPanel abstraction. The DataPanel facilitates interactive dataset manipulation, where it can house diverse data modalities and lets you evaluate models carefully with Robustness Gym. “We built DataPanels like DataFrames because they are naturally interactive and work seamlessly across development contexts: Jupiter Notebooks, Python scripts, and Streamlit,” the researchers said.

The goal is to make Meerkat DataPanel an interactive data substrate for modern machine learning across the machine learning lifecycle. 

Besides Robustness Gym, Meerkat can also be integrated into other popular benchmark datasets and works well with existing libraries and tools like WILDS, Huggingface Datasets, DOSMA, Streamlit.

What makes Meerkat different?

The data structures typically fall into two categories: those supporting complex data types and multiple modalities (PyTorch Dataset, Tensorflow Dataset), and those that support manipulation and interaction (Pandas DataFrame). “With the Meerkat DataPanel, we support all of these desiderata in one data structure,” said the researchers.

Advantages: 

  • Meerkat can store complex data types (images, graphs, videos and time series) 
  • Supports datasets that are larger than RAM (Kinetics, MIMIC-CXR, ImageNet) with efficient I/O under-the-hood  
  • Supports multimodal datasets 
  • Supports data creation and manipulation 
  • Supports data selection 
  • Support inspection in interactive environments

Comparing Meerkat with other machine learning data structures (Source: Notion Press) 

The experiment

The researchers ran an experiment to detect pneumothorax (a collapsed lung) in chest X-rays. For developing a model for this task, the researchers encountered various types of data — X-ray images to structured metadata to embeddings extracted from a trained model. 

Here, Meerkat’s DataPanel (a columnar data structure) could house all these data types under one roof. “Keeping them together enables quicker model iteration, fine-grained error analysis, and easier data exploration and inspection,” said the Stanford researchers. The codes used for running this experiment are available here

Meerkat addressed the desiderata by facilitating the inspection and manipulation of datasets that combine multiple complex data types. Meerkat provides high-level data abstractions because its data structures are written in Python and have few low-level optimisations, unlike Pandas, NumpY or Apache Arrow. 

“This does not mean that the Meerkat DataPanel is slow: each column type is as fast as the data structure it is built upon,” said the researchers.

More Great AIM Stories

Amit Raja Naik
Amit Raja Naik is a seasoned technology journalist who covers everything from data science to machine learning and artificial intelligence for Analytics India Magazine, where he examines the trends, challenges, ideas, and transformations across the industry.

More Stories

OUR UPCOMING EVENTS

8th April | In-person Conference | Hotel Radisson Blue, Bangalore

Organized by Analytics India Magazine

View Event >>

30th Apr | Virtual conference

Organized by Analytics India Magazine

View Event >>

MORE FROM AIM
Yugesh Verma
All you need to know about Graph Embeddings

Embeddings can be the subgroups of a group, similarly, in graph theory embedding of a graph can be considered as a representation of a graph on a surface, where points of that surface are made up of vertices and arcs are made up of edges

Yugesh Verma
A beginner’s guide to Spatio-Temporal graph neural networks

Spatio-temporal graphs are made of static structures and time-varying features, and such information in a graph requires a neural network that can deal with time-varying features of the graph. Neural networks which are developed to deal with time-varying features of the graph can be considered as Spatio-temporal graph neural networks. 

Vijaysinh Lendave
How to Evaluate Recommender Systems with RGRecSys?

A recommender system, sometimes known as a recommendation engine, is a type of information filtering system that attempts to forecast a user’s “rating” or “preference” for an item. In this post, we will look at RGRecSys, a library that performs constraint evaluation of recommender systems.

Yugesh Verma
A guide to explainable named entity recognition

Named entity recognition (NER) is difficult to understand how the process of NER worked in the background or how the process is behaving with the data, it needs more explainability. we can make it more explainable.

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM