Article complete

Andrew Ng Announces The Launch Of NeurIPS Data-Centric AI Workshop

If 80 per cent of machine learning work is data preparation, then ensuring data quality is the most important work of a machine learning team

Andrew Ng with angel wings, smiling against a colorful abstract background with clouds.

Key takeaways

  • Andrew Ng has announced the launch of the NeurIPS Data-Centric AI workshop to highlight significant academic research in this field.
  • The workshop encourages submissions from both academic researchers and practitioners before the deadline of September 30, 2021.
  • Data-Centric AI focuses on improving data quality, addressing gaps in tooling, and enhancing best practices in machine learning systems.
  • The organising committee comprises prominent figures from institutions like Google Research, Stanford University, and Harvard University.
  • The initiative aims to foster a vibrant community around Data-Centric AI, promoting efficient data management and engineering.

Takeaways drafted with AI from the article text.

DeepLearning.AI's Andrew Ng recently announced the launch of the NeurIPS Data-Centric AI workshop. The workshop is expected to showcase some of the best academic research work related to data-centric AI. Academic researchers and practitioners can submit their research papers on or before September 30, 2021.

https://twitter.com/AndrewYNg/status/1435278073444716550

The organising committee includes Google Research's Lora Aroyo, Stanford University professor Cody Coleman, Landing AI's Greg Diamos, Harvard University professor Vijay Janapa Reddi, Eindhoven University of Technology researcher Joaquin Vanschoren, and Google's machine learning product manager, Sharon Zhou. 

What is Data-Centric AI? 

The team strives to cultivate the DCAI community into a vibrant interdisciplinary field and tackle practical data problems with this event. The data problems include: data collection/generation, data labelling, data preprocess/augmentation, data quality evaluation, data debt, and data governance. The team believes that many of these areas are still in the early stages and hope to knit the gaps by bringing the ML community together. 

Call for Papers 

The journey of building and using datasets for AI systems is often artisanal -- painstaking and expensive. The ML community lacks high productivity and efficient open data engineering tools. To accelerate creation and iteration, alongside increasing the efficiency of use and reuse by democratising data engineering and evaluation, remains a core challenge even to this day. 

"If 80 per cent of machine learning work is data preparation, then ensuring data quality is the most important work of an ML team and therefore a vital research area," said the NeurIPS DSAI team. Further, they said human-labelled data has increasingly become the fuel and compass of AI-based software systems, while innovative efforts have mostly focused on models and code. 

However, in recent years, there has been an increased focus on scale, speed, and cost of building and improving datasets, which has, in turn, resulted in an impact on quality. Some of the major research work in the areas include 'Response-based Learning for Grounded Machine Translation,' 'Crowdsourcing with Fairness, Diversity and Budget Constraints,' Excavating AI, 'Data Excellence: Better Data for Better AI,' 'State of the Art: Reproducibility in Artificial Intelligence, and others. 

"We need a framework for excellence in 'data engineering' that does not yet exist," said the NeurIPS DCAI team, and noted that aspects like maintainability, reproducibility, reliability, validity and fidelity of datasets are often overlooked when releasing the dataset into the market. In this event, the team plans to highlight examples, case studies, and methodologies for excellence in data collection. 

The NeurIPS DCAI team said that building an active research community focused on data-centric AI is critical for defining the core problems and creating ways to measure progress in machine learning through data quality tasks.

Topics

The interested candidate can submit their papers on the following topics that include but are not limited to the following: 

New Datasets in areas: 

  • Speech, vision, manufacturing, medical, recommendation/personalisation 
  • Science 

Tools and methodologies that

  • Quantify and accelerate time to source high-quality data 
  • Ensure data is labelled consistently, such as label consensus 
  • Improve data quality more systematically. 
  • Automate the generation of high quality supervised learning training data from low-quality resources, such as forced alignment in speech recognition 
  • Produce uniform and low noise data samples, or remove labelling noise or inconsistencies from existing data 
  • Control what goes into the dataset and make high-level edits efficiently to very large datasets, like adding new words, languages, etc. 
  • Search techniques for finding suitably licensed datasets based on public resources 
  • Create training datasets for small data problems or rare classes in the long tail of big data problems 
  • Incorporate timely feedback from production systems into datasets 
  • Understand dataset coverage of important classes and editing them to cover newly identified important cases 
  • Import dataset by allowing easy combination and composition of existing datasets
  • Export dataset by making the data consumable for models and interface with model training and inference systems such as web dataset 
  • Enable composition of dataset tools like MLCube, Docker, Airflow 

Algorithms for working with limited labelled data and improving label efficiency 

  • Data selection techniques like active learning and core-set selection for identifying the most valuable examples to label 
  • Semi-supervised learning, few-shot learning, and weak supervision techniques for maximising the power of limited labelled data 
  • Self-supervised learning and transfer learning approaches for developing powerful representations used for many downstream tasks with limited labelled data. 
  • Novelty and drift detection to identify and spot when more data needs to be labelled 

Responsible AI development: 

  • Fairness, bias, diversity evaluation and analysis for dataset and algorithms/modelling 
  • Tools for 'green AI hardware-software system' design and evaluation 
  • Scalable, reliable training systems and methods 
  • Tools, methodologies, and techniques for private, secure ML training 
  • Efforts towards reproducible AI (data cards, model cars, etc.)

Instructions for submitting papers

  • Researchers can submit short papers (1-2 pages) and long papers (4 pages), addressing one or more of the topics  
  • Papers need to be formatted as per NeurIPS 2021 guidelines 
  • Papers will be peer-reviewed by the programme committee 
  • Accepted papers will be presented as lighting talks during the workshop 

Timeline 

  • Early submission deadline: 17 September 2021
  • Submission deadline: 30 September 2021 
  • Notification of acceptance: 22 October 2021 
  • Workshop: 14 December 2021 

Click here to submit your research paper. 

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

Machine Learning Latest · 5 stories
  1. TensorFlow Releases New 3D Pose Detection Model
  2. Tech Behind Facebook’s TextStyleBrush
  3. IBM Launches AI Starter Kit To Deploy ML Models Faster
  4. You are here: Andrew Ng Announces The Launch Of NeurIPS Data-Centric AI Workshop
  5. The story of Mobassir Hossen, the first Kaggle Grandmaster from Bangladesh
About the author

Editor

Amit Raja Naik, known as the 'AI Human,' serves as the editor at AIM Media House, where he leads a team of talented tech journalists who are driving and shaping technology conversations across India and around the world.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.