MLCommons Makes 2 Large Speech Datasets Freely Available

The MLCommons Association has announced the general availability of the Multilingual Spoken Words Corpus (MSWC) and People’s Speech Dataset. These permissively licensed datasets advance innovation in ML research and commercial applications. MLCommons Association also issued a call for participation in the new DataPerf benchmark suite that measures and encourages innovation in data-centric AI.

“Data is food for AI, and our systems need not just massive amounts of calories, but also high quality nutrition. We need not just big data, but good data,” said Andrew Ng, founder and CEO of Landing AI, co-founder and chairman of Coursera, founding lead of Google Brain and adjunct professor at Stanford University.

Multilingual Spoken Words Corpus, or MSWC, is a rich audio speech dataset with over 340,000 keywords in 50 languages with upwards of 23.4 million examples. It is a diverse multilingual dataset that spans languages by over 5 billion people and helps advancing the R&D of applications that include voice interfaces for a broad global audience. The contributors to the MSWC include researchers from Factored, Coqui, Harvard University, Intel, Google, Landing AI, University of Michigan and NVIDIA.

The People’s Speech Dataset is a 30,000-hour supervised conversational dataset and is among the world’s largest English speech recognition datasets, licensed for academic and commercial usage. It is released under a CC license and democratizes access to speech technology like voice assistants and transcription. Contributors to this dataset include researchers from Factored, Baidu, Harvard University, Intel, NVIDIA and Landing AI.

“Speech technology can empower billions of people across the planet, but there’s a real need for large, open, and diverse datasets to catalyze innovation,” said David Kanter, co-founder and executive director, MLCommons Association.

The new DataPerf benchmark suite supports AI innovation by measuring the quality of datasets and the impact of enhancing datasets. DataPerf fosters and measures the progress of understanding and improving datasets. To encourage participation in DataPerf in 2022, MLCommons Association will support a series of challenges with leaderboards. Contributors to the suite include researchers from Coactive.AI, Alibaba, ETH Zurich, Google, Landing.AI, Meta, Harvard University, Stanford University, and TU Eindhoven, drawing on the teams responsible for Cats4ML, the Data-Centric AI Competition, DCBench, Dynabench, and the MLPerf™ benchmarks.