Key takeaways
- Artificial intelligence is increasingly used for synthetic music generation, enhancing models to rival human composers.
- The quality of AI/ML models depends heavily on the datasets they are trained on.
- NSynth is a significant dataset containing over 300,000 musical notes from a variety of instruments.
- The MAESTRO dataset offers extensive paired audio and MIDI recordings, crucial for music AI training.
- URMP facilitates audio-visual analysis by compiling multi-instrument performances from separately recorded tracks.
Artificial intelligence has been tapped for synthetic music generation for some time now. However, the watershed moment came when music informatics met AI. Now, music AI researchers are taking advantage of the latest AI, ML, and analytics developments to develop models to create music on par with human composers.
The strength of any AI/ML model is predicated on the data it’s fed. Below, we look at the major music generation datasets doing the rounds in 2022.
The dataset probably contains the largest instrumental notes at 305,979 musical notes, including unique pitch, timbre and envelope. The musical notes were collected from 1,006 instruments from commercial sample libraries and are annotated based on (acoustic, electronic or synthetic) instrument family and sonic qualities. Instruments including bass, flute guitar, keyboard, mallet, organ, reed, string, synth lead and vocal have been used in this dataset.
MAESTRO
The MAESTRO (MIDI and Audio Edited for Synchronous Tracks and Organisation) dataset has over 200 hours of paired audio and MIDI recordings of International Piano-e-Competition in the past ten years. The MIDI data has key strike velocities, including sustain/sostenuto/una corda pedal positions.
URMP (University of Rochester Multi-Modal Musical Performance)
URMP introduced a dataset for facilitating audio-visual analysis of musical performances. The dataset contains several simple multi-instrument musical pieces assembled from separately recorded performances of individual tracks. For each piece, a musical score is provided in MIDI format.
The dataset consists of 21.6 million harmonisations submitted from the Bach Doodle and metadata about the composition, like country of origin and the feedback. It also has MIDI of user-entered melody and MIDI of the generated harmonisation. An exploration of the melodies in the dataset contains top repeated melodies from each country or the regional hits.
This dataset contains 176,581 unique MIDI files, 45,129 matched and aligned to the Million Song Dataset entries. The dataset mainly facilitates large-scale music information retrieval, both symbolic and audio content-based.
Music 21
Music21 contains music performances from 21 categories. It is a set of tools to help scholars quickly find answers to questions like, “I wonder how often Bach does that” or “Which band used these chords for the first time? Or “How to know more about Renaissance counterpoint or the Indian ragas or post-tonal pitch structures or the form of minutes”.
Datasets for Indian Music
CompMusic catalogues datasets for Indian art music. The website aims to advance the automatic description of music by emphasising cultural specificity carrying research in music information processing with a domain knowledge approach. The project mainly focused on five music traditions of the world: Hindustani (North India), Carnatic (South India), Turkish-makam (Turkey), Arab-Andalusian (Maghreb), and Beijing Opera (China).
Indian Music Tonic Dataset
The dataset contains 597 commercially available audio music recordings of Indian art music, both Hindustani and Carnatic music, where each recording is manually annotated with the tonic of the lead artist.
http://compmusic.upf.edu/iam-tonic-dataset
Carnatic Varnam Dataset
The dataset has 28 solo vocal recordings recorded to be researched on the intonation analysis of Carnatic ragas. In addition, the dataset contains audio recordings, time aligned tala cycle annotations and swara notations in a machine-readable format.
http://compmusic.upf.edu/carnatic-varnam-dataset
Carnatic Music Rhythm Dataset
This dataset is a sub-collection of 176 excerpts in four taalas of Carnatic music with audio, tala related metadata and time aligned markers to indicate the progress through the tala cycles.
http://compmusic.upf.edu/carnatic-rhythm-dataset
Hindustani Music Rhythm Dataset
The dataset is a sub-collection of 151 in four taals of Hindustani music which includes audio, taal related metadata and time aligned markers to indicate the progress through the taal cycles.
http://compmusic.upf.edu/hindustani-rhythm-dataset
Mridangam Stroke Dataset
The dataset contains 7,162 audio examples of individual strokes of the Mridangam in various tonics and ten different strokes played on Mridangams with six tonic values.
http://compmusic.upf.edu/mridangam-stroke-dataset
Mridangam Tani-avarthanam Dataset
The dataset is a transcribed collection of two tani-avarthanams played by Mridangam maestro Padmavibhushan Umayalpuram K. Sivaraman. The audio of the dataset was recorded at the IIT Madras and annotated by Carnatic percussionists. The dataset contains 24 minute of audio and 8,800 strokes.
http://compmusic.upf.edu/mridangam-tani-dataset
Saraga: Research datasets of Indian Art Music
The repository contains time aligned melody, rhythm and structural annotations for two large open datasets of Indian Art Music (Carnatic and Hindustani music).
Tabla Solo Dataset
The dataset is a transcribed collection of Tabla solo audio recordings spanning compositions from six different Gharanas of Tabla, played by Pt. Arvind Mulgaonkar. It consists of audio and time aligned bol transcriptions.
