M2D2: Exploring General-purpose Audio-Language Representations Beyond CLAP

Benchmark Model Rank Results
emotion-recognition-on-emomusicM2D-CLAP#1EmoA: 77.4EmoV: 61.9
emotion-recognition-on-emomusicM2D2#2EmoA: 76.7EmoV: 59.3
emotion-recognition-on-emomusicM2D#3EmoA: 76.1EmoV: 59.4
instrument-recognition-on-nsynthM2D-CLAP#1Accuracy: 80.6
instrument-recognition-on-nsynthM2D2 AS+#2Accuracy: 79.7
instrument-recognition-on-nsynthM2D AS#3Accuracy: 78.7
music-auto-tagging-on-magnatagatuneM2D2 AS+#1PR-AUC: 41.6ROC AUC: 91.8
audio-classification-on-esc-50M2D2 AS+#3Top-1 Accuracy: 98.5PRE-TRAINING DATASET: AudioSet,WavCaps…
audio-classification-on-audiosetM2D2#14Test mAP: 0.490