MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound

Benchmark Model Rank Results
action-classification-on-kinetics-600🍷MerlotReserve-Base (+Audio)Top-1 Accuracy: 89.7Top-5 Accuracy: 96.6
action-classification-on-kinetics-600🍷MerlotReserve-Base (no Audio)Top-1 Accuracy: 88.1Top-5 Accuracy: 95.8
action-classification-on-kinetics-600🍷MerlotReserve-Large (+Audio)Top-1 Accuracy: 91.1Top-5 Accuracy: 97.1
action-classification-on-kinetics-600🍷MerlotReserve-Large (no Audio)Top-1 Accuracy: 89.4Top-5 Accuracy: 96.3