Omnivore: A Single Model for Many Visual Modalities

Open paper
Benchmark Model Rank Results
action-classification-on-kinetics-400OMNIVORE (Swin-L)#55Acc@1: 84.1Acc@5: 96.1
action-classification-on-kinetics-400OMNIVORE (Swin-B)#56Acc@1: 84.0Acc@5: 96.2
action-recognition-in-videos-on-somethingOMNIVORE (Swin-B, IN-21K+ Kinetics400 pretrain)#28Top-1 Accuracy: 71.4Top-5 Accuracy: 93.5
action-recognition-on-epic-kitchens-100OMNIVORE (Swin-B, finetuned)#7Action@1: 49.9Verb@1: 69.5Noun@1: 61.7
image-classification-on-imagenetOmnivore (Swin-L)#175Top 1 Accuracy: 86.0%
image-classification-on-imagenetOmnivore (Swin-B)#236Top 1 Accuracy: 85.3%
image-classification-on-inaturalist-2018OMNIVORE (Swin-L)#9Top-1 Accuracy: 84.1%
semantic-segmentation-on-nyu-depth-v2OMNIVORE (Swin-L, finetuned)#15Mean IoU: 56.8%
semantic-segmentation-on-nyu-depth-v2OMNIVORE (Swin-B, finetuned)#24Mean IoU: 55.1%

Rank counts only results with a code link.