OmniVec2 - A Novel Transformer based Network for Large Scale Multimodal and Multitask Learning

Benchmark Model Rank Results
3d-point-cloud-classification-on-modelnet40-cOmniVec2–Error Rate: 0.142
3d-point-cloud-classification-on-scanobjectnnOmniVec2–Overall Accuracy: 97.2
action-classification-on-kinetics-400OmniVec2–Acc@1: 93.6
action-classification-on-moments-in-timeOmniVec2–Top 1 Accuracy: 53.1
action-classification-on-moments-in-time-2OmniVec2–Top 1 Accuracy: 53.1
action-recognition-in-videos-on-ucf101OmniVec2–3-fold Accuracy: 99.6
audio-classification-on-audiosetOmniVec2–Test mAP: 0.558
audio-classification-on-esc-50OmniVec2–Top-1 Accuracy: 99.1PRE-TRAINING DATASET: Multiple…
fine-grained-image-classification-on-oxford-iiit-petOmniVec2–Accuracy: 99.6
image-classification-on-imagenetOmniVec2–Top 1 Accuracy: 89.3%
image-classification-on-inaturalist-2018OmniVec2–Top-1 Accuracy: 94.6
image-classification-on-places365OmniVec2–Top 1 Accuracy: 65.1
semantic-segmentation-on-nyu-depth-v2OmniVec2–Mean IoU: 63.6
semantic-segmentation-on-s3dis-area5OmniVec2–mIoU: 77.1
text-summarization-on-dialogsumOmniVec2–Rouge1: 47.6Rouge2: 22.1RougeL: 41.4BertScore: 72.8
text-summarization-on-samsum-corpusOmniVec2–ROUGE-1: 59.1ROUGE-2: 34.1ROUGE-L: 63.7BertScoreF1: 65.1
zero-shot-video-retrieval-on-youcook2OmniVec2–text-to-video R@1: 26.1text-to-video R@5: 54.1…