MViTv2: Improved Multiscale Vision Transformers for Classification and Detection

Open paper
Benchmark Model Rank Results
action-classification-on-kinetics-400MViTv2-L (ImageNet-21k pretrain)#43Acc@1: 86.1Acc@5: 97.0
action-classification-on-kinetics-400MViT-B (train from scratch)#179FLOPs (G) x views: 225x5
action-classification-on-kinetics-600MViTv2-L (ImageNet-21k pretrain)#19Top-1 Accuracy: 87.9Top-5 Accuracy: 97.9
action-classification-on-kinetics-600MViTv2-L (train from scratch)#24Top-1 Accuracy: 85.5
action-classification-on-kinetics-600MViTv2-B (train from scratch)#54Top-5 Accuracy: 97.2
action-classification-on-kinetics-600MViT-L (train from scratch)#55GFLOPs: 206x5
action-classification-on-kinetics-700MViTv2-L (ImageNet-21k pretrain)#15Top-1 Accuracy: 79.4Top-5 Accuracy: 94.9
action-classification-on-kinetics-700MoViNet-A6#16Top-1 Accuracy: 79.4
action-classification-on-kinetics-700MViTv2-B#17Top-1 Accuracy: 76.6Top-5 Accuracy: 93.2
action-recognition-in-videos-on-somethingMViTv2-L (IN-21K + Kinetics400 pretrain)#21Top-1 Accuracy: 73.3Top-5 Accuracy: 94.1Parameters: 213.1
action-recognition-in-videos-on-somethingMViT-B (IN-21K + Kinetics400 pretrain)#25Top-1 Accuracy: 72.1GFLOPs: 225x3
action-recognition-in-videos-on-somethingMViTv2-B (IN-21K + Kinetics400 pretrain)#100Top-5 Accuracy: 93.4Parameters: 51.1
action-recognition-in-videos-on-somethingMViT-L (IN-21K + Kinetics400 pretrain)#103GFLOPs: 2828x3
action-recognition-on-ava-v2-2MViTv2-L (IN21k, K700)#17mAP: 34.4
image-classification-on-imagenetMViTv2-H (512 res, ImageNet-21k pretrain)#29Top 1 Accuracy: 88.8%Number of params: 667MGFLOPs: 763.5
image-classification-on-imagenetMViTv2-L (384 res, ImageNet-21k pretrain)#47Top 1 Accuracy: 88.4%Number of params: 218MGFLOPs: 140.7
image-classification-on-imagenetMViTv2-H (mageNet-21k pretrain)#61Top 1 Accuracy: 88%Number of params: 667MGFLOPs: 120.6
image-classification-on-imagenetMViTv2-L (384 res)#154Top 1 Accuracy: 86.3%Number of params: 218MGFLOPs: 140.2
image-classification-on-imagenetMViTv2-T#535Top 1 Accuracy: 82.3%Number of params: 24MGFLOPs: 4.7
instance-segmentation-on-coco-minivalMViTv2-L (Cascade Mask R-CNN, multi-scale, IN21k pre-train)#21mask AP: 50.5
instance-segmentation-on-coco-minivalMViTv2-H (Cascade Mask R-CNN, single-scale, IN21k pre-train)#33mask AP: 48.5
instance-segmentation-on-coco-minivalMViTv2-L (Cascade Mask R-CNN, single-scale)#37mask AP: 47.1
instance-segmentation-on-coco-minivalMViT-L (Mask R-CNN, single-scale)#42mask AP: 46.2
object-detection-on-coco-minivalMViTv2-L (Cascade Mask R-CNN, multi-scale, IN21k pre-train)#34box AP: 58.7
object-detection-on-coco-minivalMViTv2-H (Cascade Mask R-CNN, single-scale, IN21k pre-train)#46box AP: 56.1
object-detection-on-coco-minivalMViTv2-L (Cascade Mask R-CNN, single-scale)#57box AP: 54.3
object-detection-on-coco-minivalMViT-L (Mask R-CNN, single-scale, IN21k pre-train)#64box AP: 52.7
object-detection-on-coco-oMViTV2-H (Cascade Mask R-CNN)#16Average mAP: 30.9Effective Robustness: 5.62

Rank counts only results with a code link.