MViTv2: Improved Multiscale Vision Transformers for Classification and Detection

Open paper
Benchmark Model Rank Results
action-classification-on-kinetics-700MViTv2-L (ImageNet-21k pretrain)#15Top-1 Accuracy: 79.4Top-5 Accuracy: 94.9
action-classification-on-kinetics-700MoViNet-A6#16Top-1 Accuracy: 79.4
action-classification-on-kinetics-700MViTv2-B#17Top-1 Accuracy: 76.6Top-5 Accuracy: 93.2
object-detection-on-coco-oMViTV2-H (Cascade Mask R-CNN)#16Average mAP: 30.9Effective Robustness: 5.62
action-recognition-on-ava-v2-2MViTv2-L (IN21k, K700)#17mAP: 34.4
action-classification-on-kinetics-600MViTv2-L (ImageNet-21k pretrain)#19Top-1 Accuracy: 87.9Top-5 Accuracy: 97.9
action-classification-on-kinetics-600MViTv2-L (train from scratch)#24Top-1 Accuracy: 85.5
action-classification-on-kinetics-600MViTv2-B (train from scratch)#56Top-5 Accuracy: 97.2
action-classification-on-kinetics-600MViT-L (train from scratch)#57GFLOPs: 206x5
action-recognition-in-videos-on-something-something-v2MViTv2-L (IN-21K + Kinetics400 pretrain)#22Top-1 Accuracy: 73.3Top-5 Accuracy: 94.1Parameters: 213.1
action-recognition-in-videos-on-something-something-v2MViT-B (IN-21K + Kinetics400 pretrain)#26Top-1 Accuracy: 72.1GFLOPs: 225x3
action-recognition-in-videos-on-something-something-v2MViTv2-B (IN-21K + Kinetics400 pretrain)#104Top-5 Accuracy: 93.4Parameters: 51.1
action-recognition-in-videos-on-something-something-v2MViT-L (IN-21K + Kinetics400 pretrain)#107GFLOPs: 2828x3
instance-segmentation-on-coco-minivalMViTv2-L (Cascade Mask R-CNN, multi-scale, IN21k pre-train)#23mask AP: 50.5
instance-segmentation-on-coco-minivalMViTv2-H (Cascade Mask R-CNN, single-scale, IN21k pre-train)#39mask AP: 48.5
instance-segmentation-on-coco-minivalMViTv2-L (Cascade Mask R-CNN, single-scale)#47mask AP: 47.1
instance-segmentation-on-coco-minivalMViT-L (Mask R-CNN, single-scale)#54mask AP: 46.2
image-classification-on-imagenetMViTv2-H (512 res, ImageNet-21k pretrain)#34Top 1 Accuracy: 88.8%Number of params: 667MGFLOPs: 763.5
image-classification-on-imagenetMViTv2-L (384 res, ImageNet-21k pretrain)#52Top 1 Accuracy: 88.4%Number of params: 218MGFLOPs: 140.7
image-classification-on-imagenetMViTv2-H (mageNet-21k pretrain)#67Top 1 Accuracy: 88%Number of params: 667MGFLOPs: 120.6
image-classification-on-imagenetMViTv2-L (384 res)#170Top 1 Accuracy: 86.3%Number of params: 218MGFLOPs: 140.2
image-classification-on-imagenetMViTv2-T#586Top 1 Accuracy: 82.3%Number of params: 24MGFLOPs: 4.7
action-classification-on-kinetics-400MViTv2-L (ImageNet-21k pretrain)#44Acc@1: 86.1Acc@5: 97.0
action-classification-on-kinetics-400MViT-B (train from scratch)#194FLOPs (G) x views: 225x5
object-detection-on-coco-val2017MViTv2-L (Cascade Mask R-CNN, multi-scale, IN21k pre-train)#45box AP: 58.7
object-detection-on-coco-val2017MViTv2-H (Cascade Mask R-CNN, single-scale, IN21k pre-train)#88box AP: 56.1
object-detection-on-coco-val2017MViTv2-L (Cascade Mask R-CNN, single-scale)#123box AP: 54.3
object-detection-on-coco-val2017MViT-L (Mask R-CNN, single-scale, IN21k pre-train)#162box AP: 52.7

Rank counts only results with a code link.