Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning

Benchmark Model Rank Results
action-classification-on-charadesTubeViT-L#2MAP: 66.2
action-classification-on-kinetics-400TubeViT-H (ImageNet-1k)#4Acc@1: 90.9Acc@5: 98.9FLOPs (G) x views: 176400x4x3
action-classification-on-kinetics-400TubeVit-L (ImageNet-1k)#7Acc@1: 90.2Acc@5: 98.6FLOPs (G) x views: 95300x4x3
action-classification-on-kinetics-400TubeVit-B (ImageNet-1k)#19Acc@1: 88.6Acc@5: 97.6FLOPs (G) x views: 8700x3x4
action-classification-on-kinetics-600TubeVit-H#2Top-1 Accuracy: 91.8Top-5 Accuracy: 98.9
action-classification-on-kinetics-600TubeVit-L#4Top-1 Accuracy: 91.5Top-5 Accuracy: 98.7
action-classification-on-kinetics-600TubeVit-B#6Top-1 Accuracy: 90.9Top-5 Accuracy: 97.3
action-classification-on-kinetics-700TubeViT-L#4Top-1 Accuracy: 83.8Top-5 Accuracy: 96.6
action-recognition-in-videos-on-somethingTubeViT-L#8Top-1 Accuracy: 76.1Top-5 Accuracy: 95.2