Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling

Benchmark Model Rank Results
image-classification-on-imagenetSparK (ConvNeXt-Large, 384)#178Top 1 Accuracy: 86.0%Number of params: 198M
instance-segmentation-on-coco-2017-valSparK (ConvNeXt V1-B Mask R-CNN)#1mask AP*: 45.1mask AP: 45.1AP: 45.1
self-supervised-image-classification-on-1SparK (ConvNeXt-Large, 384)#18Top 1 Accuracy: 86.0%Number of Params: 198M
self-supervised-image-classification-on-1SparK (ConvNeXt-Large)#24Top 1 Accuracy: 85.4%Number of Params: 198M
self-supervised-image-classification-on-1ConvNeXt-Base (SparK pre-training)#27Top 1 Accuracy: 84.8%Number of Params: 89M
self-supervised-image-classification-on-1ConvNeXt-Small (SparK pre-training)#37Top 1 Accuracy: 84.1%Number of Params: 50M
self-supervised-image-classification-on-1ResNet-200 (SparK pre-training)#47Top 1 Accuracy: 83.1%Number of Params: 65M
self-supervised-image-classification-on-1ResNet-152 (SparK pre-training)#49Top 1 Accuracy: 82.7%Number of Params: 60M
self-supervised-image-classification-on-1ResNet-101 (SparK pre-training)#53Top 1 Accuracy: 82.2%Number of Params: 44M
self-supervised-image-classification-on-1ResNet-50 (SparK pre-training)#55Top 1 Accuracy: 80.6%Number of Params: 26M