The Missing Point in Vision Transformers for Universal Image Segmentation

Benchmark Model Rank Results
instance-segmentation-on-ade20k-valViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280, COCO_pretrain)#3AP: 40.7
instance-segmentation-on-ade20k-valViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280)#6AP: 37.8
instance-segmentation-on-cityscapes-valViT-P (OneFormer, ConvNeXt-L, single-scale, 512x1024, Mapillary Vistas-pretrained)#1mask AP: 49.0AP: 49.0
panoptic-segmentation-on-ade20k-valViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280, COCO_pretrain)#2PQ: 54.0
panoptic-segmentation-on-ade20k-valViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280)#7PQ: 51.9
panoptic-segmentation-on-cityscapes-valViT-P (OneFormer, InternImage-H)#1PQ: 70.8mIoU: 85.4AP: 50.6
semantic-segmentation-on-ade20kViT-P (InternImage-H)#1Validation mIoU: 63.6Params (M): 1610
semantic-segmentation-on-ade20kViT-P (OneFormer, InternImage-H)#7Validation mIoU: 61.6Params (M): 1400
semantic-segmentation-on-ade20kViT-P (OneFormer, DiNAT-L)#16Validation mIoU: 59.9Params (M): 309
semantic-segmentation-on-cityscapes-valViT-P (InternImage-H)#1mIoU: 87.4
semantic-segmentation-on-coco-1ViT-P (OneFormer, InternImage-H)#2mIoU: 69.1
semantic-segmentation-on-coco-1ViT-P (OneFormer, DiNAT-L)#4mIoU: 68.8
semantic-segmentation-on-coco-stuff-testViT-P (InternImage-H)#1mIoU: 53.5