OneFormer: One Transformer to Rule Universal Image Segmentation

Benchmark Model Rank Results
instance-segmentation-on-ade20k-valOneFormer (InternImage-H, emb_dim=1024, single-scale, 896x896, COCO-Pretrained)#1AP: 44.2APL: 64.3APM: 49.9APS: 23.7
instance-segmentation-on-ade20k-valOneFormer (DiNAT-L, single-scale, 1280x1280, COCO-pretrain)#4AP: 40.2APL: 59.7APM: 44.4APS: 19.2
instance-segmentation-on-ade20k-valOneFormer (DiNAT-L, single-scale)#7AP: 36.0
instance-segmentation-on-ade20k-valOneFormer (Swin-L, single-scale)#8AP: 35.9
instance-segmentation-on-cityscapes-valOneFormer (ConvNeXt-L, single-scale, Mapillary-Pretrained)#2mask AP: 48.7
instance-segmentation-on-cityscapes-valOneFormer (DiNAT-L, single-scale)#5mask AP: 45.6
instance-segmentation-on-cityscapes-valOneFormer (Swin-L, single-scale)#6mask AP: 45.6
instance-segmentation-on-coco-val-panopticOneFormer (InternImage-H, emb_dim=1024, single-scale)#1AP: 52.0
instance-segmentation-on-coco-val-panopticOneFormer (DiNAT-L, single-scale)#2AP: 49.2
instance-segmentation-on-coco-val-panopticOneFormer (Swin-L, single-scale)#4AP: 49.0
panoptic-segmentation-on-ade20k-valOneFormer (InternImage-H, emb_dim=256, single-scale, 896x896)#1PQ: 54.5mIoU: 60.4AP: 40.2
panoptic-segmentation-on-ade20k-valOneFormer (DiNAT-L, single-scale, 1280x1280, COCO-Pretrain)#4PQ: 53.4mIoU: 58.9
panoptic-segmentation-on-ade20k-valOneFormer (DiNAT-L, single-scale, 1280x1280)#8PQ: 51.5mIoU: 58.3AP: 37.1
panoptic-segmentation-on-ade20k-valOneFormer (Swin-L, single-scale, 1280x1280)#9PQ: 51.4mIoU: 57.0AP: 37.8
panoptic-segmentation-on-ade20k-valOneFormer (DiNAT-L, single-scale, 640x640)#11PQ: 50.5mIoU: 58.3AP: 36.0
panoptic-segmentation-on-ade20k-valOneFormer (ConvNeXt-XL, single-scale, 640x640)#12PQ: 50.1mIoU: 57.4AP: 36.3
panoptic-segmentation-on-ade20k-valOneFormer (ConvNeXt-L, single-scale, 640x640)#13PQ: 50.0mIoU: 56.6AP: 36.2
panoptic-segmentation-on-ade20k-valOneFormer (Swin-L, single-scale, 640x640)#14PQ: 49.8mIoU: 57.0AP: 35.9
panoptic-segmentation-on-cityscapes-testOneFormer (ConvNeXt-L, single-scale, Mapillary Vistas-Pretrained)#1PQ: 68.0
panoptic-segmentation-on-cityscapes-valOneFormer (ConvNeXt-L, single-scale, 512x1024, Mapillary Vistas-pretrained)#2PQ: 70.1PQst: 74.1PQth: 64.6mIoU: 84.6AP: 48.7
panoptic-segmentation-on-cityscapes-valOneFormer (ConvNeXt-L, single-scale)#3PQ: 68.51mIoU: 83.0AP: 46.5
panoptic-segmentation-on-cityscapes-valOneFormer (ConvNeXt-XL, single-scale)#6PQ: 68.4mIoU: 83.6AP: 46.7
panoptic-segmentation-on-cityscapes-valOneFormer (DiNAT-L, single-scale)#8PQ: 67.6mIoU: 83.1AP: 45.6
panoptic-segmentation-on-cityscapes-valOneFormer (Swin-L, single-scale)#11PQ: 67.2mIoU: 83.0AP: 45.6
panoptic-segmentation-on-coco-minivalOneFormer (InternImage-H,single-scale)#2PQ: 60.0PQst: 49.2PQth: 67.1AP: 52.0mIoU: 68.8
panoptic-segmentation-on-coco-minivalOneFormer (DiNAT-L, single-scale)#14PQ: 58.0PQst: 48.4PQth: 64.3AP: 49.2mIoU: 68.1
panoptic-segmentation-on-coco-minivalOneFormer (Swin-L, single-scale)#17PQ: 57.9PQst: 48.0PQth: 64.4AP: 49.0mIoU: 67.4
panoptic-segmentation-on-mapillary-valOneFormer (DiNAT-L, single-scale)#1PQ: 46.7mIoU: 61.7PQst: 54.9PQth: 40.5
panoptic-segmentation-on-mapillary-valOneFormer (ConvNeXt-L, single-scale)#2PQ: 46.4mIoU: 61.6PQst: 54.0PQth: 40.6
semantic-segmentation-on-ade20k-valOneFormer (InternImage-H, emb_dim=256, multi-scale, 896x896)#6mIoU: 60.8
semantic-segmentation-on-ade20k-valOneFormer (DiNAT-L, multi-scale, 896x896)#9mIoU: 58.6
semantic-segmentation-on-ade20k-valOneFormer (DiNAT-L, multi-scale, 640x640)#11mIoU: 58.4
semantic-segmentation-on-ade20k-valOneFormer (Swin-L, multi-scale, 896x896)#13mIoU: 58.3
semantic-segmentation-on-ade20k-valOneFormer (Swin-L, multi-scale, 640x640)#18mIoU: 57.7
semantic-segmentation-on-cityscapes-valOneFormer (ConvNeXt-XL, Mapillary, multi-scale)#10mIoU: 85.8
semantic-segmentation-on-cityscapes-valOneFormer (ConvNeXt-XL, multi-scale)#13mIoU: 84.6
semantic-segmentation-on-cityscapes-valOneFormer (Swin-L, multi-scale)#15mIoU: 84.4
semantic-segmentation-on-coco-1OneFormer (InternImage-H, emb_dim=1024, single-scale)#3mIoU: 68.8
semantic-segmentation-on-coco-1OneFormer (DiNAT-L, single-scale)#5mIoU: 68.1
semantic-segmentation-on-coco-1OneFormer (Swin-L, single-scale)#7mIoU: 67.4
semantic-segmentation-on-mapillary-valOneFormer (DiNAT-L, multi-scale)#2mIoU: 64.9