CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers

Open paper
Benchmark Model Rank Results
camouflaged-object-segmentation-on-pcod-1200CMX#1S-Measure: 0.922
image-manipulation-localization-on-casia-v1CMX (RGB+SRM)#1Average Pixel F1(Fixed threshold): .791
image-manipulation-localization-on-casia-v1CMX (RGB+Bayar)#4Average Pixel F1(Fixed threshold): .774
image-manipulation-localization-on-casia-v1CMX (RGB+NP++)#5Average Pixel F1(Fixed threshold): .761
image-manipulation-localization-on-cocoglideCMX (RGB+SRM)#1Average Pixel F1(Fixed threshold): .585
image-manipulation-localization-on-cocoglideCMX (RGB+Bayar)#3Average Pixel F1(Fixed threshold): .566
image-manipulation-localization-on-cocoglideCMX (RGB+NP++)#6Average Pixel F1(Fixed threshold): .516
image-manipulation-localization-on-columbiaCMX (RGB+NP++)#2Average Pixel F1(Fixed threshold): .884
image-manipulation-localization-on-columbiaCMX (RGB+Bayar)#3Average Pixel F1(Fixed threshold): .872
image-manipulation-localization-on-columbiaCMX (RGB+SRM)#6Average Pixel F1(Fixed threshold): .834
image-manipulation-localization-on-coverageCMX (RGB+SRM)#3Average Pixel F1(Fixed threshold): .630
image-manipulation-localization-on-coverageCMX (RGB+Bayar)#4Average Pixel F1(Fixed threshold): .592
image-manipulation-localization-on-coverageCMX (RGB+NP++)#5Average Pixel F1(Fixed threshold): .577
image-manipulation-localization-on-dso-1CMX (RGB+NP++)#2Average Pixel F1(Fixed threshold): .895
image-manipulation-localization-on-dso-1CMX (RGB+SRM)#4Average Pixel F1(Fixed threshold): .792
image-manipulation-localization-on-dso-1CMX (RGB+Bayar)#5Average Pixel F1(Fixed threshold): .776
multispectral-object-detection-on-flir-1CMX#4mAP50: 82.2%
object-detection-on-dsecCMX#4mAP: 29.1
object-detection-on-eventpedCMX#4AP: 58.0
object-detection-on-inoutdoorCMX#3AP: 62.3
object-detection-on-pku-ddd17-carCMX#12mAP50: 80.4
object-detection-on-stcrowdCMX#3AP: 61.0
pedestrian-detection-on-dvtodCMX#4mAP: 81.6
pedestrian-detection-on-llvipCMX#6AP: 0.596
semantic-segmentation-on-bjroadCMX#2IoU: 62.28
semantic-segmentation-on-cityscapes-valCMX (B4)#31mIoU: 82.6
semantic-segmentation-on-cityscapes-valCMX (B2)#38mIoU: 81.6
semantic-segmentation-on-ddd17CMX#3mIoU: 71.88
semantic-segmentation-on-deliverCMX (RGB-Depth)#10mIoU: 62.67
semantic-segmentation-on-deliverCMX (RGB-Event)#15mIoU: 56.52
semantic-segmentation-on-deliverCMX (RGB-LiDAR)#16mIoU: 56.37
semantic-segmentation-on-dsecCMX#3mIoU: 72.42
semantic-segmentation-on-event-basedCMX#2mIoU: 85.81
semantic-segmentation-on-eventscapeCMX (B4)#1mIoU: 64.28
semantic-segmentation-on-eventscapeCMX (B2)#2mIoU: 61.90
semantic-segmentation-on-gamusCMX#2mIoU: 75.23
semantic-segmentation-on-kitti-360CMX (RGB-Depth)#4mIoU: 64.43
semantic-segmentation-on-kitti-360CMX (RGB-LiDAR)#5mIoU: 64.31
semantic-segmentation-on-llrgbd-syntheticCMX (SegFormer-B2)#3mIoU: 66.52
semantic-segmentation-on-nyu-depth-v2CMX (B5)#13Mean IoU: 56.9%
semantic-segmentation-on-nyu-depth-v2CMX (B4)#17Mean IoU: 56.3%
semantic-segmentation-on-nyu-depth-v2CMX (B2)#26Mean IoU: 54.4%
semantic-segmentation-on-portoCMX#2IoU: 72.85
semantic-segmentation-on-potsdamCMX#1mIoU: 85.97
semantic-segmentation-on-replicaCMX#5mIoU: 17.0
semantic-segmentation-on-scannetv2CMX#1Mean IoU: 61.3%
semantic-segmentation-on-selmaCMX#1mIoU: 91.7
semantic-segmentation-on-spectralwasteCMX (RGB-HYPER)#1mIoU: 58.2
semantic-segmentation-on-spectralwasteCMX ( RGB-HYPER3 )#2mIoU: 56.6
semantic-segmentation-on-stanford2d3d-rgbdCMX (SegFormer-B4)#1mIoU: 62.1Pixel Accuracy: 82.6
semantic-segmentation-on-stanford2d3d-rgbdCMX (SegFormer-B2)#2mIoU: 61.2Pixel Accuracy: 82.3
semantic-segmentation-on-sun-rgbdCMX (B5)#10Mean IoU: 52.4%
semantic-segmentation-on-sun-rgbdCMX (B4)#11Mean IoU: 52.1%
semantic-segmentation-on-sun-rgbdDPLNet#17Mean IoU: 49.7%
semantic-segmentation-on-syn-udtiriCMX#1IoU: 93.31
semantic-segmentation-on-synthetic-bathingCMX-SRA#1mIoU: 94.20
semantic-segmentation-on-synthetic-bathingCMX#2mIoU: 88.23
semantic-segmentation-on-tlcgisCMX#2IoU: 84.14
semantic-segmentation-on-uplightCMX (B2 RGB-AoLP)#2mIoU: 92.13
semantic-segmentation-on-uplightCMX (B2 RGB-DoLP)#3mIoU: 92.07
semantic-segmentation-on-us3dCMX#1mIoU: 84.63
semantic-segmentation-on-vaihingenCMX#1mIoU: 82.87
semantic-segmentation-on-zju-rgb-pCMX (B4 RGB-AoLP)#2mIoU: 92.6
semantic-segmentation-on-zju-rgb-pCMX (B2 RGB-DoLP)#4mIoU: 92.2
thermal-image-segmentation-on-kp-day-nightCMX#3mIoU: 46.2
thermal-image-segmentation-on-mfn-datasetCMX (B4)#6mIOU: 59.7
thermal-image-segmentation-on-mfn-datasetCMX (B2)#14mIOU: 58.2
thermal-image-segmentation-on-noisy-rs-rgb-tCMX (B4)#3mIoU: 56.1
thermal-image-segmentation-on-rgb-t-glassCMX#2MAE: 0.029

Rank counts only results with a code link.