MARLIN: Masked Autoencoder for facial video Representation LearnINg

Benchmark Model Rank Results
deepfake-detection-on-faceforensics-1MARLIN (ViT-L)#2AUC: 0.9377
deepfake-detection-on-faceforensics-1MARLIN (ViT-B)#3AUC: 0.9305
deepfake-detection-on-faceforensics-1MARLIN (ViT-S)#4AUC: 0.8863
emotion-classification-on-cmu-moseiMARLIN (ViT-L)#1Accuracy: 80.63
emotion-classification-on-cmu-moseiMARLIN (ViT-B)#2Accuracy: 80.6
emotion-classification-on-cmu-moseiMARLIN (ViT-S)#3Accuracy: 80.38
lip-sync-on-lrs2Wav2Lip + ViT + MARLIN#1FID: 3.452LSE-D: 7.127LSE-C: 5.528
multimodal-sentiment-analysis-on-cmu-mosei-1MARLIN (ViT-L)#8Accuracy: 74.83
multimodal-sentiment-analysis-on-cmu-mosei-1MARLIN (ViT-B)#9Accuracy: 73.7
multimodal-sentiment-analysis-on-cmu-mosei-1MARLIN (ViT-S)#10Accuracy: 72.69