MARLIN: Masked Autoencoder for facial video Representation LearnINg

Benchmark Model Rank Results
emotion-classification-on-cmu-moseiMARLIN (ViT-L)#1Accuracy: 80.63
emotion-classification-on-cmu-moseiMARLIN (ViT-B)#2Accuracy: 80.6
emotion-classification-on-cmu-moseiMARLIN (ViT-S)#3Accuracy: 80.38
lip-sync-on-lrs2Wav2Lip + ViT + MARLIN#1FID: 3.452LSE-D: 7.127LSE-C: 5.528
deepfake-detection-on-faceforensics-plus-plusMARLIN (ViT-L)#2AUC: 0.9377
deepfake-detection-on-faceforensics-plus-plusMARLIN (ViT-B)#3AUC: 0.9305
deepfake-detection-on-faceforensics-plus-plusMARLIN (ViT-S)#4AUC: 0.8863
multimodal-sentiment-analysis-on-cmu-moseiMARLIN (ViT-L)#8Accuracy: 74.83
multimodal-sentiment-analysis-on-cmu-moseiMARLIN (ViT-B)#9Accuracy: 73.7
multimodal-sentiment-analysis-on-cmu-moseiMARLIN (ViT-S)#10Accuracy: 72.69