Graph Convolutions Enrich the Self-Attention in Transformers!

Benchmark Model Rank Results
graph-regression-on-pcqm4m-lscGraphormer + GFSA#9Validation MAE: 0.1193
graph-regression-on-pcqm4mv2-lscGraphormer + GFSA#12Validation MAE: 0.0860
image-classification-on-imagenetSwin-S + GFSA#466Top 1 Accuracy: 83%
image-classification-on-imagenetCaiT-S + GFSA#488Top 1 Accuracy: 82.8%
image-classification-on-imagenetDeiT-S-24 + GFSA#613Top 1 Accuracy: 81.5%
image-classification-on-imagenetDeiT-S-12 + GFSA#638Top 1 Accuracy: 81.1%
speech-recognition-on-librispeech-test-cleanBranchformer + GFSA#79Word Error Rate (WER): 2.11
speech-recognition-on-librispeech-test-otherBranchformer + GFSA#82Word Error Rate (WER): 4.94