Graph Convolutions Enrich the Self-Attention in Transformers!

Open paper
Benchmark Model Rank Results
graph-regression-on-pcqm4m-lscGraphormer + GFSA#9Validation MAE: 0.1193
graph-regression-on-pcqm4mv2-lscGraphormer + GFSA#12Validation MAE: 0.0860
speech-recognition-on-librispeech-test-cleanBranchformer + GFSA#87Word Error Rate (WER): 2.11
speech-recognition-on-librispeech-test-otherBranchformer + GFSA#94Word Error Rate (WER): 4.94
image-classification-on-imagenetSwin-S + GFSA#512Top 1 Accuracy: 83%
image-classification-on-imagenetCaiT-S + GFSA#535Top 1 Accuracy: 82.8%
image-classification-on-imagenetDeiT-S-24 + GFSA#673Top 1 Accuracy: 81.5%
image-classification-on-imagenetDeiT-S-12 + GFSA#707Top 1 Accuracy: 81.1%

Rank counts only results with a code link.