Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet

Benchmark Model Rank Results
image-classification-on-imagenetFF#905Top 1 Accuracy: 74.9