Polynomial, trigonometric, and tropical activations

Benchmark Model Rank Results
image-classification-on-imagenetConvNeXt-T-Hermite#530Top 1 Accuracy: 82.34Number of params: 28MTop 5 Accuracy: 96.03
language-modelling-on-openwebtextGPT2-Hermite#2eval_perplexity: 18.39eval_loss: 2.91parameters: 124M
language-modelling-on-openwebtextGPT2-Tropical#3eval_perplexity: 18.64eval_loss: 2.92parameters: 124M
language-modelling-on-openwebtextGPT2-Fourier#4eval_perplexity: 18.72eval_loss: 2.93parameters: 124M
language-modelling-on-openwebtextGPT2-GELU#5eval_perplexity: 19.24eval_loss: 2.95parameters: 124M
text-generation-on-openwebtextGPT2-Hermite#1eval_loss: 2.91