Deep Equilibrium Models

Benchmark Model Rank Results
language-modelling-on-penn-treebank-wordDEQ-TrellisNet#28Test perplexity: 57.1Params: 24M
language-modelling-on-wikitext-103DEQ-Transformer (medium, adaptive embed)#49Test perplexity: 23.2Number of params: 110M
language-modelling-on-wikitext-103DEQ-TrellisNet#64Test perplexity: 29.0Number of params: 180M
language-modelling-on-wikitext-103DEQ-Transformer (small)#68Test perplexity: 32.4Number of params: 138M