When Does Sparsity Mitigate the Curse of Depth in LLMs
Open paper
Benchmark
Model
Rank
Results
common-sense-reasoning-on-arc-easy
Transformer (2.3B, L=32)
#80
Accuracy: 61.74
sentence-completion-on-hellaswag
Transformer (2.3B, L=32)
#185
Accuracy: 48.63
Rank counts only results with a code link.