SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization
Open paper
Benchmark
Model
Rank
Results
reading-comprehension-on-race
SpectralShift (1.5B-A0.6B GDN-MoE, 32K extension, 10B CPT)
#32
Accuracy: 34.2
Rank counts only results with a code link.