Stepped MoE: Segment-Level Routing with Configurable Inference Complexity
Open paper
Benchmark
Model
Rank
Results
code-generation-on-humaneval
Flexible S-MoE 12B→4B (extra data)
–
Pass@1: 46.1
multi-task-language-understanding-on-mmlu
Flexible S-MoE 4B/20B
–
Average (%): 65.0
Rank counts only results with a code link.