FlexMoRE: A Flexible Mixture of Rank-heterogeneous Experts for Efficient Federatedly-trained Large Language Models
Open paper
Benchmark
Model
Rank
Results
multi-task-language-understanding-on-mmlu-pro
FlexMoRE-a4 (heterogen. MC9)
–
Accuracy: 25.64
Rank counts only results with a code link.