MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
Open paper
Benchmark
Model
Rank
Results
sentence-completion-on-hellaswag
MatryoshkaLora (Llama-3.1-8B-Instruct)
#138
Accuracy: 62.8
arithmetic-reasoning-on-gsm8k
MatryoshkaLora (Llama-3.1-8B-Instruct)
#160
Accuracy: 80.5
Rank counts only results with a code link.