DoRA: Weight-Decomposed Low-Rank Adaptation

Benchmark Model Rank Results
parameter-efficient-fine-tuning-on-boolqLLaMA2-7b#3Accuracy (% ): 81.93
parameter-efficient-fine-tuning-on-hellaswagLLaMA2-7b#3Accuracy (% ): 76.27
parameter-efficient-fine-tuning-on-winograndeLLaMA2-7b#2Accuracy (% ): 70.09