Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Benchmark Model Rank Results
question-answering-on-piqaPowerLM-3B#62Accuracy: 79.9
multi-task-language-understanding-on-mmluPowerLM-3B#65Average (%): 49.2
question-answering-on-boolqPowerLM-3B#65Accuracy: 72.0
common-sense-reasoning-on-winograndePowerLM-3B#81Accuracy: 70.0
sentence-completion-on-hellaswagPowerLM-3B#95Accuracy: 74.6
code-generation-on-humanevalPowerLM-3B#127Pass@1: 26.8
code-generation-on-mbppPowerLM-3B#147Accuracy: 33.6
math-word-problem-solving-on-mathPowerLM-3B (4-shot)#168Accuracy: 15.2
arithmetic-reasoning-on-gsm8kPowerLM-3B (8-shot CoT)#239Accuracy: 49.7