CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kCharacterFlywheel V7 (Llama 3.1 70B)–Accuracy: 92.3
code-generation-on-humanevalCharacterFlywheel V7 (Llama 3.1 70B)–Pass@1: 77.4
code-generation-on-mbppCharacterFlywheel V7 (Llama 3.1 70B)–Accuracy: 66.6
math-word-problem-solving-on-mathCharacterFlywheel V7 (Llama 3.1 70B)–Accuracy: 50.5