LFM2.5-24B-A2B: Built for Fast On-Device Agents

Benchmark Model Rank Results
agents-on-automationbench-publicLFM2.5-24B-A2B–Success Rate: 27.69
agents-on-bfclv4LFM2.5-24B-A2B–Accuracy: 65.01
agents-on-pinchbenchLFM2.5-24B-A2B–Accuracy: 83.18
code-generation-on-scicodeLFM2.5-24B-A2B–Accuracy: 32.71
code-generation-on-swe-bench-verifiedLFM2.5-24B-A2B (OpenCode)–Accuracy: 51.00
instruction-following-on-ifbenchLFM2.5-24B-A2B–Accuracy: 74.69
instruction-following-on-multi-ifLFM2.5-24B-A2B–Accuracy: 84.38
mathematical-reasoning-on-math-500LFM2.5-24B-A2B–Accuracy: 96.04
multiple-choice-question-answering-mcqa-on-gpqa-diamondLFM2.5-24B-A2B–Accuracy: 74.42
question-answering-on-humanity-s-last-exam-hleLFM2.5-24B-A2B–Accuracy: 7.40