We current SWE-Fitness center, the primary setting for coaching real-world software program engineering (SWE) brokers. SWE-Fitness center incorporates 2,438 real-world Python process situations, every comprising a codebase with an executable runtime setting, unit exams, and a process laid out in pure language. We use SWE-Fitness center to coach language mannequin primarily based SWE brokers, reaching as much as 19% absolute good points in resolve price on the favored SWE-Bench Verified and Lite check units. We additionally experiment with inference-time scaling by verifiers educated on agent trajectories sampled from SWE-Fitness center. When mixed with our fine-tuned SWE brokers, we obtain 32.0% and 26.0% on SWE-Bench Verified and Lite, respectively, reflecting a brand new state-of-the-art for open-weight SWE brokers. To facilitate additional analysis, we publicly launch SWE-Fitness center, fashions, and agent trajectories.
- * Equal contribution
- †College of California, Berkeley
- ‡ Work executed whereas at Apple
- § College of Illinois Urbana-Champaign
- ¶ Carnegie Mellon College
