Think about going to sleep after writing a single Markdown specification and waking as much as discover that an AI agent ran dozens of LLM fine-tuning experiments in a single day in your behalf – discovering optimum LoRA ranks, refining studying price schedules, tuning batch sizes and committing every verified enchancment to Git.
That is now not a fantasy. Earlier this 12 months, the autoresearch venture showcased how autonomous LLM brokers can iteratively discover pre-training in a self-contained loop. Taking inspiration from this paradigm, we created autofinetune: making use of autonomous analysis loops to LLM post-training (Supervised High-quality-Tuning and Reinforcement Studying through GRPO), utilizing Google’s full AI stack—Tunix, Gemma, and Cloud TPUs orchestrated with Antigravity CLI and Gemini Flash 3.7.
On this publish, we’ll discover how the autonomous analysis loop works for post-training and stroll by a few real-world LLM finetuning case research.
The Paradigm Shift: From Handbook Tuning to Autonomous Loops
Conventional post-training entails a repetitive, handbook cycle:
- Formulate a speculation (e.g., “Will including
attn_vec_einsumto the LoRA goal modules enhance accuracy?” or “What occurs if we alter rollout temperature throughout GRPO?”). - Edit coaching scripts and hyperparameters.
- Launch a job on accelerators.
- Monitor loss curves and benchmark evaluations.
- Manually revert failures or document successes in a spreadsheet.
As demonstrated in autoresearch, we will now automate this complete course of with the ability of AI brokers:
- Design the Area (
program.md): human defines the loop, boundary situations, analysis standards, and constraints. - Present the Execution Code (
run.py): A single, clear, self-contained finetuning script. - Let the Agent Iterate: An agent follows directions in
program.md: it modifiesrun.py, runs the coaching job, measures the goal metric, retains successful commits or reverts regressions, and logs ends inoutcomes.tsv.
Case Research 1: Supervised High-quality-Tuning (SFT) on FunctionGemma
Within the first experiment in autofinetune, we took the identical SFT setup in our earlier weblog and prolonged it by creating the autoresearch loop to optimize google/functiongemma-270m-it on the google/mobile-actions dataset.
The Setup
- {Hardware}: Cloud TPU v5e-1
- Iteration Velocity: A couple of minutes per run
- Whole Runs: 20 automated experiments in a few hours for fast experimentation
- Goal Metric: Submit-training analysis accuracy on perform name era (accuracy)
The agent was given boundaries in program.md:
- Allowed: LoRA rank/alpha, goal projection layers, studying charges, warmup/decay schedules, optimizers (e.g., AdamW/Muon, gradient clipping), batch measurement, and seeds.
- Disallowed: Altering the dataset, variety of epochs, or mannequin structure.
Pattern Trajectory
Here’s a pattern trajectory from sample_runs/SFT_results.tsv demonstrating how the agent hill climbed.
As you possibly can see, the agent is ready to robotically modify LoRA rank/alpha, optimizer, studying price, and so on. to maintain bettering the mannequin’s accuracy by way of producing right perform calls.
Case Research 2: Reinforcement Studying (GRPO) on Gemma for Math Reasoning
Supervised fine-tuning is barely a easy check. For our second case research, we took the official GRPO instance from the Tunix repository (which trains Gemma 3 1B for math reasoning utilizing GSM8K; the skilled mannequin has higher numerical accuracy and format accuracy in its solutions) and set it up for autonomous RL finetuning. Reinforcement studying is topic to hyperparameter sensitivity, instability, and longer execution occasions – making this activity more difficult and time-consuming.
The Area Setup
- {Hardware}: Cloud TPU v6e-1
- Mannequin: Gemma 3 1B
- Dataset: GSM8K
- Iteration Velocity: A few hours per run
- Run Period: 40 experiments over 2–3 days
- Goal Metric: to simplify the outer optimization loop for the agent, we selected a single synthetic analysis metric
Post_RL_metric, which is solelynumerical_accuracy + format_accuracy (you possibly can in fact use different metrics, i.e., utilizing totally different weights).
Pattern trajectory
Beneath is a pattern trajectory logged in sample_runs/RL_results.tsv, displaying the agent’s progress. The agent was in a position to determine higher LoRA configurations, rollout temperature, KL penalty, system immediate, and so on. to enhance the overall reward by ~10%.
What’s Subsequent?
We hope this venture reveals you the ability of AI brokers within the area of LLM post-training and conjures up you to consider how one can leverage them to automate your LLM finetuning workflows utilizing Tunix on TPUs. Please try the code, pattern runs, and program.md templates within the autofinetune GitHub repository, discover the Tunix library, and begin constructing your personal autonomous post-training lab immediately!
