\n<\/p>\n
Think about going to sleep after writing a single Markdown specification and waking as much as discover that an AI agent ran dozens of LLM fine-tuning experiments in a single day in your behalf – discovering optimum LoRA ranks, refining studying price schedules, tuning batch sizes and committing every verified enchancment to Git.<\/p>\n
That is now not a fantasy. Earlier this 12 months, the autoresearch<\/a> venture showcased how autonomous LLM brokers can iteratively discover pre-training in a self-contained loop. Taking inspiration from this paradigm, we created autofinetune<\/a>: making use of autonomous analysis loops to LLM post-training (Supervised High-quality-Tuning and Reinforcement Studying through GRPO), utilizing Google\u2019s full AI stack\u2014Tunix<\/a>, Gemma, and Cloud TPUs orchestrated with Antigravity CLI<\/a> and Gemini Flash 3.7.<\/p>\n<\/div>\n On this publish, we\u2019ll discover how the autonomous analysis loop works for post-training and stroll by a few real-world LLM finetuning case research.<\/p>\n Conventional post-training entails a repetitive, handbook cycle:<\/p>\n As demonstrated in Within the first experiment in autofinetune<\/a>, we took the identical SFT setup in our earlier weblog<\/a> and prolonged it by creating the autoresearch loop to optimize google\/functiongemma-270m-it<\/a> on the google\/mobile-actions<\/a> dataset.<\/p>\n The agent was given boundaries in program.md<\/a>:<\/p>\n Here’s a pattern trajectory from sample_runs\/SFT_results.tsv<\/a> demonstrating how the agent hill climbed.<\/p>\n<\/div>\n As you possibly can see, the agent is ready to robotically modify LoRA rank\/alpha, optimizer, studying price, and so on. to maintain bettering the mannequin\u2019s accuracy by way of producing right perform calls.<\/p>\n Supervised fine-tuning is barely a easy check. For our second case research, we took the official GRPO instance<\/a> from the Tunix repository (which trains Gemma 3 1B for math reasoning utilizing GSM8K; the skilled mannequin has higher numerical accuracy and format accuracy in its solutions) and set it up for autonomous RL finetuning. Reinforcement studying is topic to hyperparameter sensitivity, instability, and longer execution occasions – making this activity more difficult and time-consuming.<\/p>\n Beneath is a pattern trajectory logged in sample_runs\/RL_results.tsv<\/a>, displaying the agent\u2019s progress. The agent was in a position to determine higher LoRA configurations, rollout temperature, KL penalty, system immediate, and so on. to enhance the overall reward by ~10%.<\/p>\n<\/div>\n We hope this venture reveals you the ability of AI brokers within the area of LLM post-training and conjures up you to consider how one can leverage them to automate your LLM finetuning workflows utilizing Tunix on TPUs. Please try the code, pattern runs, and Think about going to sleep after writing a single Markdown specification and waking as much as discover that an AI agent ran dozens of LLM fine-tuning experiments in a single day in your behalf – discovering optimum LoRA ranks, refining studying price schedules, tuning batch sizes and committing every verified enchancment to Git. That is […]<\/p>\n","protected":false},"author":2,"featured_media":18657,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[56],"tags":[3112,74,9026,7308,9285],"class_list":["post-18655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-software","tag-autonomous","tag-llm","tag-posttraining","tag-tpus","tag-tunix"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/18655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=18655"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/18655\/revisions"}],"predecessor-version":[{"id":18656,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/18655\/revisions\/18656"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/18657"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=18655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=18655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=18655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}The Paradigm Shift: From Handbook Tuning to Autonomous Loops<\/b><\/h2>\n
\n
attn_vec_einsum<\/i><\/code> to the LoRA goal modules enhance accuracy?”<\/i> or “What occurs if we alter rollout temperature throughout GRPO?”<\/i>).<\/li>\nautoresearch<\/code>, we will now automate this complete course of with the ability of AI brokers:<\/p>\n\n
program.md<\/code>)<\/b>: human defines the loop, boundary situations, analysis standards, and constraints.<\/li>\nrun.py<\/code>)<\/b>: A single, clear, self-contained finetuning script.<\/li>\nprogram.md<\/code>: it modifies run.py<\/code>, runs the coaching job, measures the goal metric, retains successful commits or reverts regressions, and logs ends in outcomes.tsv<\/code>.<\/li>\n<\/ul>\n<\/div>\nCase Research 1: Supervised High-quality-Tuning (SFT) on FunctionGemma<\/b><\/h2>\n
The Setup<\/b><\/h3>\n
\n
\n
Pattern Trajectory<\/b><\/h3>\n
Case Research 2: Reinforcement Studying (GRPO) on Gemma for Math Reasoning<\/b><\/h2>\n
The Area Setup<\/b><\/h3>\n
\n
Post_RL_metric<\/code>, which is solely numerical_accuracy + format_accurac<\/code>y (you possibly can in fact use different metrics, i.e., utilizing totally different weights).<\/li>\n<\/ul>\nPattern trajectory<\/b><\/h3>\n
What\u2019s Subsequent?<\/b><\/h2>\n
program.md<\/code> templates within the autofinetune GitHub repository<\/b><\/a>, discover the Tunix library<\/b><\/a>, and begin constructing your personal autonomous post-training lab immediately!<\/p>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"