\n<\/p>\n
Reinforcement studying has emerged as a strong paradigm for unlocking reasoning capabilities in massive language fashions. Nevertheless, counting on sparse rewards makes this course of extremely sample-inefficient, as fashions should navigate huge search areas with minimal suggestions. Whereas basic curriculum studying goals to mitigate this by ordering information primarily based on complexity, the fitting ordering for a particular mannequin is commonly unclear. To deal with this, we suggest Goldilocks, a novel teacher-driven information sampling technique that goals to foretell every query\u2019s problem for the scholar mannequin. The trainer mannequin selects questions of applicable problem for the scholar mannequin, i.e., questions which are neither too simple nor too onerous (Goldilocks precept), whereas coaching the scholar with GRPO. By leveraging the scholar\u2019s efficiency on seen samples, the trainer repeatedly adapts to the scholar\u2019s evolving talents. On OpenMathReasoning dataset, Goldilocks information sampling improves the efficiency of fashions skilled with commonplace GRPO beneath the identical compute funds.<\/p>\n
Reinforcement studying has emerged as a strong paradigm for unlocking reasoning capabilities in massive language fashions. Nevertheless, counting on sparse rewards makes this course of extremely sample-inefficient, as fashions should navigate huge search areas with minimal suggestions. Whereas basic curriculum studying goals to mitigate this by ordering information primarily based on complexity, the fitting ordering […]<\/p>\n","protected":false},"author":2,"featured_media":12979,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[2895,1997,8328,616,3495,8329,5296,2914],"class_list":["post-12977","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-difficulty","tag-escape","tag-goldilocks","tag-reasoning","tag-rewards","tag-sparse","tag-task","tag-tuning"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/12977","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12977"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/12977\/revisions"}],"predecessor-version":[{"id":12978,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/12977\/revisions\/12978"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/12979"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12977"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=12977"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=12977"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}