\n
\n<\/p>\n

<\/p>\n

\n… authorities of the folks, by the folks, for the folks …<\/i>
\u00a0\u00a0\u00a0\u00a0\u2014 Abraham Lincoln, Gettysburg Handle (1863)\n<\/p>\n

The price of AI is dropping quickly. GPT-4-class capabilities value roughly $30<\/span> per million tokens in early 2023; right now the identical runs underneath $1<\/span>, and some suppliers are pushing prices under $0.10<\/span><\/a>. Throughout benchmarks, inference costs have fallen between 9x and 900x per 12 months<\/a>, with a median decline close to 50x. Even frontier fashions are getting dramatically cheaper<\/a> every technology, with open-source fashions following carefully behind. And crucially, even when \u201cNobel-Prize-winning genius-level\u201d intelligence isn\u2019t right here but, the intelligence that suffices for the overwhelming majority of information work is right here right now, and getting cheaper by the month. At this fee, we’re quickly coming into the period of nearly free intelligence<\/strong>\u2014the sort that’s greater than sufficient for on a regular basis information work.<\/p>\n

\n $\"A$ \n<\/p>\n

<\/p>\n

\nDisclosure: This publish is a perspective led by Aditya G. Parameswaran<\/a>\u2014an Affiliate Professor of EECS and co-director of the EPIC Information Lab at UC Berkeley\u2014collectively along with his collaborators. It’s half panorama survey and half perspective, and several other of the analysis instructions mentioned under (together with agentic hypothesis, structured reminiscence, and synthesizing customized information programs from scratch) draw on the authors’ personal ongoing work.\n<\/p>\n

So, what does this new period of near-free intelligence imply for information programs? We imagine three new challenges\u2014and alternatives\u2014stem from near-zero inference prices:<\/p>\n

Information Methods For<\/em> Brokers.<\/strong> Brokers will quickly develop into the dominant workload for information programs\u2014with swarms of brokers spun up in response to every end-user request. Given variations in traits between brokers and people\u2014or functions appearing on their behalf\u2014how ought to we redesign information programs for such agentic customers?<\/em><\/p>\n

Information Methods Of<\/em> Brokers.<\/strong> As brokers begin taking over the majority of information work, a brand new substrate is required for 1000’s of brokers to handle state over long-running duties, coordinate and attain consensus, and take care of failures. What do information programs that reliably and effectively run and handle agent swarms appear to be?<\/em><\/p>\n

Information Methods By<\/em> Brokers.<\/strong> Brokers are quickly changing into able to synthesizing total information programs in a single go\u2014that means we are able to rebuild customized programs for every new workload. Verifying that such programs match meant conduct is a problem. What does it take to let brokers synthesize information programs we are able to really belief?<\/em><\/p>\n

\n $\"A$

\nInformation Methods For, Of, and By Brokers
\n<\/i>\n<\/p>\n

Subsequent, we are going to focus on every in additional element, adopted by discussing the intertwined future of knowledge programs and brokers, particularly because the three challenges intersect.<\/p>\n

Information Methods For Brokers<\/h2>\nAn agent querying a database doesn\u2019t behave like an individual or a BI device. It performs what we name agentic hypothesis<\/em><\/a>: a high-volume, heterogeneous stream of labor spanning schema introspection, columnar exploration, partial after which full question formulation. With a number of brokers every exploring parts of the speculation area, every person request might quantity to 1000s of particular person SQL queries. Now, customers can situation \u2018high-level\u2019 information duties, e.g., root-cause evaluation\u2014e.g., \u2018why did espresso gross sales in Berkeley drop this 12 months\u2019\u2014or exploratory cohort evaluation\u2014e.g., \u2018which person segments are most definitely to churn subsequent quarter\u2019\u2014every involving a combinatorial area of potential joins, aggregations, and filter combos.<\/p>\n
\n $\"An$

\nInformation Methods Redesigned to Extra Successfully Help Agentic Hypothesis
\n<\/i>\n<\/p>\n
The requests from these brokers have numerous alternatives for optimization. For example, on a text-to-SQL benchmark with a number of brokers making an attempt every job, solely 10-20% of the sub-plans are distinct. Thus, 80-90% of sub-queries carry out duplicate work. The identical experiments present job success charges considerably growing with extra agentic makes an attempt\u2014so the redundancy is definitely useful. However from the information system perspective it\u2019s wasted work.<\/p>\n
An agent-first information system can exploit such properties to assist brokers make progress sooner. It might probably reuse outcomes throughout overlapping sub-plans, drawing on concepts from decades-old literature on multi-query optimization<\/a> and shared scans<\/a>. Or the information system can attempt to satisfice<\/em>, returning approximate solutions which can be ok for brokers to make progress, leveraging work from the<\/a> AQP<\/a> literature<\/a>\u2014or streaming the outcomes of the ultimate or intermediate operators to assist brokers resolve if seeing the remaining is important or useful.<\/p>\n
One other alternative right here is to rethink the question interface totally: as a substitute of brokers issuing a single SQL question at a time, they might as a substitute situation a batch of queries, every with its personal approximation necessities. Since enumerating an exponential search area (as within the root trigger or cohort evaluation examples above) isn\u2019t use of agentic reasoning means, maybe information programs ought to help higher-level primitives relatively than requiring brokers to record every SQL question explicitly. One concept right here is to attract on DBT-style Jinja macros<\/a> to offer looping-based primitives for brokers to work together with information programs.<\/p>\n
\n $\"A$

\nA Caffeinated Military of Brokers Able to Tirelessly Full Your Information Duties
\n<\/i>\n<\/p>\n
A ultimate alternative right here is to cease pondering of knowledge programs as passive executors of queries; information programs could possibly be proactive<\/a>, as they possess extra grounding in information and system traits that brokers could lack a priori\u2014they might steer brokers in numerous instructions, present outcomes for associated queries, and likewise present performance-level suggestions (e.g., as a substitute of executing an costly question, the system might first present the agent a latency estimate). The rationale we are able to do that now versus the previous is that an agent can settle for any type of textual suggestions and isn\u2019t anticipating a strict SQL question outcome. In truth, the information system might additionally put together each materialized and digital views for an agent prematurely, supplied to the agent as a part of context, as this can be cheaper or more practical than having an agent writer or use them.<\/p>\n
Information Methods Of Brokers<\/h2>\n
Beforehand, we centered on how brokers work together with information programs. Now, we contemplate every little thing else brokers have to maintain working: the place they stay, how they keep in mind, how they coordinate with one another, and the way they take care of failures of one another. This agentic substrate<\/em> is separate from the inference stack powering uncooked intelligence. Nevertheless, the inference stack itself is being abstracted away by APIs (e.g., from OpenAI or Anthropic), or, for open-weight fashions, by serving<\/a> frameworks<\/a> that disguise low-level particulars. To date, the agentic substrate has been managed by harnesses like Claude Code<\/a> and Codex<\/a>, coupled with numerous mechanisms to retailer<\/a> and retrieve<\/a> reminiscence.<\/p>\n
First, on the reminiscence entrance, the present knowledge is that recordsdata<\/a> are all you want<\/a>; brokers write to unstructured markdown (MD) recordsdata, which may then be searched utilizing grep, or through embedding-based retrieval. In truth, many argue that the answer to continuous studying is having brokers devour so much (e.g., a complete codebase, slack, firm wikis, \u2026) after which write their learnings into MD recordsdata, that are then retrieved selectively on demand. Certainly, file programs, bash scripting, and MD recordsdata are and can nonetheless be essential for brokers. Nevertheless, at scale, when brokers are doing the overwhelming majority of information work, this method will not be efficient.<\/p>\n
Given restricted context home windows, retrieving all MD file fragments that could be related and stuffing it into the context will break down in some unspecified time in the future. Even when context home windows proceed to develop, there are latency advantages to not put all data into context \u2014 and in lots of instances, e.g., when information work includes interacting with giant databases or code bases, it is going to be infeasible to serialize all related information into context.<\/p>\n
\n $\"A$

\nInformation Methods As A Substrate for Multi-Agent Swarms
\n<\/i>\n<\/p>\n
One might use a information<\/a> graph<\/a> illustration<\/a>, however information graphs undergo from the identical limitations as unstructured MD-based reminiscence because of their lack of structured search. What one wants is to have the ability to retrieve solely reminiscence that’s pertinent to the duty, throughout a number of attributes (or aspects) of curiosity. For instance, an agent debugging a flaky take a look at ought to be capable to pull solely the reminiscences tagged with the related module, language, framework, and failure mode\u2014relatively retrieving primarily based on key phrases or embedding similarity. A separate situation is what to really retrieve; uncooked agent traces with errors will not be very helpful as they’ll induce brokers to repeat the identical mistake\u2014as a substitute, we wish the retrieved reminiscence to be corrective.<\/p>\n
We not too long ago explored a associated notion of structured reminiscence<\/em><\/a>, the place we set up reminiscence throughout numerous attributes, every of which could possibly be set as *<\/code> to point common applicability, or set as an inventory of values to be matched. For a knowledge agent, the size might embody the columns and tables, sort of operation, and at last, open-ended natural-language corrective directions. So, we might embody reminiscence that solely applies to a given sort of operation (e.g., \u2018when performing date-time operations, use fiscal 12 months versus calendar 12 months conventions\u2019), or a given desk (e.g., \u2018column product_cleaned is most popular over column product when querying on product title\u2019). One open query is defining an application-specific structured reminiscence<\/em>\u2014or what others have referred to as world fashions for reminiscence<\/a>. We imagine that is akin to defining a schema for every software\u2014and maybe brokers themselves will help us outline and refine it over time.<\/p>\n
\n \nOne Attainable Approach To Retailer and Retrieve Structured Information [From Here]<\/a> \n<\/i>\n<\/p>\n
Structured reminiscence shall be helpful additionally for evolutionary<\/a> frameworks<\/a> to successfully handle search areas. Certainly, storing, structuring, and mining giant volumes of single and multi-agent traces<\/a> will help future brokers develop into rather more environment friendly\u2014probably enabling efficient recursive self-improvement by structured memory-based mechanisms.<\/p>\n
One other problem is to help concurrent edits to shared reminiscence, and concurrent edits usually, when there are numerous brokers performing transformations. Whereas there have been some helpful makes an attempt at supporting<\/a> multiversioning<\/a> and copy-on-write semantics<\/a>, it isn\u2019t clear that such strategies will suffice when 1000’s of brokers try to edit shared state on the identical time. For example, when brokers are attempting numerous potential transactions in response to a person request, the results of the overwhelming majority of those transactions have to be rolled again\u2014with solely the one \u2018appropriate\u2019 transaction\u2019s outcome persisting. Work on supporting exactly-once semantics is related right here, as are underlying strategies primarily based on CRDTs and operational transformation. For updates to fuzzy mechanisms comparable to reminiscence, we might be able to sacrifice on consistency for good correctness within the curiosity of latency. Whereas brokers can motive about semantics to compensate or roll again their actions to ultimately finalize most duties, the first problem lies within the diploma to which they step on one another\u2019s toes throughout the course of. An essential failure mode to be averted is a type of \u201clivelock,\u201d the place incessant compensating actions stop any significant progress.<\/p>\n
Past shared state, different considerations emerge when making an attempt to help a military of brokers, together with what to do when brokers fail, how brokers ought to talk with one another (instantly or by intermediate shared state), and the way we must always take care of straggler brokers. There have been some developments in supporting sturdy multi-agent execution, comparable to Temporal<\/a>, nevertheless it stays to be seen if such options will apply at scale throughout 1000’s of brokers. On the subject of communication, we’d like mechanisms to allow brokers to barter with one another. Think about 4 developer brokers making an attempt to succeed in consensus on a shared schema, with distinct however overlapping aims. In a human setting, this might contain iterative dialogue and compromise; for agentic swarms, we should outline the mechanisms that enable them to converge on a design that displays the underlying objectives of their respective principals. Or if brokers are all requiring entry to a restricted useful resource, once more communication shall be crucial. It stays to be seen if that is greatest accomplished through centralized coordination, or if a decentralized method is important.<\/p>\n
Information Methods By Brokers<\/h2>\nLastly, if intelligence is successfully free, then we are able to make use of this intelligence to synthesize new information programs from scratch. Certainly, in lots of settings, general-purpose information programs could also be overkill, as they must help each schema, question, and {hardware} goal. Given a workload, current work, together with Bespoke OLAP<\/a> and GenDB<\/a>, has proven that one can use an agentic pipeline to synthesize a whole, workload-specific analytical engine\u2014in minutes to some hours, at a value of some {dollars}. The engines are disposable: when the workload shifts, one can merely regenerate them. Analogously, our work has proven that one can synthesize customized key-value shops<\/a> from scratch, focused to the workload. In truth, trendy IDEs, comparable to Kiro<\/a>, elevate specs for programs growth to be a first-class citizen.<\/p>\n \n \nBrokers Can Synthesize Customized Information Methods From Scratch \n<\/i>\n<\/p>\n The primary situation, nevertheless, is that specs are usually imperfect, and don\u2019t cowl all nook instances. Current-day brokers will exploit the lacking specs to reward-hack their solution to a excessive efficiency metric. In our customized key-value retailer work, we discovered that one solution to alleviate that is to have auxiliary verification brokers making an attempt to generate take a look at instances that catch the exploitation of nook instances, primarily increasing the specification. One more method is to each generate a system and a proof for its correctness collectively, for which we now have discovered some early success<\/a>, however extra must be accomplished to solidify the method. Additional, it stays to be seen what’s one of the best ways to solicit human-written specs for a system\u2014can this be accomplished in an iterative, human-in-the-loop method, versus a one-shot, incomplete one. Certainly, human-written specs are incomplete even for manually authored software program, so one would anticipate that future brokers which can be extra aligned will more and more train higher judgement when making design selections.<\/p>\n \n \nOne Attainable Information System Synthesis Pipeline [From Here]<\/a> \n<\/i>\n<\/p>\n Different questions right here contain testing whether or not ranging from a mature system (e.g., Postgres) and eradicating elements\/performance can result in greater efficiency or extra person belief. Individually, is there a chance to make the design composable, comprising numerous verified elements which can be blended and matched given a workload? For instance, maybe the workload hasn\u2019t modified sufficient for the storage layer to be up to date, however maybe the question optimizer requires modifications. A maybe extra viable proposition includes using brokers coupled with proof programs to focus on essential elements of the code related to formal proofs, relatively than doing so for the complete system.<\/p>\n A ultimate alternative right here is to maneuver away from the standard information programs stack with clearly-defined interfaces (e.g., parser, question optimizer, storage supervisor, \u2026) \u2014 that had been every largely the prerogative of a single human workforce to handle. As an alternative, brokers can discover new methods to \u201cmix\u201d these elements collectively, maybe figuring out new optimization alternatives in consequence. Brokers may fill in lacking gaps in performance to make present programs rather more feature-complete, or attain feature-parity with different competing programs\u2014or analogously, constantly refining open-source programs in response to function requests or points (maybe filed by different brokers!) Doing so in a approach that prioritizes correctness, long-term upkeep, and human interpretability shall be a problem.<\/p>\n Wanting Additional Forward<\/h2>\nWithin the period of near-free intelligence, information programs matter greater than ever. As brokers tackle the majority of information work, the workload for information programs will change, the substrate they should run on must be constructed, and more and more, they’ll take part in designing information programs themselves. Every of those shifts opens up a brand new, thrilling analysis agenda.<\/p>\n \n \nCo-Evolution of Information Methods and Brokers \n<\/i>\n<\/p>\n Wanting additional out, the boundaries between brokers and information programs will probably begin to blur. For example, brokers could design the information programs they themselves run on, defining each the interfaces in addition to the system elements beneath. Each the interfaces and internals might be developed over time by brokers in a type of recursive self-improvement. There may be additionally a chance to rethink information programs as a holistic supply of fact for everything of related state: together with uncooked information, reminiscence, and coordination state, additional erasing the distinctions between the information that’s being queried by brokers and information generated because of agentic exercise. Lastly, information programs could themselves incorporate agentic elements, basically evolving from passive computation engines into clever, proactive, self-optimizing architectures. It’s onerous to foretell what the long run could maintain. We\u2019re in for a wild experience!<\/p>\nAcknowledgments<\/h2>\nThe attitude and ongoing work described on this publish are the product of joint analysis and plenty of discussions with fantastic collaborators on the EPIC Information Lab<\/a>, Information Methods & Foundations<\/a> group, and the broader Berkeley AI-Methods neighborhood. Thanks all!<\/p>\n BibTex for this publish:<\/p>\n \n\n@misc{intelligence-is-free-blog,\n title={Intelligence is Free, Now What? Information Methods for, of, and by Brokers},\n writer={Aditya G. Parameswaran and Shubham Agarwal and Kerem Akillioglu and Shreya Shankar\n and Sepanta Zeighami and Rishabh Iyer and Matei Zaharia and Alvin Cheung\n and Natacha Crooks and Joseph Gonzalez and Joseph Hellerstein and Ion Stoica},\n howpublished={url{https:\/\/bair.berkeley.edu\/weblog\/2026\/07\/07\/intelligence-is-free-now-what\/}},\n 12 months={2026}\n}\n<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"… authorities of the folks, by the folks, for the folks …\u00a0\u00a0\u00a0\u00a0\u2014 Abraham Lincoln, Gettysburg Handle (1863) The price of AI is dropping quickly. GPT-4-class capabilities value roughly $30 per million tokens in early 2023; right now the identical runs underneath $1, and some suppliers are pushing prices under $0.10. Throughout benchmarks, inference costs have […]<\/p>\n","protected":false},"author":2,"featured_media":16487,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[617,311,310,110,157,160,312,193,140],"class_list":["post-16485","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-agents","tag-artificial","tag-berkeley","tag-blog","tag-data","tag-free","tag-intelligence","tag-research","tag-systems"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/16485","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16485"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/16485\/revisions"}],"predecessor-version":[{"id":16486,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/16485\/revisions\/16486"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/16487"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16485"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16485"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16485"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}