There’s a variety of pleasure proper now about agent frameworks,
\n orchestration patterns, and protocols. All of it issues, however nearly none of
\n it delivers worth in the event you skip the info layer. Earlier than any agent framework can
\n produce helpful outcomes, your knowledge must be in a form {that a} machine can
\n devour, belief, and act on. On this article, we focus on what your
\n knowledge must appear to be for agentic AI to derive worth from it.<\/p>\n
We have spent fairly a little bit of time constructing knowledge architectures for the human For over three many years, we have been constructing knowledge techniques for people. A human analyst is aware of what \u201cincome\u201d means in your explicit A human hesitates at knowledge that appears improper; an agent acts on it Brokers have none of it. They can not lean on the tribal information and For a human client, the info solely needed to be adequate<\/i>; the All 5 come right down to the identical concept. Every is a job people used to do None of those attributes builds itself. The remainder of the article works We’ll take them one subject at a time, and present what it takes to construct People have a scent check for dangerous knowledge. They discover when a quantity Take into account a concrete situation. A pricing agent is requested for the The agent would not hesitate, it retrieves $49.99, quotes the The leaders most assured their knowledge is AI-ready additionally title knowledge A human gross sales rep would have paused: \u201cWait, did not we replace this So how do you forestall AI brokers from accessing dangerous or stale This reverses a decade of \u201cschemaless is versatile\u201d considering, Within the Open Information Contract Normal, that contract is proven Enforcement occurs alongside three dimensions.<\/p>\n Discover how this adjustments the sooner pricing situation, it Defining a contract is one factor. What occurs when knowledge The circulation works like this. Uncooked knowledge arrives from supply If it passes all three, it flows into the licensed, agent Unhealthy knowledge lands in a dead-letter queue, by no means in entrance of the The purpose is that the agent by no means sees the dangerous knowledge. It A medallion structure is an analytical knowledge design sample for Unhealthy knowledge will get quarantined, however the place does the good<\/i> knowledge go? For agentic architectures, there is a helpful fourth tier value Determine 1: Medallion tiers for brokers: Bronze and Silver are for people; brokers see solely Gold and The important thing architectural precept is that brokers ought to solely Every little thing to this point has appeared like a desk, costs, currencies, The stale-price situation has a twin right here. A coverage doc will get Contracts<\/i> transfer from the content material to the encircling metadata. High quality gates<\/i> get checks suited to textual content, reject empty or Whether or not the info is a priced row or an embedded paragraph, the Contracts, quarantine, and the medallion structure deal with the The agent processes a request and assesses knowledge high quality alerts, Let’s return to the pricing situation one final time. The worth \u201cI am not assured this worth is present. Routing to a human for Information high quality alerts ought to drive the In different phrases, knowledge high quality alerts ought to drive the The laborious half is popping these high quality alerts right into a single You do not have to construct all of this without delay, and most groups We have made knowledge reliable. However when brokers act autonomously on that knowledge, Even with excellent knowledge, autonomous motion raises a tougher query, when a Image a financial institution working agentic AI for commerce finance, the place the governance An agent processes a letter of credit score. It checks KYC knowledge, verifies the Conventional audit logs can inform you what<\/b> occurred, Conventional audit logs can inform you what<\/i> occurred, which tables have been The best way to shut this audit hole is agentic lineage<\/i>, an extension of Concretely, for the commerce finance case, a single hint represents the That is precisely what a regulator wants. Not \u201cthe This is not a theoretical train. The EU AI Act is essentially the most Collectively, Articles 12 and 19 translate into three obligations to your The EU is furthest forward, and for now no different jurisdiction has a legislation fairly Understanding you want audit trails is one factor; rolling this out safely is
\n client. We’re about at hand these architectures to a really completely different sort
\n of client, and most of them aren’t prepared for it.<\/p>\nThe customers of your knowledge are altering<\/h2>\n
\n Dashboards, experiences, analyst queries, all of it designed round an individual
\n sitting in entrance of a display screen. And it labored, as a result of people convey an
\n huge quantity of implicit context, together with the curiosity to trace
\n down no matter they’re lacking from the individuals round them.<\/p>\n
\n group. They know which tables to question and which of them to keep away from.
\n They discover when a quantity seems to be off, when a complete is suspiciously spherical,
\n when a date falls on a public vacation, or when a worth appears too low. That
\n intuition is doing a considerable amount of invisible context and information
\n work.<\/p>\n
\n anyway<\/p>\n<\/div>\n
\n sample recognition individuals accumulate over years, in order that they want context
\n made express, entry in actual time, and high quality they’ll depend on. And the
\n distinction that issues most is that this: when the info feels improper, a human
\n double-checks; an agent confidently acts on it. That behavioral hole is
\n what the remainder of this dialogue is constructed round.<\/p>\n<\/section>\nWhat \u201cAI-ready\u201d has to imply now<\/h2>\n
\n analyst did the remaining. The that means, the sanity test, and the judgment
\n about whether or not a quantity might be trusted all lived in an individual’s head.
\n When the identical knowledge is handed to an agent, each little bit of that implicit
\n labor has to maneuver into the info itself. That exhibits up as 5
\n attributes, every the flip facet of one thing a human used to do for
\n free.<\/p>\n\n
\n acts on it. The arrogance a human used to produce must be inbuilt, so the
\n knowledge should be correct, recent, and validated earlier than the agent ever sees it.<\/li>\n
\n taken out, and that your fiscal 12 months begins in February; an agent must be instructed
\n each. Which means that used to reside in somebody’s head must be made express within the
\n knowledge.<\/li>\n
\n an agent decides in 30 seconds, that reasoning is gone except you seize it as
\n it occurs. You’ve gotten to have the ability to reconstruct what the agent did and why.<\/li>\n
\n an agent’s must be bounded by design. Entry should be scoped, managed, and
\n auditable.<\/li>\n
\n one thing; an agent must be ready<\/i> to do the one thing. The information cannot simply be
\n readable, it must be actionable.<\/li>\n<\/ul>\n
\n with out considering, now pushed into the info itself. Miss one, and the
\n agent will not degrade gracefully the way in which an individual would. It fails
\n confidently.<\/p>\n
\n by 4 subjects that do, roughly within the order you must sort out
\n them.<\/p>\n\n
\n as a result of a single improper reality poisons each layer constructed on high of it.<\/li>\n
\n what it will possibly attain, making knowledge Traceable<\/i> and Ruled<\/i>.<\/li>\n
\n making knowledge Contextual<\/i>.<\/li>\n
\n write again, making knowledge Operational<\/i>.<\/li>\n<\/ol>\n
\n every attribute in. Work by all 4, and the 5 attributes cease
\n being summary objectives. They turn out to be one thing you may engineer,
\n turning bizarre knowledge into AI-ready knowledge.<\/p>\n<\/section>\nInformation Contracts and High quality: Brokers Cannot Scent Unhealthy Information<\/h2>\n
\n seems to be off, when a date is senseless, or when a worth appears improper.
\n Brokers haven’t any such intuition. As Simon Willison places it, language fashions are gullible<\/a>, they consider
\n no matter they’re handed and act on it. Feed an AI agent a improper
\n worth, and it will not pause to surprise, it can use the quantity and
\n produce a assured, improper reply. With out trusted knowledge, nothing else
\n in agentic AI works, so that is the place we start.<\/p>\nBrokers deal with each worth as reality<\/h3>\n
\n present worth of Product X<\/i>. Yesterday, the worth was up to date from
\n $49.99 to $59.99. However the agent’s knowledge supply hasn’t refreshed, it
\n nonetheless exhibits the previous quantity.<\/p>\n
\n buyer, the shopper buys, and the corporate loses $10 on each unit
\n bought. Each step the agent took was technically appropriate. It adopted
\n its workflow completely. The knowledge<\/i> it accessed was the
\n downside.<\/p>\n
\n readiness their largest barrier<\/p>\n<\/div>\n
\n final week?\u201d They’d double-check. They’ve institutional reminiscence and a
\n really feel for when one thing’s off. The agent has neither. Errors do not
\n set off warnings; they cascade silently by the workflow. And
\n this is not a uncommon edge case. Within the 2026 State
\n of Information Integrity and AI Readiness<\/a> report, Exactly and Drexel
\n College’s LeBow School of Enterprise surveyed 505 knowledge and analytics
\n leaders, of whom 87% believed their knowledge was prepared for AI, but 43% named
\n knowledge readiness as the one largest barrier to getting worth from it. That
\n hole between confidence and readiness is the organization-level model
\n of the pricing agent, certain of itself and improper. A separate KPMG World AI Pulse<\/a> survey of two,145 leaders factors the
\n identical method, with almost half of executives now seeing AI’s prices exceed its
\n advantages. Most enterprises are one stale area away from the situation
\n above.<\/p>\n<\/section>\nSchema is legislation: knowledge contracts as code<\/h3>\n
\n knowledge? The reply is knowledge contracts<\/i>, treating schema as legislation,
\n not a well mannered suggestion.<\/p>\n
\n for human customers, free schemas are merely inconvenient, whereas
\n for AI brokers, they’re harmful. An information contract, written within the
\n Open Information Contract
\n Normal<\/a>,
\n the format the Information Contract CLI<\/a>
\n makes use of (and advisable in Thoughtworks tech radar
\n 33<\/a>),
\n defines the foundations explicitly. A product_pricing<\/code> contract may
\n specify:<\/p>\n\n
worth<\/code> should be better than zero.<\/li>\nforeign money<\/code> that rejects something exterior USD, EUR, or
\n GBP.<\/li>\n
\n inside the final 24 hours.<\/li>\n<\/ul>\n
\n under.<\/p>\napiVersion: v3.1.0\nsort: DataContract\nid: product-pricing\ntitle: Product Pricing\nmodel: 1.0.0\nstanding: lively\nschema:\n - title: product_pricing\n physicalType: desk\n properties:\n - title: product_id\n logicalType: string\n physicalType: varchar(64)\n required: true\n distinctive: true\n primaryKey: true\n primaryKeyPosition: 1\n - title: worth\n logicalType: quantity\n physicalType: decimal\n required: true\n high quality:\n - sort: sql\n description: Each worth should be better than zero\n question: SELECT min({property}) FROM {object}\n mustBeGreaterThan: 0\n - title: foreign money\n logicalType: string\n physicalType: varchar(3)\n required: true\n high quality:\n - sort: sql\n description: Forex should be a supported ISO code\n question: SELECT depend(*) FROM {object} WHERE {property} NOT IN ('USD', 'EUR', 'GBP')\n mustBe: 0\n - title: ingested_at\n logicalType: timestamp\n physicalType: timestamp\n required: true\nslaProperties:\n # the rule that might have caught the stale-price situation\n - property: latency\n worth: 24\n unit: h\n component: product_pricing.ingested_at\n<\/pre>\n\n
\n express by the contract.<\/li>\n
\n nightly batch updates aren’t sufficient when an agent solutions in actual time.
\n Key the SLA to when the info was final efficiently loaded, not when a worth final
\n modified, in order that regular knowledge is not flagged as stale and a stalled pipeline cannot
\n masquerade as recent.<\/li>\n
\n they fail.<\/li>\n<\/ul>\n
\n prevents it by design. If the pricing knowledge hasn’t been refreshed
\n in 24 hours, the contract is violated earlier than the agent ever sees
\n the info.<\/i><\/p>\n<\/section>\nThe quarantine sample<\/h3>\n
\n violates it? You want a circuit breaker and that is the
\n quarantine sample.<\/p>\n
\n techniques, APIs, databases, streams. Earlier than it enters the agent
\n accessible knowledge retailer, it passes by a contract validation
\n gate that checks three issues, does it match the schema, is it
\n inside the freshness SLA, and does it cross the standard
\n guidelines?<\/p>\n
\n prepared tier. If it fails any one in all them, it is quarantined,
\n routed to a lifeless letter queue for human assessment, with alerts
\n fired.<\/p>\n
\n agent<\/p>\n<\/div>\n
\n would not get poisoned by stale costs or corrupted embeddings. In
\n the pricing situation, if the ingested_at<\/code> timestamp is older
\n than 24 hours the contract is violated and the report is
\n quarantined, so when requested concerning the worth the agent says, \u201cI
\n haven’t got present pricing knowledge\u201d slightly than confidently quoting
\n the improper quantity. That could be a much better failure mode. And it is a job
\n for the info structure, not the mannequin. A greater mannequin will not rescue
\n you from dangerous knowledge.<\/p>\n<\/section>\nMedallion structure for brokers<\/h3>\n
\n organizing knowledge in a lakehouse<\/a>,
\n popularized by Databricks<\/a>.<\/p>\n
\n That is what the medallion structure organizes, and its first three
\n tiers are properly established:<\/p>\n\n
\n and lineage.<\/li>\n
\n are enforced, and that is the place the quarantine sample lives.<\/li>\n
\n entry is ruled, and metrics are trusted.<\/li>\n<\/ul>\n
\n including: Adaptive Gold<\/b> the place brokers turn out to be lively
\n members in knowledge curation slightly than passive customers (proven
\n within the determine under). They monitor their very own question patterns,
\n establish often accessed combos, and materialize optimized
\n datasets, successfully constructing their very own warehouse views primarily based on
\n actual utilization. The concept brokers can actively curate knowledge, slightly
\n than solely learn it, is already in manufacturing, at DataHub’s CONTEXT 2025 summit<\/a>, Apple
\n described brokers performing as \u201cdigital stewards\u201d of its knowledge catalog,
\n repeatedly scanning metadata, flagging gaps, and proposing
\n updates, turning passive documentation into an lively governance
\n companion. Apple’s brokers curate the catalog<\/i>; Adaptive Gold factors
\n that very same active-curation sample on the datasets<\/i> themselves. That
\n final step is an extrapolation, however a modest one from one thing
\n already working.<\/p>\n![](\"https:\/\/martinfowler.com\/articles\/making-data-ready-for-agentic-ai\/medallion.svg\")
<\/p>\n
\n knowledge flows from uncooked Bronze by validated Silver to licensed
\n Gold and agent curated Adaptive Gold, whereas brokers are restricted to
\n Gold and above.<\/p>\n<\/div>\n
\n above<\/p>\n<\/div>\n
\n entry Gold tier or above. Bronze and Silver exist for
\n lineage, debugging, and human investigation. Exposing uncooked or
\n partially validated knowledge to brokers invitations the
\n pricing downside again in.<\/p>\n<\/section>\nThe identical guidelines for unstructured knowledge<\/h3>\n
\n timestamps, however most of what brokers devour is not tabular. It is
\n paperwork, wikis, PDFs, and help tickets, chunked and embedded
\n right into a vector retailer for retrieval. In case your brokers do RAG, that is
\n the info they run on, and it wants the identical belief ensures, even
\n although you may’t write worth > 0<\/code> on a paragraph. The patterns
\n carry over, solely the standard dimensions change.<\/p>\n
\n up to date, however the vector index is not re-embedded, so the agent
\n retrieves the previous model and solutions confidently from it, the identical
\n failure because the stale worth, solely now it is an embedding slightly than a
\n row. The freshness SLA<\/i> carries over, however be exact about what
\n the clock measures, the purpose is not when the content material final modified,
\n it is when the index was final efficiently rebuilt towards its
\n sources. A 24-hour SLA means the re-indexing job will need to have accomplished
\n inside the final 24 hours, if it hasn’t, the index is stale and
\n quarantined even when nothing seems to have modified, as a result of a
\n silently failed indexer is precisely when you may’t inform whether or not
\n one thing did. That one heartbeat catches each the up to date however
\n unindexed<\/i> doc and the pipeline that quietly stopped.<\/p>\n
\n You’ll be able to’t constrain the prose, however you may require that each chunk
\n carry a supply, a model, a timestamp, and an entry scope, and
\n reject something that does not. That metadata can be what makes
\n retrieval traceable and governable later.<\/p>\n
\n truncated chunks, catch near-duplicate paperwork that skew
\n retrieval, flag failed extractions and OCR rubbish, and look ahead to
\n embedding drift. A malformed or empty embedding warps similarity
\n search, so it by no means reaches the shop, for a similar purpose a foul
\n worth by no means reaches the agent, a warped index makes the agent
\n retrieve confidently improper content material.<\/p>\n
\n job is equivalent. The structure has to scent what’s dangerous earlier than
\n the agent does.<\/p>\n<\/section>\nConfidence-threshold routing<\/h3>\n
\n clear instances. However there is a grey space, knowledge that is not clearly dangerous,
\n however is not absolutely reliable both. That is the place
\n confidence-threshold routing is available in, bridging full autonomy
\n and full human management.<\/p>\n
\n and checks not simply mannequin<\/i> confidence, however data-level<\/i> alerts
\n like freshness, completeness, and consistency. If confidence is at
\n or above the brink (say 85%), the agent proceeds autonomously.
\n Under it, the agent defers to a human. The edge is configurable
\n per use case, for instance, pricing may demand 90%, whereas an
\n inside FAQ is ok at 70%.<\/p>\n
\n knowledge is three days stale; the freshness SLA says 24 hours. The SLA
\n violation robotically drives the arrogance rating under the
\n threshold, no matter how assured the mannequin itself feels about
\n its reply. The agent ought to reply by pulling a human in:<\/p>\n
\n verification.\u201d<\/p>\n
\n threshold, not simply the mannequin’s personal confidence<\/p>\n<\/div>\n
\n threshold, not simply the mannequin’s personal confidence. A mannequin may be
\n certain of a stale reply, and the freshness SLA overrides that
\n misplaced certainty.<\/p>\n
\n rating and weighing it towards the mannequin’s personal confidence. That is an
\n open design downside, not a solved one. Begin with a tough gate slightly
\n than a easy composite. Any contract or SLA breach forces a human,
\n no matter how the opposite alerts look. Add weighted scoring later,
\n and solely as soon as you may present it beats that easy rule.<\/p>\n<\/section>\nThe place to start out<\/h3>\n
\n cannot. Contracts, quarantine gates, a medallion structure, and
\n confidence-threshold routing are so much to face up in a single go. The
\n excellent news is that they are additive, every one lowers threat by itself,
\n and you’ll layer in the remaining over time. Start with the best
\n leverage strikes and develop from there.<\/p>\n\n
\n dataset can have completely different freshness necessities per client, comparable to a
\n pricing desk that is fantastic on nightly batches for a dashboard might have close to actual
\n time updates when a quoting agent relies on it.<\/li>\n
\n enters agent accessible storage. Begin together with your highest threat datasets comparable to
\n pricing, stock, buyer information.<\/li>\n
\n CI\/CD, outline contracts as YAML, validate robotically, block deployments on
\n failure. Deal with knowledge contracts with the identical rigor you’d give an API
\n contract.<\/li>\n
\n threshold, defer to a human. Begin excessive (round 90%) and regulate downward as you
\n construct belief and observe accuracy.<\/li>\n<\/ol>\n
\n who’s watching?<\/p>\n<\/section>\n<\/section>\nTraceability and Governance: Auditing Autonomous Brokers<\/h2>\n
\n regulator asks why the agent did what it did, are you able to reply? Conventional
\n techniques report what occurred. Agentic ones have to clarify why. That shift, from
\n what<\/i> to why<\/i>, is the place governance will get laborious.<\/p>\nThe audit hole<\/h3>\n
\n structure is the true innovation.<\/p>\n
\n buyer is not on a sanctions checklist, evaluates the credit score phrases, and approves a
\n $2.4 million transaction, all in about 30 seconds. Six months later, a regulator
\n asks a easy query, \u201cWhy was this accepted?\u201d<\/p>\n
\n however they can not inform you why<\/b>.<\/p>\n<\/div>\n
\n queried, at what time, by which service account. What they can not inform you is
\n why<\/i>. Why did the agent test the sanctions checklist earlier than the credit score phrases? Why
\n did it approve regardless of a minor documentation discrepancy? What options did
\n it contemplate and reject? The hole between \u201cwhat\u201d and \u201cwhy\u201d is the place regulatory
\n threat arises, and the EU AI
\n Act<\/a>‘s Article
\n 12<\/a> requires high-risk techniques
\n to maintain computerized logs for precisely this purpose, so the \u201cwhy\u201d may be
\n reconstructed after the actual fact. Closing that hole is what agentic lineage
\n is for.<\/p>\n<\/section>\nAgentic lineage<\/h3>\n
\n conventional knowledge lineage. The place conventional lineage tracks which sources have been
\n accessed, agentic lineage tracks why the agent determined to entry X, as a result of it
\n discovered Y in supply Z.<\/p>\n
\n end-to-end workflow of processing letter of credit score LC-4892. Inside that hint,
\n every span is a person step:<\/p>\n\n
\n verified.<\/li>\n
\n limits.<\/li>\n
\n reasoning chain hooked up.<\/li>\n<\/ul>\n
\n agent accessed the compliance database at 14:32:07 UTC\u201d however \u201cthe
\n agent checked KYC first, then sanctions, then credit score phrases, and
\n accepted as a result of all three handed.\u201d The traces and spans
\n mannequin<\/i> is borrowed immediately from distributed techniques
\n observability, so engineers already perceive the psychological mannequin from
\n instruments like Jaeger<\/a> and Zipkin<\/a>. For the agentic equal, Langfuse<\/a>, Arize
\n Phoenix<\/a>, and OpenTelemetry<\/a>
\n for AI are the rising selections. All three function on the
\n Thoughtworks Know-how Radar, OpenTelemetry at Undertake, Langfuse at
\n Trial, and Arize Phoenix at Assess.<\/p>\n<\/section>\nThe regulatory tooth are actual<\/h3>\n
\n particular regulation on the books. Article 12<\/a> requires
\n high-risk AI techniques to robotically log occasions over their lifetime
\n so their operation may be traced, and Article
\n 19<\/a> requires
\n suppliers to maintain these logs for no less than six months. Breaching
\n these record-keeping obligations falls within the Act’s center penalty
\n tier, as much as \u20ac15 million or 3% of worldwide annual turnover,
\n whichever is greater. For a big firm, even 3% of worldwide turnover
\n runs into the a whole lot of hundreds of thousands.<\/p>\n
\n structure:<\/p>\n\n
\n the way it operated, not simply remoted timestamps.<\/li>\n
\n observability infrastructure has to deal with long-term storage.<\/li>\n
\n logs; making them reply a regulator’s query is on you. Meaning
\n capturing the complete reasoning chain, which sources have been consulted, what logic was
\n utilized, and which options the agent weighed and rejected.<\/li>\n<\/ul>\n
\n prefer it. However you do not have to guess on the place regulation lands to see the purpose.
\n Ultimately one thing will drive the query of why an agent did what it
\n did, whether or not that is a regulator, an auditor, a buyer disputing a call, or
\n simply your personal workforce attempting to debug one. The protected assumption is not {that a}
\n explicit legislation is coming, it is that you will need to reply that query
\n regardless. A system you may’t clarify is one you may’t absolutely belief,
\n defend, or repair.<\/p>\n<\/section>\nStaged autonomy<\/h3>\n
\n one other. You do not deploy an agent with full autonomy on day one, any extra
\n than you’d hand a model new worker unrestricted entry. Autonomy is earned
\n in levels:<\/p>\n
![](\"https:\/\/martinfowler.com\/articles\/making-data-ready-for-agentic-ai\/context-layer.svg\")
<\/p>\n![](\"https:\/\/martinfowler.com\/articles\/making-data-ready-for-agentic-ai\/semantic-layer-seq.svg\")
<\/p>\n![](\"https:\/\/martinfowler.com\/articles\/making-data-ready-for-agentic-ai\/po-payment.svg\")
<\/p>\n![](\"https:\/\/martinfowler.com\/articles\/making-data-ready-for-agentic-ai\/stack.svg\")
<\/p>\n