This publish is co-authored with OneAdvanced group
Deploying AI brokers on a United Kingdom (UK)-sovereign AWS structure requires cautious selections about mannequin internet hosting, information residency, and agent orchestration. OneAdvanced, a UK-based enterprise software program supplier serving over 10,000 prospects, wanted to ship AI capabilities whereas ensuring that no information would depart the UK. On the time, the precise fashions they needed, Llama 4 Maverick and Llama Guard 4, weren’t but accessible by means of managed providers within the UK area. OneAdvanced took a distinct path: self-hosting open-weight massive language fashions (LLMs) on AWS infrastructure they absolutely management.
On this publish, we describe how OneAdvanced constructed a UK-sovereign AI answer utilizing Llama 4 Maverick and Llama Guard 4 on Amazon SageMaker AI. The answer pairs a Retrieval Augmented Era (RAG) pipeline backed by Amazon Aurora PostgreSQL-Suitable Version with the pgvector extension, over 50 specialised brokers powered by Strands Brokers SDK, and a device layer working on Amazon Elastic Container Service (Amazon ECS). We stroll by means of the structure, key implementation selections, and outcomes.
The problem: Information sovereignty and mannequin internet hosting
OneAdvanced supplies sector-focused software program as a service (SaaS) options to organizations in healthcare, authorized, and lots of different regulated industries. Their prospects deal with delicate information day by day, together with affected person data, authorized case information, and compliance documentation. These prospects anticipate that AI tooling meets strict information residency, safety, and privateness requirements.
As Andrew Henderson, CTO of OneAdvanced, explains within the OneAdvanced AI launch video:
“Information sovereignty, significantly within the UK, is a tough requirement for a lot of of our prospects, particularly these within the public sector and extremely regulated industries. They should know precisely the place their information is, who has entry to it, and that it resides throughout the UK’s authorized and regulatory framework to assist complete compliance and belief.”
OneAdvanced initially prototyped with Amazon Bedrock and noticed speedy outcomes inside a two-week dash: chat completion, an Amazon Bedrock Agent for querying UK statute regulation, Snowflake information integration, and chart technology.
Nonetheless, to satisfy their sovereignty necessities, OneAdvanced wanted to host fashions solely in their very own UK-based AWS accounts. On the time of the engagement, the fashions they needed, Llama 4 Maverick and Llama Guard 4, weren’t but accessible by means of AWS managed providers within the UK area. Self-hosting meant deploying, serving, and scaling these fashions whereas constructing a production-grade answer round them. This included content material moderation, doc retrieval, agent orchestration, and a no-code agent builder for non-technical customers.
Answer overview
The next diagram reveals the high-level structure of the OneAdvanced AI answer.
Determine 1: Excessive-level structure of the OneAdvanced UK-sovereign AI answer
This structure enabled OneAdvanced to attain full UK information sovereignty and speedy agent deployment, supporting their ISO 42001 certification for AI governance whereas sustaining full management over their mannequin serving infrastructure.
The answer consists of 4 parts. vLLM serves Llama 4 Maverick (FP8) and Llama Guard 4 on Amazon SageMaker AI, working on p5.48xlarge situations within the London (eu-west-2) area. Over 50 Strands brokers run on Amazon ECS, every with its personal system immediate, device configuration, and optionally available enter kind, with agent configuration saved in Amazon DynamoDB. Paperwork uploaded to Amazon Easy Storage Service (Amazon S3) are transformed to markdown, chunked, and embedded into pgvector for retrieval. Llama Guard 4 checks consumer inputs for dangerous content material earlier than the request reaches the primary mannequin.
A typical request flows like this: the consumer sends a message, and Llama Guard checks it for dangerous content material (evaluated earlier than the primary inference mannequin). The request routes to the suitable Strands agent on Amazon ECS. The agent calls instruments and retrieves related paperwork from pgvector and Amazon S3 as wanted, or invokes specialist instruments together with net search.
Conditions
To self-host an AI mannequin on AWS, you want the next sources and abilities:
- An AWS account with entry to
p5.48xlargesituations in your goal Area. For details about requesting GPU occasion quota, seek advice from the Amazon SageMaker AI Developer Information. - Intermediate information of deploying containers on Amazon ECS and managing storage with Amazon S3. For getting began, seek advice from the Amazon ECS Developer Information.
- Entry to your chosen AI mannequin weights (Llama 4 Maverick and Llama Guard 4 require license acceptance on Hugging Face).
- Expertise with Python-based machine studying (ML) frameworks and container orchestration.
- A PostgreSQL database with the pgvector extension enabled for vector similarity search.
Mannequin deployment on Amazon SageMaker AI
OneAdvanced serves Llama 4 Maverick (meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8) and Llama Guard 4 (meta-llama/Llama-Guard-4-12B) utilizing vLLM on Amazon SageMaker AI endpoints. They deploy on p5.48xlarge situations within the London area, utilizing Hugging Face fashions with AWS Deep Studying Containers.
Longer context home windows drove the transfer to P5 situations. OneAdvanced targets 120K–128K token context lengths to assist use instances like massive doc evaluation and multi-turn conversations. In the course of the advisory engagement with AWS, load testing with vLLM on P5 situations validated that the infrastructure may deal with their throughput necessities. OneAdvanced began on p4d.24xlarge situations and moved to p5.48xlarge for manufacturing, additionally profiting from reserved occasion reductions on GPU compute.
Llama Guard 4 changed an earlier deployment of Llama Guard 3 after OneAdvanced noticed excessive false rejection charges with the earlier model. The guard mannequin runs serially earlier than the primary mannequin, screening consumer inputs for dangerous content material earlier than inference begins.
Constructing over 50 brokers with Strands Brokers SDK
A particular side of the OneAdvanced answer is its agent library: over 50 task-specific brokers spanning healthcare, authorized, HR, advertising, logistics, and extra. Brokers embody a Care Incident Response assistant, a Scientific Security Bulletin generator, a scheme of labor generator for schooling, an operational state of affairs simulation, a efficiency overview assistant, a Doc Comparability device, and an AWS Architect Agent, amongst many others. OneAdvanced went from their first agent to over 50 in solely three weeks, with most brokers in-built lower than a day.
OneAdvanced evaluated a number of agentic frameworks, together with LangChain, LangGraph, and others, earlier than selecting Strands Brokers SDK. The choice got here down to a couple elements: Strands takes a model-first method with no inflexible workflow definitions, it helps turn-taking and interview-style interactions, and it allowed OneAdvanced to maneuver shortly from concept to deployed agent. As Nick Heap, Principal Software program Engineer at OneAdvanced, explains:
“Put up-evaluation, Strands stood out because the clear frontrunner for the mission. Its complete suite of instruments not solely met our necessities but in addition supplied a future-proof answer that intently aligned with our in-house imaginative and prescient.”
Every agent is outlined with a system immediate, a set of instruments, and an optionally available structured enter kind. Brokers are containerized and deployed on Amazon ECS, with runtime configuration saved in Amazon DynamoDB. Customers browse the agent catalog and choose the one that matches their job.
OneAdvanced additionally constructed a no-code agent builder that non-developers can use to create and configure brokers by means of a visible interface. Customers outline the agent’s persona, design enter kinds with drag-and-drop fields, write system prompts with references to kind fields (utilizing an @ syntax to inject kind values), and choose from a library of accessible instruments. This design makes agent creation accessible to product managers, clinicians, and enterprise analysts with out writing code.
Brokers draw from a shared device library that features calculator, chart creation, file content material reader, mermaid diagram generator, group and private information search, spreadsheet question (together with Snowflake integration), textual content file question, UK statute regulation search, and net search (opt-in per group and per consumer question). For instance, an agent can question a Snowflake database after which generate a chart from the ends in a single interplay.
OneAdvanced additionally makes use of sub-agents and superior agentic patterns to keep up the context window throughout complicated interactions. A notable design sample is the interview-style agent. Some brokers, just like the Strategic Pondering assistant, ask structured questions earlier than offering recommendation: “Query 1 of three: What particular ache factors does your AI answer intention to handle?” This builds context by means of turn-taking reasonably than anticipating customers to supply the whole lot upfront. Targeted, guided interactions produce higher outcomes than open-ended prompts.
RAG pipeline
The RAG system provides brokers entry to each private and organization-level doc collections. Customers add paperwork by means of the file administration interface into both a private area or a shared group area on Amazon S3. Paperwork are transformed to markdown and chunked into 2,048-token segments earlier than embedding into pgvector for vector similarity search. For longer paperwork, recursive summarization handles content material that exceeds the chunk dimension.
For embeddings, OneAdvanced makes use of the intfloat/multilingual-e5-large-instruct mannequin, chosen for its robust multilingual assist and instruction-following capabilities. Slightly than adopting an off-the-shelf RAG framework, OneAdvanced constructed their very own retrieval system, internally known as “Llamadex,” giving them full management over the pipeline. They initially selected pgvector for its simplicity and the pace it supplied in attending to manufacturing. As the answer matures, OneAdvanced is reviewing this technique to judge options.
When an agent must reply a query grounded in uploaded paperwork, it calls the suitable retrieval device, “Private information search” or “Group information search,” which queries pgvector and returns related chunks together with supply references. Customers can see precisely which paperwork have been used to generate a response, supporting transparency and belief.
Safety and accountable AI
Safety runs by means of each layer of the OneAdvanced AI answer, a direct consequence of serving prospects who deal with delicate information day by day in sectors like well being and authorized.
The infrastructure runs within the London AWS Area. No consumer information leaves the UK. No consumer queries or responses are retained or used for mannequin coaching. As Nick Heap wrote:
“An AI service that’s solely hosted within the UK and information just isn’t used to coach the AI mannequin and OneAdvanced engineers would not have entry to the info.”
Uploaded paperwork in private and group areas are absolutely remoted and never readable even by OneAdvanced workers. Llama Guard 4 screens consumer inputs for dangerous content material earlier than they attain the primary mannequin. Privateness controls are customizable at organizational ranges. On the infrastructure aspect, Amazon GuardDuty supplies risk detection on the Amazon S3 doc storage layer.
OneAdvanced holds ISO 42001 certification for AI governance—a certification they report being among the many first organizations within the UK and Europe to attain. They’re additionally a signatory to the EU AI Pact, reflecting a proactive method to accountable AI throughout their merchandise and operations.
Outcomes
By the AWS advisory engagement, OneAdvanced went from prototype to manufacturing on a UK-sovereign AI answer. OneAdvanced deployed Llama 4 Maverick on P5 situations within the London area, giving them full management over mannequin serving infrastructure with no information leaving the UK. They constructed and shipped over 50 Strands brokers in solely three weeks, masking use instances throughout healthcare, authorized, HR, advertising, and extra. The answer has been working in manufacturing since July 2025, serving prospects for over a 12 months. It met its goal efficiency metrics. OneAdvanced publicly launched this because the UK’s first non-public sovereign AI for enterprise, with no consumer information retained, skilled on, or logged. The engagement obtained a 5/5 buyer satisfaction rating.
“We actually admire and loved the engagement. We discovered lots which helped us launch a singular providing to the market powered and supported by our strategic accomplice. Thanks very a lot to all concerned instantly and not directly.”
— Alex Savage, Head of Integration, OneAdvanced
Conclusion and subsequent steps
On this publish, we confirmed how OneAdvanced constructed a UK-sovereign AI answer by self-hosting Llama 4 Maverick and Llama Guard 4 on Amazon SageMaker AI, orchestrating over 50 specialised brokers with Strands Brokers SDK on Amazon ECS, and grounding responses in buyer paperwork by means of a RAG pipeline backed by pgvector. The result’s a manufacturing AI answer that meets strict information sovereignty necessities whereas delivering sensible AI capabilities to enterprise prospects in regulated industries.
OneAdvanced has not too long ago applied an analysis framework utilizing LLM-as-a-judge with sentiment evaluation for steady high quality enchancment. Their roadmap additionally consists of backend methods to assist customers refine their prompts, and working parallel LLM fashions throughout model transitions to keep up stability for finish customers.
When you’re contemplating an identical method, begin by evaluating your information sovereignty necessities and figuring out which fashions it’s worthwhile to self-host. OneAdvanced started with a two-week prototype on Amazon Bedrock to validate their use instances earlier than committing to self-hosted infrastructure. For extra details about deploying fashions on Amazon SageMaker AI, seek advice from the Amazon SageMaker AI Developer Information. To get began with agentic workflows, go to the Strands Brokers SDK on GitHub.
To learn extra about OneAdvanced’s journey constructing their AI answer, see Nick Heap’s publish The AI race and the way we absolutely embraced velocity on the OneAdvanced careers weblog.
If in case you have an identical use case and wish to discover sovereign AI deployment on AWS, contact your AWS account group.
In regards to the authors
