Getting a dwell agent into manufacturing is about greater than demo. It has to take the suitable actions throughout a spoken dialog, flip after flip, the place timing and restoration matter as a lot as content material. Habits that sounded good yesterday can quietly change on the subsequent immediate tweak or mannequin iteration. Instruments cease firing. Context slips between turns. Interjections go ignored. Transport with confidence takes repeatable proof that the agent holds up throughout the conversations actual customers will even have.
That’s why we’re bringing native dwell analysis to ADK. Now you can drive a dwell, voice-based agent with a simulated consumer that speaks its turns as audio, rating the spoken replies, and do all of it inside the identical eval loop you already run for textual content brokers. This submit takes a dwell agent from “it really works in a demo” to “it is measured and trusted” with out leaving ADK.
Consider your first dwell agent
To see this in motion, we’ll construct an entire dwell analysis loop: creating the agent, authoring an eval case, operating the eval, and inspecting the recorded outcomes.
Step 1: The agent beneath check
Our instance makes use of a graph-based workflow: three single-purpose dwell brokers sequenced collectively, with every stage operating on gemini-live-2.5-flash-native-audio.
from google.adk.brokers.llm_agent import Agent
from google.adk.instruments.tool_context import ToolContext
from google.adk.workflow import START, Workflow
from pydantic import BaseModel, Subject
LIVE_MODEL = "gemini-live-2.5-flash-native-audio"
def validate_date_of_birth(dob: str, tool_context: ToolContext) -> dict:
"""Validate a confirmed date of start towards data (mocked)."""
match = dob == "1985-07-12"
tool_context.state["dob_verified"] = match
return {"match": match}
greeter_agent = Agent(
mannequin=LIVE_MODEL,
title="greeter_agent",
mode="job",
instruction="You're Sam, a pleasant care-team assistant. Greet the caller "
"and ensure you are talking with John Doe earlier than sharing the rest. "
"Ask one query per flip, then full your job with the confirmed title.",
)
dob_verifier_agent = Agent(
mannequin=LIVE_MODEL,
title="dob_verifier_agent",
mode="job",
instruments=[validate_date_of_birth],
instruction="Ask for the caller's date of start, learn it again to substantiate, "
"then name validate_date_of_birth in YYYY-MM-DD format. Full your job "
"with 'verified' or 'unverified'.",
)
goals_agent = Agent(
mannequin=LIVE_MODEL,
title="goals_agent",
mode="job",
instruction="Identification is verified. Proactively share the upcoming "
"appointment on Tuesday, June sixteenth at 3 PM with Dr. Instance, reply any "
'questions, then wrap up warmly and finish with "Goodbye."',
)
root_agent = Workflow(
title="live_workflow",
edges=[
(START, greeter_agent),
(greeter_agent, dob_verifier_agent),
(dob_verifier_agent, goals_agent),
],
)Python
Every stage is an strange dwell agent, with the workflow merely orchestrating them and carrying output from one stage to the subsequent. This stream walks by way of three steps with a software name within the center, so it generates a wealthy multi-turn trajectory price grading. As management strikes between brokers, the consumer by no means notices a handoff. The audio stream stays open throughout your complete interplay, and ADK carries the gathered session state and dialog historical past ahead so every agent picks up in context fairly than beginning chilly.
Step 2: Creator the eval set
An eval set is a JSON file containing your check instances. Take a look at instances are decoupled from how they run, so you’ll be able to combine two distinct types: dialog eventualities and glued conversations.
The primary is a dialog state of affairs: you describe a objective and a persona, and the consumer simulator improvises the turns.
{
"eval_id": "example_scenario_case",
"conversation_scenario": {
"starting_prompt": "Hiya?",
"conversation_plan": "You're John Doe. Affirm your title when greeted. When requested to your date of start, give July twelfth, 1985, and ensure it when learn again. Take heed to the appointment particulars, ask what you must convey to the go to, then say you haven't any different questions and let the decision wrap up.",
"user_persona": "NOVICE"
},
"session_input": {
"app_name": "live_workflow",
"user_id": "test_user_id",
"state": {}
}
}JSON
The user_persona shapes how the simulated consumer communicates. ADK ships with just a few built-in personas, and NOVICE tells the simulator to share solely high-level objectives and look ahead to the agent to ask for specifics, testing how effectively the agent drives the dialog. Personas are prompt-driven fairly than hardcoded, so you’ll be able to prolong the set with your individual personas. The simulator ends a state of affairs by itself as soon as the conversation_plan is happy, so that you script the objective and let it resolve when the decision is completed. As a safeguard towards run-off conversations, max_allowed_invocations caps the full variety of turns, giving each dynamic case a predictable higher sure.
You too can writer a mounted dialog and script the consumer’s turns verbatim. A static case is simply as legitimate an enter to a dwell run as a simulated consumer.
{
"eval_id": "example_fixed_case",
"dialog": [
{
"user_content": {
"role": "user",
"parts": [{ "text": "Hi, yes, this is John Doe." }]
}
},
{
"user_content": {
"position": "consumer",
"components": [{ "text": "My date of birth is July 12th, 1985." }]
}
}
]
}JSON
Step 3: Activate dwell and audio
In your test_config.json, add a live_model_config and level ADK on the llm_audio consumer simulator. Every consumer flip from the instances above is synthesized to speech with the Gemini TTS voice you decide and streamed to the dwell agent.
{
"standards": {
"rubric_based_multi_turn_trajectory_quality_v1": {
"threshold": 0.7,
"judge_model_options": { "judge_model": "gemini-3.7-flash" },
"rubrics": [
{
"rubric_id": "verifies_identity_first",
"rubric_content": {
"text_property": "Across the call, the agent confirms the caller's name and validates their date of birth before disclosing any appointment details."
}
}
// ... further end-to-end rubrics
]
}
},
"live_model_config": {
"timeout_seconds": 300
},
"user_simulator_config": {
"sort": "llm_audio",
"mannequin": "gemini-3.7-flash",
"max_allowed_invocations": 10,
"audio_model": "gemini-3.1-flash-tts-preview",
"audio_model_configuration": {
"response_modalities": ["AUDIO"],
"speech_config": {
"voice_config": {
"prebuilt_voice_config": { "voice_name": "Kore" }
},
"language_code": "en-US"
}
}
}
}JSON
A number of issues price calling out:
live_model_configpermits dwell mode. Omitting this runs the very same check instances in normal textual content mode.mannequinvs.audio_model:mannequinpowers the simulated consumer’s turn-taking logic, whereasaudio_modelsynthesizes these turns into speech. Altervoice_nameandlanguage_codeto check agent efficiency towards totally different voices and accents.standardsconfigures metrics and move/fail thresholds. Rubric-based LLM judges (like trajectory high quality) consider the dialog finish to finish—supreme for multi-agent graphs. You too can connect per-turn metrics to attain particular person responses or software executions.
A spoken reply might be appropriate in a whole bunch of various phrasings. Pure-language rubrics choose intent the best way a human reviewer would, captured as soon as and utilized robotically throughout each dialog in your suite.
Step 4: Run it
Along with your agent, eval set, and configuration prepared, run the analysis from the CLI:
uv run adk eval
contributing/samples/dwell/live_workflow
contributing/samples/dwell/live_workflow/live_workflow.evalset.json
--config_file_path contributing/samples/dwell/live_workflow/test_config.jsonShell
Observe: Be sure you have the eval extras put in (uv pip set up -e ".[eval]") and API credentials configured for each the Reside API and Gemini TTS.
This identical pipeline might be known as programmatically by way of AgentEvaluator, making it simple to drop dwell voice evaluations into your CI/CD pipeline to catch regressions earlier than transport.
Step 5: Examine the ends in ADK Internet
For interactive debugging, ADK Internet now natively helps dwell evaluations. The run setup dialog features a Commonplace | Reside mode toggle. Choosing Reside reveals enter modality choices (Audio or Textual content) alongside voice and language settings for the simulated consumer.
As soon as the run completes, ADK rebuilds the dwell audio stream right into a clear transcript. Every flip renders in a devoted message bubble full with transcript textual content and an inline playable audio clip, so you’ll be able to consider how your agent sounded, not simply what it stated.
Get began
Prepared to check your dwell agent? Clone the live_workflow pattern, run adk eval, and look at your ends in ADK Internet.
Take a look at the ADK documentation for deeper guides on consumer simulation, artificial audio profiles, and customized analysis metrics. Your voice agent would not should ship on vibes—now it may well ship measured.
