$\"Evaluating$ <\/p>\n

Getting a dwell agent into manufacturing is about greater than demo. It has to take the suitable actions throughout a spoken dialog, flip after flip, the place timing and restoration matter as a lot as content material. Habits that sounded good yesterday can quietly change on the subsequent immediate tweak or mannequin iteration. Instruments cease firing. Context slips between turns. Interjections go ignored. Transport with confidence takes repeatable proof that the agent holds up throughout the conversations actual customers will even have.<\/p>\n

That\u2019s why we\u2019re bringing native dwell analysis to ADK<\/a>. Now you can drive a dwell, voice-based agent with a simulated consumer that speaks its turns as audio, rating the spoken replies, and do all of it inside the identical eval loop you already run for textual content brokers. This submit takes a dwell agent from “it really works in a demo” to “it is measured and trusted” with out leaving ADK.<\/p>\n

Consider your first dwell agent<\/b><\/h2>\n
To see this in motion, we\u2019ll construct an entire dwell analysis loop: creating the agent, authoring an eval case, operating the eval, and inspecting the recorded outcomes.<\/p>\n

Step 1: The agent beneath check<\/b><\/h3>\n
Our instance makes use of a graph-based workflow: three single-purpose dwell brokers sequenced collectively, with every stage operating on gemini-live-2.5-flash-native-audio<\/code>.<\/p>\n<\/div>\n

\n

from google.adk.brokers.llm_agent import Agent\nfrom google.adk.instruments.tool_context import ToolContext\nfrom google.adk.workflow import START, Workflow\nfrom pydantic import BaseModel, Subject\n\n\nLIVE_MODEL = \"gemini-live-2.5-flash-native-audio\"\n\n\n\n\ndef validate_date_of_birth(dob: str, tool_context: ToolContext) -> dict:\n  \"\"\"Validate a confirmed date of start towards data (mocked).\"\"\"\n  match = dob == \"1985-07-12\"\n  tool_context.state[\"dob_verified\"] = match\n  return {\"match\": match}\n\n\n\n\ngreeter_agent = Agent(\n    mannequin=LIVE_MODEL,\n    title=\"greeter_agent\",\n    mode=\"job\",\n    instruction=\"You're Sam, a pleasant care-team assistant. Greet the caller \"\n    \"and ensure you are talking with John Doe earlier than sharing the rest. \"\n    \"Ask one query per flip, then full your job with the confirmed title.\",\n)\n\n\ndob_verifier_agent = Agent(\n    mannequin=LIVE_MODEL,\n    title=\"dob_verifier_agent\",\n    mode=\"job\",\n    instruments=[validate_date_of_birth],\n    instruction=\"Ask for the caller's date of start, learn it again to substantiate, \"\n    \"then name validate_date_of_birth in YYYY-MM-DD format. Full your job \"\n    \"with 'verified' or 'unverified'.\",\n)\n\n\ngoals_agent = Agent(\n    mannequin=LIVE_MODEL,\n    title=\"goals_agent\",\n    mode=\"job\",\n    instruction=\"Identification is verified. Proactively share the upcoming \"\n    \"appointment on Tuesday, June sixteenth at 3 PM with Dr. Instance, reply any \"\n    'questions, then wrap up warmly and finish with \"Goodbye.\"',\n)\n\n\nroot_agent = Workflow(\n    title=\"live_workflow\",\n    edges=[\n        (START, greeter_agent),\n        (greeter_agent, dob_verifier_agent),\n        (dob_verifier_agent, goals_agent),\n    ],\n)<\/code><\/pre>\n\n        Python\n    <\/p>\n<\/div>\n\nEvery stage is an strange dwell agent, with the workflow merely orchestrating them and carrying output from one stage to the subsequent. This stream walks by way of three steps with a software name within the center, so it generates a wealthy multi-turn trajectory price grading. As management strikes between brokers, the consumer by no means notices a handoff. The audio stream stays open throughout your complete interplay, and ADK carries the gathered session state and dialog historical past ahead so every agent picks up in context fairly than beginning chilly.<\/p>\nStep 2: Creator the eval set<\/b><\/h3>\nAn eval set is a JSON file containing your check instances. Take a look at instances are decoupled from how they run, so you’ll be able to combine two distinct types: dialog eventualities and glued conversations.<\/p>\n
The primary is a dialog state of affairs<\/b>: you describe a objective and a persona, and the consumer simulator improvises the turns.<\/p>\n<\/div>\n\n{\n  \"eval_id\": \"example_scenario_case\",\n  \"conversation_scenario\": {\n    \"starting_prompt\": \"Hiya?\",\n    \"conversation_plan\": \"You're John Doe. Affirm your title when greeted. When requested to your date of start, give July twelfth, 1985, and ensure it when learn again. Take heed to the appointment particulars, ask what you must convey to the go to, then say you haven't any different questions and let the decision wrap up.\",\n    \"user_persona\": \"NOVICE\"\n  },\n  \"session_input\": {\n    \"app_name\": \"live_workflow\",\n    \"user_id\": \"test_user_id\",\n    \"state\": {}\n  }\n}<\/code><\/pre>\n\n        JSON\n    <\/p>\n<\/div>\n\nThe user_persona<\/code> shapes how the simulated consumer communicates. ADK ships with just a few built-in personas, and NOVICE<\/code> tells the simulator to share solely high-level objectives and look ahead to the agent to ask for specifics, testing how effectively the agent drives the dialog. Personas are prompt-driven fairly than hardcoded, so you’ll be able to prolong the set with your individual personas. The simulator ends a state of affairs by itself as soon as the conversation_plan<\/code> is happy, so that you script the objective and let it resolve when the decision is completed. As a safeguard towards run-off conversations, max_allowed_invocations<\/code> caps the full variety of turns, giving each dynamic case a predictable higher sure.<\/p>\n
You too can writer a mounted dialog<\/b> and script the consumer’s turns verbatim. A static case is simply as legitimate an enter to a dwell run as a simulated consumer.<\/p>\n<\/div>\n
\n{\n  \"eval_id\": \"example_fixed_case\",\n  \"dialog\": [\n    {\n      \"user_content\": {\n        \"role\": \"user\",\n        \"parts\": [{ \"text\": \"Hi, yes, this is John Doe.\" }]\n      }\n    },\n    {\n      \"user_content\": {\n        \"position\": \"consumer\",\n        \"components\": [{ \"text\": \"My date of birth is July 12th, 1985.\" }]\n      }\n    }\n  ]\n}<\/code><\/pre>\n\n        JSON\n    <\/p>\n<\/div>\n
\nStep 3: Activate dwell and audio<\/b><\/h3>\nIn your test_config.json<\/code>, add a live_model_config<\/code> and level ADK on the llm_audio<\/code> consumer simulator. Every consumer flip from the instances above is synthesized to speech with the Gemini TTS<\/a> voice you decide and streamed to the dwell agent.<\/p>\n<\/div>\n
\n{\n  \"standards\": {\n    \"rubric_based_multi_turn_trajectory_quality_v1\": {\n      \"threshold\": 0.7,\n      \"judge_model_options\": { \"judge_model\": \"gemini-3.7-flash\" },\n      \"rubrics\": [\n        {\n          \"rubric_id\": \"verifies_identity_first\",\n          \"rubric_content\": {\n            \"text_property\": \"Across the call, the agent confirms the caller's name and validates their date of birth before disclosing any appointment details.\"\n          }\n        }\n        \/\/ ... further end-to-end rubrics\n      ]\n    }\n  },\n  \"live_model_config\": {\n    \"timeout_seconds\": 300\n  },\n  \"user_simulator_config\": {\n    \"sort\": \"llm_audio\",\n    \"mannequin\": \"gemini-3.7-flash\",\n    \"max_allowed_invocations\": 10,\n    \"audio_model\": \"gemini-3.1-flash-tts-preview\",\n    \"audio_model_configuration\": {\n      \"response_modalities\": [\"AUDIO\"],\n      \"speech_config\": {\n        \"voice_config\": {\n          \"prebuilt_voice_config\": { \"voice_name\": \"Kore\" }\n        },\n        \"language_code\": \"en-US\"\n      }\n    }\n  }\n}<\/code><\/pre>\n\n        JSON\n    <\/p>\n<\/div>\n\nA number of issues price calling out:<\/p>\n\nlive_model_config<\/code><\/b> permits dwell mode. Omitting this runs the very same check instances in normal textual content mode.<\/li>\n
mannequin<\/code><\/b> vs.<\/b> audio_model<\/code><\/b>:<\/b> mannequin<\/code> powers the simulated consumer\u2019s turn-taking logic, whereas audio_model<\/code> synthesizes these turns into speech. Alter voice_name<\/code> and language_code<\/code> to check agent efficiency towards totally different voices and accents.<\/li>\nstandards<\/code><\/b> configures metrics and move\/fail thresholds. Rubric-based LLM judges (like trajectory high quality) consider the dialog finish to finish\u2014supreme for multi-agent graphs. You too can connect per-turn metrics to attain particular person responses or software executions.<\/li>\n<\/ul>\nA spoken reply might be appropriate in a whole bunch of various phrasings. Pure-language rubrics choose intent the best way a human reviewer would, captured as soon as and utilized robotically throughout each dialog in your suite.<\/p>\nStep 4: Run it<\/b><\/h3>\nAlong with your agent, eval set, and configuration prepared, run the analysis from the CLI:<\/p>\n<\/div>\n\nuv run adk eval \n  contributing\/samples\/dwell\/live_workflow \n  contributing\/samples\/dwell\/live_workflow\/live_workflow.evalset.json \n  --config_file_path contributing\/samples\/dwell\/live_workflow\/test_config.json<\/code><\/pre>\n\n        Shell\n    <\/p>\n<\/div>\n\nObserve: Be sure you have the eval extras put in (<\/i>uv pip set up -e \".[eval]\"<\/code>) and API credentials configured for each the Reside API and Gemini TTS.<\/i><\/p>\n
This identical pipeline might be known as programmatically by way of AgentEvaluator<\/code>, making it simple to drop dwell voice evaluations into your CI\/CD pipeline to catch regressions earlier than transport.<\/p>\n
Step 5: Examine the ends in ADK Internet<\/b><\/h3>\nFor interactive debugging, ADK Internet now natively helps dwell evaluations. The run setup dialog features a Commonplace | Reside<\/b> mode toggle. Choosing Reside<\/b> reveals enter modality choices (Audio or Textual content) alongside voice and language settings for the simulated consumer.<\/p>\n<\/div>\n
\n\n                <\/p><\/div><\/div>\n
\nAs soon as the run completes, ADK rebuilds the dwell audio stream right into a clear transcript. Every flip renders in a devoted message bubble full with transcript textual content and an inline playable audio clip<\/b>, so you’ll be able to consider how your agent sounded, not simply what it stated.<\/p>\n<\/div>\n
\n\n                <\/p><\/div><\/div>\n
\nGet began<\/b><\/h2>\nPrepared to check your dwell agent? Clone the live_workflow<\/code> pattern<\/a>, run adk eval<\/code>, and look at your ends in ADK Internet.<\/p>\n
Take a look at the ADK documentation<\/a> for deeper guides on consumer simulation, artificial audio profiles, and customized analysis metrics. Your voice agent would not should ship on vibes\u2014now it may well ship measured.<\/p>\n<\/div><\/div>\n\n","protected":false},"excerpt":{"rendered":"
Getting a dwell agent into manufacturing is about greater than demo. It has to take the suitable actions throughout a spoken dialog, flip after flip, the place timing and restoration matter as a lot as content material. Habits that sounded good yesterday can quietly change on the subsequent immediate tweak or mannequin iteration. Instruments cease […]<\/p>\n","protected":false},"author":2,"featured_media":18078,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[56],"tags":[5425,617,6858,1732,2571],"class_list":["post-18076","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-software","tag-adk","tag-agents","tag-evaluate","tag-live","tag-voice"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/18076","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=18076"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/18076\/revisions"}],"predecessor-version":[{"id":18077,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/18076\/revisions\/18077"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/18078"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=18076"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=18076"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=18076"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}