Google ADK

cli_sdk.integrations.google_adk puts a calibrated check in front of the tools of a Google Agent Development Kit agent. Before a guarded tool runs, the guard evaluates the proposed call and either allows it, sends it to a person through ADK’s native tool confirmation, or blocks it. Each decision comes from a query calibrated on your labelled examples and carries its guarantee card: an error-rate bound for a Gate, a calibrated probability interval for a Belief, both on data like the calibration set. The adapter plugs in per agent (before_tool_callback) or app-wide (CLIGuardPlugin on the App), and works with any ADK agent model: Gemini, OpenAI, Azure OpenAI, Claude, or Gemma served by vLLM or SGLang. It was verified end to end against google-adk 2.9.2.

Why calibrate instead of picking a threshold

Suppose an AML agent auto-holds any account the model scores above 0.9. Nobody can say what fraction of those holds are unwarranted, and the answer changes when the model, the prompt or the traffic changes. In the AML example, the guard instead learns its threshold from 360 labelled alerts (0.61 for that evidence model) and attaches the statement it was calibrated to: “With 90% confidence, at most 0.1 of auto-approved decisions are wrong (Learn-then-Test, fixed-sequence, n=360).” Every hold below the threshold goes to an analyst. If the calibration cannot support the target, or the evidence model or the prompt has changed since calibration, no hold is placed automatically.

A guarantee bounds a rate over many decisions on data exchangeable with the calibration set. It never makes one hold, or one eligibility mark, correct. Read “at most 10% of automatic holds like these are unwarranted”, never “this hold is safe”.

Install

Python 3.10 or later. Install the SDK with the google-adk extra, plus what your models need:

pip install "cci-sdk[google-adk]"                                      # keyless mock only
pip install "cci-sdk[google-adk,openai]"                               # Gemini agent; OpenAI, Azure, Gemini, vLLM or SGLang evidence
pip install "cci-sdk[google-adk,openai]" "google-adk[extensions]"      # OpenAI, Azure OpenAI or Gemma agent (LiteLLM)
pip install "cci-sdk[google-adk,anthropic]" "google-adk[extensions]"   # Claude as the agent and the evidence model

Plain google-adk covers Gemini and custom BaseLlm models; google-adk[extensions] adds LiteLLM and the Anthropic SDK. The openai extra installs the OpenAI SDK that the OpenAI, Azure OpenAI, Gemini, vLLM and SGLang evidence backends use.

How decisions map onto ADK

Guard decision	The hook returns	What ADK does	What the model sees
allow	`None`	Runs the tool	The tool’s result
escalate	`{"status": "pending_human_review", ...}`, after calling `tool_context.request_confirmation(hint=..., payload=...)` and setting `tool_context.actions.skip_summarization = True`	Emits an `adk_request_confirmation` function call and ends the turn; the tool does not run	Nothing: the turn ends before the model is called again
reviewer approves	`None` (the hook sees `tool_context.tool_confirmation.confirmed`)	Re-runs the original call, and the tool runs	The tool’s result
reviewer rejects	`{"status": "rejected_by_reviewer", ...}`	Skips the tool	That a reviewer declined the call, and not to retry it
block	`{"status": "blocked", "message": ..., "cli": ...}`	Skips the tool	Why the call was blocked, and not to retry it

Which decisions a rule can produce depends on its primitive: a Gate allows or escalates (and blocks when it abstains); a Belief rule allows, escalates or blocks by where its Venn-Abers interval falls relative to allow_above and block_below. Any answer without a guarantee (missing, too small or stale calibration) escalates, and so does any error while building the context or evaluating it. Tools without a rule, such as read-only lookups, always run.

Two ways to install the guard:

One agent: Agent(..., before_tool_callback=cli_before_tool_callback(guard)).
The whole app: App(name=..., root_agent=agent, plugins=[CLIGuardPlugin(guard)]), run with Runner(app=app, session_service=...). The plugin guards every agent and sub-agent. Plugins run before agent callbacks, and a plugin that returns a value skips them.

Every decision, including the reviewer’s answer, is appended to session.state["cli_guard_audit"]: the action, tool, arguments, reason, the guarantee card (type, method, statement, profile, calibration size) and the evidence. ADK persists it through the tool event’s state delta.

Set it up

Choose the agent model

The agent model reads the conversation and proposes tool calls. It is not part of the calibration, so any model with good tool calling works. The examples build it in examples/agents/_shared/adk_models.py, from environment variables with placeholder defaults; a real provider stops with a clear message before any network call while its key is still a placeholder. In each tab, s = settings(provider).require_key() holds the model name, key and endpoint read from the environment.

export OPENAI_API_KEY="<your key>"       # OPENAI_MODEL defaults to gpt-4.1-mini
python examples/agents/google_adk/aml_account_hold_agent.py --provider openai

from google.adk.models.lite_llm import LiteLlm
 
s = settings(provider).require_key()
extra = {"api_base": s.base_url} if s.base_url else {}
return LiteLlm(model=f"openai/{s.model}", api_key=s.api_key, **extra)

A non-reasoning model such as gpt-4.1-mini also returns token log-probabilities, so the same provider can be the L1 evidence model.

export AZURE_OPENAI_API_KEY="<your key>"
export AZURE_OPENAI_ENDPOINT="https://<your-resource>.openai.azure.com"
export AZURE_OPENAI_DEPLOYMENT="<your deployment name>"   # AZURE_OPENAI_API_VERSION defaults to 2024-10-21
python examples/agents/google_adk/aml_account_hold_agent.py --provider azure

return LiteLlm(model=f"azure/{s.model}", api_base=s.base_url, api_version=s.api_version,
               api_key=s.api_key)

The name after azure/ is your deployment name, not the model family. Pin the deployment’s version-upgrade policy so a silent model upgrade does not change the evidence model behind a calibration profile.

export ANTHROPIC_API_KEY="<your key>"    # ANTHROPIC_MODEL defaults to claude-sonnet-5
python examples/agents/google_adk/aml_account_hold_agent.py --provider anthropic
python examples/agents/google_adk/aml_account_hold_agent.py --provider anthropic --evidence-provider vllm

from anthropic import AsyncAnthropic
from google.adk.models.anthropic_llm import AnthropicLlm
 
return AnthropicLlm(model=s.model, client=AsyncAnthropic(api_key=s.api_key))

AnthropicLlm calls the Anthropic API directly. Do not pass a bare "claude-..." model string: ADK resolves it to Claude on Vertex AI. Claude has no log-probabilities, so as the evidence model it works at L0 (sampling); the second command keeps Claude as the agent and scores with Gemma on vLLM at L1.

export GOOGLE_API_KEY="<your key>"       # or GEMINI_API_KEY; GEMINI_MODEL defaults to gemini-2.5-flash
python examples/agents/google_adk/aml_account_hold_agent.py --provider gemini

from google.adk.models.google_llm import Gemini
 
return Gemini(model=s.model, client_kwargs={"api_key": s.api_key})

ADK’s native Gemini client; no LiteLLM involved. As the evidence model, Gemini runs at L0 (sampling) through its OpenAI-compatible endpoint.

vllm serve google/gemma-4-12B-it --max-logprobs 20 --generation-config vllm \
    --enable-auto-tool-choice --tool-call-parser gemma4
python examples/agents/google_adk/aml_account_hold_agent.py --provider vllm

return LiteLlm(model=f"hosted_vllm/{s.model}", api_base=s.base_url, api_key=s.api_key)

--enable-auto-tool-choice --tool-call-parser gemma4 lets the agent call tools; without them it never proposes a call and the guard never runs. --max-logprobs 20 --generation-config vllm are for the same server as an L1 evidence model. The name after hosted_vllm/ must equal the served model name (VLLM_MODEL, default google/gemma-4-12B-it, at VLLM_BASE_URL, default http://localhost:8000/v1). Use hosted_vllm/, not vllm/, which is LiteLLM’s in-process route.

python -m sglang.launch_server --model-path google/gemma-4-12B-it --port 30000 \
    --tool-call-parser gemma4
python examples/agents/google_adk/aml_account_hold_agent.py --provider sglang

return LiteLlm(model=f"openai/{s.model}", api_base=s.base_url, api_key=s.api_key)

LiteLLM has no sglang/ provider; SGLang speaks the OpenAI API, so the openai/ route with api_base works. It needs some API key: the examples send SGLANG_API_KEY, default EMPTY (without one LiteLLM raises “Missing credentials”). The name after openai/ must equal the served model name (SGLANG_MODEL).

Keyless by default: --provider mock uses ScriptedLlm, a BaseLlm subclass that reads the conversation and proposes tool calls deterministically, so every example runs offline.

Choose the evidence model and calibrate

The guard scores each proposed call with its own evidence model: one fixed, closed question about the case, asked the same way at calibration and at serving time. It cannot reuse the agent’s responses: ADK’s LiteLLM adapter does not pass token log-probabilities through, and the agent’s free-form turns are not a fixed scoring function. The evidence model answers at one of two access levels.

L1, log-probabilities: one request per score, reading the first-token probability of each answer option. OpenAI and Azure OpenAI on non-reasoning configurations, and Gemma on vLLM or SGLang.
L0, sampling: sample_count sampled answers per score (8 in the examples), for models without log-probabilities: Claude and Gemini. More requests, and coarser scores.

Calibration scores every labelled example once and stores the scores in a JSON profile, so it costs one pass over the calibration set per evidence model:

Example	Calibration examples	Requests at L1	Requests at L0 (8 samples)	Per guarded call afterwards
AML account holds	360	360	2,880	1 (8 at L0)
Trial pre-screening	320	320	2,560	1 (8 at L0)
AML `--batch`	none (reuses the AML profile)	40 per batch	320 per batch	not applicable

The profile records a fingerprint of the evidence model, its settings and the query’s instructions. If any of them changes, the profile is stale: every guarded call escalates until you recalibrate (the examples’ ensure_calibrated helper does it automatically). The example builds both models first, so a placeholder key stops the run before anything is scored:

evidence_provider = args.evidence_provider or args.provider
# Both models are built before anything else, so a placeholder key stops the run here.
model = None if args.batch else adk_models.agent_model(args.provider, policy=investigator_policy)
backend = providers.evidence_backend(evidence_provider, mock_scorer=mock_hold_evidence)
client = LocalCLIClient(backend, store=calibration.default_store(__file__, args.store),
                        sample_count=providers.sample_count(evidence_provider))

examples = calibration.load_jsonl("aml_alerts.jsonl")
profile = calibration.ensure_calibrated(client, [HOLD_GATE], examples, recalibrate=args.recalibrate)[0]

With --provider mock, the evidence model is MockEvidenceBackend with a domain scorer (mock_hold_evidence) that reads the case facts the way a fallible model would, with deterministic noise keyed on the case id. It never sees the label. See Bring your own model for every evidence backend and Local mode for profiles and fingerprints.

Wire the guard

A GuardRule names the tool, the calibrated query that decides it, and a context builder that turns the proposed arguments into the context the query evaluates. In the AML example the query is a Gate in fdr mode:

HOLD_GATE = Gate(
    instructions=HOLD_QUESTION,
    calibration_profile="aml-holds-v1",
    guarantee="fdr",
    target=0.10,
    delta=0.10,
)

The context builder looks the alert up in the case system and renders it with alert_context(), the same function the dataset generator used to build every calibration context:

def hold_context(self, args: dict[str, Any]) -> dict[str, Any]:
    """The guard's context for a proposed hold: the case facts, rendered like the calibration set.
 
    A hold on an unknown alert, or on an account that is not the alert's,
    raises here; the guard then escalates instead of guessing.
    """
    alert = self.alerts.get(args.get("alert_id"))
    if alert is None:
        raise ValueError(f"alert {args.get('alert_id')!r} is not in the case system")
    if args.get("account_id") != alert["account_id"]:
        raise ValueError(f"account {args.get('account_id')!r} is not the account on alert {alert['alert_id']}")
    return alert_context(alert)

def alert_context(alert: dict[str, Any]) -> dict[str, Any]:
    """The context the guard evaluates for one alert, at calibration and at serving time.
 
    Only case facts go in; free text written by the agent (such as the
    ``reason`` argument of a hold) never does, because the calibration set
    was scored without it.
    """
    return {
        "guideline": list(GUIDELINE),
        "alert": {
            "alert_id": alert["alert_id"],
            "account_id": alert["account_id"],
            "monitoring_rule": alert["monitoring_rule"],
            "customer": dict(alert["customer"]),
            "activity_30d": dict(alert["activity_30d"]),
            "documentation": dict(alert["documentation"]),
            "prior_alerts_12m": alert["prior_alerts_12m"],
        },
    }

App-wide, with the plugin:

cases = CaseSystem(SCENARIO_ALERTS)
guard = ToolGuard(client, [
    GuardRule(tool="place_account_hold", query=HOLD_GATE, context=cases.hold_context),
])
agent = Agent(name="aml_investigator", model=model, instruction=INSTRUCTION,
              tools=list(make_tools(cases)))
# App-wide: the plugin guards place_account_hold for every agent the runner drives.
app = App(name=APP, root_agent=agent, plugins=[CLIGuardPlugin(guard)])
sessions = InMemorySessionService()
runner = Runner(app=app, session_service=sessions)

Per agent, with the callback (the trial example, a Belief rule with two thresholds):

ELIGIBILITY = Belief(instructions=ELIGIBILITY_QUESTION, calibration_profile="trial-screening-v1")

guard = ToolGuard(client, [
    GuardRule(tool="mark_eligible", query=ELIGIBILITY, context=system.eligibility_context,
              allow_above=0.85, block_below=0.15),
])

def build_agent(model: Any, guard: ToolGuard, system: ScreeningSystem) -> Agent:
    return Agent(
        name="trial_prescreener",
        model=model,
        instruction=INSTRUCTION,
        tools=list(make_tools(system)),
        before_tool_callback=cli_before_tool_callback(guard),
    )

The guarantee holds only for the scoring function it was calibrated with, and the context is part of that function. Build calibration and serving contexts with one function, so they have the same keys, the same policy text and the same rendering. Evaluate the facts from your system of record rather than the agent’s summary of them, and keep the agent’s free text (such as reason) out of the context, since the calibration set was scored without it. See Exchangeability.

Handle reviews and resume

An escalated call pauses the turn with an adk_request_confirmation function call. confirmation_requests(events) lists them, each with the request id, the tool, its arguments, the hint and, under cli, the guard’s decision with its guarantee card. Answer on the same session with confirmation_response(request_id, approved, payload):

for request in confirmation_requests(events):
    reviewed = reviewer.decide(case_id, request)
    answer = {"reviewer": "aml-analyst", "decision": "approved" if reviewed else "rejected"}
    events = await adk_models.run_turn(runner, user_id=USER, session_id=session.id,
                                       message=confirmation_response(request["id"], reviewed, answer))
    adk_models.print_turn(events, ran_status="hold_placed")

reviewer.decide is the examples’ stand-in review queue (scripted, or interactive with --interactive). ADK checks that the answer matches a pending request and that the original call’s name and arguments match the session history, then re-runs the original call. The guard sees tool_context.tool_confirmation, runs the tool only if it was approved, and records human_approved or human_rejected in the audit trail. Behind adk api_server or adk web, send the same function_response part (the request id, name adk_request_confirmation, and a response with confirmed) as the new message on the session.

In production the answer may arrive hours later, from another process. Keep the session, the request id and the decision payload in your review queue until a reviewer answers.

Example: AML account holds

examples/agents/google_adk/aml_account_hold_agent.py is an anti-money-laundering alert investigator. It reads an alert with get_alert (unguarded) and proposes place_account_hold(account_id, alert_id, reason) whenever a red-flag pattern from the written guideline is present or close; CLIGuardPlugin decides every proposed hold.

Primitive: Gate(guarantee="fdr", target=0.10, delta=0.10), a Learn-then-Test selective threshold. With 90% confidence over the calibration draw, at most 10% of the holds it places automatically are unwarranted, on alerts like the calibration set. It bounds the error rate among automatic holds, not over all alerts. The Gate never blocks here: it places the hold or sends it to an analyst.
Data: data/aml_alerts.jsonl, 360 synthetic alerts (191 warranted a hold, 169 did not) from data/generate_aml_alerts.py, labelled under a fictional investigation guideline: structuring, rapid movement of funds, wires to high-risk jurisdictions, activity above profile, and whether documentation explains them. The generator states how it adds genuine ambiguity: investigators decided on the full transaction history, of which the alert summary is a slightly lossy view.
Scenarios: A-5001, clear structuring, is held automatically. A-5002, a large deposit with a verified closing statement, escalates and the analyst rejects it. A-5003, with every measure just below a threshold, escalates and the analyst approves it.

python examples/agents/google_adk/aml_account_hold_agent.py                # mock: offline, keyless
python examples/agents/google_adk/aml_account_hold_agent.py --batch        # nightly queue with gate_batch
python examples/agents/google_adk/aml_account_hold_agent.py --interactive  # you are the analyst
python examples/agents/google_adk/aml_account_hold_agent.py --provider vllm

Output of a second run in mock mode. The first run is identical except that it prints calibrated now instead of cached, and reports calibration progress on stderr:

==============================================================================
Google ADK: AML account-hold investigator with a calibrated FDR guard
==============================================================================
Synthetic data for demonstration only. Not legal or regulatory advice. Account
actions and regulatory filings remain the responsibility of qualified
compliance staff.

agent model: mock; evidence model: mock
calibration: data/aml_alerts.jsonl (360 synthetic alerts)
profile: aml-holds-v1, LTT-selective, n=360 (minimum 40), serving, cached
------------------------------------------------------------------------------
A-5001: six cash deposits just under 10,000 USD at three branches in 30 days
    agent: get_alert(alert_id="A-5001")
    agent: place_account_hold(account_id="ACC-40317", alert_id="A-5001",
           reason="6 cash deposits between 8,000 and 9,999 USD; 88% of
           deposits sent out within 48 hours; deposits 14 times the expected
           monthly profile.")
    guard: ALLOW place_account_hold: auto-approved under the calibrated risk
           bound (confidence 1.000, calibrated threshold 0.610).
           guarantee: With 90% confidence, at most 0.1 of auto-approved
                      decisions are wrong (Learn-then-Test, fixed-sequence,
                      n=360).
    tool:  place_account_hold ran
    agent: Placed hold HOLD-A-5001 on ACC-40317 for alert A-5001.
    audit: session.state['cli_guard_audit'] = ['allow']
------------------------------------------------------------------------------
A-5002: a 243,000 USD deposit with a verified property-sale closing statement
    agent: get_alert(alert_id="A-5002")
    agent: place_account_hold(account_id="ACC-72950", alert_id="A-5002",
           reason="deposits 41 times the expected monthly profile.")
    guard: ESCALATE place_account_hold: below the calibrated auto-approval
           threshold (confidence 0.105, calibrated threshold 0.610).
           guarantee: With 90% confidence, at most 0.1 of auto-approved
                      decisions are wrong (Learn-then-Test, fixed-sequence,
                      n=360).
    tool:  place_account_hold did not run; waiting for review
    ADK:   adk_request_confirmation issued; the turn ends until a reviewer
           answers
    review request for A-5002:
      action:    place_account_hold({"account_id": "ACC-72950", "alert_id": "A-5002", "reason": "deposits 41 times the expected monthly profile."})
      why:       below the calibrated auto-approval threshold (confidence 0.105, calibrated threshold 0.610).
      guarantee: With 90% confidence, at most 0.1 of auto-approved decisions are wrong (Learn-then-Test, fixed-sequence, n=360).
    [simulated AML analyst] rejected
    guard: reviewer rejected the call
    tool:  place_account_hold did not run (rejected_by_reviewer)
    agent: An analyst declined the hold, so none was placed. I have not
           retried it; the alert stays with the analyst.
    audit: session.state['cli_guard_audit'] = ['escalate', 'human_rejected']
------------------------------------------------------------------------------
A-5003: every measure just below a guideline threshold; partial documentation
    agent: get_alert(alert_id="A-5003")
    agent: place_account_hold(account_id="ACC-18864", alert_id="A-5003",
           reason="2 cash deposits between 8,000 and 9,999 USD; 79% of
           deposits sent out within 48 hours; 18,500 USD wired to high-risk
           jurisdictions.")
    guard: ESCALATE place_account_hold: below the calibrated auto-approval
           threshold (confidence 0.373, calibrated threshold 0.610).
           guarantee: With 90% confidence, at most 0.1 of auto-approved
                      decisions are wrong (Learn-then-Test, fixed-sequence,
                      n=360).
    tool:  place_account_hold did not run; waiting for review
    ADK:   adk_request_confirmation issued; the turn ends until a reviewer
           answers
    review request for A-5003:
      action:    place_account_hold({"account_id": "ACC-18864", "alert_id": "A-5003", "reason": "2 cash deposits between 8,000 and 9,999 USD; 79% of deposits sent out within 48 hours; 18,500 USD wired to high-risk jurisdictions."})
      why:       below the calibrated auto-approval threshold (confidence 0.373, calibrated threshold 0.610).
      guarantee: With 90% confidence, at most 0.1 of auto-approved decisions are wrong (Learn-then-Test, fixed-sequence, n=360).
    [simulated AML analyst] approved
    guard: reviewer approved; ADK re-ran the original call
    tool:  place_account_hold ran
    agent: Placed hold HOLD-A-5003 on ACC-18864 for alert A-5003.
    audit: session.state['cli_guard_audit'] = ['escalate', 'human_approved']
------------------------------------------------------------------------------
The bundled calibration sets are small so the example runs quickly. For
production, calibrate on at least the recommended number of labelled examples
from your own traffic (about 1,000 for alpha=0.10), audit on held-out data,
and monitor for drift.

A-5002’s confidence (0.105) and A-5003’s (0.373) are below the calibrated threshold of 0.61, so both go to an analyst, who can weigh what the alert summary leaves out and decides them either way. Each turn pauses on adk_request_confirmation without running the tool, and resumes when the analyst answers.

Nightly batch: finite-sample FDR across a queue

--batch decides a queue of 40 alerts at once with client.gate_batch(contexts, HOLD_GATE): each alert gets a conformal p-value against the calibration alerts that did not warrant a hold, and Benjamini-Hochberg at level target selects the alerts to hold. The guarantee is about the batch: in expectation, at most 10% of the alerts it auto-holds are unwarranted. The queue comes from the calibration set’s generator, so it is exchangeable with it. No agent model is involved.

queue = generate(NIGHTLY_SIZE, seed=NIGHTLY_SEED, first_id=7001)
answers = client.gate_batch([row["context"] for row in queue], HOLD_GATE)
held = [row for row, answer in zip(queue, answers) if answer.approved]

==============================================================================
Google ADK: AML account-hold investigator with a calibrated FDR guard
==============================================================================
Synthetic data for demonstration only. Not legal or regulatory advice. Account
actions and regulatory filings remain the responsibility of qualified
compliance staff.

agent model: (not used in --batch); evidence model: mock
calibration: data/aml_alerts.jsonl (360 synthetic alerts)
profile: aml-holds-v1, LTT-selective, n=360 (minimum 40), serving, cached
------------------------------------------------------------------------------
Nightly queue: 40 synthetic alerts from the calibration set's generator (so
exchangeable with it), decided together with client.gate_batch().
    auto-held:        16
                      A-7004 A-7005 A-7009 A-7010 A-7014 A-7015 A-7016 A-7017
                      A-7019 A-7020 A-7024 A-7026 A-7028 A-7032 A-7036 A-7040
    sent to analysts: 24
    guarantee: Across this batch of 40, the expected fraction of approved
               items that are wrong is at most 0.1 (conformal selection with
               Benjamini-Hochberg, n=360).
    demo check, possible only because the labels are synthetic: 1 of the 16
    automatic holds was unwarranted, and 15 of the 18 warranted holds in the
    queue were placed automatically (the rest wait for an analyst). The bound
    is on the expected fraction across batches, so one batch can land above or
    below it.
------------------------------------------------------------------------------
The bundled calibration sets are small so the example runs quickly. For
production, calibrate on at least the recommended number of labelled examples
from your own traffic (about 1,000 for alpha=0.10), audit on held-out data,
and monitor for drift.

Example: clinical-trial pre-screening

examples/agents/google_adk/clinical_trial_screening_agent.py is a pre-screening assistant for a fictional protocol, SYN-CKD-201 (type 2 diabetes with stage 3 chronic kidney disease). It reads a record with get_prescreening_record and proposes mark_eligible(patient_id, trial_id), a pre-screening flag the study team confirms at the screening visit. The guard is a per-agent before_tool_callback.

Primitive: Belief(instructions="Does this patient meet every inclusion criterion and no exclusion criterion of the protocol in the context?"), which returns a Venn-Abers (IVAP) interval $[p_0, p_1]$ that brackets a calibrated probability under exchangeability. The rule marks automatically only when the whole interval is at or above 0.85, blocks only when it is entirely below 0.15, and sends everything else to a research coordinator. A Belief is a calibrated probability, not an error-rate bound: it does not promise that any patient is eligible, and it does not cap the error rate of automatic marks. For a stated bound on wrong automatic decisions, use a Gate.
Data: data/trial_screening.jsonl, 320 synthetic records (176 eligible at the screening visit, 144 not) from data/generate_trial_screening.py. The visit re-measures eGFR and HbA1c, so records near a cutoff are genuinely uncertain from the record alone.
Scenarios: P-6001, clearly eligible, is marked. P-6002, with eGFR 59 and HbA1c 10.4% at the edges of their ranges, escalates and the coordinator approves. P-6003 stopped an SGLT2 inhibitor 5 weeks ago, inside the protocol’s 12-week exclusion, and is blocked: the agent is told it cannot mark the patient. The scripted agent checks age, diagnosis and labs but not medications, which is why it proposes P-6003 at all; that is the kind of mistake the guard is there for.

python examples/agents/google_adk/clinical_trial_screening_agent.py                # mock: offline, keyless
python examples/agents/google_adk/clinical_trial_screening_agent.py --interactive  # you are the coordinator
python examples/agents/google_adk/clinical_trial_screening_agent.py --provider anthropic --evidence-provider vllm

Output of a second run in mock mode (the first differs only in calibrated now and the calibration progress on stderr):

==============================================================================
Google ADK: clinical-trial pre-screening with a Venn-Abers guard
==============================================================================
Synthetic data for demonstration only. Not medical advice and not a medical
device. A calibrated guarantee bounds error rates on data like the calibration
set; it does not make any single recommendation safe. Keep a qualified
clinician in the loop and follow your institution's clinical governance.

agent model: mock; evidence model: mock
calibration: data/trial_screening.jsonl (320 synthetic records)
profile: trial-screening-v1, IVAP, n=320 (minimum 20), serving, cached
rule: allow if p0 >= 0.85; block if p1 < 0.15; otherwise a coordinator decides
------------------------------------------------------------------------------
P-6001: age 58, eGFR 44, HbA1c 8.1%, no exclusions on record
    agent: get_prescreening_record(patient_id="P-6001")
    agent: mark_eligible(patient_id="P-6001", trial_id="SYN-CKD-201")
    guard: ALLOW mark_eligible: calibrated probability in [0.93, 1.00],
           entirely at or above 0.85.
           guarantee: Venn-Abers pair [0.928, 1.000]: the probability computed
                      under the true label is calibrated on data exchangeable
                      with the 320 calibration examples; a wide pair means the
                      calibration data cannot pin the probability down.
    tool:  mark_eligible ran
    agent: Marked P-6001 as pre-screen eligible for SYN-CKD-201; the study
           team confirms eligibility at the screening visit.
    audit: session.state['cli_guard_audit'] = ['allow']
------------------------------------------------------------------------------
P-6002: eGFR 59 and HbA1c 10.4%, both at the edge of re-measured ranges
    agent: get_prescreening_record(patient_id="P-6002")
    agent: mark_eligible(patient_id="P-6002", trial_id="SYN-CKD-201")
    guard: ESCALATE mark_eligible: calibrated probability in [0.62, 0.67]: the
           calibration data cannot settle it.
           guarantee: Venn-Abers pair [0.619, 0.667]: the probability computed
                      under the true label is calibrated on data exchangeable
                      with the 320 calibration examples; a wide pair means the
                      calibration data cannot pin the probability down.
    tool:  mark_eligible did not run; waiting for review
    ADK:   adk_request_confirmation issued; the turn ends until a reviewer
           answers
    review request for P-6002:
      action:    mark_eligible({"patient_id": "P-6002", "trial_id": "SYN-CKD-201"})
      why:       calibrated probability in [0.62, 0.67]: the calibration data cannot settle it.
      guarantee: Venn-Abers pair [0.619, 0.667]: the probability computed under the true label is calibrated on data exchangeable with the 320 calibration examples; a wide pair means the calibration data cannot pin the probability down.
    [simulated research coordinator] approved
    guard: reviewer approved; ADK re-ran the original call
    tool:  mark_eligible ran
    agent: Marked P-6002 as pre-screen eligible for SYN-CKD-201; the study
           team confirms eligibility at the screening visit.
    audit: session.state['cli_guard_audit'] = ['escalate', 'human_approved']
------------------------------------------------------------------------------
P-6003: dapagliflozin (an SGLT2 inhibitor) stopped 5 weeks ago
    agent: get_prescreening_record(patient_id="P-6003")
    agent: mark_eligible(patient_id="P-6003", trial_id="SYN-CKD-201")
    guard: BLOCK mark_eligible: calibrated probability in [0.00, 0.02],
           entirely below 0.15.
           guarantee: Venn-Abers pair [0.000, 0.019]: the probability computed
                      under the true label is calibrated on data exchangeable
                      with the 320 calibration examples; a wide pair means the
                      calibration data cannot pin the probability down.
    tool:  mark_eligible did not run (blocked)
    agent: I cannot mark P-6003 as eligible for SYN-CKD-201: the calibrated
           check blocked it. The record stays in the coordinator's standard
           screening queue.
    audit: session.state['cli_guard_audit'] = ['block']
------------------------------------------------------------------------------
The bundled calibration sets are small so the example runs quickly. For
production, calibrate on at least the recommended number of labelled examples
from your own traffic (about 1,000 for alpha=0.10), audit on held-out data,
and monitor for drift.

A block is not a clinical judgment. It means the calibrated probability that a record like this one qualifies is entirely below 0.15, so the automated path does not mark it; the record stays in the coordinator’s normal screening workflow.

Pitfalls

Resume with the confirmation id, on the same session. Answer with the id of the adk_request_confirmation call (what confirmation_requests returns), not the original tool call’s id. ADK also checks the original call’s name and arguments against the session history and raises ValueError on a mismatch.
A paused turn has no final text. The adk_request_confirmation event reports is_final_response() as True but carries only a function call. Code that takes the first final response’s text gets nothing; scan the turn’s events, as confirmation_requests does.
Tool confirmation is experimental in ADK 2.x. It emits an [EXPERIMENTAL] warning, works with InMemorySessionService, and ADK’s documentation lists DatabaseSessionService and VertexAiSessionService as unsupported. Check the current ADK documentation before relying on a persistent session store for paused reviews.
Register the plugin on the App. Runner(plugins=[...]) still works in ADK 2.9.2 but emits a DeprecationWarning; pass App(name=..., root_agent=..., plugins=[CLIGuardPlugin(guard)]) to Runner(app=...) instead.
Writing your own hook? Any non-None return from a before_tool_callback skips the tool, even {}. When you escalate with request_confirmation, also set tool_context.actions.skip_summarization = True, or ADK calls the model again in the same turn and a real model narrates the pending placeholder. Plugin hooks receive tool_args=, not args=. The adapter handles all three.
after_tool_callback runs for skipped calls too. It receives the guard’s blocked or pending_human_review dict as tool_response, so audit code there must not assume the tool ran.
FunctionTool(require_confirmation=...) is yes or no only. It cannot express block versus escalate, and its predicate runs three times per confirmed call. Use the callback or plugin for calibrated three-way decisions.
Model strings. A bare "claude-..." string routes to Claude on Vertex AI; use AnthropicLlm or LiteLlm("anthropic/...") for an Anthropic API key. Use hosted_vllm/, not vllm/, for a vLLM server, and openai/ plus api_base and an API key for SGLang. The served model name must match the name after the prefix, and self-hosted servers need their tool-call parser enabled or the agent never calls a tool.
google-adk alone does not install LiteLLM or the Anthropic SDK. Use google-adk[extensions] for OpenAI, Azure OpenAI, Claude and Gemma agent models.

Local mode: LocalCLIClient, profiles, fingerprints, gate_batch and fail-closed behavior.
Bring your own model: agent and evidence models, access levels, and every provider.
Risk-sensitive domains: choosing a primitive for a decision, and the governance around it.
Gate and Belief: the two primitives these examples use.
Guarantees: what each guarantee states, and what it does not.

LangGraph Microsoft Agent Framework