Microsoft Agent Framework
cli_sdk.integrations.agent_framework puts a calibrated check in front of
the tools that carry risk in a Microsoft Agent Framework 1.x agent. One
FunctionMiddleware, CLIGuardMiddleware, asks a
ToolGuard about every proposed call to a guarded
tool and maps the answer onto the framework’s own mechanisms: the tool
runs, the run pauses with a native tool-approval request for a person, or
the tool is skipped and the model is told why. Tools without a rule run as
usual.
The two examples at the end run offline, without keys, on synthetic data,
and the output shown is what they printed with agent-framework-core
1.19.0. The self-hosted path (OpenAIChatCompletionClient plus a vLLM
evidence backend) was also exercised against a local stand-in server; the
other connectors were constructed with dummy keys but not called, and no
live provider API was used.
Why a calibrated guard, not a confidence threshold
The usual first guardrail for an agent action is a hand-set threshold: record the credit tier when the model says it is at least 0.9 sure. That number has no stated meaning. Model confidence is not calibrated, it moves when the model, the prompt or the traffic changes, and nobody can say how often a call that clears it is wrong.
The guard replaces the number with a statement tied to labelled data. In the credit example below, the calibrated prediction set contains the tier the policy assigns for at least 90% of applications like the calibration files, separately within each application channel, and the tool runs only when that set is exactly the allowed tier the agent proposed. The threshold behind it is computed from labelled examples, the profile records which evidence model and prompt produced it (change either and the profile goes stale, so the guard escalates every call until you recalibrate), and it can be audited on fresh data. Because an automatic tier is wrong only when its set missed, at most 10% of applications like the calibration files are recorded automatically with a wrong tier. That is a rate over all applications on data like the calibration set; it does not make any single decision correct, which is why everything the guard cannot settle goes to a person.
Install
pip install "cci-sdk[agent-framework,openai]" agent-framework-openaiThe agent-framework extra installs agent-framework-core>=1.19, which
provides Agent, AgentSession, tool, FunctionMiddleware and the tool
approval flow. Use 1.19.0 or later: it contains the fixes for approvals
being bypassed when another call precedes an approval-required call in the
same batch (#8079) and for resuming an approval when the agent has a
context provider (#8410). The model connectors are separate packages, and
everything needs Python 3.10 or later.
| Agent model | Connector package | Evidence model extra |
|---|---|---|
| OpenAI, Azure OpenAI, Gemma on vLLM or SGLang | agent-framework-openai | cci-sdk[openai] |
| Anthropic Claude | agent-framework-anthropic (beta, 1.0.0b260918) | see the note below |
| Google Gemini | agent-framework-gemini (beta, 1.0.0b260910) | cci-sdk[openai] (Gemini evidence uses its OpenAI-compatible endpoint) |
The Anthropic and Gemini connectors are pre-releases with date-stamped
versions; pin the exact version you tested. Do not install the
agent-framework meta-package for this: it pulls in about 30
sub-packages, and agent-framework-core plus the connector you use is
enough.
agent-framework-anthropic 1.0.0b260918 requires anthropic<0.117, and
the cci-sdk[anthropic] extra requires anthropic>=1.0, so pip cannot
install both in one environment. With Claude as the agent model, install
pip install "cci-sdk[agent-framework,openai]" agent-framework-anthropic
and score with a log-probability evidence model
(--provider anthropic --evidence-provider openai, or vllm). If you
want Claude as the evidence model too, AnthropicEvidenceBackend sent the
same request with anthropic 0.116.0 (the version the connector installs)
as with 1.8.0 in a check against a local stand-in server; that check did
not call the live API.
How decisions map onto the framework
| Guard decision | What CLIGuardMiddleware does | What the framework does |
|---|---|---|
allow | await call_next() | The tool runs and the model sees its result. |
escalate | Stores a ticket (the request id and a SHA-256 fingerprint of the tool name and arguments) in session.state["cli_guard"], sets context.result to a function_approval_request and raises MiddlewareTermination | The run pauses. result.user_input_requests holds the request, with the guard’s decision in additional_properties["cli"]. |
block | Sets context.result to a JSON not_executed result with the reason, without calling the tool | The model receives that result and carries on (it should explain, not retry). |
| Reviewer approves | On replay, runs the tool only if a ticket exists for that request and the arguments still match its fingerprint; otherwise raises MiddlewareFailure | The paused call runs. |
| Reviewer rejects | Not called again | The model receives Error: Tool call invocation was rejected by user. and the tool never runs. |
| Escalation without a session | Raises MiddlewareFailure | The run aborts and the exception reaches the caller of agent.run; the tool does not run. |
The guard fails closed: a missing or stale calibration profile, an
evidence-model error or a context that cannot be built all escalate, and
never allow. pending_reviews(result) lists the paused requests as
{"request", "tool", "arguments", "cli"}, and review_message(review, approved) builds the message that answers one. Guarded tools keep the
default approval_mode ("never_require"): the middleware decides which
calls need a person.
Set up a guarded agent
Choose the agent model
The agent model only has to call tools well; the guard scores each
proposed call with its own evidence model (next step), which can be a
different provider. The examples build the agent model with
chat_client(provider) from
examples/agents/_shared/maf_models.py, reading keys from
environment variables (see examples/agents/.env.example). While a key is
still a placeholder, the script stops with a message before any request.
pip install "cci-sdk[agent-framework,openai]" agent-framework-openai
export OPENAI_API_KEY=... # your key
export OPENAI_MODEL=gpt-4.1-mini # default
python examples/agents/agent_framework/credit_underwriting_agent.py --provider openaiOpenAIChatClient calls the OpenAI Responses API. The same
gpt-4.1-mini default serves as a log-probability (L1) evidence model;
prefer a dated snapshot such as gpt-4.1-mini-2025-04-14 for the
evidence model, so a model update never silently changes the scoring
function behind a profile.
The factory, from examples/agents/_shared/maf_models.py:
def chat_client(provider: str, *, planner: Optional[Planner] = None) -> Any:
"""The agent's chat client for ``provider``. ``planner`` drives ``--provider mock``.
Real providers stop with a clear message while their key is a placeholder,
before any connector is imported or any request is sent.
"""
if provider == "mock":
if planner is None:
raise ValueError("--provider mock needs a planner for the scripted model")
return ScriptedChatClient(planner=planner)
s = settings(provider).require_key()
try:
if provider == "openai":
from agent_framework.openai import OpenAIChatClient
return OpenAIChatClient(model=s.model, api_key=s.api_key, base_url=s.base_url)
if provider == "azure":
from agent_framework.openai import OpenAIChatCompletionClient
# model is the deployment name. For Entra ID, pass credential=... instead of api_key.
return OpenAIChatCompletionClient(model=s.model, azure_endpoint=s.base_url, api_key=s.api_key,
api_version=s.api_version)
if provider == "anthropic":
from agent_framework.anthropic import AnthropicClient
return AnthropicClient(model=s.model, api_key=s.api_key)
if provider == "gemini":
from agent_framework.gemini import GeminiChatClient
return GeminiChatClient(model=s.model, api_key=s.api_key)
if provider in ("vllm", "sglang"):
from agent_framework.openai import OpenAIChatCompletionClient
return OpenAIChatCompletionClient(model=s.model, base_url=s.base_url, api_key=s.api_key)
except ImportError as exc:
raise SystemExit(f"--provider {provider} needs the Agent Framework connector: "
f"pip install {CONNECTORS[provider]} ({exc})") from exc
raise SystemExit(f"no Agent Framework chat client for provider {provider!r}")--provider mock (the default) uses ScriptedChatClient, a keyless
BaseChatClient composed with the framework’s FunctionInvocationLayer,
ChatMiddlewareLayer and ChatTelemetryLayer, so the tool loop, the
middleware and the approval flow are the framework’s own. It proposes a
scripted call for each case and summarizes the tool result.
Choose the evidence model and calibrate
The evidence model answers one closed question about each proposed call, and the guard calibrates its answers on labelled examples. Two access levels matter here (the access ladder has the rest):
- L1, token log-probabilities. One request per score, at temperature 0, reading the probability of each option’s letter. OpenAI and Azure OpenAI non-reasoning configurations, and Gemma on vLLM or SGLang.
- L0, sampling.
sample_countsampled answers per score (the examples use 8 for Anthropic and Gemini), turned into smoothed frequencies. Claude and Gemini. Scores are coarser, so more calls escalate.
Calibration scores every labelled example once and is cached as a JSON
profile. For the credit example’s 320 files that is 320 requests at L1 or
2,560 at L0; for the pharmacovigilance example’s 300 cases, 300 or 2,400.
Each guard check afterwards costs one request at L1 or eight at L0. The
query, from examples/agents/agent_framework/credit_underwriting_agent.py:
TIER_QUERY = Set(
instructions={
"question": "Which risk tier does the tiering policy assign to this consumer-loan application?",
"tiering_policy": list(credit_data.TIERING_POLICY),
},
options=TIER_OPTIONS,
calibration_profile="credit-tiers-v1",
alpha=0.10,
method="APS",
group_by="channel",
)and the calibration, in run():
store = calibration.default_store(__file__, args.store)
client = LocalCLIClient(evidence, store=store, sample_count=providers.sample_count(evidence_provider))
examples = calibration.load_jsonl(DATASET)
try:
calibration.ensure_calibrated(client, [TIER_QUERY], examples, recalibrate=args.recalibrate)
except EvidenceError as exc:
raise SystemExit(f"calibration stopped: the {evidence_provider} evidence model failed ({exc})") from excevidence is providers.evidence_backend(evidence_provider, ...). The
profile is rebuilt when the query’s instructions or options change, or when
a different evidence model or configuration is used; until then, a stale
profile gives heuristic answers and the guard escalates every call. See
Local mode for sizes, fingerprints and audits.
Wire the guard
The rule names the tool, the query, and how to build the evaluation context from the call. Build it from your system of record, not from the agent’s own description of the case, and with the same function that built the calibration contexts: the guarantee holds only for contexts scored the way the calibration contexts were (see Exchangeability).
@tool
def assign_risk_tier(
application_id: Annotated[str, Field(description="The application id, for example L-7001.")],
tier: Annotated[Literal["A", "B", "C", "D", "E"],
Field(description="Risk tier from A (lowest risk) to E (highest risk).")],
) -> str:
"""Record the risk tier for a consumer-loan application in the loan origination system."""
TIER_ASSIGNMENTS[application_id] = tier
return f"Tier {tier} recorded for application {application_id}."
def guard_context(arguments: Mapping[str, Any]) -> dict[str, Any]:
"""The evaluation context for a proposed call.
It is the application file from the system of record, rendered by the
same ``application_context`` function that built the calibration data,
so inference and calibration contexts have identical keys and wording.
"""
return credit_data.application_context(APPLICATIONS[arguments["application_id"]])
def build_guard(client: LocalCLIClient) -> ToolGuard:
return ToolGuard(client, [GuardRule(
tool=TOOL,
query=TIER_QUERY,
context=guard_context,
allow_labels=["A", "B", "C"], # D and E always go to an underwriter
match_argument="tier", # the set must be exactly the tier the agent proposed
)])credit_data is examples/agents/data/generate_credit_applications.py,
the generator that wrote the calibration data. Then register the
middleware on the agent:
def make_agent() -> Agent:
return Agent(client=agent_model, name="underwriting-assistant", instructions=INSTRUCTIONS,
tools=[assign_risk_tier], middleware=[CLIGuardMiddleware(guard)])A context that cannot be built (an unknown application id, a value outside the calibrated vocabulary) escalates.
Handle reviews and resume
Run the agent with an AgentSession: approvals are bound to it. When the
guard escalates, agent.run returns with the request pending. To resume
later, possibly in another process, persist both the session and the
request, then answer the request on the restored session. From the credit
example:
def save_pending(path: Path, session: AgentSession, review: Mapping[str, Any]) -> None:
"""Persist everything needed to resume after the review: session (history and ticket) and request."""
path.parent.mkdir(parents=True, exist_ok=True)
record = {"session": session.to_dict(), "request": review["request"].to_dict(), "cli": review["cli"]}
path.write_text(json.dumps(record, indent=2, sort_keys=True, default=str) + "\n", encoding="utf-8") reviews = pending_reviews(result)
if reviews:
# The run is paused. Persist the session and the approval request; the process could exit here.
path = pending_dir / f"{case_id}.json"
save_pending(path, session, reviews[0])
print(f" run paused for an underwriter; session and approval request saved to "
f"{path.parent.name}/{path.name}")
# Later, possibly in another process: load, decide, and resume with a fresh agent.
saved = json.loads(path.read_text(encoding="utf-8"))
approved = reviewer.decide(case_id, saved)
resumed_agent = make_agent()
resumed_session = AgentSession.from_dict(saved["session"])
request = Content.from_dict(saved["request"])
result = await resumed_agent.run(review_message(request, approved), session=resumed_session)
path.unlink()The saved session holds the conversation, the guard’s ticket and the framework’s approval state. The conversation includes the case data, so store the file with the same access controls as the system of record. When the process stays up, answer on the same session instead, as the pharmacovigilance example does:
reviews = pending_reviews(result)
if reviews:
# Paused for a safety physician. Here the session stays in memory and the
# run resumes in the same process; see credit_underwriting_agent.py for
# persisting the session to JSON and resuming elsewhere.
approved = reviewer.decide(case_id, reviews[0])
result = await agent.run(review_message(reviews[0], approved), session=session)reviewer is the examples’ stand-in review queue; in production the
decision comes from your reviewers’ queue. With stream=True, collect the
run with async for update in stream, then call
pending_reviews(await stream.get_final_response()); the answer is sent
the same way.
Example: credit underwriting
examples/agents/agent_framework/credit_underwriting_agent.py
A consumer-lending underwriting assistant records a risk tier from A
(lowest risk) to E (highest) for each application. The guard is a Set
over the five tiers with APS at alpha=0.10 and group_by="channel":
Mondrian calibration fits one threshold per application channel (branch,
online, broker), so the 90% coverage holds within each channel rather than
only on average across them. The rule allows the call only when the set is
exactly the proposed tier and that tier is A, B or C; tiers D and E always
go to an underwriter.
The data is examples/agents/data/credit_applications.jsonl, 320 synthetic
files written by generate_credit_applications.py (seeded, standard
library only). Labels come from a written, invented tiering policy (points
for credit score band, debt-to-income, payment history, a thin file and
unverified income, plus overrides), with about 4% of files moved one tier
by a documented underwriter override that the file does not explain. The
broker channel is deliberately small (40 files) so its per-channel status
is visibly less certain.
git clone https://github.com/rahvis/cci-sdk && cd cci-sdk
python examples/agents/agent_framework/credit_underwriting_agent.pyThe first run calibrates once and prints progress to stderr
(calibrating 'credit-tiers-v1' on 320 labelled examples ...); later runs
reuse .cli_profiles/credit-tiers-v1.json and print the same decisions.
The output, in mock mode:
==============================================================================
Credit underwriting assistant (Microsoft Agent Framework) with a calibrated tier guard
==============================================================================
Synthetic data for demonstration only. Not financial, credit or legal advice.
Calibrated guarantees support, but do not replace, your model-risk-management,
fair-lending and compliance review.
agent model: mock; evidence model: mock
calibration: credit-tiers-v1: APS, n=320 (minimum 9, recommended 1000), realized-coverage CI [0.871, 0.926], status=serving
per-channel (Mondrian) status, alpha=0.1:
branch n=140 calibrated realized coverage 90% interval [0.856, 0.939]
broker n=40 calibrated realized coverage 90% interval [0.817, 0.965]
online n=140 calibrated realized coverage 90% interval [0.856, 0.939]
------------------------------------------------------------------------------
L-7001: strong file, branch channel
agent proposed: assign_risk_tier({"application_id": "L-7001", "tier": "A"})
guard: ALLOW assign_risk_tier -> the calibrated prediction set is exactly {A}.
guarantee: Contains the correct answer at least 90% of the time on profile 'credit-tiers-v1' (n=140).
group-conditional: channel 'branch', n=140 of the profile's 320 files
tool executed: yes, tier A recorded
agent: Done. Tier A recorded for application L-7001.
------------------------------------------------------------------------------
L-7002: thin credit file, online channel
agent proposed: assign_risk_tier({"application_id": "L-7002", "tier": "B"})
guard: ESCALATE assign_risk_tier -> the calibrated prediction set is {B, C}, not a single allowed label.
guarantee: Contains the correct answer at least 90% of the time on profile 'credit-tiers-v1' (n=140).
group-conditional: channel 'online', n=140 of the profile's 320 files
run paused for an underwriter; session and approval request saved to pending_reviews/L-7002.json
review request for L-7002:
action: assign_risk_tier({"application_id": "L-7002", "tier": "B"})
why: the calibrated prediction set is {B, C}, not a single allowed label.
guarantee: Contains the correct answer at least 90% of the time on profile 'credit-tiers-v1' (n=140).
[simulated underwriter] approved
resumed from L-7002.json with a new agent instance; pending file removed
tool executed: yes, tier B recorded
agent: Done. Tier B recorded for application L-7002.
------------------------------------------------------------------------------
L-7003: weak file submitted by a broker
agent proposed: assign_risk_tier({"application_id": "L-7003", "tier": "D"})
guard: ESCALATE assign_risk_tier -> the calibrated prediction set is {D, E}, not a single allowed label.
guarantee: Contains the correct answer at least 90% of the time on profile 'credit-tiers-v1' (n=40).
group-conditional: channel 'broker', n=40 of the profile's 320 files
run paused for an underwriter; session and approval request saved to pending_reviews/L-7003.json
review request for L-7003:
action: assign_risk_tier({"application_id": "L-7003", "tier": "D"})
why: the calibrated prediction set is {D, E}, not a single allowed label.
guarantee: Contains the correct answer at least 90% of the time on profile 'credit-tiers-v1' (n=40).
[simulated underwriter] rejected
resumed from L-7003.json with a new agent instance; pending file removed
tool executed: no, nothing recorded
agent: The reviewer rejected the proposed action, so it was not carried out.
------------------------------------------------------------------------------
The bundled calibration sets are small so the example runs quickly. For
production, calibrate on at least the recommended number of labelled examples
from your own traffic (about 1,000 for alpha=0.10), audit on held-out data,
and monitor for drift.
Fair-lending note: the features exclude protected characteristics (age, sex,
race, ethnicity, religion, national origin, marital status) and obvious
proxies such as ZIP code. Per-channel calibration shows the coverage guarantee
holds within each channel, not only on average. It is not a fair-lending
analysis: review the policy, and the guard's escalation rates by segment, with
your fair-lending and model-risk teams.What happened:
- Per-channel status. Every channel has at least the minimum 9 files
for
alpha=0.10, so each iscalibratedand gets its own threshold. The realized-coverage interval shows how much the delivered coverage varies with the calibration draw at each size: 90% of calibration sets of 40 files deliver between 81.7% and 96.5% coverage, against 85.6% to 93.9% with 140 files. The guarantee (at least 90% on average over draws) holds either way; the smaller channel is simply less certain. A channel below the minimum would be listed asunderpoweredand fall back to the marginal threshold, and its guarantee cards would carry no group. - L-7001 is a strong branch file. The set is
{A}, the tier the agent proposed, so the tool ran without review. - L-7002 has a thin credit file. The set is
{B, C}: at 90% coverage the data cannot tell the two tiers apart for this file, so the run paused. The session and request were written topending_reviews/L-7002.json, read back into a new agent instance, and the underwriter’s approval ran the call the guard had paused. - L-7003 is a weak broker file with a proposed tier D. D is never
allowed automatically, and the set
{D, E}would not be a single tier anyway. The underwriter rejected the proposal and nothing was recorded; the tier is then set by the underwriter outside the agent.
The guarantee printed with each decision is about sets, over applications
like that channel’s calibration files: at most 10% of them get a set that
misses the policy tier. An automatically recorded tier is wrong only when
its set missed, so at most 10% of those applications are recorded
automatically with a wrong tier. That bound is over all applications, not
over the recorded ones: when few files are recorded automatically, the
error rate among them can be higher. To bound the error rate among
automatic decisions directly, use a Gate with
guarantee="fdr". None of this is a statement about one application in
isolation, and channel-conditional coverage is one input to a fair-lending
review, not a substitute for one.
Example: pharmacovigilance
examples/agents/agent_framework/pharmacovigilance_agent.py
A drug-safety case processing assistant decides whether to submit an
expedited report for an adverse-event case. The guard is an Interval over
the ordered levels grade_1 to grade_5 (mild, moderate, severe,
life-threatening, fatal), ordinal APS at alpha=0.10: the interval
contains the grade an assessor records for at least 90% of cases like the
calibration set. The rule looks at the whole interval:
| Interval | Decision | Why |
|---|---|---|
entirely grade_3 to grade_5 | allow: submit automatically | Serious under the synthetic guide; over-reporting is the conservative direction. |
entirely grade_1 to grade_2 | block: no expedited report | The case stays in routine case processing and periodic reporting. |
| crosses the boundary | escalate: a safety physician decides | The calibration data cannot settle which side the case is on. |
def build_guard(client: LocalCLIClient) -> ToolGuard:
return ToolGuard(client, [GuardRule(
tool=TOOL,
query=SERIOUSNESS_QUERY,
context=guard_context,
allow_levels=SERIOUS, # the whole interval is serious: submit automatically
block_levels=NON_SERIOUS, # the whole interval is non-serious: routine periodic reporting
)]) # anything that crosses the boundary: a safety physician decidesThe data is examples/agents/data/adverse_events.jsonl, 300 synthetic
cases for a fictitious product, written by generate_adverse_events.py.
Labels follow an invented grading guide (the highest grade any finding
meets: outcome, hospitalization, treatment, laboratory changes, effect on
daily activities). Overnight observation stays are left to the assessor’s
judgement, with narrative wording that only partly settles them, and about
3% of cases carry a one-grade coding difference.
git clone https://github.com/rahvis/cci-sdk && cd cci-sdk
python examples/agents/agent_framework/pharmacovigilance_agent.py==============================================================================
Drug-safety case processing assistant (Microsoft Agent Framework) with a calibrated seriousness guard
==============================================================================
Synthetic data for demonstration only. Not medical advice and not a medical
device. A calibrated guarantee bounds error rates on data like the calibration
set; it does not make any single recommendation safe. Keep a qualified
clinician in the loop and follow your institution's clinical governance.
agent model: mock; evidence model: mock
calibration: ae-seriousness-v1: ordinal-aps, n=300 (minimum 9, recommended 1000), realized-coverage CI [0.871, 0.927], status=serving
------------------------------------------------------------------------------
AE-8001: hospital admission with intravenous therapy
agent proposed: submit_expedited_report({"case_id": "AE-8001", "seriousness_grade": "grade_3"})
guard: ALLOW submit_expedited_report -> the calibrated interval [grade_3, grade_3] lies inside the allowed levels.
guarantee: Contains the correct answer at least 90% of the time on profile 'ae-seriousness-v1' (n=300).
calibrated interval: [grade_3, grade_3] (severe)
tool executed: yes, expedited report submitted (grade_3)
agent: Done. Expedited report for case AE-8001 submitted with grade_3.
------------------------------------------------------------------------------
AE-8002: overnight observation stay, borderline
agent proposed: submit_expedited_report({"case_id": "AE-8002", "seriousness_grade": "grade_3"})
guard: ESCALATE submit_expedited_report -> the calibrated interval [grade_2, grade_3] crosses a decision boundary.
guarantee: Contains the correct answer at least 90% of the time on profile 'ae-seriousness-v1' (n=300).
calibrated interval: [grade_2, grade_3] (moderate to severe)
review request for AE-8002:
action: submit_expedited_report({"case_id": "AE-8002", "seriousness_grade": "grade_3"})
why: the calibrated interval [grade_2, grade_3] crosses a decision boundary.
guarantee: Contains the correct answer at least 90% of the time on profile 'ae-seriousness-v1' (n=300).
[simulated safety physician] approved
tool executed: yes, expedited report submitted (grade_3)
agent: Done. Expedited report for case AE-8002 submitted with grade_3.
------------------------------------------------------------------------------
AE-8003: mild rash treated over the counter
agent proposed: submit_expedited_report({"case_id": "AE-8003", "seriousness_grade": "grade_2"})
guard: BLOCK submit_expedited_report -> the calibrated interval [grade_1, grade_1] lies inside the blocked levels.
guarantee: Contains the correct answer at least 90% of the time on profile 'ae-seriousness-v1' (n=300).
calibrated interval: [grade_1, grade_1] (mild)
tool executed: no; not expedited, the case stays in routine periodic reporting
agent: The action was not carried out automatically: the calibrated interval [grade_1, grade_1] lies inside the blocked levels.
------------------------------------------------------------------------------
The bundled calibration sets are small so the example runs quickly. For
production, calibrate on at least the recommended number of labelled examples
from your own traffic (about 1,000 for alpha=0.10), audit on held-out data,
and monitor for drift.The scripted agent is deliberately over-cautious and proposes a report for
every case, which is the failure the block rule exists for; a real model
may not propose one for AE-8003 at all. AE-8002’s overnight observation is
exactly the judgement call the guide leaves open, and the interval
[grade_2, grade_3] says so, so a physician decided. Here the run resumed
on the same in-memory session.
Blocking is the direction that can under-report. Because the interval
misses the recorded grade for at most 10% of cases like the calibration
set, at most 10% of all such cases can be blocked while their recorded
grade is serious. That is a demonstration setting. A real deployment
would use a far smaller alpha for this decision (for example 0.01,
which needs at least 99 calibration cases and about 2,500 are
recommended), or send blocked cases to routine physician review as well.
The guard decides whether a report is expedited, not which grade the
agent writes into it. The synthetic guide is not a regulatory standard:
real expedited reporting depends on seriousness criteria, expectedness
and causality, and stays with qualified safety staff.
Pitfalls
- Older API names do not import in 1.19. Blog posts and older samples
use
ChatAgent,@ai_function,AgentThreadandget_new_thread(),model_id=,AzureOpenAIChatClient,agent.run_stream(),FunctionApprovalRequestContentand middleware written asawait next(context). UseAgent,@tool,AgentSessionandagent.create_session(),model=,OpenAIChatCompletionClient(azure_endpoint=...),agent.run(..., stream=True),Contentwithtype == "function_approval_request", andawait call_next()with no argument. OpenAIChatClientspeaks the Responses API. Pointed at vLLM or SGLang, which serve Chat Completions, it fails with aChatClientException. UseOpenAIChatCompletionClient(base_url=..., api_key="EMPTY", model=<served name>)for self-hosted models.- Escalation needs an
AgentSession. Without one, approval state lives only in the transcript you send back, and a caller could forge a request and approval pair. The middleware therefore refuses to escalate without a session (MiddlewareFailure) and, with one, runs a call only for an approval it issued, for the same arguments. - Persist the session and the request. Resuming needs
session.to_dict()and the request’sto_dict(). Editing the stored request’s arguments does not change what runs: the framework executes the call it paused, which is the call the guard escalated. - A plain exception in function middleware fails open. The framework
turns it into a tool error (
Error: Function failed.) and the loop keeps going. If you write your own middleware, raiseMiddlewareFailureto abort the run, or setcontext.resultand return to skip the tool, asCLIGuardMiddlewaredoes. - Rejected requests keep their ticket. On rejection the framework does
not call the middleware again, so the ticket stays in
session.state["cli_guard"]. It cannot be replayed: the framework ignores an approval for a request that is no longer pending and logsIgnored an approval response ... did not match the active approval occurrence identity. Prune the tickets if you keep sessions for a long time. - Several guarded calls in one turn. When the model proposes an allowed call and an escalated call in the same turn, the allowed call runs at once and the escalated one pauses; after the review, the model receives both results.
- Self-hosted Gemma needs the tool-call parser. Start vLLM with
--enable-auto-tool-choice --tool-call-parser gemma4and SGLang with--tool-call-parser gemma4. Without them, tool calls come back as plain text and the guard never runs. - Test doubles need the framework layers. A bare
BaseChatClientsubclass returns function calls without running them, or the middleware. ComposeFunctionInvocationLayer, ChatMiddlewareLayer, ChatTelemetryLayerin front of your class, in that order, and write_inner_get_responseas a plaindefthat returns an awaitable (or aResponseStreamwhen streaming), asScriptedChatClientinexamples/agents/_shared/maf_models.pydoes. - Avoid a non-approval
MiddlewareTerminationfor blocking. Ending a run that way can drop earlier tool-loop turns from the history (open issue #8455). The middleware blocks by returning a result instead; keep it that way if you extend it.
Related
- Local mode: calibration, profiles, fingerprints and
audits for
LocalCLIClient. - Bring your own model: evidence backends and access levels per provider.
- Risk-sensitive domains: how to use calibrated guards in finance and healthcare workflows.
SetandInterval: the primitives behind the two examples.- Guarantees: what each guarantee card means.