Gate

Replace a hand-picked confidence threshold with a calibrated accept / escalate / abstain decision that carries a stated, auditable guarantee: a bound on expected risk, a high-probability bound on risk, or a false-discovery-rate bound on the decisions it approves.

Request

from cli_sdk import AnthropicBackend, CLIClient, Gate
 
with CLIClient() as client:
    result = client.evaluate(
        context={"transfer_request": "Send 4,800 EUR to the supplier on file.", "account_id": "acct_1042"},
        backend=AnthropicBackend(model="claude-sonnet-5"),
        queries={
            "approve": Gate(
                instructions="Should this transfer be auto-approved?",
                calibration_profile="high-stakes-transfers",
                guarantee="risk_high_probability",
                target=0.01,
                delta=0.05,
            ),
        },
    )
 
gate = result.answers["approve"]
if gate.approved:                      # decision == "auto_approve"
    execute_transfer()
elif gate.decision == "abstain":
    ask_user_to_confirm()
else:                                  # "escalate"
    route_to_human()

Response

{
  "approve": {
    "type": "gate",
    "decision": "escalate",
    "guarantee": {
      "type": "risk_high_probability",
      "target": 0.01,
      "delta": 0.05,
      "method": "RCPS",
      "calibration_profile": "high-stakes-transfers",
      "calibration_n": 2400
    }
  }
}

decision is one of auto_approve, escalate, or abstain. The SDK fails closed: any other value in a response is read as escalate, never as approval.

Guarantee modes

The loss on one request is 1 when the gate auto-approves it and the approved action is wrong, and 0 otherwise (escalated and abstained requests incur no loss). loss can name a different bounded loss. The two risk modes therefore bound the rate of wrong auto-approvals per request; "fdr" bounds the wrong fraction among approved decisions. If a gate approves a fraction $\pi$ of requests, a per-request risk of at most target allows an error rate among approvals of up to $\text{target}/\pi$ .

`guarantee`	Statement	Method
`"risk"`	$\mathbb{E}[\text{loss}] \le \text{target}$ , over the calibration draw and the request	Conformal risk control (CRC)
`"risk_high_probability"`	With probability $\ge 1-\delta$ over the calibration draw, the deployed threshold’s risk is $\le \text{target}$	RCPS / Learn-then-Test
`"fdr"`	Among the decisions the gate auto-approves, the expected fraction that are wrong is $\le \text{target}$	Conformal selection; e-BH when decisions may be dependent

"risk" bounds an expectation. "risk_high_probability" is the stronger, audit-friendly statement most compliance reviews want: it bounds the risk of the threshold you actually deployed, with a stated confidence over the calibration draw, at the cost of a larger calibration set. At target=0.01, delta=0.05, no threshold can be certified with fewer than about 400 calibration examples, even if none of them is an error.

Different gates for different stakes

Calibrate a separate profile for each action type rather than reusing one threshold everywhere:

from cli_sdk import Gate
 
balance_check = Gate(
    instructions="Can this read-only request be completed without confirmation?",
    calibration_profile="low-stakes-actions",
    guarantee="risk",
    target=0.15,
)
wire_transfer = Gate(
    instructions="Should this transfer be auto-approved?",
    calibration_profile="high-stakes-transfers",
    guarantee="risk_high_probability",
    target=0.01,
    delta=0.05,
)

Each profile carries its own labelled examples, its own target and $\delta$ , and its own audit history, so every threshold is the output of a calibration procedure a risk committee can evaluate, not a number picked by feel. The high-stakes approvals guide builds this end to end.

Answer attributes

Attribute	Type	Meaning
`decision`	`str`	`"auto_approve"`, `"escalate"`, or `"abstain"`
`approved`	`bool`	`decision == "auto_approve"`
`guarantee`, `is_heuristic`		The guarantee card, and whether it is heuristic

Parameters

Parameter	Type	Description
`instructions`	`str \| dict \| list`	Required. The accept / escalate decision to make.
`calibration_profile`	`str`	Required.
`guarantee`	`"risk" \| "risk_high_probability" \| "fdr"`	Default `"risk"`.
`target`	`float`	Required, in $(0, 1)$ . The risk bound, or the FDR level for `"fdr"`.
`delta`	`float`	Required for `"risk_high_probability"`. Failure probability over the calibration draw.
`loss`	`str`, optional	Named loss function; defaults to 0/1 error on approved decisions.

The constructor raises ValueError when instructions or calibration_profile is missing, target is outside $(0, 1)$ , or guarantee="risk_high_probability" has no delta.

Interval Claim