Belief

A single-statement probability, returned as a Venn-Abers calibrated interval $[p_0, p_1]$ instead of a raw model probability or a heuristic confidence number.

When to use it

Use Belief wherever you would ask a yes/no question and need to know not just the model’s estimate, but how well-supported that estimate is — for example, whether a message expresses urgency, whether two records refer to the same entity, or whether a generated statement is consistent with a source document.

Request

from cli_sdk import Belief, CLIClient, OpenAIBackend
 
with CLIClient() as client:
    result = client.evaluate(
        context={"message": "Help! My payouts have been failing for 3 days."},
        backend=OpenAIBackend(model="gpt-4.1-2025-04-14"),
        queries={
            "urgent": Belief(
                instructions="Does this message convey urgency?",
                calibration_profile="urgency-v1",
                criteria={"true": "The sender needs action within a day."},
            ),
        },
    )
 
urgent = result.answers["urgent"]
print(urgent.venn_abers)        # (0.9, 0.96)
print(urgent.probability)       # 0.906
print(round(urgent.interval_width, 2))    # 0.06
print(urgent.straddles(0.5))    # False

Response

{
  "urgent": {
    "type": "belief",
    "probability": 0.906,
    "venn_abers": [0.90, 0.96],
    "guarantee": {
      "type": "calibration",
      "method": "IVAP",
      "calibration_profile": "urgency-v1",
      "calibration_n": 850,
      "last_audited": "2026-09-18T00:00:00Z"
    }
  }
}

venn_abers is the interval $[p_0, p_1]$ — the object that carries the statistical guarantee. probability is the interval merged to one number with $p_1 / (1 - p_0 + p_1)$, for code that needs a scalar. The guarantee card has no alpha: a Venn-Abers interval is not a $1-\alpha$ confidence interval. See Venn-Abers calibration for exactly what is guaranteed.

Reading the interval

• Narrow interval near 0 or 1 — the calibration data strongly and consistently supports this answer.
• Wide interval — the answer is genuinely ambiguous, or falls in a region the calibration set does not densely cover. Treat this as “not enough evidence”, not as a value between the endpoints.
• Interval spanning a decision threshold — straddles(t) is True. This is the case to route to a human or a stronger backend. Act only when $p_0 \ge t$, decline only when $p_1 \le t$. The entity matching guide builds a full pipeline on this rule; for a stated error-rate bound on the decisions themselves, use Gate.

Answer attributes

Parameters