RAG claim filtering

Problem

A support copilot generates long-form answers from retrieved documentation. Product wants a statement like “at most 5% of shipped answers contain a claim the documentation does not back”, not “the answer scored 0.8 on an internal quality rubric”.

Scoring the whole answer cannot give that statement: one unsupported sentence inside an otherwise good answer is exactly the failure to catch.

Primitives

Claim: decomposes the draft into atomic claims, scores each one’s support, and keeps the subset that clears a calibrated threshold, with $P(\text{all retained claims true}) \ge 1-\alpha$ over exchangeable question/answer pairs.

Code

Label a calibration set

Each calibration example is one question, its retrieved documents, a draft answer, and a true/false label for every atomic claim in the draft, checked against the documents. Draw the questions at random from real traffic.

from cli_sdk import AnthropicBackend, CalibrationExample, CLIClient
 
PROFILE = "rag-factuality-v1"
BACKEND = AnthropicBackend(model="claude-sonnet-5", sample_count=10)
 
with CLIClient() as client:
    client.calibration_profiles.create(
        name=PROFILE,
        backend=BACKEND,
        method="conformal-factuality",
        alpha=0.05,
    )
    client.calibration_profiles.add_examples(
        PROFILE,
        examples=[
            CalibrationExample(
                context={
                    "question": item["question"],
                    "retrieved_docs": item["docs"],
                    "draft_answer": item["draft"],
                },
                label=[{"claim": c["text"], "supported": c["supported"]} for c in item["claims"]],
                source="human",
            )
            for item in annotated_items          # 50 to 200 annotated pairs
        ],
    )

Filter every draft before it ships

from cli_sdk import Claim, ClaimAnswer, CLIClient, EvaluateResponse
 
class FilteredAnswer(EvaluateResponse):
    filtered_answer: ClaimAnswer
 
FALLBACK = "I could not find a documented answer to that. A teammate will follow up."
 
def answer_question(client: CLIClient, question: str, docs: list[str], draft: str) -> str:
    result = client.evaluate(
        context={"question": question, "retrieved_docs": docs, "draft_answer": draft},
        backend=BACKEND,
        queries={
            "filtered_answer": Claim(
                instructions="Filter the draft answer to only claims fully supported by retrieved_docs.",
                calibration_profile=PROFILE,
                alpha=0.05,
                support_source="retrieved_docs",
            ),
        },
        response_model=FilteredAnswer,
    )
    answer = result.filtered_answer
    log_dropped(question, [(d.text, d.reason, d.score) for d in answer.dropped_claims])
 
    if answer.is_heuristic or not answer.retained_claims:
        return FALLBACK
    return answer.as_text()
 
with CLIClient() as client:
    print(answer_question(
        client,
        "What's your refund policy for annual plans?",
        docs=["Annual plans can be refunded within 30 days of purchase.",
              "Refunds are returned to the original payment method."],
        draft=("Annual plans can be refunded within 30 days of purchase. Refunds are issued to "
               "the original payment method. Refunds typically take 2 business days."),
    ))
    # Annual plans can be refunded within 30 days of purchase. Refunds are issued to the original payment method.

Walkthrough

The profile is seeded with around 140 fully annotated question/answer pairs. That is small next to a classification profile, because every claim in every draft has to be checked by hand — but the guarantee is about the answer as a whole, so each annotated pair is one calibration example.

At request time, Claim splits the draft into atomic claims and scores each one’s support. Claude offers no token probabilities (access level L0), so the score comes from resampling (sample_count=10 generations) and agreement across samples, plus direct claim-to-source support from the retrieved_docs field named in support_source. The extra generations show up in result.usage.backend_calls. On an L2 backend (prompt scoring, for example self-hosted vLLM) the same guarantee is built from claim log-likelihoods, and on an L1 backend from generated-token log-probabilities, both at lower cost.

Claims whose support clears the calibrated threshold are kept; the rest are returned in dropped_claims with a reason and score, so you can log what was removed and why. When nothing survives, the copilot says it does not know rather than shipping an unsupported answer.

A smaller alpha keeps fewer claims: at $\alpha = 0.01$ the threshold must hold for 99% of answers, so borderline claims are dropped more often. Track answer.retention_rate alongside the guarantee to see what the stricter bound costs in answer completeness.

What the guarantee card means

Inside answer_question, the card reads:

card = result.filtered_answer.guarantee
card.type         # "risk"
card.target       # 0.05
card.method       # "conformal-factuality"
card.describe()   # Expected loss at most 0.05 on profile 'rag-factuality-v1' (n=140).

The loss for one answer is 1 if any retained claim is false, 0 otherwise. Expected loss at most 0.05 means that, over questions exchangeable with the calibration set, at least 95% of filtered answers contain no false claim.
It is a statement about the population of answers you ship, not about this answer, and not about each claim individually.
“False” means what the calibration labels meant: not supported by the retrieved documents. If retrieval misses the right document, a correct claim can be dropped; the guarantee does not cover retrieval quality.
It holds while questions, retrieval, and the draft generator stay exchangeable with the calibration pairs. Changing the retriever or the drafting prompt is a pipeline change: recalibrate.

Support routing with an FDR gate High-stakes approvals