Quickstart

Install the SDK

pip install cci-sdk

The package installs the cli_sdk Python module, the offline statistics engine cli_sdk.stats, and the cli command-line tool.

Set your API key

export CLI_API_KEY=sk_live_...

To use a self-hosted deployment instead, also set CLI_BASE_URL. No key is required when CLI_BASE_URL points at localhost.

Create a calibration profile

Every primitive needs a named, labelled calibration profile. Create one and add examples:

from cli_sdk import CLIClient, OpenAIBackend
 
backend = OpenAIBackend(model="gpt-4.1-2025-04-14")
 
with CLIClient() as client:
    client.calibration_profiles.create(
        name="support-routing-v1",
        backend=backend,
        method="APS",
        alpha=0.10,
    )
    profile = client.calibration_profiles.add_examples(
        "support-routing-v1",
        examples=[
            {"context": {"ticket": "I was charged twice."}, "label": "billing"},
            {"context": {"ticket": "Your app crashes on launch."}, "label": "technical"},
            # ... a few hundred to a few thousand more
        ],
    )
    print(profile.n, profile.minimum_n, profile.recommended_n)

See Calibration profiles for sizing: CLI refuses to serve a guarantee below $n \ge (1-\alpha)/\alpha$ examples (9 at $\alpha = 0.10$ ) and recommends roughly 1,000 for a stable $\alpha = 0.10$ guarantee.

Ask a guaranteed question

from cli_sdk import CLIClient, OpenAIBackend, Set
 
DEPARTMENTS = {
    "billing": "Payments, invoicing, refunds",
    "technical": "Bugs, outages, integrations",
    "sales": "Pricing, upgrades, new accounts",
}
 
with CLIClient() as client:
    result = client.evaluate(
        context={"ticket": "I was charged twice for order A-104."},
        backend=OpenAIBackend(model="gpt-4.1-2025-04-14"),
        queries={
            "department": Set(
                instructions="Which team should handle this ticket?",
                options=DEPARTMENTS,
                calibration_profile="support-routing-v1",
                alpha=0.10,
            ),
        },
    )
 
answer = result.answers["department"]
print(answer.set)                    # e.g. ['billing']
print(answer.guarantee.alpha)        # 0.1
print(answer.guarantee.calibration_n)
print(answer.guarantee.describe())

Give every option a short description. A None description is sent as null and works, but a few words usually help the backend score the options.

Act on the guarantee, not the number

if answer.is_heuristic:
    route_to_human(answer.set)            # profile too small: no formal guarantee
elif answer.is_singleton:
    route_automatically(answer.top)
else:
    route_to_human(answer.set)

The 90% guarantee is about the set: over exchangeable tickets, the set contains the correct team at least 90% of the time. It is marginal over all tickets and not conditional on the set’s size, so the error rate among tickets that happen to get a singleton set is not bounded by 10%. A singleton is a useful signal, but to bound the error rate of the tickets you route automatically, gate the auto-route with Gate(guarantee="fdr"), as in the support routing guide.

Exchangeability — the one assumption every guarantee rests on.
Primitives — all seven query types.
Guides — end-to-end walkthroughs, from support routing to air-gapped calibration.
Backends — OpenAI, Azure OpenAI, Anthropic Claude, Gemini, Bedrock, OpenRouter, vLLM, SGLang, or your own model.

Introduction Exchangeability