API reference

Base URL: https://cci.gitdate.ink/api/v1 (or your self-hosted deployment’s URL). Every body is JSON.

Authentication

Authorization: Bearer <CLI_API_KEY>

The Python SDK reads the key from CLI_API_KEY and the base URL from CLI_BASE_URL. A self-hosted deployment on localhost accepts requests without a key.

Evaluate

POST /v1/evaluate
Content-Type: application/json

Field	Type	Required	Description
`context`	`string \| object \| array`	Yes	The content every query is evaluated against.
`backend`	`object`	See note	The model backend. Required unless every query declares its own `cascade` (every `route` query, and a `judge` query with `cascade`).
`queries`	`map<string, Query>`	Yes	One or more named queries. Each query is a flat object, described below.

Example

curl https://cci.gitdate.ink/api/v1/evaluate \
  -H "Authorization: Bearer $CLI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": {"ticket": "My payouts have been failing for 3 days.", "account_tier": "enterprise"},
    "backend": {"provider": "openai", "model": "gpt-4.1-2025-04-14", "access_hint": "auto"},
    "queries": {
      "department": {
        "type": "set",
        "instructions": "Which team should handle this ticket?",
        "options": {
          "billing": "Payments, invoicing, refunds",
          "technical": "Bugs, outages, integrations",
          "sales": "Pricing, upgrades, new accounts"
        },
        "calibration_profile": "support-routing-v3",
        "alpha": 0.1,
        "method": "APS"
      },
      "route": {
        "type": "gate",
        "instructions": "Auto-route this ticket without human review?",
        "calibration_profile": "support-routing-v3",
        "guarantee": "fdr",
        "target": 0.05
      }
    }
  }'

The SDK emits exactly this body: each query object is what Query.to_payload() returns, with type first and every field left as None omitted.

Query objects

Every query has a type. The remaining fields are the primitive’s constructor arguments, at the top level of the query object.

`type`	Required fields	Optional fields
`belief`	`instructions`, `calibration_profile`	`criteria`
`set`	`instructions`, `options` (2 or more), `calibration_profile`	`alpha`, `method` (`LAC`, `APS`, `RAPS`), `group_by`, `backend_access_hint`
`interval`	`instructions`, `levels` (2 to 10), `calibration_profile`	`alpha`, `method` (`CQR`, `ordinal-aps`)
`gate`	`instructions`, `calibration_profile`, `guarantee` (`risk`, `risk_high_probability`, `fdr`), `target`	`delta` (required for `risk_high_probability`), `loss`
`claim`	`instructions`, `calibration_profile`	`alpha`, `support_source`
`judge`	`instructions`, `calibration_profile`	`alpha`, `cascade`
`route`	`cascade`, `calibration_profile`, `guarantee` (`cost_budget`, `accuracy`)	`target_cents` (required for `cost_budget`), `alpha`

instructions may be a string, object, or array (claim and judge take a string). A cascade is an array of stages, each {"backend": <backend object> | "human_queue"}.

Backend objects

Field	Description
`provider`	`openai`, `azure-openai`, `anthropic`, `gemini`, `bedrock`, `openrouter`, `vllm`, `sglang`, or `client_evidence`
`model`	Model identifier (hosted providers)
`access_hint`	`auto` (default), `sampling`, `logprobs`, `prompt-scoring`, `exact`, `hidden-state`
`connection`	Stored provider credential to use
provider fields	For example `deployment`, `api_version`, `reasoning_effort`, `effort`, `sample_count`, `thinking_level`, `region`, `custom_model_import`, `pin_upstream`, `upstream`, `quantization`, `base_url`, `engine_version`, `logprobs_mode`, `deterministic` (see Backends)

When your own model computes the evidence (the SDK’s CustomBackend), the backend object is {"provider": "client_evidence", "name": "...", "access_level": "L1"} and every non-route query carries an evidence field:

{
  "type": "set",
  "instructions": "Which team should handle this ticket?",
  "options": {"billing": "Payments, invoicing, refunds", "technical": "Bugs, outages, integrations"},
  "calibration_profile": "support-routing-v3",
  "evidence": {"option_probabilities": {"billing": 0.81, "technical": 0.19}}
}

Evidence key	Sent when
`option_probabilities`	`set`, `belief`, `gate`, `judge` at L1 and above
`level_probabilities`	`interval` at L1 and above
`samples`	The same queries at L0, and `claim` at every level
`hidden_states`	Added at L4 when the backend implements it
`claim_scoring`	`"client"` on `claim` queries at L2 and above

Response

Field	Description
`answers`	Map from query id to answer object. Every answer has `type` and `guarantee`.
`backend`	The backend that served the request.
`usage`	`backend_calls` and `backend_tokens`.
`warnings`	Array of strings, for example a fingerprint mismatch on a non-strict profile.
`request_id`	Also returned in the `x-request-id` header.

Answer `type`	Fields
`belief`	`probability`, `venn_abers` (`[p0, p1]`)
`set`	`set`, `probabilities`, `venn_abers` (per option)
`interval`	`point_estimate`, `interval`, `legend`
`gate`	`decision` (`auto_approve`, `escalate`, `abstain`)
`claim`	`retained_claims`, `dropped_claims` (objects with `text`, `reason`, `score`)
`judge`	`winner`, `escalated_to`
`route`	`output`, `served_by`, `escalated`, `cost_cents`

The guarantee object is specified on Guarantee reference.

Calibration profiles

Method	Path	Body	Returns
`POST`	`/v1/calibration-profiles`	`name`, `backend`, `method`, `alpha`, `strict_fingerprint`, optional `group_by`, `prompt_template_hash`	Profile
`GET`	`/v1/calibration-profiles`		`{"profiles": [...]}`
`GET`	`/v1/calibration-profiles/{name}`		Profile
`POST`	`/v1/calibration-profiles/{name}/examples`	`{"examples": [{context, label, source?, group?, metadata?}]}`	Profile
`POST`	`/v1/calibration-profiles/{name}/label-with-judge`	`judge`, `unlabelled_examples` (`[{context}]`), `human_labelled_sample_size`	Labelling job
`POST`	`/v1/calibration-profiles/{name}/audit`	`{"examples": [...]}`	Audit result
`POST`	`/v1/calibration-profiles/{name}/monitors`	`type`, `false_alarm_rate`, optional `target`, `labelled_sample_rate`, `use_judge_pseudo_labels`	Monitor
`GET`	`/v1/calibration-profiles/{name}/monitors`		`{"monitors": [...]}`
`GET`	`/v1/calibration-profiles/{name}/monitors/{id}/alerts`		`{"alerts": [...]}`

A profile object has name, version, method, alpha, n, minimum_n, recommended_n, realized_coverage_ci, last_audit, backend_fingerprint, group_by, groups ({group: {n, status}}), and status (collecting, serving, stale).

An audit result has result (pass or fail), realized_coverage, ci ([lower, upper]), sample, target, and date.

A monitor has id, profile, type, target, false_alarm_rate, labelled_sample_rate, and status. An alert has type, severity, n, e_value, target, false_alarm_rate, detected_at, and message.

curl https://cci.gitdate.ink/api/v1/calibration-profiles \
  -H "Authorization: Bearer $CLI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "support-routing-v3",
    "backend": {"provider": "openai", "model": "gpt-4.1-2025-04-14"},
    "method": "APS",
    "alpha": 0.1,
    "strict_fingerprint": true,
    "prompt_template_hash": "sha256:9f2c..."
  }'

Backends

Method	Path	Returns
`GET`	`/v1/backends`	`{"backends": [...]}`: configured backends and their detected access level

Errors

Error bodies carry a human-readable message (or detail / error); 422 bodies also name the offending field.

Status	Meaning	SDK exception
`401 Unauthorized`	Missing or invalid API key	`AuthenticationError`
`422 Unprocessable Entity`	Request failed validation	`ValidationError` (`.field`)
`424 Failed Dependency`	A query’s profile does not yet have enough examples for the requested guarantee; the body still contains `heuristic`-labelled answers	Returned as heuristic answers; `InsufficientCalibrationError` in strict mode
`429 Too Many Requests`	Rate limit exceeded; honor `retry-after`	`RateLimitError` (`.retry_after`) after retries
`502 Backend Error`	The model backend errored, or reached a lower access level than the query required	`BackendError` after retries
`529 Overloaded`	The service is temporarily overloaded	`BackendError` after retries

The Python SDK retries 429, 502, and 529 with full-jitter exponential backoff by default. See Errors and retries.

Command-line tool