Complete example¶
Independent implementation of the documented System One wire format — not affiliated with TypeSafe.
This is the one page that shows every public option at once: the program below is
examples/complete_call.py,
embedded here as written rather than copied. The documentation tests run it against a local
stub and check that every public parameter name appears in the source; they do not prove that
each option is used correctly or that every prose claim and wire field is covered.
Getting started is still the page to read first: it is one question, one call, one answer. This page is for when you want to see the whole surface and what each part of it does.
Before you run it¶
pip install jevper openai
# The complete_call.py program uses the OpenAI client and Chat Completions only.
# To use Anthropic's Messages surface, install anthropic and pin api="messages".
The program reads three environment variables, so no secret is ever written down:
| Variable | Default | Meaning |
|---|---|---|
JEVPER_BASE_URL |
https://api.openai.com/v1 |
Where the provider client points. A local server's URL carries the /v1 |
JEVPER_API_KEY |
replace-me |
The SDK's key. Local servers ignore it; the SDK still wants one |
JEVPER_MODEL |
gpt-5.6-terra |
The model id sent with every call |
JEVPER_API_KEY=sk-... \
JEVPER_MODEL=gpt-5.6-terra \
python examples/complete_call.py
JEVPER_API_KEY=local \
JEVPER_BASE_URL=http://127.0.0.1:11434/v1 \
JEVPER_MODEL=qwen3.5:9b \
python examples/complete_call.py
Two choices in the program are deliberate, and neither is the default. method="structured" is
pinned so the program answers the same way on a server with logprobs and one without, and
api="chat_completions" is pinned so the request fields named below are the ones that go on the
wire. method="auto" and api="auto" — the defaults — try to find the best surface the provider
has, and the tables further down say what each one resolves to.
"""One jevper call with every public option set explicitly.
This file is the program the "Complete example" page embeds. ``tests/test_docs.py`` runs
it against a local stub server, so the page and a program that works cannot drift apart.
"""
import json
import os
from openai import OpenAI
from jevper import (
Choice,
Example,
Noul,
ReasoningConfig,
RetryPolicy,
Score,
SystemOneClient,
)
BASE_URL = os.environ.get("JEVPER_BASE_URL", "https://api.openai.com/v1")
API_KEY = os.environ.get("JEVPER_API_KEY", "replace-me")
MODEL = os.environ.get("JEVPER_MODEL", "gpt-5.6-terra")
# Few-shot demonstrations are a state and the answer it earned. They can be
# attached at three levels: on the client (``examples``), on the call
# (``examples``), and on the question itself (``Choice(examples=...)``). The
# first non-empty level wins, in the order question, call, client. A mapping is
# keyed by question id; a sequence applies to every question, which only works
# when they are all the same type.
CLIENT_EXAMPLES = {
"intent": [
Example(
state="The app signs me out every time I open it.",
answer="technical",
probabilities={"billing": 0.02, "technical": 0.94, "sales": 0.04},
)
],
"sentiment": [
Example(
state="Three outages this week. I am done paying for this.",
answer=2,
probabilities={0: 0.0, 1: 0.1, 2: 0.9},
)
],
}
QUESTIONS = {
# Choice picks one of your keys and reports a probability for each. One to
# 255 keys; the key order is the answer order, and it decides a tie.
"intent": Choice(
instructions="Pick the intent of the message.",
criteria={
"billing": "money, invoices, refunds or charges",
"technical": "errors, crashes, login or performance problems",
"sales": "pricing, plans, purchasing or an upgrade",
},
examples=[
Example(
state="Where do I download the invoice for last month?",
answer="billing",
probabilities={"billing": 0.91, "technical": 0.06, "sales": 0.03},
)
],
),
# Noul answers with one probability: 1.0 is true, 0.0 is false. The criteria
# are optional descriptions of each end, not the answer.
"needs_human": Noul(
instructions="Does this need a person to answer it today?",
criteria={
"true": "the customer is waiting on an answer only a person can give",
"false": "the documented answer is enough",
},
),
# Score rates on an ordered scale of 2 to 10 levels and reports a probability
# per level. The answer is the probability-weighted level index, levels
# counted from zero.
"sentiment": Score(
instructions="Rate how angry the customer is.",
criteria=["calm", "frustrated", "angry"],
),
}
def main() -> None:
# The provider client is yours. jevper never creates, closes or reconfigures
# one: it calls the object you hand it, through a copy whose own retry loop
# is off. ``RetryPolicy`` handles transient failures; ``n_retry_malformed``
# handles unreadable answers, and ``usage.n_retries`` counts only transient
# retries. ``debug["llm_attempts"]`` records every provider request. The
# ``timeout`` is the SDK's own, and it applies to each request it makes.
provider = OpenAI(base_url=BASE_URL, api_key=API_KEY, timeout=60.0)
try:
with SystemOneClient(
provider,
# Required. The model id every call sends; also the key jevper
# remembers capability verdicts under, so a second model on one client
# is judged on its own first refusals.
model=MODEL,
# How the decision is elicited: "auto" (the default) asks for logprobs
# and falls back to JSON where the provider has none, or one of
# "logprobs", "grammar", "structured", "discrete". Pinned here so the
# program behaves the same on every server. "logprobs" reads a
# one-token label on Chat Completions and Responses; "grammar" does
# so on Chat Completions only. "structured" asks for a JSON
# distribution and works everywhere; "discrete" asks for one option
# and reports it as one-hot.
method="structured",
# The wire surface: "auto" (the default) prefers responses, then
# chat_completions, then messages, and falls back when a route is
# missing; or pin one. The choice decides which provider fields exist,
# which is why the output budget below is named per surface.
api="chat_completions",
# Reasoning. mode="native" is one call carrying the provider's own
# reasoning fields, "two_step" is an analysis call and then the answer
# call, and the default "auto" picks native on Responses and on
# Messages when a budget is set, two_step elsewhere; reasoning=None
# is no reasoning at all. effort and summary are the Responses
# parameters, context is llama.cpp's, and budget_tokens is the
# Messages surface's ``thinking`` budget. A field with no counterpart
# on the selected surface is not sent, and current Claude models
# reject any non-default temperature on Messages, thinking or not.
reasoning=ReasoningConfig(
mode="native",
effort="low",
summary="auto",
context="auto",
budget_tokens=1024,
),
# Default few-shot examples for every call this client makes.
examples=CLIENT_EXAMPLES,
# Send a strict JSON schema for the answer (``response_format`` here,
# ``text.format`` on the Responses surface, ``output_config.format`` on
# Messages). False leaves the schema in the prompt and asks for plain
# JSON; a server that refuses the strict schema gets that same fallback
# on its own, whichever way this is set.
structured_outputs=True,
# Rescale a structured distribution that misses 1 by more than 1e-6,
# keeping the model's numbers in ``debug["original_probabilities"]``.
# False reports them verbatim, sum and all. An all-zero distribution
# then has no meaningful argmax: Python's ``max`` tie behavior picks
# the first criterion key, while confidence treats it as uniform.
normalize_probabilities=True,
# Alternatives requested for the ``logprobs`` and ``grammar``
# readouts, 0 to 20. A label readout needs at least two: one
# alternative to compare the sampled token against. Ignored by the
# structured readouts this program uses.
top_logprobs=20,
# Questions answered at once. Questions are independent, so this is a
# thread pool here and an asyncio semaphore in AsyncSystemOneClient.
max_concurrency=8,
# Corrective retries when an answer cannot be read: the client
# describes the failure and asks for a conforming answer. The turn
# does not quote the model's previous reply and is separate from the
# transient-failure retries below.
n_retry_malformed=1,
# Transient-failure retries per provider call: HTTP 408, 409, 429 and
# any 5xx, plus connection and timeout errors, with exponential
# backoff that honours Retry-After when the server sends one. These
# are the defaults, written out.
retry=RetryPolicy(
n_retries=2,
base_delay=0.5,
max_delay=8.0,
respect_retry_after=True,
),
# Sampling temperature, sent only when set. 0.0 is what a
# classification wants: the distribution should be the model's
# belief, not a sample from it. The OpenAI surfaces take it as a typed
# field; on Messages it travels in the body, and current Claude models
# reject any non-default value, with jevper also leaving it out when
# a thinking budget is on.
# Provider request fields jevper has no parameter for, merged into
# every request body. This is where the output budget lives, and its
# name is the surface's own: max_completion_tokens here,
# max_output_tokens on responses, max_tokens on messages, where it is
# also required and defaults to 1024. A field named here is the value
# that reaches the wire, so naming response_format, text or
# output_config also moves the schema into the prompt.
extra_body={"max_completion_tokens": 2048},
# Headers sent with every request. A name spelled the way the client
# spells its own replaces the default instead of joining it.
# Credential headers are redacted in ``debug``.
extra_headers={"x-jevper-example": "complete"},
# The provider's cache-routing key. Left unset, jevper derives a
# stable one per question from the parts of the prompt that do not
# change between calls, so a rubric's requests share a cached prefix.
# Set it to group requests your own way — and pass your own when the
# derived key is a fingerprint of your rubric at the provider.
prompt_cache_key="complete-example",
) as client:
# The state under judgement. A string, a list of chat turns,
# {"messages": [...]}, or any JSON value; it is rendered after the
# question block, except when its last turn is an assistant turn, in
# which case the question follows it.
state = [
{
"role": "system",
"content": "Support inbox for a subscription product.",
},
{
"role": "user",
"content": (
"I was charged twice this month and the second charge is "
"not on my card statement. I need this fixed today."
),
},
]
# Non-None per-call options and non-empty examples override the
# client's values for this call. Omitted options and empty examples
# inherit the client values. The values repeat the client's here to
# show where an override sits, not that the two should differ.
response = client.system_one(
state=state,
questions=QUESTIONS,
examples={
"intent": [
Example(
state=(
"Why is my invoice higher than the plan I "
"signed up for?"
),
answer="billing",
)
]
},
model=MODEL,
method="structured",
api="chat_completions",
reasoning=ReasoningConfig(mode="native", effort="low"),
temperature=0.0,
prompt_cache_key="complete-example",
)
# One typed answer per question, in the order the questions were given.
# NoulAnswer carries only ``noul``. ChoiceAnswer also carries
# ``choice``, ``probabilities`` and ``confidence``; ScoreAnswer also
# carries ``score`` and ``legend``.
for question_id, answer in response.answers.items():
print(
question_id, answer.type,
json.dumps(answer.model_dump(mode="json")),
)
# Usage counts successful provider results, including the analysis
# pass, the answer call and each corrective retry. Failed requests,
# fallback probes and route misses are recorded in
# ``debug["llm_attempts"]`` instead. The other counters aggregate
# token counts, transient retries, latency and provider cache reads.
print("usage:", response.usage)
# What the call actually did. These keys are always present:
# ``method``, ``api``, ``reasoning_mode``, ``llm_attempts``,
# ``retry_reasons``, ``probability_errors``,
# ``original_probabilities`` and ``labels_missing``. ``methods`` and
# mixed-surface keys are conditional. Each attempt holds the request
# kwargs (with credential headers redacted), provider response, error
# and parsed readout.
debug = response.debug
print(
"debug:",
debug["method"],
debug["api"],
debug["reasoning_mode"],
len(debug["llm_attempts"]),
"attempt(s)",
)
if debug.get("server_limits"):
print(
"the server refused these fields, and jevper stopped sending them:",
debug["server_limits"],
)
# The whole response is the Jev wire shape, so it serializes to what the
# hosted API returns.
print(response.model_dump_json(exclude_none=True))
finally:
# Leaving the ``with`` block closed jevper's own thread pool. The provider
# client is yours to close; jevper never closes it, on either facade.
provider.close()
if __name__ == "__main__":
main()
It prints one line per answer, the aggregated usage, what the call resolved to, and the whole
response as the Jev wire shape. The comments explain behavior and contracts as well as options;
the reference is where names are defined. A Noul answer has only its noul value;
ChoiceAnswer and ScoreAnswer add their documented choice, probability, confidence, score and
legend fields.
The output budget, per surface¶
jevper has no max_tokens parameter: the three surfaces do not share a name for it, so the field
belongs to the provider and travels in extra_body. What each one calls it is:
| Surface | Field in extra_body |
Notes |
|---|---|---|
chat_completions |
{"max_completion_tokens": 2048} |
OpenAI's current name, and it counts reasoning tokens; max_tokens is the older field, still what most local servers take, and refused by OpenAI's reasoning models |
responses |
{"max_output_tokens": 2048} |
Also counts reasoning tokens; the builder does not validate an arbitrary max_tokens in extra_body, so whether a server rejects that key is provider behavior |
messages |
{"max_tokens": 2048} |
Required by the API. With resolved native reasoning and a budget_tokens, jevper defaults to 1024 plus that budget; otherwise it sends 1024 |
The budget is a ceiling, not a target: a distribution is a label or a small object, so 2048 is room
for a reasoning trace to finish. When a provider runs out of it, jevper raises
IncompleteAnswerError naming the surface's own field, rather than reporting a missing answer.
Current Claude models reject any non-default temperature on the Messages surface, whether or not
thinking is enabled. thinking: {type: "enabled", budget_tokens} is deprecated on Claude 4.6 and
rejected with 400 on 4.7 and later.
Which option reaches which surface¶
Not every option applies to every combination, and jevper says so by leaving a field out rather than sending one the surface does not have:
| Option | chat_completions |
responses |
messages |
|---|---|---|---|
method="logprobs" |
logprobs=true, top_logprobs |
top_logprobs and include=["message.output_text.logprobs"]; native reasoning may add reasoning.encrypted_content to include |
refused: the API returns no logprobs |
method="grammar" |
logprobs=true, top_logprobs, plus a GBNF grammar in the body |
refused: grammar is Chat-Completions-only | refused: the API returns no logprobs |
method="structured", "discrete" |
response_format with a strict schema |
text.format with the same schema |
output_config.format through the body |
structured_outputs=False |
response_format={"type": "json_object"}, schema in the prompt |
text={"format": {"type": "json_object"}}, schema in the prompt |
schema in the prompt |
reasoning effort |
reasoning_effort |
reasoning.effort |
no equivalent field; not sent |
reasoning summary, context |
no equivalent field; not sent | reasoning.summary, reasoning.context |
not sent |
reasoning budget_tokens |
not sent | not sent | thinking={"type": "enabled", "budget_tokens": n} |
temperature |
typed field | typed field | body; current Claude models reject a non-default value |
top_logprobs |
only with logprobs or grammar |
only with logprobs |
not sent |
examples |
chat turns before the question block | same | same; question-level non-empty examples take precedence over per-call examples, then client examples |
prompt_cache_key |
request body; OpenResponses documents a 64-character maximum, while OpenAI's API reference states no length limit | request body; the same schema/reference distinction applies | the Messages surface never carries this option; an extra_body key of that name is still forwarded |
extra_headers |
every request | every request | every request |
max_concurrency, n_retry_malformed, retry |
client-side; no wire field | client-side | client-side |
Two extra_body keys are refused before any request: model, which would disagree with the model
id jevper keys its cache and its capability memory on, and a truthy stream, because jevper reads
the answer from one whole response. A key named in extra_body wins over the typed field for the
same name, so naming response_format, text or output_config there also moves the schema into
the prompt. The corrective retry turn uses only the failure reason and an instruction; it does not
quote the model's previous reply.
Switching surface¶
The program above is Chat Completions. The other two are three lines each — and the client changes with the surface, because the Messages API is Anthropic's:
# Responses: the budget field is max_output_tokens, and effort/summary travel natively
with SystemOneClient(provider, model=MODEL, method="structured", api="responses",
reasoning=ReasoningConfig(mode="native", effort="low", summary="auto"),
extra_body={"max_output_tokens": 2048}) as client:
...
# Messages: the anthropic SDK, the budget field is max_tokens, and only the thinking budget applies
from anthropic import Anthropic
provider = Anthropic(base_url=BASE_URL, api_key=API_KEY)
with SystemOneClient(provider, model=MODEL, method="structured", api="messages",
reasoning=ReasoningConfig(mode="native", budget_tokens=1024),
extra_body={"max_tokens": 2048}) as client:
...
method="logprobs" and "grammar" cannot be pinned to messages — the API has no logprobs — and
grammar sends a GBNF grammar body field, so the Chat Completions server must accept that field.
api="auto" needs no choice: it prefers responses, falls back to chat_completions, then
messages, and remembers a route that answered 404. What each server implements is measured per
server in local-servers.md.
Async is a delta, not a second program¶
Same constructor, same system_one, one await. The async client holds an asyncio semaphore
instead of a thread pool, and aclose() is a no-op because it owns nothing to close:
from anthropic import AsyncAnthropic
from jevper import AsyncSystemOneClient
async with AsyncSystemOneClient(AsyncAnthropic(base_url=BASE_URL, api_key=API_KEY),
model=MODEL) as client:
response = await client.system_one(state=state, questions=QUESTIONS)
The provider client is still yours to close, on both facades: jevper never closes the object it was
handed. close() shuts down jevper's own thread pool and nothing else.
A client that is not an SDK¶
jevper imports neither openai nor anthropic. It calls whatever object it is handed, reading each
field as an attribute or a mapping key, so the structural contract is small: a Chat Completions
object returns choices[0].message.content and choices[0].finish_reason; a Responses object
returns message items under output; a Messages object returns content blocks under content.
Each surface also supplies its usage counts. api="auto" follows what the object exposes, so a
chat-only object is never asked for a Responses route.
Run it with no server at all — the model is a constant:
"""A client that is not an SDK: the whole structural contract, in one class.
jevper never imports ``openai`` or ``anthropic``. It calls whatever object it is
handed, reading attributes or mapping keys with the same helper, so a
twenty-line object can stand in for a full SDK client. This is the smallest one
that answers a question, and the "Using an existing or duck-typed client" page
embeds it.
"""
from __future__ import annotations
from types import SimpleNamespace
from jevper import Noul, SystemOneClient
class ConstantModel:
"""``client.chat.completions.create(**kwargs)``, returning what jevper reads
off a Chat Completions answer: ``choices[0].message.content``,
``choices[0].finish_reason`` and ``usage.prompt_tokens`` /
``usage.completion_tokens``. Every one may be a mapping key instead of an
attribute. A real client adds the rest — streaming, retries, auth — none of
which jevper requires."""
class _Completions:
def create(self, **kwargs: object) -> dict[str, object]:
assert kwargs["model"] == "constant-model", kwargs["model"]
return {
"choices": [
{
"index": 0,
"finish_reason": "stop",
"message": {"role": "assistant", "content": '{"noul": 0.8}'},
}
],
"usage": {"prompt_tokens": 41, "completion_tokens": 7},
}
def __init__(self) -> None:
self.chat = SimpleNamespace(completions=self._Completions())
def main() -> None:
with SystemOneClient(
ConstantModel(),
model="constant-model",
method="structured", # ask for JSON, so the answer needs no logprobs
# api="auto" is the default and the surface follows the object: this one
# exposes no responses.create and no messages.create, so every call goes
# to chat_completions without probing a route that is not there.
) as client:
response = client.system_one(
state="The status page says all systems are operational.",
questions={
"is_outage": Noul(
instructions="Is the service down right now?"
)
},
)
answer = response.answers["is_outage"]
print(answer.noul) # 0.8
print(response.usage.n_calls) # 1
print(response.debug["api"]) # chat_completions
if __name__ == "__main__":
main()
The next level up is the same contract with the parts a real client adds — streaming, retries, auth,
connection pooling — which is what both official SDKs give you, and what
examples/incident-triage
is built on: a support-ticket triage service that installs jevper as a dependency, uses nothing
but its public API, and runs against five local servers.
Where each option is defined¶
| What | Where |
|---|---|
| Every constructor option, with defaults | API reference |
| Every per-call option | API reference |
RetryPolicy |
API reference |
ReasoningConfig and mode resolution |
Reasoning, API reference |
The debug record and its conditional keys |
API reference |
| Every error and what raises it | Troubleshooting, API reference |
| Choosing a method or a surface | Methods |
| Few-shot examples | Few-shot examples |
| Per-server behaviour, measured | Local servers |
| Tracing and hosting | MLflow |