The EvalShift capture SDK installs inside your agent’s process and records what it actually does — model calls, tool calls, retrievals — then writes a CLI-valid trace to disk. It turns real production behavior into the source side of a migration suite, instead of hand-authoring JSONL traces.

It is a separate package from the CLI, and disk is the only interface between the two: the SDK writes capture files, the CLI reads them without calling SDK code. The CLI (evalshift 0.14.0+) depends on the SDK, so one environment holds both.

The SDK is open source under MIT — the code lives at babaliauskas/evalshift-sdk ↗.

## Install

The distribution is evalshift-sdk; the import name is evalshift (so the public API reads from evalshift import capture). It needs Python 3.10+ and has no required runtime dependencies — stdlib only, so it stays light enough to embed in any agent.

# uv (recommended)
uv add evalshift-sdk

# or pip
pip install evalshift-sdk

## Quickstart

Decorate your agent entry point. That’s the whole integration for a simple agent:

from evalshift import capture

@capture.agent(suite="support_agent", redact=True, tools=[])
def handle_ticket(query: str) -> str:
    ...  # your agent: model calls, tools, retrieval

handle_ticket("my refund hasn't arrived")

redact= is required at every capture point — there is no default and no process-wide setter. True masks with the built-in default_redactor, False records payloads verbatim on purpose, and a (value) -> value callable is your own redactor. Anything else, None included, raises TypeError at the capture point — whether or not the gate is on. See Redaction.

tools= is also required at every capture point: the toolset the agent (or an individual model call) was actually offered. [] asserts “no tools” on purpose — a real value, not a default — so a capture can tell “this agent had no tools” apart from “we don’t know”. See Toolsets.

## Off by default

Capture is off unless EVALSHIFT_CAPTURE is set to a truthy value (1, true, yes, or on). With the gate off every entry point is a thin pass-through — zero files, zero overhead — so it is safe to leave the decorators in production code and flip the gate only where you want traces. The gate is re-read at every call, not frozen at decoration time, so enabling EVALSHIFT_CAPTURE after import works.

# nothing is recorded unless the gate is on
EVALSHIFT_CAPTURE=1 python run_agent.py
# -> .evalshift/captures/support_agent/cap_<id>.json

From here, see Instrumenting agents for tools, model calls, and async; Redaction for masking PII before it hits disk; and Config, sinks & hygiene for where files go and how the firehose self-manages.

## What ships today

The capture core (sync + async, sinks, redaction, hygiene) is shipped, along with:

• +Provider client wrappers — wrap_openai, wrap_anthropic, wrap_genai proxy a client you already built and record one model_call per request made inside a session (sync, async, streaming; OpenAI-compatible servers included). See Adapters.
• +The LangChain adapter — zero-instrument capture via a callback handler. See Adapters.
• +Capture lifecycle in the CLI — evalshift capture list / promote / clean / diff turns a capture into a golden suite case. See Captures → golden cases.
• +Suites from captures — the end-to-end capture → promote → run loop.
• +Schema migration — upgrade-on-read loaders migrate older captures along any registered chain. The current envelope is schema_version 2.1.0 — additive over 2.0.0 (requested_tool_calls on model_call), so 2.0.0 captures still read. Captures written by a pre-2.0.0 SDK are refused (ObsoleteSchemaVersionError) rather than migrated — captures are regenerated state, so re-run the instrumented agent to re-capture.

## Roadmap

Still planned but not built yet:

• +More framework adapters — LlamaIndex and the OpenAI Agents SDK.