Adapters record model calls without writing record_model_call by hand. Two kinds ship: client wrappers that proxy an OpenAI, Anthropic, or Gemini client you already built, and a LangChain callback handler that records a whole chain. Both emit the same capture file — same span tree, same versioned envelope, same gate, sampling, redaction, and hygiene.
## Provider client wrappers
wrap_openai(client), wrap_anthropic(client), and wrap_genai(client) return a drop-in proxy over a client instance. Wrap it once; every intercepted call made inside an active capture session (@capture.agent, agent_session, …) records one model_call. Outside a session the wrapper is inert. Each module needs its extra and is import-guarded, so the SDK stays stdlib-only at runtime.
# one extra per provider; the SDK runtime stays stdlib-only uv add 'evalshift-sdk[openai]' # openai>=1.40 uv add 'evalshift-sdk[anthropic]' # anthropic>=0.40 uv add 'evalshift-sdk[google-genai]' # google-genai>=1.0
### OpenAI
from openai import OpenAI
from evalshift import capture
from evalshift.adapters.openai import wrap_openai
client = wrap_openai(OpenAI()) # or AsyncOpenAI(); use the proxy exactly like the client
@capture.agent(suite="support_agent", redact=True, tools=[])
def handle_ticket(query: str) -> str:
r = client.chat.completions.create(
model="gpt-4o-mini", messages=[{"role": "user", "content": query}]
)
return r.choices[0].message.content or ""
# OpenAI-compatible servers (Ollama, vLLM, Groq, OpenRouter, ...) need no wrapper of their own
local = wrap_openai(OpenAI(base_url="http://localhost:11434/v1", api_key="ollama"))### Anthropic
from anthropic import Anthropic
from evalshift import capture
from evalshift.adapters.anthropic import wrap_anthropic
client = wrap_anthropic(Anthropic()) # or AsyncAnthropic()
@capture.agent(suite="support_agent", redact=True, tools=[])
def handle_ticket(query: str) -> str:
r = client.messages.create(
model="claude-opus-4-8", max_tokens=512,
messages=[{"role": "user", "content": query}],
)
return r.content[0].text### Google Gemini
from google import genai
from evalshift import capture
from evalshift.adapters.genai import wrap_genai
client = wrap_genai(genai.Client()) # sync and client.aio both covered
@capture.agent(suite="support_agent", redact=True, tools=[])
def handle_ticket(query: str) -> str:
r = client.models.generate_content(model="gemini-2.5-flash", contents=query)
return r.text or ""What is intercepted — sync, async, and streaming. Everything else on the client is forwarded untouched and not recorded:
| Wrapper | Intercepted |
|---|---|
wrap_openai | chat.completions.create, responses.create — sync, async, stream=True |
wrap_anthropic | messages.create (sync, async, stream=True), messages.stream (sync and async managers) |
wrap_genai | models.generate_content, models.generate_content_stream, and both under client.aio |
Each recorded model_call carries model_id, the tools offered, requested_tool_calls (what the model asked to call), input, output, token usage, latency, and the allow-listed generation settings. input is always a messages-style list: Anthropic’s system, the Responses API’s instructions and Gemini’s system_instruction become a leading system message, so the CLI recovers system prompt, history, and current turn the same way for every provider.
- +Wraps the instance, never the module. Nothing is monkeypatched; a client you did not wrap is untouched.
- +Record-only. A wrapper opens no session and takes no
suite/redact— the boundary and the masking choice stay on@capture.agent. A wrapped client can be shared between captured and uncaptured code paths. - +
toolsis asserted per call. The call’s owntoolskwarg (Gemini:config.tools) is recorded, or[]when the request carried none — the session’stools=is never inherited, because the wrapper knows exactly what the provider was sent. - +The real call is never guarded. Provider exceptions propagate exactly as before; a wrapper fault means “this call was not recorded”, never a broken client. A request that raised records nothing.
- +Streaming returns a proxy that records once when the stream is exhausted, closed, or fails — with whatever output had arrived, and usage from the final chunk when the provider sends one (OpenAI chat streams need
stream_options={"include_usage": True}, else tokens stay 0). A stream simply abandoned records nothing. - +
cost_usdstays 0. The CLI prices tokens at promote time; a model with no price entry (local, self-hosted) legitimately stays at 0. - +OpenAI-compatible servers — Ollama, vLLM, Groq, OpenRouter, and the like — are covered by
wrap_openaiwith abase_url;model_idis whatever string you passed, and a server that omitsusagerecords zero tokens.
@capture.agent alongside @capture.tool is the intended pairing. Do not also call record_model_call for the same request, or it is recorded twice.## LangChain
EvalShiftCallbackHandler is a LangChain BaseCallbackHandler. Drop it into any callbacks=[...] list and the root chain, model/chat calls, tools, and retrievers are all recorded into one capture, finalized when the root run ends — no @capture.agent needed.
# the adapter needs the optional langchain extra uv add 'evalshift-sdk[langchain]' # or: pip install "evalshift-sdk[langchain]"
from langchain.agents import AgentExecutor
from evalshift.adapters.langchain import EvalShiftCallbackHandler
handler = EvalShiftCallbackHandler(
suite="support_agent", redact=True, tools=bound_tools
)
# pass it anywhere LangChain takes callbacks — per call or per object
agent: AgentExecutor = build_agent()
agent.invoke({"input": "my refund hasn't arrived"}, config={"callbacks": [handler]})The constructor is keyword-only and takes the same required redact= as @capture.agent — True for default_redactor, False for verbatim, or your own callable; anything else raises TypeError. If the redactor raises at capture time the capture is dropped rather than written unredacted (fail-closed). tools= is also required — the toolset this chain was offered ([] if it never binds tools). Unlike the manual recorders it has no per-call override: LangChain callbacks carry no user-supplied tools kwarg, so the one constructor value is stamped onto every model_call span the handler opens. Add code_version to stamp the envelope. The handler does not accept the conversation-identity kwargs.
# redact= takes True / False / your own (value) -> value callable
handler = EvalShiftCallbackHandler(
suite="support_agent",
redact=my_redactor, # required; fail-closed if it raises
tools=bound_tools, # required; [] if this chain binds no tools
code_version="git-sha-or-tag", # optional, stamped into the envelope
)One handler instance is safe to reuse across many invocations and across threads: per-run state is keyed by the LangChain run_id and lock-guarded. Parentage is resolved from each callback’s parent_run_id chain (LangChain callbacks fire flat, not nested), so tool and model spans land under the right enclosing tool. Generation config is auto-extracted from LangChain’s invocation_params, filtered through the same ten-key allow-list as the manual recorders — tool_choice, parallel_tool_calls and tool_config included, so bind_tools(tool_choice=..., parallel_tool_calls=False) is recorded with no extra work.
The handler also records requested_tool_calls for free: on_llm_end reads AIMessage.tool_calls, which LangChain has already normalised across providers. A chat model that asked for nothing records []; a plain text completion, which cannot ask, records nothing; invalid_tool_calls are excluded — they are parse failures, not requests.
@capture.tool records it twice — use one or the other.[langchain] extra installed does not fail, and the SDK stays stdlib-only at runtime.## Roadmap
These adapters are planned but not built yet:
- +LlamaIndex — capture query engines and agents via its callback/instrumentation hooks.
- +OpenAI Agents SDK — capture runs without manual decorators.
Until then, instrument those agents with a wrapped client or directly with the decorators and helpers.
