// sdk · redaction

Redaction & data boundary

mask before disk

Redaction runs in-process, before any byte hits disk. The redactor is applied during serialization to every payload field — tool arguments, results and errors; model input, output and requested_tool_calls; retrieval queries and documents; final-output text; guardrail reasons; and error messages — so payloads are masked in the written capture, and the tool input_hash is derived from the redacted arguments.

Required at every capture point

redact= is a required keyword on @capture.agent, capture.agent_session, capture.agent_session_async, and the LangChain EvalShiftCallbackHandler. There is no default and no process-wide setter: a capture records the inside of an agent run, which routinely holds PII, so masking is a decision made — and reviewable — at each capture point rather than one silence selects.

Value	Effect
`True`	Mask with the built-in default_redactor.
`False`	Record payloads verbatim, on purpose.
`(value) -> value`	Your own redactor callable.
`anything else`	`TypeError` at the capture point — `None` included, and whether or not the gate is on.

python

from evalshift import capture

# True -> default_redactor: emails, sk-..., AKIA..., Bearer ...
@capture.agent(suite="support_agent", redact=True, tools=[])
def handle_ticket(query: str) -> str: ...

# False -> capture payloads verbatim, on purpose
@capture.agent(suite="fixtures", redact=False, tools=[])
def replay_fixture(case: dict) -> str: ...

# omitting redact= is a TypeError, gate on or off
@capture.agent(suite="support_agent", tools=[])
def broken(query: str) -> str: ...

The keyword is typed RedactSetting (Redactor | bool, exported from evalshift for annotating your own wrappers), so mypy flags a missing or wrong redact= statically as well.

What default_redactor covers

default_redactor is a conservative masker for common secrets in strings. It walks dicts, lists, and tuples recursively, returns copies, and never mutates its input.

+Email addresses → [REDACTED_EMAIL]
+OpenAI-style sk- keys, AWS AKIA… keys, and Bearer tokens → [REDACTED_KEY]

Those patterns are all redact=True covers. Structured secrets — SSNs, account numbers, internal id formats — need a redactor of your own.

Custom redactors

A redactor is any (value) -> value callable, passed in place of True:

python

def scrub(value):
    # a (value) -> value callable; return a redacted copy, don't mutate
    if isinstance(value, dict):
        return {k: ("***" if k == "ssn" else scrub(v)) for k, v in value.items()}
    if isinstance(value, str):
        return value.replace(account_number, "[REDACTED_ACCT]")
    return value

@capture.agent(suite="support_agent", redact=scrub, tools=[])
def handle_ticket(query: str) -> str: ...

−

fail-closed

If a redactor raises, the capture is dropped rather than written unredacted — masking failure never leaks raw data. The host agent still returns its real value; only the telemetry is sacrificed.

Upgrading from 0.2.0

configure(redact=...) and evalshift.config.active_redactor() were removed in 0.3.0. Move the process-wide redactor to each capture point — one way to set masking, so no global state can change what a given agent records:

python

# 0.2.0
configure(redact=default_redactor)
@capture.agent(suite="support_agent")

# 0.3.0 — same masking, decided at the call site
@capture.agent(suite="support_agent", redact=True, tools=[])

What a written capture may still contain

A redactor masks payloads. It does not remove structure. After redaction a capture file still contains:

+The shape of the run — which tools fired, in what order, with what parentage.
+Envelope metadata — suite, capture id, timestamps, code_version, schema_version.
+The one-way input_hash (computed from redacted arguments) — it identifies a call but cannot be reversed to the input.
+Model-call config — the toolset fields (tools_offered, toolset_ref and the sidecar tool schemas they point to) and the allow-listed generation_config. Config is deliberately outside the redactor’s reach: it is what the call was set up with, not payload. requested_tool_calls is not config — its arguments are model-generated payload, so it is redacted alongside input and output.

Note that a session’s agent_input is identity only: it is hashed into the envelope’s input_hash as-is and never stored raw, so redaction does not apply to it.

Plan accordingly: choose a redactor that covers your sensitive fields before turning the gate on in any environment that handles real user data.