// 07 · deep
Changelog
release notes
Headlines only, on two tracks. The full entries live in each repository’s CHANGELOG.md, which stays canonical: evalshift-cli ↗ (the CLI, package evalshift, currently 1.0.0) and evalshift-sdk ↗ (the capture SDK, currently 0.4.0). The two are versioned independently; since CLI 0.14.0 pip install evalshift installs both.
## CLI — evalshift
# evalshift (CLI)
1.0.0 · 2026-09-18 · first stable release — a written compatibility contract names what is public (evalshift.yaml, command names and flags, exit codes, documented artifact fields, the run bundle) and what is not · evalshift all renamed to evalshift compare, with all kept as a permanent working alias · suite selection now prints a ready-to-run command per wired suite
0.15.0 · 2026-09-15 · license switch AGPL-3.0-or-later → Apache-2.0 (0.4.0 through 0.14.1 remain AGPL, 0.3.0 and earlier remain MIT)
0.14.0 · 2026-09-10 · samples_per_example repeats every call and scores the mean · teacher-forced multi-round replay · runs record generation parameters a model cannot honour (fail_on_dropped_params) · tool-choice constraints replayed · judge-family and CI pin-drift warnings in doctor · import package renamed to evalshift_cli (breaking)
0.13.1 · 2026-08-28 · init --ci scaffolds a production-shaped GitHub Actions workflow: suite discovery, one job per suite, a single gate join check
0.13.0 · 2026-08-28 · initial public release
0.12.0 · 2026-08-21 · captured generation config replayed at dispatch (temperature, structured output) · init's default budgets loosened for a first migration · max_tool_argument_drift gains a Wilson interval · docs lead with the capture SDK
0.11.0 · 2026-08-14 · replay history carries the agent loop (tool calls + results) · tool_arguments against: expected · tool_selection mode: expected_set · promotion recovers expected text from the last model call · skipped tool-only turns no longer read as equivalent
0.10.2 · 2026-08-08 · capture sync dedup spans runs · identical slices collapsed before the BH family is built · latency budget labelled as a mean · AI-agent references documented in the repo
0.10.1 · 2026-08-04 · evaluate scores concurrently under defaults.concurrency · embeddings and judge verdicts hit the response cache
0.10.0 · 2026-08-03 · run insights (machine-written narrative, verified figures) · plan-limit upgrade prompt on push · leaner bundle contract — older CLIs are rejected by EvalShift Cloud
0.9.x · 2026-07-23 · GitHub Action reference wired into agent files · doctor row names the failing check
0.8.0 · 2026-07-23 · evalshift tools sync — AST-extract tool definitions from your Python and wire tools_path · field-level push validation errors
0.7.x · 2026-07-09 · run retention + evalshift runs clean · multi-turn suites (history, conversation_id, turn_index) with teacher-forced replay · evalshift --version reads the installed package
0.6.0 · 2026-07-03 · migration policy + migration_decision.json + --policy-gate · defaults.max_tokens and truncation detection (truncated calls excluded from statistics) · failure taxonomy
0.5.0 · 2026-05-18 · Cloud CLI flows: login, whoami, bundle, push, all --push · project + thresholds config · init --ci scaffolds the GitHub Action
0.4.0 · 2026-05-16 · license switch MIT → AGPL-3.0-or-later (0.3.0 and earlier remain MIT)
0.3.0 · 2026-05-12 · evalshift all single-command pipeline · reusable stage cores · orchestrator progress callback
0.2.0 · agent migrations · tool_selection / tool_arguments / tool_trace_structure evaluators
0.1.0 · structural / semantic / llm-judge evaluators · BH-FDR analysis · HTML report
## Capture SDK — evalshift-sdk
# evalshift-sdk (capture SDK)
0.4.0 · 2026-09-10 · provider client wrappers wrap_openai / wrap_anthropic / wrap_genai · requested_tool_calls on model_call · tool_choice, parallel_tool_calls and strict recorded · trace schema 2.1.0 · pip install evalshift now installs the SDK alongside the CLI
0.3.0 · 2026-08-23 · first release from the public repository · redact= required at every capture entry point (breaking) · toolset capture as content-addressed sidecars · trace schema 2.0.0, 1.x captures refused (breaking)
0.2.0 · 2026-08-07 · first release on PyPI · capture decorators, record_model_call and redaction · multi-turn conversation envelope (trace schema 1.1.0)
The CLI is Apache-2.0 from 0.15.0 onward — any use, commercial included, with no share-back requirement. Earlier releases keep the license they shipped with: 0.4.0 through 0.14.1 under AGPL-3.0-or-later, 0.3.0 and earlier under MIT. The capture SDK is MIT.