EvalShift is a local-first CLI. Runs, scoring, statistics, and reports all happen on your machine under .evalshift/ — no account, no sign-up, nothing leaves your laptop. The only network calls are the model API calls you ask for. The hosted service exists for teams that want shared runs, diffs, and PR comments — it is strictly opt-in, and everything below works without it.

## 1. Install the CLI

EvalShift is a Python package (Python 3.14+). Install it with uv or pip:

# uv (recommended)
uv pip install evalshift

# or pip
pip install evalshift

evalshift --version

## 2. First run — free, offline, no API keys

The fastest first run costs nothing and needs no keys — the demo scaffold ships canned fixtures, so the whole pipeline replays deterministically:

mkdir evalshift-demo && cd evalshift-demo
evalshift demo                       # scaffold a runnable demo project
evalshift all --offline --yes --open # full pipeline, zero API calls

evalshift all drives doctor → run → evaluate → analyze → report under one live progress display, then opens report.html: a single-file HTML report with per-prompt and per-slice comparisons, effect sizes with 95% CIs, and a migration-policy verdict panel. evalshift demo writes a complete customer-support agent project:

## 3. Run it live

The same demo against real models — a Gemini key is enough for the demo config:

export GEMINI_API_KEY=...
evalshift all --open

# or pick models per run:
evalshift all --from gemini-2.5-flash --to gemini-3.1-flash-lite-preview --open

## 4. Your own project

For a real migration you won’t start from the demo — you’ll start from evalshift init and your own captured traffic:

cd your-project
evalshift init        # minimal, capture-first evalshift.yaml

init writes a minimal, capture-first evalshift.yaml. The intended flow: instrument your agent with the capture SDK, record real production runs, promote them into a golden suite with evalshift capture sync, evaluate candidates against real recorded behaviour, and — once the suite earns your trust — gate pull requests with the GitHub Action. That loop is the recommended workflow — read it next.

## Where to next

• +Recommended workflow — SDK capture → CLI evaluation → Action gating, the loop we suggest for real projects.
• +CLI overview — the five-stage pipeline, run artefacts, cache, and resume.
• +Golden suite — the example format, expected tools, and multi-turn history.
• +Hosted setup — optional: shared runs, baselines, and diffs for teams.
• +GitHub Action — optional: the golden suite as a merge gate on every PR.