EvalShift Cloud stores pushed runs, diffs them across branches, and comments on PRs — useful once a team wants shared reports, baselines, and PR gating. It is strictly opt-in: the CLI works fully without an account, model calls always run locally with your own provider keys, and nothing leaves your machine unless you run push (or all --push). Provider API keys are never uploaded.

## 1. Sign up and create a token

Open EvalShift Cloud, create an account, then create an API token from your org settings. For CI, prefer a project-scoped token when the project already exists; use an org-scoped token if the Action should auto-create the project.

## 2. Authenticate the CLI

evalshift login                       # device-code browser flow
evalshift login --token es_...        # or paste a token (verified via GET /me)
evalshift whoami

Credentials live in ~/.evalshift/credentials (owner-only permissions). Precedence: CLI flags (--host/--token) > env (EVALSHIFT_HOST/EVALSHIFT_TOKEN) > credentials file. --no-browser prints the approval URL for remote shells.

## 3. Point config at a project

Projects are org/project slugs — from --project, or the project: key in config. Missing projects are auto-created when permissions allow. Add the project path and the regression budget you want the Cloud diff to enforce:

version: 1
project: acme/customer-router
migration_policy:
  max_overall_regression_rate: 0.03
  max_critical_regressions: 0
  min_equivalence_rate: 0.95

## 4. Push the first Cloud run

all --push runs the local pipeline, builds run_bundle.json.gz, uploads it, finalizes it, and prints the Cloud run URL. Pushes are idempotent per project on the local run id, and that URL carries the server-minted id instead. evalshift bundle builds the artefact without uploading if you want to inspect it first.

evalshift compare --yes --push

# or push an existing local run
evalshift push <run-id>

## 5. Install the GitHub Action

evalshift init --ci writes this shape of workflow. The Action pushes the candidate run, finds a compatible baseline on the PR base branch, posts or updates one PR comment, and sets the evalshift/regression commit status. Add EVALSHIFT_TOKEN plus your provider key (e.g. GEMINI_API_KEY) as GitHub repository secrets. Full inputs, gating modes, and recipes live in the GitHub Action section.

permissions:
  contents: read
  pull-requests: write
  issues: write
  statuses: write

steps:
  - uses: actions/checkout@v4
  - uses: babaliauskas/evalshift-action@v0
    with:
      token: ${{ secrets.EVALSHIFT_TOKEN }}
      fail-on: regression

## 6. Review diffs and thresholds

Open the run URL to inspect the report. When a compatible baseline exists, the PR comment links to a Cloud diff with aggregate, slice, and per-example deltas. fail-on: regression fails only when the Cloud diff reports regressed examples.

## Troubleshooting

• +401 from the CLI or Action means the token is missing, revoked, or scoped to the wrong project.
• +No diff link means no compatible baseline exists yet on the base branch. Push one run on main, then open another PR.
• +Browser run viewing needs storage CORS to allow the web origin.