The Action gates on the hosted diff between this run and the latest compatible run on the base branch, and reports through one PR comment and one commit status. This page covers the fail-on modes, what lands on the pull request, and how branches and baselines resolve.

## The fail-on modes

any-slice-regression is not simply “stricter than regression” in every case; it’s a different question. A run where the aggregate regression count is above zero but no individual slice moved down will fail under regression and pass under any-slice-regression. If you want both guarantees, run the Action twice with different modes (and comment: "false" on one of them), or keep regression and rely on slices for diagnosis rather than gating.

When no compatible baseline run exists on the base branch, there’s nothing to compare against: the check passes, regression_count is 0, and the PR comment says so explicitly.

## What lands on the pull request

One comment, updated in place. The Action maintains exactly one comment per PR, marked with a hidden HTML marker so it edits itself on every push instead of stacking up. With a baseline present it shows the conclusion, the hosted run link, the regression count, the diff link, the pass-rate movement, and up to five regressed slices, worst first. When nothing regressed you get a single No regressed slices row. Percentages are rounded for display only — gating uses the raw values, so a slice can appear as 0 pts and still count as a regression. Without a baseline, the table is replaced by a single line saying no compatible baseline run was found.

A commit status. Context evalshift/regression, linking to the hosted diff (or the run, when there’s no diff). This is what you add to branch protection to make EvalShift a required check. It is set on push events too, not just pull requests.

## How it works, step by step

1. +Install — actions/setup-python, then pip install evalshift==<version>. No pip caching, so budget roughly 20–60 seconds.
2. +Run — evalshift all --yes --config … --suite …, the full local pipeline. Artifacts land in .evalshift/runs/<run-id>/, including report.html.
3. +Push — evalshift push uploads the run bundle, creating the project if create-project allows it. Git metadata travels with the bundle so the server can pair this run with base-branch runs later.
4. +Find a baseline — asks the hosted API for the latest compatible run on the base branch. “Compatible” is a server-side judgement — a suite that changed shape can’t be diffed against an older one.
5. +Fetch the diff — aggregate and per-slice deltas from the hosted API.
6. +Report — writes the five outputs, upserts the PR comment, sets the commit status.
7. +Gate — exits non-zero when fail-on says the diff is a regression.

## Branch and baseline resolution

The Action figures out two branch names, and both matter:

• +branch — the candidate, recorded on the hosted run. From the PR head ref, else the pushed ref.
• +base-branch — where to look for a baseline. From the PR base ref, else the current ref.

On a push event, base-branch falls back to the branch being pushed. A push to main therefore diffs against the previous main run. That’s intentional: it tracks trunk drift over time, and it’s how baselines get recorded in the first place. Override branch/base-branch only when your branch naming genuinely differs from your git refs. If base-branch resolves to an empty string, the Action skips the baseline lookup entirely and always passes. Baselines and diffs themselves are covered in Baselines & diffs.