Eval-driven development: what it is, how it differs from TDD, and a loop that works

Eval-driven development is a way of building LLM applications and agents in which the eval comes before the change. You write down what a new capability or a fix should do as a set of tasks with checks, run them, make the change, and accept it only when the results improve against a baseline without breaking what already worked. The name is borrowed from test-driven development, and the borrowing causes most of the confusion: an eval is not a unit test.

The term is also used in two senses, and two of the best-known teachers of LLM evals advise against the stricter one. This post covers what eval-driven development means, where the term comes from, how it differs from test-driven development, a six-step loop with a worked example, the ways the practice goes wrong, and where it lives in CI. A disclosure: I build EvalShift, a tool for one part of this loop, and it has a short section near the end. The rest works with any tool or none.

What eval-driven development is

Eval-driven development (EDD) comes down to three habits.

1. +The expectation is written first. Before a prompt edit, a new tool or a model swap, someone writes the tasks and the checks that say what done means.
2. +A change is accepted on a comparison. The new version runs on the same tasks as the old one, and the difference decides. Reading five outputs and finding them fine does not count.
3. +Failures flow back. A bug report, a bad production trace or a surprising transcript becomes a task.

"Eval driven development", "eval-driven development", "evaluation-driven development" and EDD all name the same practice. The parts of an eval (task, trial, grader, transcript, outcome) are defined in AI evals for agents. This post is about the order of work.

Having evals is not the same thing. A team can own a dashboard full of scores and still ship on instinct.

Where the term comes from

I could not find who used the phrase first. These are the dated sources I read in November 2026, oldest first.

Two meanings of the term

The loose meaning: evals decide every change. This is how Vercel uses it. Its post describes evals as "like end-to-end tests for AI and other probabilistic systems" and says the v0 team iterates on prompts almost daily, adds new failing prompts to the eval set, and includes eval results in every pull request that touches the output pipeline. OpenAI's version is one line: "Evaluate early and often." Its evals guide ↗ compares the process to behavior-driven development, where behavior is specified before it is implemented.

The strict meaning: the eval exists before the capability. This is Anthropic's. Its guide recommends that teams "build evals to define planned capabilities before agents can fulfill them" and then iterate until the agent performs well. Anthropic adds that it often builds features as bets on what models will do in a few months, and that evals starting at a low pass rate show which bets paid off when a new model arrives.

The objection

Hamel Husain and Shreya Shankar teach a course on AI evals, and their FAQ answers the strict meaning with "Generally no." Their argument is that failure modes in ordinary software are predictable and failure modes in LLM systems are not, so you cannot anticipate what will break. Their advice: "Write evaluators for errors you discover, not errors you imagine." They allow one exception, a constraint where you know exactly what success looks like, such as never mentioning a competitor.

There is research behind the objection. Shankar and colleagues studied ↗ people building LLM-assisted evaluators in April 2024 and named a pattern they call criteria drift: people need criteria in order to grade outputs, and grading outputs is what teaches them the criteria.

Both sides are right, about different kinds of check. What an agent must have done by the end of a task, and the rules it must never break on the way, are knowable before any code exists. Whether its replies are good is not knowable until you have read a few dozen of them. The loop below writes the first kind of check before the change and the second kind after it.

Eval-driven development vs test-driven development

What carries over

The discipline carries over. Writing the expectation first forces you to decide what done means. Martin Fowler's description ↗ of test-driven development, which Kent Beck developed in the late 1990s, includes a step people forget: writing out a list of test cases before the first one is implemented. That list maps directly onto a task set. So do small steps, one change between runs, and keeping every past success as a regression test.

What does not carry over

Results are rates. There is no red and no green. A suite of 50 cases that each pass 98% of the time comes back fully green in only 36% of runs, which is why LLM testing recommends three to five runs per case and a gate on the comparison. Small sets are noisier still: 16 passes out of 20 is 80%, with a 95% interval of roughly 58% to 92%, so going from 16 to 17 tells you nothing on its own.

Graders need validating. A unit test's assertion is correct by inspection. An LLM judge is a model with its own error rate, and it returns a confident verdict whether or not its criterion makes sense. When to trust an LLM judge has the procedure for checking one against human labels before it blocks anything.

The model changes under you. In test-driven development the code under test changes only when you change it. An LLM application depends on a model that can be re-pointed, retired or served differently while your repository stays the same, so the regression suite is also the acceptance test for the provider's next release. The deprecation schedule lists the dates you do not choose.

The eval-driven development loop

Six steps. The first three happen before the change, which is the part that makes this eval-driven and not just evaluated.

Step 1: Write the change down as tasks before you build it

Describe the capability or the fix as 20 to 50 tasks, each with an input and a statement of what success means in terms of the world, not the reply. Anthropic's guide gives the same starting size and the same sources: real failures, the checks you already run by hand before a release, the bug tracker and the support queue. A capability that does not exist yet has no failures, so use real requests that people or an older system handle today.

Two properties make a task set worth running. Each task has to be unambiguous: Anthropic's test is that two domain experts would independently reach the same pass or fail verdict. And the set has to be balanced, with tasks where the behavior should happen and tasks where it should not. Anthropic learned this building web search for Claude.ai and puts it in five words: "One-sided evals create one-sided optimization." If a task is hard to write, the requirement is not finished.

Step 2: Grade outcomes with code first, and leave judges for later

Write checks only for what you can know in advance:

• +The outcome. The record exists, the status changed, the field has the right value.
• +The rules. Steps that must never be skipped and actions that must never repeat.
• +The structure. The output parses and validates.

These are code, they cost nothing to run, and you know what they should say before the agent exists. Do not write an LLM judge for "helpful" or "on-brand" yet. You have not seen the outputs, so the criterion would be a guess. That judge is written in step 5, from failures you have read.

Pick the numbers you will report now as well. Agent evaluation metrics defines the candidates. For a customer-facing agent the one that matters is the share of tasks that pass on every trial.

Step 3: Run the baseline before you change anything

Run the new tasks against the current system. For a new capability most of them should fail. This is the red step of test-driven development, with the same purpose: a check that cannot fail is not checking anything. Anthropic notes the opposite case too. With a frontier model, a task that fails on every one of many trials is more often a broken task than an incapable agent.

Then run the existing regression suite a few times with nothing changed. The spread between those runs is your noise floor, and without it a two-point drop after your change cannot be read. How many eval cases do you need covers what suite size lets a comparison conclude.

Step 4: Make one change and compare it with the baseline

Change one thing: the prompt, or a tool definition, or the model. Run the new tasks and the regression suite on the old and the new version, on the same cases, several trials each. Then read two results.

• +The capability tasks should go up. That is the hill you are climbing.
• +The regression suite should not go down by more than the noise floor from step 3.

If both the prompt and the model changed, you cannot attribute the result. Make the second change a second run.

Step 5: Read the transcripts and fix the eval too

A score tells you that something moved. A transcript tells you what. Read every failure on the new tasks and a sample of the passes. Each failure is one of three things:

• +The agent is wrong. Fix the prompt, the tool or the code, and run again.
• +The eval is wrong. The grader rejected a valid solution, or the task was impossible. Anthropic's guide expects this: when grading is unfair or a task is ambiguous, revise the eval.
• +Nobody anticipated it. This is the case Husain and Shankar describe. Add a task, and if code cannot check it, write a judge now: one criterion, pass or fail, checked against 30 to 50 trials you labeled by hand, and advisory until that comparison holds up. LLM-as-a-judge covers how.

In test-driven development the third step is refactoring the code. Here it is refactoring the eval.

Step 6: Graduate passing tasks and gate every later change

When a capability task passes on every trial, move it into the regression suite. Anthropic calls this graduating: a task that asked whether the agent can do something now asks whether it still can. The capability set then needs refilling, because an eval at 100% tracks regressions and gives no signal for improvement.

Then make the regression suite automatic: it runs on every pull request that touches the prompt, the model name, the tool definitions or the suite, and compares the branch with the base branch. Keep feeding it, too. LLM monitoring covers finding production failures, and building a golden suite from production traffic covers turning recorded runs into cases.

A worked example: adding cancellation to a support agent

This example is illustrative. It shows the order of work, and none of it is a measurement.

A support agent answers billing questions. The next feature lets it cancel a subscription through a new cancel_subscription tool. Today those requests go to a person.

Step 1. Pull 24 real requests from the support queue and write each as a task. Twelve are plain requests to cancel. Eight are requests where the agent must not cancel: a question about what cancelling would do, a request to pause, an account the customer does not own. Four are phrased in ways the prompt will not expect.

- id: cancel-explicit-monthly
  input: "Please cancel my subscription. Account 7741."
  expect: {account: "7741", status: cancelled, cancel_calls: 1}

- id: cancel-question-only
  input: "If I cancelled today, would I lose my saved reports?"
  expect: {account: "7741", status: active, cancel_calls: 0}

Step 2. Three code checks, one result each: the outcome in the sandbox database, the number of cancel calls, and the rule that a lookup comes before a cancel. No judge yet, because nobody has seen what the agent says when it cancels.

def grade(trial, db, expect):
    names = [call.name for call in trial.tool_calls]
    cancels = names.count("cancel_subscription")
    first_cancel = names.index("cancel_subscription") if cancels else len(names)

    return {
        "outcome": db.subscription(expect["account"]).status == expect["status"],
        "cancel_count": cancels == expect["cancel_calls"],
        "lookup_first": cancels == 0 or "lookup_subscription" in names[:first_cancel],
    }

Step 3. Run the 24 tasks on the current agent, three trials each. The tool does not exist, so the twelve should-cancel tasks fail on outcome by construction, and the eight should-not tasks should pass. If one of the eight fails, the task or the grader is wrong, and you found out before writing any feature code.

Step 4. Add the tool and one paragraph to the system prompt, and run both sets on the old and the new version. Suppose the should-cancel tasks now mostly pass and two of the should-not tasks fail, because the agent cancelled when a customer asked a hypothetical question. Twelve happy-path tasks alone would have reported a clean success.

Step 5. Read the transcripts. Suppose they show two things nobody wrote a task for: a customer with two subscriptions gets the wrong one cancelled, and the replies never say when access ends. The first becomes three new tasks with an outcome check. The second cannot be checked by code, so it becomes a judge with one criterion, labeled against 30 trials and kept advisory.

Step 6. Tasks that pass on all three trials move to the regression suite, and the ones that still fail stay in the capability set. The cancellation rules now run on every pull request, including the one that swaps the model next quarter.

How eval-driven development goes wrong

Tuning the prompt to the eval set

Twenty rounds of prompt edits against the same 30 tasks produce a prompt that passes those 30 tasks. The authors of What We've Learned From a Year of Building with LLMs ↗ open their warning about over-weighted evals with Goodhart's law: "When a measure becomes a target, it ceases to be a good measure." Three habits limit the damage. Keep a held-out slice of tasks that you run and never read failures from. Refill the set from production every few weeks. And treat a prompt that names specific test inputs as a bug.

A score nobody reads

A pass rate on a dashboard is not a decision. If no change has ever been blocked or reverted because of the suite, the suite is decoration. The fix is steps 5 and 6: someone reads the transcripts, and the comparison sits on the pull request where the change is reviewed.

A judge nobody validated

OpenAI lists not calibrating automated metrics against human evals among its anti-patterns. An unvalidated judge fails in both directions: it blocks good changes, which teaches the team to override it, and it passes bad ones with a reassuring number attached.

An absolute threshold that breaks when the suite grows

"Pass rate must stay above 90%" works until the suite improves. Take 100 cases passing at 92%. Add 15 hard cases from production, of which the agent passes 5. The suite is now better and the pass rate is 84.3%, with no change to the agent. The gate turns red, and the team learns to stop adding hard cases. Gate on the comparison with the base branch, and keep thresholds for budgets that are meant to be absolute, such as zero violations of a rule.

Evals written after the fact

An eval written after the feature ships encodes what the system does, not what it should do. Anthropic's version: "Evals get harder to build the longer you wait." It is still worth doing: the same guide says the Bolt team started after its agent was widely used and had an eval system in three months. Start from failures, not from current behavior.

Where the loop lives in CI

The loop runs at three speeds.

Vercel's post describes the middle row: every pull request that affects the output pipeline carries eval results. OpenAI's guide calls it continuous evaluation. Restrict the trigger to the files that can move the result, because every run costs model calls. LLM regression testing in CI walks through a gate on GitHub Actions, and prompt regression testing covers where the prompt text has to live for a prompt edit to stay comparable with the base branch.

Keep the tasks and the graders in the repository, next to the prompt they test. The OpenAI page that recommends eval-driven development carries a notice, as of November 2026, that OpenAI is deprecating its hosted Evals platform: read-only for existing users from October 31, 2026, and scheduled to shut down on November 30, 2026. A suite that lives as files survives a change of vendor.

Eval-driven development with EvalShift

Disclosure again: I'm the founder. EvalShift covers the regression half of the loop: steps 3, 4 and 6 for a model change, and the pull-request gate. It is not a harness for capability evals on a feature that does not exist yet. The CLI runs a fixed suite on two models, the current one and a candidate, and it never runs your tools: it replays recorded runs, feeding both models the tool results captured in production. An outcome check against a database, like the one in the worked example, stays in your own tests.

The capture SDK records real runs, and evalshift capture sync --rounds all turns them into a golden suite that carries each run's recorded tool results. The rules from step 2 go into the config as trace rules that both models are held to:

defaults:
  samples_per_example: 3   # every case runs three times on each model

evaluators:
  trace_invariants:
    - name: cancellation_contract
      owner: "@support-team"
      rules:
        - id: lookup-before-cancel
          type: order
          before: lookup_subscription
          after: cancel_subscription
        - id: one-cancel
          type: call_count
          tool: cancel_subscription
          max_calls: 1

migration_policy:
  max_invariant_violations: 0

evalshift compare --policy-gate runs both models over the suite, scores each side, tests whether the differences are larger than the noise, and exits non-zero when the migration policy returns fail or conditional pass. Judges that evalshift init writes start as advisory (blocking: false), which matches step 5. The GitHub Action runs the suite on a pull request, pushes the run, compares it with a baseline run on the base branch, and by default gates on the policy verdict. The CLI and SDK are open source and run locally with your own provider keys.

FAQ

What is eval-driven development? Eval-driven development is a way of building LLM applications and agents in which the eval is written before the change. You describe a new capability or fix as tasks with checks, run them against the current system, make the change, and accept it only when results improve against that baseline without regressing what already worked.

What is the difference between eval-driven development and test-driven development? Both write the expectation before the code. In test-driven development a test passes or fails the same way every time and the assertion is trusted. In eval-driven development a result is a pass rate that varies between runs, LLM judges need validating against human labels, each run costs model calls, and the model can change while your code stays the same.

Where does the term eval-driven development come from? Its origin is unclear. One of the earliest prominent uses is a Vercel post from October 17, 2024 describing how v0 is built. A paper on evaluation-driven development of LLM agents followed on November 21, 2024, OpenAI's evaluation best practices recommend it, and Anthropic recommended it for agent teams on January 9, 2026.

Should you write evals before building a feature? Write the checks you can know in advance: the outcome a task must reach and the rules that must never be broken. Hamel Husain and Shreya Shankar advise against writing evaluators for imagined errors, so leave quality criteria and LLM judges until you have read real outputs and seen what fails.

How many evals do you need to start eval-driven development? 20 to 50 tasks drawn from real failures or real requests are enough to start. Balance them between cases where a behavior should happen and cases where it should not. A set that small is noisy: 16 passes out of 20 has a 95% interval of roughly 58% to 92%.

How do you avoid overfitting a prompt to your evals? Keep a held-out slice of tasks that you run but never read failures from, refill the suite from production traffic every few weeks, and treat a prompt that names specific test inputs as a bug. Gate on a comparison with the base branch, not on an absolute score.

Keep reading

• +AI evals for agents: the parts of an eval and seven steps to write one.
• +LLM testing: the types of LLM tests and why every case is a pass rate.
• +LLM regression testing in CI: step 6 as a pull-request check.
• +Prompt edits deserve the same gate as a model swap: keeping prompt changes comparable with the base branch.
• +How many eval cases do you need?: sizing the suite so a comparison returns a verdict.