LLM drift: how to detect model drift in AI agents, including tool-calling drift
LLM drift detection for agents: output, tool-selection, argument and call-count drift, silent provider updates vs upgrades, trace replay and a CI gate.
LLM drift is a change in what an application built on a language model does, caused by the model and not by your code. It comes from two places: the provider changes what runs behind a model name you call, or you switch to a new model on purpose. To detect it, run a fixed set of real cases through the model on a schedule and before every upgrade, and compare four things against a baseline: the output, which tools the model calls, the arguments it passes, and how many calls it makes.
For an agent, the last three matter more than the first. A model that skips the order lookup before issuing a refund can still write "I've refunded your order", and every check that reads only the reply will pass it. This post covers the types of drift and how to measure each, the difference between silent provider updates and planned upgrades, how to replay recorded agent traces against another model, and a CI gate you can copy.
A disclosure: I build EvalShift, a tool for testing model changes on agents. It has a short section at the end. Everything before it works without it.
What LLM drift is, and how it differs from classic model drift
In classic machine learning, drift has a settled vocabulary. Evidently AI's guide ↗ defines data drift as "a shift in the distributions of the ML model input features", concept drift as a change in "the relationships between input and target variables", and prediction drift as "the distribution shift in the model outputs". Detection compares distributions: a Kolmogorov-Smirnov test for numeric features, a chi-square test for categorical ones, or a distance such as the Population Stability Index.
All three assume you own the model. The weights only change when you retrain, so drift there means the world moved away from a model that stayed still. With an API model, the model itself can move. That adds a fourth kind, and for most teams it is the one that bites:
| Kind of drift | What changed | Example | Who changes it |
|---|---|---|---|
| Data drift | The inputs | Users start asking about a product you launched last week | Your users |
| Concept drift | What the right answer is | A refund policy changes, and old correct answers become wrong | Your business |
| Prediction drift | The distribution of outputs | More replies end in a handoff to a human | A symptom of the others |
| Model drift (LLM) | The model behind the name you call | A provider updates a snapshot, moves an alias or reroutes a retired model | The provider, or you on upgrade day |
The first three are monitoring problems: you watch production and notice the change. LLM monitoring covers them. The fourth can be tested for directly, because you can send the same inputs to the old and the new model and compare what comes back. That is what the rest of this post is about.
Silent provider updates vs planned upgrades
Model drift arrives in two ways, and they need different detection.
Silent changes happen to a model name you did not change.
- +The model was updated in place. Researchers at Stanford and Berkeley compared ↗ the March and June 2023 versions of GPT-4 on the same tasks. Accuracy on identifying prime numbers fell from 84.0% to 51.1%, and the share of generated code that ran as returned fell from 52.0% to 10.0%. The paper's summary: "GPT-4's ability to follow user instructions has decreased over time". Providers now offer dated snapshots for this reason, but a name that is not pinned can still change under you.
- +The serving stack broke. In September 2025 Anthropic published a postmortem ↗ of three infrastructure bugs that degraded Claude's responses. One misrouted 0.8% of Sonnet 4 requests when it was introduced on August 5, and 16% in its worst hour on August 31. Anthropic wrote that "the evaluations we ran simply didn't capture the degradation users were reporting".
- +A tuning change shipped and was rolled back. OpenAI rolled out a GPT-4o update in ChatGPT on April 24 and 25, 2025, and began rolling it back on April 28 after users found it sycophantic. Its follow-up ↗ says "our offline evaluations—especially those testing behavior—generally looked good", and that its offline evals "weren't broad or deep enough to catch sycophantic behavior". That was the ChatGPT product, not a pinned API snapshot, but it shows how a change can pass the provider's own tests.
- +An alias moved. Google's model docs ↗ say a
latestalias "will get hot-swapped with every new release", with two weeks' notice by email, and its changelog recordsgemini-pro-latestswitching togemini-3-pro-previewon January 21, 2026. Anthropic's older aliases move too:claude-sonnet-4-5resolves ↗ to "the most recent dated snapshot for that minor version". From the 4.6 generation on, Anthropic says each model ID, dateless ones included, is a pinned snapshot. - +A retired model was rerouted. Google's
changelog ↗ says that from December 1, 2026,
requests to
gemini-2.5-prowill automatically route togemini-3.8-flash, and requests togemini-2.5-flashtogemini-3.5-flash-lite. Those requests will keep returning 200, and the changelog warns that the breaking changes "include function calling andthought_signaturevalidation". On October 8, 2026, requests togemini-3.5-flashwere already being routed togemini-3.6-flash.
Planned upgrades are changes you make: a new model ID in a config file, usually because the old one is being retired or the new one is cheaper. You know the date, and you can test before it ships. The deprecation schedule lists the dates that force these for OpenAI, Anthropic and Google.
| Silent change | Planned upgrade | |
|---|---|---|
| What changes | The model behind a name, or the system serving it | The name itself |
| Notice | None, or an email about an alias | You choose the date, or a deprecation notice sets it |
| When you find out | From users, or from a scheduled check | Before release, if you test |
| What to compare | Today's behavior against a stored baseline | The new model against the current one, on the same inputs |
| Main defense | Pin snapshots, record the model served, run a check on a schedule | A paired comparison gated in CI |
The defenses overlap. Pinning a snapshot turns most silent changes into planned ones, because the
name you call stops moving until you move it. OpenAI's
text generation guide ↗ recommends pinning
production applications to snapshots such as gpt-5.5-2026-04-23, and Google's says "most
production apps should use a specific stable model". Pinning does not protect you from serving
bugs like Anthropic's, and it does not survive the snapshot's retirement date. Anthropic's model
ID docs say as much: if a pinned ID starts behaving differently, "an infrastructure update is the
most likely cause."
The four types of drift in an agent
An agent's behavior is a trace, not a reply: which tools it called, in what order, with what arguments, and what it said at the end. Each part drifts on its own.
| Type | What changes | Example | How to measure | Needs ground truth? |
|---|---|---|---|---|
| Output drift | The final reply | Answers get longer, change format, or state a different fact | Code checks on format, an LLM judge on a criterion, embedding similarity to the old reply | For correctness, yes. For "did it change", no |
| Tool-selection drift | Which tools are called | issue_refund without lookup_order first | Compare the set of tool names per case with the baseline's | No |
| Argument drift | The values passed to a tool | order_id: "a1042" where the baseline passed "A-1042" | Per-field comparison: exact for IDs and enums, a tolerance for numbers, similarity for free text | No |
| Call-count drift | How many calls are made | A search repeated five times, or a confirmation step skipped | Count of calls per case, compared with a tolerance | No |
None of the last three needs a hand-written expected answer to detect a change. The baseline model is the reference: a case drifted if the new model does something different from what the old one did on the same input. Whether the change is a regression is a second question, and the answer depends on whether the old behavior was right. Keep the two apart, or every case where the old model was wrong reads as a regression when the new one fixes it.
Output drift
Measure output with the cheapest check that answers the question. A JSON reply either parses and matches the schema or it does not, and that check costs nothing and never disagrees with itself. For prose, an LLM judge with one pass-or-fail criterion per check, compared pairwise with the baseline's reply, tells you whether the new reply is worse. When to trust an LLM judge covers how to check the judge.
Embedding similarity between the old and the new reply measures change, not quality. A reworded correct answer scores low. Use it to find what moved, and do not gate on it alone.
Tool-selection drift
Compare the set of tool names each model called on the same case. A set and not a sequence, because a model that calls the same two tools in parallel in a different order has not drifted in any way that matters. Switch to an ordered comparison only for tools where order is the point, such as a lookup that must come before a write.
This is the type with the worst failures, because a tool call is an action. A skipped verification step or an extra write does not show up in the reply.
Providers' own migration notes say tool use changes between versions, in both directions:
- +Anthropic's Sonnet 5 prompting guide ↗ says it "is more agentic than Claude Sonnet 4.6 by default and reaches for tools … more readily". Its general best practices ↗ warn that prompts written to stop older models undertriggering tools may make newer ones "overtrigger".
- +OpenAI's GPT-5 prompting guide ↗ calls GPT-5 "thorough and comprehensive when trying to gather context in an agentic environment", which means more calls before an answer. Its GPT-4.1 guide ↗ says that model follows instructions "more closely and more literally than its predecessors".
- +OpenAI's function calling guide ↗
notes that one snapshot,
gpt-4.1-nano-2025-04-14, can repeat a call to the same tool when parallel tool calls are on.
A prompt tuned against one model's habits encodes them. When the habits change, the prompt is tuned for a model you no longer run.
Argument drift
Same tool, different values. Compare field by field, and pick the comparison from the field's type:
- +IDs, enums, booleans and dates: exact match after trimming whitespace and case-folding.
A-1042anda-1042are the same order.A-1042and1042are not. - +Numbers: a relative tolerance. A refund of 49.99 and one of 50.00 may be fine. One of 499.90 is not.
- +Free text, such as a search query: a similarity score, reported and not gated. Two different wordings of the same query are expected.
Schema enforcement narrows this but does not remove it. OpenAI's function calling guide says
strict: true makes calls "reliably adhere to the function schema". That guarantees the shape: an
order_id that is a string. It does not guarantee the value is the right order.
Call-count drift
Count calls per case and allow a tolerance of one, because models legitimately differ by a call: one fetches two records in one call, the other in two. A difference of three or more, or a case that hits your step limit, is a different strategy. Look at p95 as well as the median, since loops show up in the tail first.
Measure the noise before you call anything drift
A model sampled twice on the same input does not always return the same thing, so some of what looks like drift is noise. Before trusting a comparison, run the current model against itself: the same cases, twice, with the same settings. The rate at which a model disagrees with itself is your noise floor, and drift is what exceeds it.
Three habits keep the comparison honest:
- +Pair by case. Compare each case's new result with the same case's old result, not one average with another. Pairing removes the variation between easy and hard cases.
- +Report a rate with its interval. Three drifted cases out of 40 is 7.5%, and the 95% Wilson interval runs from 2.6% to 19.9%. Twelve out of 400 is 3%, with an interval of 1.7% to 5.2%. The first says almost nothing. How many eval cases do you need works through sizing a suite so the interval is narrow enough to decide.
- +Fix the thresholds before you look. Decide the budget for each type of drift before the run. A threshold chosen after seeing the numbers is chosen to make them pass.
Replay agent traces against a different model
The most useful baseline is your own production traffic. The question is how to send a recorded agent run to a different model, given that an agent run is a loop: the model calls a tool, the tool result goes back in, the model calls another tool.
Rerunning the whole loop live has two problems. The tools have side effects, so a replay that
executes issue_refund issues a refund. And once the new model makes a different call in round
one, everything after it is a different conversation, so the two runs no longer compare.
The method that works is teacher-forced replay:
- +Record each production run in full. The input, the tools offered, every model call, every tool call with its arguments, and every tool result. Remove personal data before it is written.
- +Replay round by round. For round one, send the new model the recorded input and the recorded toolset, and compare its tool calls with the recorded ones.
- +Feed back the recorded results, never live ones. For round two, send the recorded conversation up to that point, including the recorded tool results, and compare again. No tool is executed, so the replay has no side effects.
- +Score each round. Both models see identical context at every round, so each comparison isolates one model decision.
- +Stop where the recording ends. If the new model calls a tool the original run never called, there is no recorded result to feed back. Record that as tool-selection drift in that round, and go no further down that path.
Teacher forcing has a known limit: it tests each decision given the original model's history, not the path the new model would take on its own. A new model that recovers well from its own mistakes, or compounds them, looks the same in a teacher-forced replay. That is the price of a comparison that stays paired and has no side effects, and for detecting drift it is the right trade. For the end-to-end path, run a small set of tasks against a sandbox with fake tools.
Replay also needs the request reproduced, not just the messages. Carry over the generation
settings the original call used, such as tool_choice, parallel tool calls and response format.
A replay that drops tool_choice: required will report drift that is really a missing setting.
Build a golden suite from production traffic covers turning recordings into a suite, and evaluating agent tool calls covers scoring the traces.
Catch silent drift with a scheduled check
Silent drift has no pull request to hang a check on, so it needs a schedule. Four steps:
- +Pin snapshots in production. Call a snapshot ID, not an alias, so the name you call only changes when you change it.
- +Record the model that answered. Each provider's response names it. OpenAI's
modelfield is "the model used for the chat completion", and Gemini'smodelVersionis "the model version used to generate the response". Chart responses by that value and alert on a new one. OpenAI'ssystem_fingerprint, once the usual signal for backend changes, is now marked deprecated. - +Run a fixed suite on a schedule. Nightly or weekly, run the same cases through the model you serve, and compare the tool calls and outputs with the run stored from the day you last accepted it. A drift rate whose interval sits above the noise floor is a change.
- +Run the alias alongside the snapshot. If you pin
gpt-5.5-2026-04-23, run thegpt-5.5alias on the same schedule. While the two agree, the alias has not moved. When they diverge, you have your upgrade test results before anyone asked you to upgrade.
A CI gate for drift
Before a model change merges, the same comparison runs as a gate. The script below takes two JSONL files of recorded traces, one line per case with the tool calls each model made, and fails the job when the drift rate for a check is above budget with 95% confidence. It reports inconclusive when the interval straddles the budget, so a small suite does not pass by default.
import json
import math
import sys
BUDGET = 0.05 # at most 5% of cases may drift on each check
ID_FIELDS = ("order_id", "customer_id") # arguments that must match exactly
def load(path):
with open(path) as f:
return {row["id"]: row for row in map(json.loads, f)}
def wilson(k, n, z=1.96):
"""95% interval for k drifted cases out of n."""
if n == 0:
return 0.0, 1.0
p = k / n
centre = (p + z * z / (2 * n)) / (1 + z * z / n)
half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
return max(0.0, centre - half), min(1.0, centre + half)
def drifted(base, cand):
b_names = [c["name"] for c in base["calls"]]
c_names = [c["name"] for c in cand["calls"]]
b_args = {c["name"]: c["args"] for c in base["calls"]}
c_args = {c["name"]: c["args"] for c in cand["calls"]}
return {
"tool_selection": set(b_names) != set(c_names),
"arguments": any(
b_args[tool].get(field) != c_args[tool].get(field)
for tool in b_args.keys() & c_args.keys()
for field in ID_FIELDS
),
"call_count": abs(len(b_names) - len(c_names)) > 1,
}
base, cand = load(sys.argv[1]), load(sys.argv[2])
rows = [drifted(base[i], cand[i]) for i in sorted(base.keys() & cand.keys())]
failed = False
for check in ("tool_selection", "arguments", "call_count"):
k = sum(row[check] for row in rows)
lo, hi = wilson(k, len(rows))
if lo > BUDGET:
verdict, failed = "FAIL", True
elif hi > BUDGET:
verdict = "inconclusive"
else:
verdict = "pass"
print(f"{check}: {k}/{len(rows)} drifted, 95% CI {lo:.1%} to {hi:.1%}: {verdict}")
sys.exit(1 if failed else 0)On 120 cases where the candidate skipped the lookup in 14 and changed an order ID's case in two, it prints:
tool_selection: 14/120 drifted, 95% CI 7.1% to 18.6%: FAIL arguments: 2/120 drifted, 95% CI 0.5% to 5.9%: inconclusive call_count: 0/120 drifted, 95% CI 0.0% to 3.1%: pass
The argument check compares raw values, so a case-only difference counts as drift. Normalize IDs before comparing if your system treats them as the same. Wire the script into a workflow that runs on pull requests touching the model config, and on a schedule for silent drift:
on:
pull_request:
paths: ["agent/models.yaml", "prompts/**"]
schedule:
- cron: "0 5 * * 1" # Mondays, 05:00 UTC
jobs:
drift:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- run: python replay.py --suite suite/ --out candidate.jsonl
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
- run: python drift_gate.py baseline/traces.jsonl candidate.jsonlreplay.py stands for your replay harness: whatever runs the suite through the model and writes one
line per case. LLM regression testing in CI covers the
pull-request side in more depth.
Doing this with EvalShift
Disclosure again: I'm the founder. EvalShift is built around the planned-upgrade half of this post, and the scheduled check is the same command on a timer.
- +Recording. The capture SDK records production agent runs, with every model call, tool call and tool result, to local disk or a bucket you own. Redaction runs in process before anything is written.
- +Replay.
evalshift capture syncpromotes captures into a golden suite, and--rounds allkeeps every tool round. A run then replays teacher-forced: recorded tool results are fed back, never live ones, and the recordedtool_choiceand parallel-call settings are replayed with each request. - +The four types.
tool_selectioncompares the tools each model called, as a set by default.tool_argumentscompares values field by field, choosing exact, numeric or similarity matching from the tool's schema.tool_trace_structurechecks call counts with a tolerance of one, parallelism and refusals. Output is covered by schema checks, a pairwise LLM judge and an embedding similarity evaluator that measures how far the new reply moved from the current model's, not whether it is right. The evaluators docs cover all of them. - +The gate. A
migration_policysets a budget per type, includingmax_tool_divergenceandmax_tool_argument_drift. The verdict is pass, conditional pass, fail or inconclusive, and inconclusive means the suite was too small to decide. The GitHub Action runs it on pull requests, and runs under aschedule:trigger too, without the PR comment.
One command runs the comparison:
evalshift compare --suite-name support --from gpt-5.5-2026-04-23 --to gpt-5.5 --policy-gate
What it does not do: watch production. EvalShift has no dashboards and no alerts on live traffic. Run a monitoring tool for that, and the trends page for how scheduled runs compare over time.
FAQ
What is LLM drift? LLM drift is a change in what an application built on a language model does, caused by the model rather than by your code. It happens when a provider updates the model behind a name you call, moves an alias or reroutes a retired model, and when you upgrade to a new model yourself.
What is the difference between model drift and data drift? Data drift is a change in the inputs a model receives. Model drift in the LLM sense is a change in the model itself while the inputs stay the same. Classic machine learning models only change when you retrain them, so their drift is mostly data and concept drift. An API model can change underneath you.
How do you detect model drift in an LLM application? Run a fixed suite of real cases through the model on a schedule and before every upgrade, and compare each case with a stored baseline: the output, the tools called, the arguments and the number of calls. Report each drift rate with its interval, and measure the model against itself first to know the noise floor.
What is tool-calling drift? Tool-calling drift is a change in which tools an agent calls, the arguments it passes or how many calls it makes, after a model update or upgrade. It is often invisible in the final reply: an agent that skips a verification step can write the same answer as one that ran it.
How do you replay agent traces against a different model? Use teacher-forced replay: record each production run with its tool results, then send the new model each round's recorded context, feeding back recorded tool results rather than executing tools. Compare each round's tool calls with the recorded ones, and stop where the new model calls a tool the recording has no result for.
Do LLM providers change models without notice? Snapshots are meant to stay fixed, but aliases move, retired models get rerouted, and serving bugs change behavior: Anthropic's September 2025 postmortem described three infrastructure bugs that degraded responses for weeks. Pin snapshots, record the model named in each response, and run a scheduled check.
Keep reading
- +What actually breaks when you switch LLMs: the change surface beyond the answer text.
- +Evaluating agent tool calls: scoring the trace, not the reply.
- +LLM monitoring: the production half of drift detection.
- +LLM model deprecation schedule: the dates that force planned upgrades.
