Braintrust alternatives for LLM evals in 2026: Langfuse, LangSmith, Promptfoo and more

The best Braintrust alternative depends on why you are leaving. As of November 2026: to self-host for free, Langfuse, or Opik if you want Apache-2.0 all the way down; if you build on LangChain or LangGraph, LangSmith; to standardize on OpenTelemetry, Arize Phoenix; for teams on Databricks, MLflow; if you only need evals in your test suite and not a platform, DeepEval or Promptfoo. The reason most teams start looking is that Braintrust cannot be self-hosted by new customers: the only way to keep data in your own cloud is an Enterprise deployment that Braintrust operates.

This post covers why teams leave Braintrust, what it does well, how the alternatives compare on license, hosting and price, a worked cost example for an agent, Promptfoo alternatives for teams choosing an eval framework instead of a platform, and what moving off Braintrust takes.

A disclosure: I build EvalShift. It is not a Braintrust replacement, so it is not on the list of alternatives; it comes up only where a model swap needs a check before it merges. Every claim about another tool comes from that tool's own pricing page, docs or announcements, checked in November 2026.

Why teams look for a Braintrust alternative

Braintrust is a strong product, and none of the reasons people leave are that it stopped working. They are almost always one of five:

1. +You cannot run it yourself. Braintrust has two deployment options ↗: SaaS, and BYOC, where the data plane runs in your cloud account. BYOC needs an Enterprise plan, Braintrust still operates it, and it is a private preview on AWS for a limited set of customers. Even then the control plane (the UI, authentication, and metadata such as project and experiment names) stays in Braintrust's cloud. Customer-operated self-hosting ↗ is documented only for organizations that already run a data plane. The pricing page still lists "on-prem" under Enterprise, but the deployment docs offer new organizations only SaaS and BYOC. If you need an install you own end to end, Braintrust is not an option.
2. +The bill follows your logs. Braintrust charges no seats, but it meters processed data and scores ↗: every byte you ingest, measured at ingestion, and every score from offline evals, online evals and human review. The free Starter plan includes 1 GB and 10k scores a month, then $4 per GB and $2.50 per thousand scores. Pro is $249 a month with 5 GB and 50k scores, then $3 per GB and $1.50 per thousand. Enterprise is custom. Agents that log long contexts on every span, or score every production request, grow both meters.
3. +Retention. Logs are kept for 14 days on Starter and 30 days on Pro, extendable to 180 days at $0.50 per GB per month. Experiments are kept up to 365 days on every plan.
4. +A proprietary platform. The autoevals ↗ scorer library is MIT, but the platform is not. Some teams want the option to fork, audit or run the whole thing.
5. +You only needed evals. If what you use is Eval() in CI, a platform with tracing, a playground and an agent is more than the job requires, and a framework that runs in your test suite is free.

What Braintrust does well

Worth listing before replacing it, because an alternative has to cover what you actually use:

• +Experiment comparison. You run an Eval() with a dataset, a task and scorers, and compare experiments ↗ against a base row by row. Each experiment also gets a grade, improvement, regression, trade-off or tie, that weighs its scores against latency, cost, errors and load.
• +No seat pricing. Every plan, the free one included, has unlimited users, so product managers and domain experts can review results without a seat budget.
• +Loop. An AI agent ↗ that reads your logs, experiments and datasets, and creates or edits prompts, scorers, datasets and dashboards for you.
• +SDK coverage. TypeScript and Python, plus Go, Java, Ruby and C#, which is wider than any alternative here.
• +One product for the loop. Playgrounds, versioned prompts, online scoring of production traces, human review, OpenTelemetry ingestion and an eval-action ↗ that posts results on a pull request.

Braintrust alternatives at a glance

Braintrust itself, for reference: proprietary, BYOC on Enterprise only, free Starter plan with unlimited users, Pro at $249 a month, billed by processed data and scores.

Braintrust vs Langfuse

Langfuse is the alternative most teams compare first, and Braintrust's own alternatives article ↗ calls it "the closest single-tool competitor". Both trace production, run evals on datasets and live traffic, manage prompts and route traces to human reviewers. They differ on hosting, licensing and the meter.

Sources: Langfuse pricing ↗, Langfuse self-hosted pricing ↗, Braintrust pricing ↗, Braintrust plans and limits ↗, Braintrust deployment options ↗.

What the same agent costs on each

The two bill on different meters, so the list prices do not compare directly. A worked example: 100,000 agent requests a month, each with ten model and tool calls, scored online by two evaluators.

• +Braintrust Pro. The data meter depends on how much you log. At 50 KB per request, 100,000 requests is 5 GB, inside Pro's allowance; every extra GB is $3. The two scores per request are 200,000 scores, 150,000 beyond the included 50,000, at $1.50 per thousand: $225. About $474 a month, with 30 days of logs.
• +Langfuse Core. Each request is 13 units: the trace, ten observations and two scores. That is 1.3M units: $29, plus $72 for the units from 100k to 1M and $21 for the 300k above that, about $122 a month, with 90 days of data. If both scores come from Langfuse's LLM-as-a-judge, each judge run also records its own billable trace ↗, so a request is at least 17 units and the month about $150.

Either way, an LLM judge's own model calls are billed by the model provider on top. Three things move this a lot. On Braintrust, score a sample instead of every request: at 10%, the same traffic is 20,000 scores and the bill is the $249 base. On Langfuse, the cost grows with every span an agent emits: a 40-step agent is 43 units per request instead of 13, and the same traffic costs about $332 a month instead of $122. And self-hosted Langfuse removes the usage bill entirely, in exchange for running Postgres, ClickHouse, Redis and blob storage.

Which to pick

Choose Langfuse when traces must stay in infrastructure you run, you want an open-source platform you could fork, or you want more than 30 days of logs without paying per GB.

Choose Braintrust when your team works through experiments, wants Loop and playgrounds in the same product, uses one of the SDK languages only Braintrust covers, or would rather not operate a platform at all.

Other Braintrust alternatives

Langfuse

Langfuse ↗ is covered in the comparison above. The one thing to budget for if you self-host is operations: a production deployment is two application containers plus Postgres, ClickHouse, Redis or Valkey, and S3-compatible blob storage. Docker Compose is fine for trying it on one VM; production runs on Kubernetes with Helm or the AWS, Azure and GCP guides.

Choose it when owning the deployment matters more than Braintrust's experiment workflow.

LangSmith

LangSmith ↗ is the closest commercial match to Braintrust: tracing, datasets, experiments compared against a baseline, pairwise evaluators, online evaluators and annotation queues. It adds the deepest LangChain and LangGraph integration of any platform, ready-made agent trajectory evaluators in the MIT-licensed agentevals ↗, and LangSmith Deployment for running agents. Evals run as pytest, Vitest or Jest tests.

It bills the other way from Braintrust: Plus is $39 per seat a month with 10k base traces, kept 14 days, or 180 days as extended traces. Self-hosting is an Enterprise add-on, so it does not solve a hosting problem.

Choose it when you build on LangChain or LangGraph, or want trajectory evaluators you do not have to write. LangSmith alternatives goes the other direction, with a full Langfuse vs LangSmith comparison.

Arize Phoenix

Phoenix ↗ is built on OpenTelemetry and the OpenInference conventions, so instrumentation is portable to any OTel backend if you move again. It runs LLM-judged evals, datasets and experiments, with prebuilt evals for agents' tool selection and tool invocation, and self-hosts with pip, Docker or Helm.

The license is the Elastic License 2.0 ↗: source-available, free to run yourself, not OSI open source. Arize AX is the commercial product, with a free tier and Pro at $50 a month.

Choose it when OpenTelemetry is your standard and you want to own the tracing layer.

Opik

Opik ↗, from Comet, is Apache-2.0 and the full platform is free to self-host. It has 43 built-in metrics, including Trajectory Accuracy, Agent Task Completion and Tool Correctness, a pytest integration, and online evaluation rules for production traces. The cloud has a free tier, and Pro ↗ starts at $19 a month; guardrails for PII and prompt injection are on Opik Cloud's Enterprise plan.

Choose it when you want an open-source Braintrust alternative under a permissive license with no enterprise folder, and agent metrics you do not have to write.

MLflow

MLflow ↗ added GenAI tracing and evaluation to the ML platform many data teams already run: mlflow.genai.evaluate() with built-in judges, scorers from DeepEval, Ragas and Phoenix, and judges on incoming production traces. It is Apache-2.0 and free to self-host; Databricks runs the managed version.

Choose it when your organization already tracks models in MLflow or works in Databricks, and another vendor is a harder sell than another feature.

DeepEval and Confident AI

DeepEval ↗ is an Apache-2.0 Python library that treats evals as unit tests: assert_test calls with metrics, run by deepeval test run, which wraps pytest and fails when a metric misses its threshold. It ships 50+ metrics, including Tool Correctness, Argument Correctness, Task Completion and Plan Adherence for agents.

Confident AI ↗, from the same team, is the hosted platform for results and production tracing. It bills per organization rather than per seat: a free plan with 2 seats, Starter at $200 a month with unlimited seats, and Team at $2,000 a month. Self-hosting is on Enterprise.

Choose it when what you use Braintrust for is scoring in CI, and you would rather keep evals in the pytest suite you already run.

Two more you will see on these lists

• +LangWatch ↗ is an agent observability and evals platform under Apache-2.0, except its enterprise/ modules, and self-hosts with Docker Compose. Its cloud has a free tier with 50k events and 14 days of data, and Growth at €29 per core seat a month.
• +W&B Weave moved into CoreWeave with the rest of Weights & Biases; CoreWeave's docs describe Agent Lens ↗, in public preview, as its successor. Check its status before starting on it.

Promptfoo alternatives

Some teams comparing Braintrust are really choosing between a platform and a framework, and the framework they look at first is Promptfoo. It is an MIT-licensed CLI: prompts, providers and test cases in YAML, a matrix of every prompt against every model, trajectory assertions for agents, red teaming, and an official GitHub Action. Evals run locally against your own provider keys.

Promptfoo announced ↗ on March 9, 2026 that it is joining OpenAI, and said it will remain open source. The repository ↗ is still MIT and still releasing. The reasons teams look elsewhere are different ones:

• +A vendor-neutral harness. Some teams prefer that the tool comparing OpenAI, Anthropic and Google models is not owned by one of them.
• +Python, not YAML. Promptfoo is a Node CLI configured in YAML. DeepEval and Inspect AI keep evals in Python next to the code they test.
• +Security focus. The homepage ↗ now leads with agent security, and evals are one part of the product.
• +Production. Promptfoo evaluates before release; continuous monitoring, with scheduled evaluations and alerts, is on the Enterprise plan ↗.

The alternatives, by what you need:

The best LLM evaluation tools compares all of these on what they grade, and EvalShift vs Promptfoo goes dimension by dimension.

Moving off Braintrust

Live tracing moves in a day; datasets, scorers and history take longer.

1. +Instrumentation. If your app exports OpenTelemetry spans to Braintrust, repoint the exporter: Langfuse and Phoenix are built on OTel, and LangSmith ingests it. If it uses Braintrust's SDK wrappers, swap to the new tool's integration for the same client library.
2. +Datasets. Export them through the SDK or API and load them into the new tool. Langfuse accepts CSV upload or the SDK; DeepEval and Promptfoo read files you keep in the repository.
3. +Scorers. autoevals is an MIT library, so scorers built on it can keep running from your own code or inside the new tool's evaluator hooks. Scorers written as Braintrust functions need to be ported.
4. +History. Logs past your plan's retention window are already gone; experiments last up to 365 days. Export anything you will need to compare against before you cancel.
5. +Run both for a couple of weeks and compare what each shows for the same requests before you switch the old one off.

The question most of these leave to you

Braintrust and every alternative above answer "what is production doing?" and "how does this dataset score?" well. The question they mostly leave to you is narrower: if I swap the model under this agent, does it still call the same tools with the same arguments, and is the difference in scores real or noise?

Braintrust grades each experiment against the base, and Langfuse and LangSmith mark or count the examples that moved; none of their docs describe a significance test on the difference. Of the platforms here, only Confident AI says its comparisons include one: its changelog ↗ added statistical significance to every test-run comparison in July 2026, without naming the test. It matters because on a suite of 200 cases, a pass rate of 86% has a 95% confidence interval of roughly 80.5% to 90.1%, so a three-point drop can be noise.

That is the job EvalShift does. It runs your captured production cases through both models, scores tool selection, arguments and trace rules on both sides, and tests each per-example difference for significance: a paired t-test or Wilcoxon, an effect size with a confidence interval, and a false-discovery-rate correction across the run. Then it fails the pull request when the migration policy says the candidate is not safe to ship.

What that catches is the failure an experiment score hides. In one run, the candidate model was 58.8% cheaper and 75.2% faster with no failed calls, and the verdict was still FAIL: it picked a different tool, or a tool where none was needed, on 25% of comparisons against a 10% limit. Every call succeeded at the API level, so a dashboard of cost, latency and errors showed a clean win.

It runs alongside whichever platform you keep; the SDK writes captures to disk, independent of where your traces go. What it does not do: trace production, monitor live traffic or red-team. The CLI and SDK are free and open source; Pro, the hosted gate with run history and baselines, is $199 a month.

How to choose

Then instrument one real agent with your top two, send them the same traffic for a week, and compare the trace views, the eval setup and the bill.

FAQ

What is the best Braintrust alternative? It depends on why you are leaving. Langfuse to self-host tracing and evals for free; Opik for the same under Apache-2.0; LangSmith for LangChain and LangGraph teams; Arize Phoenix for OpenTelemetry-native tracing; MLflow on Databricks; DeepEval or Promptfoo if you only need evals in CI.

Is there an open-source Braintrust alternative? Yes. Langfuse (MIT, except its ee folders), Opik (Apache-2.0) and MLflow (Apache-2.0) are open-source platforms with tracing and evals that you can self-host for free. Arize Phoenix is source-available under the Elastic License 2.0. DeepEval (Apache-2.0) and Promptfoo (MIT) are open-source eval frameworks without production tracing.

Can you self-host Braintrust? Not as a new customer. Braintrust offers SaaS, or BYOC on the Enterprise plan, where Braintrust operates the data plane inside your cloud account; BYOC is a private preview on AWS. The control plane stays in Braintrust's cloud either way. Customer-operated self-hosting is documented only for organizations that already run a data plane.

How much does Braintrust cost? The Starter plan is free with unlimited users, 1 GB of processed data and 10k scores a month, then $4 per GB and $2.50 per thousand scores, with 14 days of retention. Pro is $249 a month with 5 GB and 50k scores, then $3 per GB and $1.50 per thousand scores, with 30 days of retention. Enterprise is custom.

Is Langfuse better than Braintrust? For self-hosting, licensing and long retention, usually yes: Langfuse is MIT-licensed, self-hosts for free and keeps 90 days of data on its $29 plan. Braintrust is stronger for experiment-driven teams, with an improvement, regression, trade-off or tie grade for each experiment, Loop, playgrounds and SDKs in six languages, and needs no infrastructure of your own.

Does Braintrust test whether a difference between experiments is statistically significant? Its documentation describes no significance test. Braintrust compares experiments row by row and grades each one against the base as an improvement, regression, trade-off or tie. Of the platforms compared here, only Confident AI says its comparisons include a significance check, added in July 2026 without naming the test. EvalShift runs a paired t-test or Wilcoxon on each per-example difference, with an effect size, a confidence interval and a false-discovery-rate correction.

What is the best Promptfoo alternative? DeepEval if you want evals as pytest tests in Python; Inspect AI for agent evals in sandboxes with standard errors; DeepTeam for red teaming; Braintrust, LangSmith or Langfuse if you want a platform with tracing and experiments.

Is Promptfoo still open source? Yes. Promptfoo announced in March 2026 that it is joining OpenAI and said it will remain open source, and its repository is still MIT-licensed and actively released.

Keep reading

• +LangSmith alternatives: Langfuse vs LangSmith on pricing, self-hosting and retention.
• +The best LLM evaluation tools: frameworks and platforms compared by the job they do.
• +Build a golden eval suite from production traffic: turning the traces any of these platforms collect into test cases.