2026年LLM可观测性与评估平台对比：Langfuse、LangSmith等

LLM applications fail in ways traditional software does not. The same prompt can produce different outputs. A retrieval step can return the wrong document while every HTTP status reads 200. An agent can loop through fourteen tool calls, burn thousands of tokens, and deliver a confidently wrong answer. Standard application performance monitoring (APM) alone does not capture this semantic behavior — prompt and output quality, retrieval relevance, or agent-level reasoning traces.

This is the gap LLM observability and evaluation platforms fill. They record every span of an LLM pipeline — prompts, completions, retrievals, tool calls, token counts, latencies, and costs — and then score outputs for quality using automated evaluators. In 2026, this category has moved from optional tooling to core infrastructure for any team running AI in production.

The market data reflects the shift. The Business Research Company sizes the LLM observability platform market at $2.69 billion in 2026, up from $1.97 billion in 2025, and projects $9.26 billion by 2030 at a 36.2% forecast CAGR. Gartner predicts that by 2028, LLM observability investments will account for 50% of GenAI deployments, up from 15% in early 2026. LangChain’s State of Agent Engineering survey of 1,300+ professionals found that 57% of respondents now run agents in production. Nearly 89% have implemented observability for their agents. Evaluation lags behind: 52.4% run offline evaluations, 37.3% run online evaluations, and 29.5% report no evaluation at all. Quality was cited by 32% as the top barrier to production deployment.

This article compares the leading platforms across three axes: tracing depth, evaluation capability, and production monitoring. Figures were checked against primary sources (company documentation, press pages, and announcements) as of August 2026; where only secondary reporting exists, it is linked and identified as such. Rankings and “best for” judgments are editorial assessments, not measured benchmarks.

How the Category is Structured in 2026

The market has split into four camps, and understanding the split matters more than any individual feature list.

• AI-native observability platforms: Langfuse, LangSmith, Braintrust, Arize, Opik — treat the LLM trace as the primary object. They capture nested spans across agents, retrievers, and tools, and attach evaluation scores to production traffic.
• Open-source and source-available evaluation libraries and platforms: Arize Phoenix, DeepEval (Confident AI), MLflow, RAGAS — focus on scoring outputs: faithfulness, hallucination, answer relevance, and task completion, often via LLM-as-a-judge.
• AI gateways: Helicone, Portkey, LiteLLM — sit as a proxy between the application and model providers. They add logging, caching, cost tracking, and routing with minimal code changes.
• APM extensions: Datadog LLM Observability, New Relic, Dynatrace — bolt LLM tracing onto existing infrastructure monitoring so AI signals correlate with CPU, memory, and network metrics.

One standard now connects all four camps. The OpenTelemetry GenAI semantic conventions define vendor-neutral gen_ai.* span attributes for model calls, token usage, agent steps, and tool executions. OpenTelemetry, a CNCF project, maintains these conventions, which are adopted by platforms including Google Cloud, AWS, Azure, and Datadog. The conventions now live in a dedicated repository, with the GenAI registry under active development as of August 2026. Coding agents are converging on the standard too: GitHub Copilot’s agent telemetry exposes gen_ai.* span trees, Claude Code provides opt-in OpenTelemetry tracing, and Codex includes native OpenTelemetry export support. Instrumenting once against gen_ai.* improves backend portability and reduces vendor-specific instrumentation, even if implementations still differ. Buyers in 2026 should treat OTel compatibility as a hard requirement, not a nice-to-have.

The Three Axes: Tracing, Evals, and Production Monitoring

Because vendors use these terms loosely, precise definitions help before comparing platforms:

• Tracing is the record of everything an LLM application did. A trace contains nested spans: the user input, each retrieval call, each model invocation with its exact prompt and parameters, each tool execution, and the final output. Depth matters because agent traces are deeply nested with heavy payloads — a single conversation can generate megabytes of data across dozens of runs and tool calls. Non-determinism makes tracing non-negotiable: the same prompt produces different outputs, so an issue cannot be reproduced without capturing the exact input, model parameters, and temperature at call time.
• Evals answer the question tracing cannot: was the output any good? Offline evals score curated datasets before deployment, catching regressions when a prompt, model, or retrieval index changes. Online evals score live production traffic, typically via LLM-as-a-judge, sampling traces and grading them for faithfulness, relevance, toxicity, or task completion. The hardest failures are outputs that are technically valid but wrong for the domain — a hallucinated policy, a drifting tone, a retrieval miss that produces a confident but incorrect answer. Traditional latency, error-rate, and availability metrics do not detect these semantic quality failures.
• Production monitoring closes the loop: dashboards, cost attribution per model and user, latency percentiles, drift detection across prompts and use cases, and alerting when quality scores fall. The best platforms feed production traces back into eval datasets, so every real-world failure becomes a future regression test.

A platform can be strong on one axis and weak on another. Gateways excel at monitoring but skip deep tracing. Eval libraries score outputs but do not watch production. The platforms below are ranked on how completely they cover all three.

Langfuse describes itself as the most widely adopted LLM engineering platform, and its open-source adoption numbers back a strong claim.

Tracing: Langfuse captures nested traces for LLM calls, retrieval, embedding, and agent actions through OpenTelemetry, LangChain, OpenAI SDK, and LiteLLM integrations. Its signature nested trace view collapses a multi-step RAG or agent run into a stepable tree with per-span latencies and token counts. An observations-centric data model shipped in March 2026, delivering 10x+ dashboard performance gains and laying the groundwork for Langfuse v4, which the company says runs up to 165x faster.

Evals: The platform supports LLM-as-a-judge evaluators, human annotation queues, custom scores, and dataset-based regression testing that runs in CI via GitHub Actions. Evaluator templates cover hallucination, toxicity, and relevance.

Production monitoring: Cost breakdowns by model, user, or session, plus session replays for conversational agents.

Deployment: MIT-licensed core, self-hostable via Docker Compose in minutes, or managed on Langfuse Cloud with a free tier. Langfuse is widely regarded as the self-host leader in this category.

Best for: teams that want a full-featured, open-source, framework-agnostic platform with strict data-residency control.

LangSmith is LangChain’s commercial platform for observing, evaluating, and deploying agents. It is framework-agnostic with Python, TypeScript, Go, and Java SDKs plus OpenTelemetry support, but it is the default backend for LangChain 1.0 and LangGraph 1.0, where integration requires near-zero glue code.

Tracing: Full conversation and agent-run traces expose every step, tool call, and intermediate state. Polly, a built-in AI assistant, summarizes large traces to pinpoint problems. LangSmith Engine clusters production failures into prioritized issues, locates root causes in traces and code, and proposes fixes for review.