AI系统可观测性分层解析与Opik实践
Layers of observability in AI systems, explained visually:
RAG工程落地必备指南，清晰拆解了从嵌入到生成的全链路监控指标，Opik方案可直接复用，建议工程师对照检查现有链路。
Layers of observability in AI systems, explained visually:
AI 系统中可观测性的分层，图解说明：
If an LLM app is serving real users, its input and output are not enough to debug it.
如果一个 LLM 应用正在服务真实用户，仅靠其输入和输出不足以进行调试。
Consider a RAG pipeline where a query passes through embedding, retrieval, context assembly, and generation.
考虑一个 RAG 管道，查询依次经过嵌入、检索、上下文组装和生成阶段。
Every operation adds latency, may call a paid API, and can fail while still producing a valid-looking response.
每个操作都会增加延迟，可能调用付费 API，并且可能在产生看似有效响应的同时失败。
Traces and spans provide visibility.
追踪（Traces）和跨度（Spans）提供可见性。
- A trace records the full path of one request. The Trace column runs from query to response.
- A span records one operation within that trace. The colored boxes are spans.
- 一次追踪记录单个请求的完整路径。Trace 列从查询运行到响应。
- 一个跨度记录该追踪内的单个操作。彩色方框即为跨度。
Each span captures:
每个跨度捕获以下信息：
> Query span
> 查询跨度
The input, timestamp, session identifier, and request metadata.
输入内容、时间戳、会话标识符和请求元数据。
> Embedding span
> 嵌入跨度
The model, input size, latency, retries, and rate-limit errors.
模型、输入大小、延迟、重试次数和速率限制错误。
> Retrieval span
> 检索跨度
The retrieved chunks, document IDs, relevance scores, filters, top-k value, and latency. Many RAG failures originate here. Without these fields, there is no evidence that retrieval selected the wrong documents.
检索到的文本块、文档 ID、相关性分数、过滤器、top-k 值和延迟。许多 RAG 故障源于此。如果没有这些字段，就无法证明检索选择了错误的文档。
> Context span
> 上下文跨度
The context assembled from retrieved chunks, instructions, and conversation history. This catches truncated documents, duplicated chunks, missing citations, and prompts exceeding the token budget.
由检索到的文本块、指令和对话历史组装而成的上下文。这能捕捉到截断的文档、重复的文本块、缺失的引用以及超出令牌预算的提示词。
> Generation span
> 生成跨度
The model, token counts, time to first token, total latency, finish reason, retries, and estimated cost.
模型、令牌计数、首令牌时间、总延迟、结束原因、重试次数和预估成本。
With these details, a bad response can now be traced to retrieval, context assembly, or generation.
有了这些细节，现在可以将不良响应追溯到检索、上下文组装或生成环节。
To use this in practice, Opik already implements this observability infrastructure for LLM apps and is open source.
要在实践中使用此功能，Opik 已经为 LLM 应用实现了这种可观测性基础设施，且它是开源的。
It captures traces and spans across LLM calls, retrieval steps, and tool executions, with latency, token usage, and cost attached to each operation.
它跨 LLM 调用、检索步骤和工具执行捕获追踪和跨度，并为每个操作附加延迟、令牌使用量和成本。
GitHub repo: https://github.com/comet-ml/opik
GitHub 仓库：https://github.com/comet-ml/opik
(don't forget to star it ⭐)
（别忘了给它点个星 ⭐）
In Opik, every operation belonging to one request carries the same Trace ID. If the app processes 1,000 requests, it creates 1,000 traces, each containing its own spans.
在 Opik 中，属于同一请求的每个操作都携带相同的 Trace ID。如果应用处理 1,000 个请求，它将创建 1,000 个追踪，每个追踪包含其各自的跨度。
This makes cost analysis more useful. Instead of aggregate spend, teams can identify the model calls, retries, or oversized prompts responsible.
这使得成本分析更具实用性。团队不再仅关注总体支出，而是能够识别出导致问题的模型调用、重试或过大的提示词。
Over time, changes in retrieval scores, embedding latency, or context size become visible before they turn into broader quality problems.
随着时间的推移，检索分数、嵌入延迟或上下文大小的变化会在演变为更广泛的质量问题之前变得显而易见。
That said, observability is one of eight areas I would learn for building production LLM systems.
话虽如此，可观测性是我在构建生产级 LLM 系统时需要学习的八个领域之一。
I covered all eight in the 2026 LLM Engineering Roadmap, with free and open-source resources for each one.
我在《2026 LLM 工程路线图》中涵盖了全部八个领域，并为每个领域提供了免费和开源的资源。
Read it below.
请在下方阅读。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力