LLM 功能上线后实际失败的常见原因与避坑指南
THings that actually fail when an LLM feature meet real users [from experience]
一线开发者总结的 LLM 工程落地踩坑实录，涵盖评估、护栏、成本、数据清洗与可观测性五个核心维度，全是可落地的实操建议，做 Agent 的同学必看。
Demo always looks fine always because it never sees or meets the messy stuff but in prod the failures are almost never the model itself but the layers around it we never think of, although now there are several monitoring tools for this but prevention is always better than remedy so here are a few things that break worth sharing :
Demo 之所以总是表现良好，是因为它从未接触过那些混乱的情况；但在生产环境中，失败几乎从来不是模型本身的问题，而是我们未曾考虑到的周边层。尽管现在已有多种监控工具可用，但预防始终胜于补救，因此以下是几个值得分享的常见故障点：
- No eval set- every prompt or model change is vibes til you have a small scored set you can rerun or else you cant even say if a change helped or just rotated the errors the other direction
- prompt level guardrails which are treated as enforcement, a model talks itself out a rule when its confident enough. if a limit actually matters it has to be deterministic and in code
- costing: this is a scary part, retries with no idempotency or no caching and all just routed to the biggest model regardless of how trivial the task is. overkill
- garbage input: people keep on controversing over models but if it happens like you feed it raw pdfs scraped pages and htmls with noise you wont get a nice output. if your thing needs document processing have a parser like docling or llamaparse in for the docs which can also help in reducing hallucination
- silent failures: plausible and wrong returns and you find it out from a user ticket which is quite embarrassing. so withut per step tracing you cant see where it went off. there are even tools for it to look for but this is wholly up to you, this can also be handled manually
- 缺乏评估集——每次提示词或模型的更改都仅凭感觉（vibes），直到你拥有一个可重新运行的小规模评分数据集，否则你甚至无法判断某项更改是带来了帮助，还是仅仅将错误旋转到了另一个方向。
- 提示词层面的护栏若被当作强制规则执行，当模型足够自信时，它会自行规避规则。如果某个限制确实重要，则必须在代码中以确定性方式实现。
- 成本问题：这是一个令人担忧的部分，缺乏幂等性或缓存的重试机制，以及无论任务多么简单都将请求路由至最大模型的做法，都属于过度设计（overkill）。
- 垃圾输入：人们常常争论模型的好坏，但如果输入的是从 PDF、抓取页面和带有噪声的 HTML 中直接获取的原始数据，你将无法获得良好的输出。如果你的应用需要文档处理，应集成如 Docling 或 LlamaParse 这样的解析器来处理文档，这也有助于减少幻觉。
- 静默失败：返回看似合理但实际错误的结果，而你只能通过用户工单才发现，这相当尴尬。因此，如果没有逐步追踪，你就无法发现问题出在哪里。虽然有相关工具可用于检测，但这完全取决于你自己，此类问题也可以手动处理。
These are mostly the things that break altho thats not all, different projects face different scenarios
这些主要是导致故障的原因，当然并非全部，不同项目会面临不同的场景。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力