本地RAG应用构建实践：文件不出本机与CPU嵌入方案
Building a RAG app where your files never leave the computer: what worked for me
做本地RAG的同学必看，这篇给出了CPU嵌入性能、SQLite存储和引用校验的具体参数与踩坑经验，直接可复用。
A lot of people want to use AI with their own documents but don't want to upload them anywhere. I've been building a desktop app around that idea, and I want to share the approach in case it helps anyone building something similar. No vendor pitch here, these are just the decisions and numbers I measured.
很多人希望在自己的文档上使用 AI，但不想将文档上传到任何地方。我一直围绕这个理念开发桌面应用，我想分享这种方法，希望能对构建类似产品的人有所帮助。这里没有厂商推销，只是我做出的决策和测量的数据。
1. Index locally, send only the relevant chunks.
1. 本地索引，仅发送相关的片段。
Reading files, splitting them into chunks, creating embeddings and searching all happen on the user's machine. Only the top few chunks that answer a question go to the language model. The full files are never uploaded.
读取文件、将其拆分为片段、创建嵌入向量以及搜索都在用户机器上完成。只有回答问题的前几个片段会发送给语言模型。完整文件永远不会被上传。
2. A small embedding model on CPU is enough.
2. CPU 上的小型嵌入模型就足够了。
I tested several. EmbeddingGemma (quantized ONNX, chunks of 128 tokens) got 87–94 % of the recall of my server setup, running on CPU with no GPU. Indexing about 1,000 office files took around 12 minutes and 660 MB of RAM. The smaller models I tried (e5-small, MiniLM) were faster but lost too much recall.
我测试了几种。EmbeddingGemma（量化 ONNX 格式，每块 128 个 token）在 CPU 上运行且无 GPU 的情况下，达到了我的服务器设置 87–94% 的召回率。索引大约 1,000 个办公文件耗时约 12 分钟，占用 660 MB 内存。我尝试的较小模型（e5-small、MiniLM）速度更快，但召回率损失过多。
3. You don't need a vector database server.
3. 你不需要向量数据库服务器。
SQLite with the vectors stored as blobs and encrypted at rest works fine at this scale. One less thing to install for the user.
SQLite 将向量存储为 blob 并在静止状态下加密，在此规模下工作良好。这样用户就少了一项安装负担。
4. Clean the data before chunking.
4. 在分块之前清理数据。
Huge table cells (GeoJSON inside a CSV, for example) broke the embedding step. Capping chunk size and limiting tables to their header plus about 200 rows fixed most of the slow and failed files.
巨大的表格单元格（例如 CSV 中的 GeoJSON）会导致嵌入步骤出错。限制块大小并将表格限制为表头加上约 200 行，解决了大多数缓慢和失败的文件问题。
5. Check the answer, not just the retrieval.
5. 检查答案，而不仅仅是检索结果。
A source number next to a sentence doesn't mean the source says that. A second model checks each sentence against the chunk it cites, and anything unsupported gets flagged instead of hidden.
句子旁边的来源编号并不意味着来源确实说了那句话。第二个模型会将每个句子与其引用的片段进行核对，任何缺乏支持的内容都会被标记出来，而不是被隐藏。
If you're working on local or private RAG and you're stuck on something specific (choosing an embedding model for CPU, chunk size, indexing speed, checking citations, mixed languages), ask in the comments. Happy to share what worked and what didn't.
如果你正在处理本地或私有 RAG 并卡在某个具体问题（如为 CPU 选择嵌入模型、块大小、索引速度、检查引用、混合语言），请在评论中提问。我很乐意分享哪些方法有效，哪些无效。
For reference, this is what I built with it: ScrapeVantageAI Desktop, available on the Microsoft Store. You can try it locally with your own files: https://apps.microsoft.com/store/detail/9PDBP54WB0BJ
作为参考，这是我用它构建的产品：ScrapeVantageAI Desktop，可在 Microsoft Store 获取。你可以使用自己的文件在本地试用：https://apps.microsoft.com/store/detail/9PDBP54WB0BJ
And if you're curious what a RAG system looks like when you keep adding pieces on top of it, the full web version is here: https://scrapevantageai.edatasoluciones.com
如果你对不断在上面添加组件后 RAG 系统长什么样感到好奇，完整的 Web 版本在这里：https://scrapevantageai.edatasoluciones.com
The newest piece is a "map of meaning": news from 300 sources in 11 countries, grouped by what they actually say instead of the words they use, so the same topic in Spanish, English or Portuguese lands in the same cluster, and you can read all of it in Spanish, English, Portuguese or German. On top of that map, an agent recommends what to read and what to look into next, and every recommendation links back to its source. I'd love feedback on that part, it's the most experimental one.
最新的一个功能是‘意义地图’：整合来自11个国家300个来源的新闻，按照它们实际表达的内容而非所用词汇进行分组，因此西班牙语、英语或葡萄牙语中的同一主题会归入同一个聚类，你可以用西班牙语、英语、葡萄牙语或德语阅读所有内容。在此地图之上，一个智能代理会推荐你阅读什么以及下一步关注什么，每条推荐都链接回其原始来源。我很希望得到关于这一部分的反馈，它是最具实验性的部分。
Thanks for your time!
感谢你的时间！
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力