Flama：让模型与LLM成为一等公民的Python框架及架构论文
Flama: 8 years building a Python framework where ML models and LLMs are first-class, now written up on arXiv [P]
这是一篇极具参考价值的工程实践分享，Flama提出的“模型即文件”与多协议统一封装思路非常巧妙，适合正在构建LLM应用基础设施的同学深入研究其架构设计与取舍。
I started Flama in 2018 to take a trained model to a production API with as little ceremony as possible. With 2.0 LLMs became first-class alongside classic models, and together with Miguel Durán-Olivencia we've now written up the whole architecture in an 83-page paper on arXiv (2608.18733).
我于2018年启动了Flama项目，旨在以尽可能少的繁琐流程将训练好的模型部署为生产级API。随着2.0版本中LLM成为与经典模型并列的一等公民，我和Miguel Durán-Olivencia现在已在arXiv上撰写了一篇83页的论文（2608.18733），详细阐述了整体架构。
A model is a file
模型即文件
Every model, whether a scikit-learn pipeline, a PyTorch network or an LLM, is packaged into a single .flm: the weights, the artifacts it needs at inference time (tokenizer, config) and its metadata, including the training metrics you record. The metadata sits ahead of the weights, so flama model <file> inspect reads it without loading anything, and a CI job can gate promotion on those metrics without standing up a server. flama model <file> run scores a reference set offline on the same code path the server uses. The inference engine isn't baked in either: the same LLM file runs on vLLM or MLX, depending on where you deploy it.
每个模型，无论是scikit-learn管道、PyTorch网络还是LLM，都被打包成一个单一的.flm文件：包含权重、推理时所需的工件（如tokenizer、config）及其元数据，包括你记录的训练指标。元数据位于权重之前，因此`flama model <file> inspect`可以在不加载任何内容的情况下读取它，CI作业可以基于这些指标来限制发布流程，而无需启动服务器。`flama model <file> run`会在同一代码路径下离线对参考集进行评分，该路径与服务端使用的相同。推理引擎也未硬编码：同一个LLM文件可以根据部署位置在vLLM或MLX上运行。
One model, four protocols
一个模型，四种协议
from flama import Flama app = Flama() app.models.add_model( path="/llm/", model="google_gemma-4-E2B-it.flm", name="assistant", serving=("native", "openai", "anthropic", "ollama"), )That model answers OpenAI (/llm/openai/v1/chat/completions), Anthropic (/llm/anthropic/v1/messages) and Ollama (/llm/ollama/api/chat) routes at the same time, with a streaming chat UI at /llm/chat/, next to your own endpoints and your classic models.
该模型同时响应OpenAI（/llm/openai/v1/chat/completions）、Anthropic（/llm/anthropic/v1/messages）和Ollama（/llm/ollama/api/chat）的路由，并在/llm/chat/提供流式聊天UI，旁边是你的自定义端点和经典模型。
Three independent layers
三个独立层
A request goes through a dialect, a canonical transport and a backend (vLLM on Linux with CUDA, MLX on Apple Silicon). Because the layers only meet in the middle, any dialect works on any backend, which is also why tool calls and token usage come out right in every protocol.
请求依次经过方言层、规范传输层和后端层（Linux上的CUDA vLLM，Apple Silicon上的MLX）。由于各层仅在中间交汇，任何方言都可以适配任何后端，这也是为什么工具调用和Token使用量在每个协议中都能正确输出的原因。
The rest of the framework
框架的其他部分
- Dependency Injection resolved from type hints
- Schemas with Pydantic, Marshmallow or Typesystem, and OpenAPI docs generated from your handlers
- CRUD APIs generated from SQLAlchemy tables
- DDD building blocks: repositories and workers (units of work)
- JWT authentication
- MCP: turn any app into an MCP server
- `flama upgrade`, codemods that migrate your code between versions
- A Rust core for routing, serialising, parsing, crypto and compressing
- 基于类型提示解析依赖注入
- 使用Pydantic、Marshmallow或Typesystem构建Schema，并从你的处理器生成OpenAPI文档
- 从SQLAlchemy表生成CRUD API
- DDD构建块：仓库和工作单元（units of work）
- JWT认证
- MCP：将任何应用转变为MCP服务器
- `flama upgrade`，用于在不同版本间迁移代码的代码修改工具（codemods）
- 用于路由、序列化、解析、加密和压缩的Rust核心
The paper walks through each subsystem and compares Flama with web frameworks, model servers and LLM inference engines (Table 1).
论文详细介绍了每个子系统，并将Flama与Web框架、模型服务器和LLM推理引擎进行了比较（表1）。
Paper: https://arxiv.org/abs/2608.18733
论文：https://arxiv.org/abs/2608.18733
Code: https://github.com/vortico/flama
代码：https://github.com/vortico/flama
If you find it useful, a star on GitHub helps more people find it, and I'm happy to answer anything about the design.
如果你觉得它有用，在GitHub上点个星可以帮助更多人发现它，我也很乐意回答关于设计方面的任何问题。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力