基于Jev构建轻量级框架无关意图与工具路由库
Built a lightweight, framework-agnostic intent & tool router using Jev AI (to speed up agent loops)
Hey everyone,
大家好，
Lately, while building agent architectures, I kept running into the same bottleneck: waiting for large generative LLMs to decide which tool to call or which path to take at every step is both costly and slow. Especially when low latency matters, traditional generative LLMs tend to bloat.
最近，在构建 Agent 架构时，我反复遇到同一个瓶颈：等待大型生成式 LLM 在每个步骤中决定调用哪个工具或选择哪条路径，既昂贵又缓慢。特别是在对延迟敏感的场景下，传统的生成式 LLM 往往会带来性能开销。
TypeSafe's newly released "System One" decision model, Jev (typesafe/jev-1.13), offers a pretty solid alternative for this. Since it returns typed Choice and Score primitives instead of generating text, it's remarkably fast. However, there was a gap: there wasn't a standard, independent (framework-agnostic) router layer that you could just plug and play behind existing agent frameworks. Everyone was hacking together their own custom solutions.
TypeSafe 新发布的 "System One" 决策模型 Jev（typesafe/jev-1.13）为此提供了一个相当可靠的替代方案。由于它返回的是类型化的 Choice 和 Score 基元，而非生成文本，因此速度极快。然而，存在一个空白点：缺乏一个标准的、独立的（与框架无关的路由层），以便直接嵌入现有的 Agent 框架之后使用。大家都在自行拼凑定制解决方案。
So, I built a lightweight Python library that you can integrate right in front of LangChain, CrewAI, or entirely custom agent loops: jev-route
因此，我构建了一个轻量级的 Python 库，你可以将其集成到 LangChain、CrewAI 或完全自定义的 Agent 循环之前：jev-route
What does it do?
它是做什么的？
- Takes the incoming prompt or state and classifies it in milliseconds via the Jev engine.
- Has built-in confidence-gate support; if Jev's confidence score falls below your threshold, it automatically triggers a fallback mechanism or routes to the main LLM.
- Built async-first with zero bloat dependencies (only requires httpx and pydantic).
- 通过 Jev 引擎在毫秒级内对传入的提示词或状态进行分类。
- 内置置信度门控支持；如果 Jev 的置信度分数低于你的阈值，它将自动触发回退机制或路由至主 LLM。
- 采用异步优先设计，零冗余依赖（仅需 httpx 和 pydantic）。
I've just released the first version (v0.2.0), complete with unit tests and basic examples. Thought it might be useful or interesting for others building similar agent architectures.
我刚发布了第一个版本（v0.2.0），包含单元测试和基本示例。觉得这对其他正在构建类似 Agent 架构的人可能有用或有趣。
You can check out the code and details here: 👉 GitHub: [https://github.com/huzeyfe07/jev-route]
你可以在这里查看代码和详情：👉 GitHub: [https://github.com/huzeyfe07/jev-route]
I'm completely open to feedback, ideas, or PRs. What are you guys currently using for speed in your agent loops?
我非常欢迎反馈、想法或 PR。你们目前在 Agent 循环中使用什么来提速？
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力