Not Diamond 发布模型路由方法,降低 Agent 成本
Not Diamond just released the methodology behind their model routing which gets…
Agent 开发者必看,这套将路由建模为序列决策的方法直接解决了 KV Cache 浪费和成本痛点,有具体指标和逻辑,值得参考落地。
Not Diamond just released the methodology behind their model routing which gets Opus xhigh quality while cutting agent costs 20–80%.
Not Diamond 刚刚发布了其模型路由背后的方法论,该方法在保持 Opus 高输出质量的同时,将代理成本降低了 20%–80%。
Model routing becomes much harder decision for long-running coding agents. So Not Diamond is benchmarking the router across simulated user turns, changing task complexity, and response delays that can expire the KV cache.
对于长期运行的编码代理而言,模型路由成为一个更为复杂的决策问题。因此,Not Diamond 正在模拟用户交互轮次、变化的任务复杂度以及可能导致 KV 缓存过期的响应延迟等场景下,对路由器进行基准测试。
And harder than it seems, switch models too aggressively and you can throw away the KV cache. Pick one model from the opening prompt and you are assuming task complexity stays fixed for the rest of the session.
这比看起来要难得多:如果过于激进地切换模型,可能会浪费掉 KV 缓存;而如果仅根据初始提示选择一个模型,则意味着假设整个会话期间任务复杂度保持不变。
So Not Diamond is treating routing as a sequential decision problem instead.
因此,Not Diamond 转而将路由视为一个序列决策问题。
At each step, its router predicts future reward and cost for a particular model and reasoning effort, using current and previous session state, message and token counts, task complexity, KV-cache state, and intermediate reward signals.
在每个步骤中,其路由器利用当前和之前的会话状态、消息与 token 数量、任务复杂度、KV 缓存状态以及中间奖励信号,预测特定模型及推理努力程度带来的未来奖励与成本。
Read more detail on their technical report.
请参阅其技术报告以了解更多细节。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力