sPTC:推测式程序化工具调用,让工具延迟与生成重叠
Recommended reading and a really cool idea.
Recommended reading and a really cool idea.
推荐阅读,以及一个非常酷的想法。
There are a lot of interesting harness designs that are starting to emerge around tool calling and code execution. RLM is one of them. But so is this Speculative Programmatic Tool Calling approach (from the same author of RLM).
围绕工具调用和代码执行,出现了许多有趣的框架设计。RLM 是其中之一,但这个推测性程序化工具调用方法(出自 RLM 的同一作者)也是如此。
There are plenty of ways to gain efficiencies at the harness layer.
在框架层面,有很多方法可以提高效率。
Harnesses make agents wait: the model streams a block of code, and tool calls inside it only run once generation finishes. sPTC launches the safe calls early against a copy of the environment, so tool latency overlaps with token generation instead of adding on top of it. Bad guesses get thrown away. So far it's 1 to 1.2x speedup. Very promising.
框架让代理等待:模型流式生成一段代码,其中的工具调用只有在生成完成后才会执行。sPTC 会在环境副本上提前启动安全调用,使工具延迟与令牌生成重叠,而不是叠加在其之上。错误的猜测会被丢弃。目前速度提升为 1 到 1.2 倍。非常有前景。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力