Import AI 479: Realtime-Bench评测LLM低延迟能力
Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4
Realtime-Bench 首次系统量化了前沿模型在低延迟实时交互中的能力边界，数据极具参考价值；同时 DIA 报告揭示了 AI 算力基础设施（太空数据中心）面临的宏观地缘风险，值得从业者关注。
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.
欢迎订阅 Import AI，这是一份关于人工智能研究的通讯。Import AI 的运行依赖于 arXiv、卡布奇诺咖啡以及读者的反馈。如果您想支持我们，请订阅。
Subscribe now
立即订阅
LLMs are very bad at low-latency tasks:
大型语言模型（LLMs）在低延迟任务方面表现很差：
…Realtime-Bench says it’ll be a while till AI systems can play games or do online auctions for you…
……Realtime-Bench 表示，AI 系统要能为你玩游戏或参与在线拍卖，还需等待一段时间……
Researchers have built Realtime-Bench, “a suite of carefully curated dynamic tasks where delayed actions directly affect success”. The test helps evaluate how well modern, general-purpose AI systems can perform tasks where decisions need to be made in real-time.
研究人员构建了 Realtime-Bench，“这是一个精心策划的动态任务套件，其中延迟的行动会直接影响成功与否”。该测试有助于评估现代通用 AI 系统在需要实时做出决策的任务中的表现能力。
The benchmark consists of 50 interactive game and web tasks, split across 30 web tasks in four categories (travel, e-commerce, service, and security), and 20 game tasks (arcade, puzzle, strategy, and simulation).
该基准测试包含 50 个交互式游戏和网络任务，分为 30 个网络任务（涵盖旅游、电子商务、服务和安全四个类别）和 20 个游戏任务（涵盖街机、益智、策略和模拟）。
Who did the research: The benchmark was built by people at the University of Texas at Austin, Ohio State University, University of Illinois at Urbana-Champaign, NVIDIA, Northwestern University, and Steadyworks.
研究团队：该基准测试由德克萨斯大学奥斯汀分校、俄亥俄州立大学、伊利诺伊大学厄巴纳-香槟分校、NVIDIA、西北大学和 Steadyworks 的研究人员构建。
What it measures: The test is designed to probe for three qualities: temporal state awareness (”RT-Perceive”), where models can use observations to track changes in the world; adaptive deliberation (”RT-Deliberate”), where models alter their reasoning length and depth according to available time; and time-aware action selection (”RT-Act”) where models need to appropriately time their actions for the situation.
测量指标：该测试旨在探究三种特质：时间状态感知（“RT-Perceive”），即模型能否利用观察结果来追踪世界的变化；自适应深思熟虑（“RT-Deliberate”），即模型能否根据可用时间调整推理的长度和深度；以及时间感知的行动选择（“RT-Act”），即模型需要根据情境恰当地把握行动时机。
“For example, an agent purchasing a ticket must track the sale status and assess whether observed availability is still current (RT-Perceive), limit further deliberation when the purchase window is closing (RT-Deliberate), and choose between waiting for sales to open and purchasing while tickets remain available (RT-Act).”
“例如，一个购买票证的智能体必须跟踪销售状态并评估所观察到的可用性是否仍然有效（RT-Perceive），在购买窗口即将关闭时限制进一步的深思熟虑（RT-Deliberate），并在等待销售开放与在票证仍有库存时购买之间做出选择（RT-Act）。”
They have three different conditions: Wall clock, which is designed to test out performance under real-world latency. Paused clock, which lets you arbitrarily pause things so you can get a sense of baseline capability, and Token clock, which “advances the environment at a fixed rate per generation token. An action is applied at the simulated time reached when it is committed, so consuming more tokens delays execution regardless of actual inference latency”.
他们设置了三种不同的条件：Wall clock（墙钟），旨在测试现实世界延迟下的性能；Paused clock（暂停钟），允许你任意暂停操作，以便了解基线能力；以及 Token clock（Token 钟），它“以每个生成 Token 固定的速率推进环境。当动作被提交时，会在达到模拟时间时应用该动作，因此消耗更多的 Token 会延迟执行，无论实际的推理延迟如何”。
How well do models do? The authors tested out models from OpenAI (GPT-6-Astra, GPT-5.6 Sol, GPT-Realtime-2, OpenAI Decisions), Anthropic (Claude Fable 5.1, Claude Opus 5), Google (Gemini 3.8 Flash, Gemini 3.1 Flash Live, Gemini 3.8 Live), Jev 1.13, as well as Chinese models from Deepseek (V4.1) and Moonshot AI (Kimi-K3).
模型的表现如何？作者测试了来自 OpenAI（GPT-6-Astra、GPT-5.6 Sol、GPT-Realtime-2、OpenAI Decisions）、Anthropic（Claude Fable 5.1、Claude Opus 5）、Google（Gemini 3.8 Flash、Gemini 3.1 Flash Live、Gemini 3.8 Live）、Jev 1.13，以及来自 Deepseek（V4.1）和 Moonshot AI（Kimi-K3）的中国模型。
GPT-6 Astra (high) got the highest pass rate on the ‘wall clock’ evaluation setting, with 36%.
在“挂钟”评估设置下，GPT-6 Astra (high) 取得了最高的通过率，为 36%。
By comparison, with a paused clock, models with high reasoning get closer to 70% or even 80% success.
相比之下，在暂停计时的情况下，具备高推理能力的模型成功率更接近 70% 甚至 80%。
Why this matters: This benchmark is an effective proxy for where AI agents will break – the low scores here point to tasks where AI agents cannot yet be expected to perform well.
为何这很重要：该基准测试是预测 AI 智能体何处会失效的有效代理指标——此处的低分数表明，在这些任务中，目前还无法期望 AI 智能体能表现出色。
Check out the leaderboard: Realtime-Bench (official website).
查看排行榜：Realtime-Bench（官方网站）。
Read the paper: Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems? (arXiv).
阅读论文：Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?（arXiv）。
The future of spacewar, courtesy of the Defense Intelligence Agency:
太空战争的愿景， courtesy of the Defense Intelligence Agency：
…Anti-satellite tech, space nukes, and more…
……反卫星技术、太空核武器等……
Working in AI can be stressful so sometimes to relax myself I like to read about other topics to get a sense for what horrors lie elsewhere. I take the same approach with depression, where if I feel sad I will try and listen to music by people sadder than me in an attempt to cheer myself up (e.g., Radiohead, Neil Young, and in very dark moments Bill Callahan / Smog, etc).
从事 AI 工作可能会带来压力，因此有时为了放松自己，我喜欢阅读其他主题的内容，以了解其他地方潜伏着怎样的恐怖。我对待抑郁也采取同样的方法：如果感到悲伤，我会尝试听比我更悲伤的人创作的音乐，试图让自己振作起来（例如 Radiohead、Neil Young，以及在非常黑暗的时刻，Bill Callahan / Smog 等）。
So with that in mind I recently read “Challenges to Security in Space”, by the Defense Intelligence Agency.
因此，考虑到这一点，我最近阅读了由 Defense Intelligence Agency 撰写的《太空安全面临的挑战》。
What’s going on in space: As with conflict on Earth, space is defined by conflict between the US versus China, Russia, Iran, and North Korea, with China the most technologically sophisticated, and Russia the most seemingly gung ho.
太空中正在发生什么：与地球上的冲突一样，太空的格局由美国与中国、俄罗斯、伊朗和朝鲜之间的冲突所定义，其中中国的技术最为先进，而俄罗斯则显得最为激进。
“China and Russia are developing a formidable spectrum of offensive capabilities to degrade or destroy space services of other countries, from kinetic destruction to sophisticated jamming and cyber operations,” the DIA writes. “China has more than tripled its space launches per year in the past decade, with 22 launches in 2016 and 93 launches in 2025… Beijing judges space is central to the United States and its allies’ operations. Beijing also considers space an offensive military domain, which has shaped integral components of PLA military planning and campaigns.”
DIA 写道：“中国和俄罗斯正在开发一系列强大的进攻性能力，以削弱或摧毁其他国家的太空服务，从动能摧毁到复杂的干扰和网络行动。”“在过去十年中，中国的年度太空发射次数增加了三倍多，2016 年为 22 次，2025 年为 93 次……北京方面认为太空对于美国及其盟友的行动至关重要。北京还将太空视为一个进攻性的军事领域，这塑造了中国人民解放军军事规划和战役中的核心组成部分。”
Spacewar: The general vibe you get from reading this 100+ page report is there is a huge amount of ugly, frightening stuff happening in space, or being done to things in space by things on the ground. Some example quotes:
太空战争：阅读这份超过 100 页的报告后，你感受到的总体氛围是，太空中正在发生大量丑陋且令人恐惧的事情，或者地面上的事物正在对太空中的物体进行此类操作。以下是一些示例引言：
- “By the late 2020s, China may field space-based directed energy systems and higher-powered ground-based directed energy systems capable of threatening satellites in higher orbits”
- “China is developing space-based technology, including on-orbit counterspace weapons, capable of holding space systems at risk”
- “Russia’s Peresvet, a ground-based laser system, is designed to dazzle enemy satellites’ optical systems to mask the movement of Russia’s strategic missile systems”
- “Russia has launched several satellites over the last few years which conducted test activities on orbit for technologies that could enable attacks on other countries’ satellites and result in temporary or permanent damage”
- China has a couple of spaceplanes, akin to the US’s hyper-secret X-37B vehicle, and they appear to be getting more sophisticated. In 2023, one of China’s aforementioned vehicles, “released and interacted with an object.”
- “到2020年代末，中国可能部署天基定向能系统和功率更高的地基定向能系统，能够威胁高轨道卫星”
- “中国正在开发天基技术，包括在轨反空间武器，能够对空间系统构成风险”
- “俄罗斯的‘佩列斯韦特’（Peresvet）是一种地基激光系统，旨在致盲敌方卫星的光学系统，以掩护俄罗斯战略导弹系统的机动”
- “过去几年，俄罗斯发射了多颗卫星，这些卫星在轨进行了测试活动，其技术可能用于攻击其他国家的卫星，并造成暂时或永久性损坏”
- 中国拥有几艘空天飞机，类似于美国高度机密的X-37B飞行器，且它们似乎正变得越来越复杂。2023年，上述中国飞行器之一“释放并与一个物体进行了交互”。
Satellite-on-satellite fights: It’s obvious throughout the report that we’re in a world where rival nations have satellites that try to go up and mess with other satellites from other people.
卫星对卫星的对抗：整份报告显而易见地表明，我们身处这样一个世界：敌对国家拥有试图升空并干扰他人卫星的卫星。
- “In 2019, Russia launched satellite Cosmos 2542, which released a second satellite, Cosmos 2543. Cosmos 2543 then began following a U.S. national security satellite and approached it multiple times at a distance close enough to create potentially dangerous operating conditions,” the DIA writes. “Russia is developing kinetic orbital systems to permanently disable, or “kill,” satellites. Moscow’s Burevestnik is a developmental, co-orbital air-launched ASAT weapon for killing spacecraft in LEO”.
- “In February 2022, Russia launched a satellite called Cosmos, which Moscow claimed at the time was testing the effects of space radiation on electronic systems. This satellite was launched into an unusual and isolated orbit not used by any other spacecraft”.
- “2019年，俄罗斯发射了卫星‘宇宙2542号’（Cosmos 2542），该卫星释放了一颗第二颗卫星‘宇宙2543号’（Cosmos 2543）。随后，‘宇宙2543号’开始跟随一颗美国国家安全卫星，并多次接近至足以造成潜在危险运行条件的距离，”国防情报局（DIA）写道。“俄罗斯正在开发动能轨道系统，以永久禁用或‘摧毁’卫星。莫斯科的‘锆石’（Burevestnik）是一种处于研发阶段的、共轨的空射反卫星（ASAT）武器，用于摧毁低地球轨道（LEO）中的航天器”。
- “2022年2月，俄罗斯发射了一颗名为‘宇宙’的卫星，当时莫斯科声称该卫星正在测试太空辐射对电子系统的影响。这颗卫星被发射到一个异常且孤立的轨道中，没有任何其他航天器使用该轨道。”
Space nukes: Russia is experimenting with nukes in space. “A nuclear detonation in space would create an electromagnetic pulse, which would immediately affect satellites within range and could impact civilian infrastructure,” the DIA writes. “A nuclear detonation in space could be tailored to impact satellites in most orbits—LEO for commercial and reconnaissance satellites, GEO for critical communications, and MEO for navigation and timing satellites”.
太空核武器：俄罗斯正在试验太空核武器。“太空中的核爆炸会产生电磁脉冲，这将立即影响范围内的卫星，并可能对民用基础设施造成影响，”国防情报局（DIA）写道。“太空中的核爆炸可以定制以影响大多数轨道上的卫星——商业和侦察卫星所在的低地球轨道（LEO）、关键通信所在的地球静止轨道（GEO），以及导航和授时卫星所在的中地球轨道（MEO）”。
Why this matters – infrastructure for the next phase of the AI power struggle: As people spend more resources on building space-based data centers (and eventually, perhaps, lunar-based ones), space will become ever more strategically and commercially valuable and ever more contested. Research like this gives us a sense of the hardware which is already in space and will arrive in the future. Perhaps in some far-off future an AI might seek to take control of such space assets as well.
为何此事重要——为 AI 权力斗争下一阶段的基础设施：随着人们投入更多资源建设太空数据中心（未来或许还包括月球基地），太空的战略和商业价值将日益凸显，争夺也将愈发激烈。此类研究让我们对目前已进入太空及未来将抵达的硬件有了清晰的认识。也许在遥远的未来，AI 也会试图控制这些太空资产。
Read more: DIA Releases Challenges to Security in Space (Defense Intelligence Agency, press release).
延伸阅读：国防情报局发布《太空安全挑战》（美国国防情报局，新闻稿）。
Read the report: Challenges to Security in Space (DIA, PDF).
阅读报告：《太空安全挑战》（美国国防情报局，PDF 格式）。
Metis: A startup trying to fuse symbolic and neural reasoning:
Metis：一家致力于融合符号推理与神经推理的初创公司：
…Towards Vulcan-like machine minds…
……迈向类似《星际迷航》中瓦肯人的机器心智……
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力