OpenAI DevDay:优化AI成本的方法论与Luna Maxing
Stop Overpaying for Intelligence | DevDay 2026
这是极具实操价值的工程指南,直接纠正了开发者常见的计费误区,给出了具体的评估指标和选型框架,做Agent和后端开发的同学值得收藏参考。
Hello everyone, hope you're having an amazing DevDay so far. On the outdoor stage now we're going to have a panel side discussion with two people from our AI deployment engineering team to talk about how to optimize AI costs. So we're going to talk about things like you know what is the right what is the right metric to look at even when you think of optimizing costs, how do you right-size a model for a particular task, levers beyond the model itself, and throughout we'll be drawing on examples from real customers that OpenAI works with.
大家好,希望大家在 DevDay 度过了一段美好的时光。现在我们在户外舞台将举行一场小组讨论,邀请我们 AI 部署工程团队的两名成员,共同探讨如何优化 AI 成本。我们将讨论诸如:在考虑优化成本时,应该关注哪些正确的指标;如何为特定任务对模型进行合理 sizing(规模设定);以及模型之外的其他杠杆因素。在整个过程中,我们将引用 OpenAI 合作客户的实际案例。
So with that please welcome to the stage Mandeep and Sapto. Thank you for being here. Thanks for the introduction, Arty. Thank you. Of course. So let's start with the number one mistakes that we see developers make which is starting with even the metric. You know most people think of cost and look at the list price of the input and the output tokens on our website. But talk to us Mandeep about what do those numbers miss?
那么,有请 Mandeep 和 Sapto 上台。感谢你们来到这里。谢谢 Arty 的介绍。当然。那么让我们从开发者常犯的首要错误开始谈起,那就是连使用的指标都没搞对。大多数人想到的是成本,并查看我们网站上输入和输出 token 的标价。Mandeep,请跟我们谈谈,这些数字遗漏了哪些信息?
Yeah, by the way quick introduction Mandeep. I lead the applied AI engineering team based out of here in San Francisco and welcome to the DevDay. A model that is cheap on cost per token basis may not be the cheapest model on cost per task basis. And there are many reasons for this. Two main reasons are a cheaper model may consume more tokens or produce more tokens to complete the task or the model may not complete the task at all and a human has to intervene to complete the task.
顺便做个简短介绍,Mandeep 领导着位于旧金山的应用 AI 工程团队,欢迎参加 DevDay。一个按 token 计算成本低廉的模型,未必是按任务计算成本最低的模型。原因有很多。主要有两点:一是较便宜的模型可能需要消耗更多 token 或生成更多 token 才能完成任务;二是该模型可能根本无法完成任务,需要人工介入才能完成。
So the metric that developers and businesses should be looking at is cost per task not cost per token. Got it. So cost per task not cost per token. Can you give us a few examples of what this looks like in practice? Yes, I'll give a very personal example. Last week I had to travel to New York City, had to take a flight. Who here has been stranded on an airport with a canceled flight? And well, I guess that misfortune has happened to most of us at one point in our life.
因此,开发者和企业应该关注的指标是“单次任务成本”,而非“每个 token 的成本”。明白了。所以是单次任务成本,而不是每个 token 的成本。能否给我们举几个实践中的例子?好的,我分享一个非常个人的例子。上周我需要前往纽约市,必须乘坐航班。在座的有多少人曾在机场因航班取消而滞留?嗯,我想这种不幸的经历在我们大多数人的生活中都发生过一次。
And I look at the flights online change, but couldn't find the flight that actually matches the schedule. I had to call the airline and I'm greeted by a customer support chatbot. Now I have a soft spot for customer support chatbots because we're in business of building customer support chatbots. So I very patiently explained to the AI chatbot that I wanted to reschedule my flight. I did not want a red eye, but I could not make it understand.
我在网上查看了航班变更情况,但找不到符合我行程的航班。我不得不致电航空公司,接电话的是一位客户支持聊天机器人。我对客户支持聊天机器人情有独钟,因为我们正是从事构建客户支持聊天机器人的业务。于是,我非常耐心地告诉 AI 聊天机器人我想改签航班。我不想坐红眼航班,但它就是无法理解这一点。
So I literally had to say, I need to speak with a human. So a human had to intervene and complete the transaction, answer my query, book me in a flight through Houston, and I arrived before my team offsite. So while the model that the airline may have been using may have been cheaper on cost per token basis, but the cost to actually complete the task was the cost of the tokens plus the human that had to intervene to actually complete the task.
所以我不得不直接说:我需要与真人沟通。于是必须由人工介入来完成交易、回答我的问题,并为我预订经休斯顿的航班,最终我在团队外出会议之前抵达。因此,虽然航空公司可能使用的模型在每 token 成本上更便宜,但实际完成任务的成本是 token 成本加上必须介入以真正完成该任务的人工成本。
So the metric that airlines should have been looking at is the cost per task, which includes the human, the time that human had to spend to complete the task as well. Unfortunately, a very relatable example, and I hope the rest of you had a smoother journey here to DevDay. Yeah, and I'll quote a couple of customer examples. So perplexity, we're familiar with the product, a customer we work very closely with. On their research benchmark, they reported six Astra produces 9% more accurate results at half the cost.
因此,航空公司本应关注的指标是每次任务的成本,这还包括了人工为完成任务所花费的时间。这是一个非常贴切的例子,我希望你们其他人前往 DevDay 的旅程更加顺利。是的,我将引用几个客户案例。Perplexity 是我们熟悉的产品,也是我们紧密合作的客户。在他们的研究基准测试中,他们报告称 Astra 6 产生的结果准确率提高了 9%,而成本减半。
So while compared with the previous model. So while six Astra may be more expensive on token per cost basis, but on a cost per task basis, it's actually much more efficient. It uses fewer tokens and gets the task done. Likewise, Notion, a customer we work very closely with, we're familiar with the Notion product, I use it every day. They are very good at using the migrating to the latest series of models. For example, they migrated from 5.5 to 5.6 sold, reported better accuracy, half the cost per task.
因此,与前一个模型相比。虽然 Astra 6 在每 token 成本上可能更高,但在每次任务的成本基础上,它实际上效率更高。它使用更少的 token 就能完成任务。同样,Notion 也是我们要紧密合作的客户,我们熟悉 Notion 产品,我每天都在使用它。他们在迁移到最新系列模型方面做得非常好。例如,他们从 5.5 迁移到 5.6 版本后,报告的准确率更高,每次任务的成本减半。
Makes sense. You've mentioned, you know, cost per task. So is the task really done successfully? But then the other thing you called out is token efficiency. We continue to invest a lot in making not just our models performant, but also token efficient. And that really plays in. Speaking of, let's start with, you know, we've talked about the right metric here, which is cost per task. The biggest decision you have to make, of course, is which model to pick.
很有道理。你提到了每次任务的成本。那么,任务是否真的成功完成了呢?但你指出的另一件事是 token 效率。我们持续投入大量资源,不仅让我们的模型具备高性能,还要实现 token 高效。这确实至关重要。说到这个,让我们先从正确的指标开始讨论,即每次任务的成本。当然,你必须做出的最大决策是选择哪个模型。
So how do you think through right sizing a model for a particular task? Yeah. So when we advise developer on business on right sizing the model, we recommend begin with crisply defining the task and what accuracy business or a developer expects from that task. So from the airline example, it could have been we want 80% of inquiries to be resolved by the chatbot. So if they had used, for example, GPT Live with 6.0 as a delegate model, they might have been able to resolve my inquiry without delegating it to a human.
那么,你认为如何为特定任务正确匹配模型规模呢?是的。当我们就为业务正确匹配模型规模向开发者提供建议时,我们建议首先清晰地定义任务,以及业务或开发者对该任务期望达到的准确率。以航空公司的例子为例,可以是我们希望 80% 的咨询由聊天机器人解决。如果他们曾使用 GPT Live with 6.0 作为代理模型,他们或许能够直接解决我的咨询,而无需将其转交给人工。
So begin with define the task and then define what is the accuracy we expect from the task. So the goal here is to find the model harness and the configuration that can achieve that accuracy at the best cost possible. Here, it'll be good to introduce a concept, Pareto curve. Many of you may have heard of it. It's essentially plotting two variables that are dependent on each other and you have to optimize for one or the other.
因此,首先定义任务,然后定义对该任务的预期准确率。此处的目标是找到能以最低成本实现该准确率的模型编排(harness)和配置。这里引入一个概念:帕累托曲线(Pareto curve)。你们中许多人可能听说过它。它本质上是将两个相互依赖的变量进行绘图,而你需要针对其中一个或另一个进行优化。
In this case, the two variables that we would focus on is accuracy and cost. The idea here is try different models, try different settings, different configurations, different harnesses, figure out what is the accuracy per task with those configurations and figure out which model configuration and harness meets that accuracy threshold with the best cost possible. Got it. So you touched on, you know, there are some tasks where you might know upfront what model to use.
在这种情况下,我们关注的两个变量是准确率和成本。这里的思路是尝试不同的模型、不同的设置、不同的配置以及不同的编排方案,弄清楚在这些配置下每个任务的准确率是多少,并找出能以最低成本满足该准确率阈值的模型配置和编排方案。明白了吗?所以你提到,你知道有些任务在前期就能确定使用哪个模型。
Like let's say if it is a simple, simpler task, you might want Luna to let's say route. Or if it's a more complex example, you might want Astra to handle a lot of the planning. But what if it is not clear upfront what model to use for a task? Yes. So while luckily here at OpenAI, we have models that are at the Pareto frontier for accuracy and cost. So starting with six Astra, which is our frontier model, very high intelligence, 6.1 Sol provides a very high level of intelligence, very cost efficient per task.
比如,如果是一个简单或较简单的任务,你可能希望 Luna 来处理路由。或者如果是一个更复杂的示例,你可能希望 Astra 处理大量的规划工作。但如果一开始并不清楚某个任务该使用哪个模型呢?是的。因此,幸运的是,在 OpenAI,我们拥有处于准确率和成本帕累托前沿的模型。因此从六代 Astra(six Astra)开始,这是我们的前沿模型,具有极高的智能;6.1 Sol 提供了非常高级别的智能,且每个任务的成本效益非常高。
And then Luna and we've heard Luna maxing. So to your point, if the task is somewhat, many tasks you would intuitively know which model to use. If it is simple classification, or it is extracting a few fields out of a document, you would know that Luna is probably so I would in those cases, I would test a smaller, more efficient model early on in the process. If the task is more requires more intelligence, I would begin with six Astra.
然后是 Luna,我们也听到了关于 Luna maxing 的情况。所以正如你所说,对于许多任务,你会直觉地知道该使用哪个模型。如果是简单的分类任务,或是从文档中提取几个字段,你会知道 Luna 可能是合适的选择。在这些情况下,我会在流程早期测试更小、更高效的模型。如果任务需要更多的智能,我会从六代 Astra 开始。
And then as I test different models, I would plot that Pareto frontier and figure out what is that cutoff where the model and the configuration meets the performance threshold with the best cost efficiency per task. Makes sense. So start with evaluation sets, try to find the model that meets the accuracy threshold that you want, and then try to optimize for costs. Yes, exactly. Got it. You also mentioned Luna maxing.
然后,随着我测试不同的模型,我会绘制那条帕累托前沿曲线,并确定那个临界点,即模型和配置以每个任务的最佳成本效率满足性能阈值的地方。说得通。所以先从评估集开始,尝试找到满足你所需准确率阈值的模型,然后尝试优化成本。是的,完全正确。明白了。你还提到了 Luna maxing。
How many people have heard of that term? Okay, quite a few people. But the overall takeaway there is Luna is actually quite capable model. And it's found to beat a lot of models, like larger models from previous generations. But yeah, Mandib, do you want to quickly tell us about what Luna maxing is? Yes. So a lot of customers are pleasantly surprised that Luna with a high or extra high reasoning effort can actually beat many bigger models, especially from previous generation or competitors to a lot of customers offer Luna as the primary way for their developers to code and only default to other models when a higher intelligence is needed.
有多少人听说过这个术语?好的,有不少人。但总体结论是,Luna 实际上是一个非常强大的模型。它被发现能够击败许多其他模型,包括上一代的大型模型。不过,Mandib,你能快速给我们介绍一下 Luna maxing 是什么吗?是的。因此,许多客户对 Luna 在高或超高推理努力下能够击败许多更大的模型感到惊喜,尤其是与上一代或竞争对手的模型相比。对于许多客户来说,Luna 是他们开发人员编码的首选方式,只有在需要更高智能时才默认使用其他模型。
So we hear that a lot from our customers that Luna for 80-90% of their workflows, it works really well for our customers. So don't discount Luna and give it a shot. It might do more than you think it can do. And today we also announ
所以我们从客户那里听到很多反馈,Luna 在他们的 80-90% 的工作流程中表现非常出色。所以不要低估 Luna,不妨试一试。它可能比你想象的能做得更多。而且今天我们还宣布
原文超出正文长度上限,此处截断——上游还有内容,完整版见上方「原文 ↗」。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力