跳到主内容
精选75Tibo技巧与观点

论 token 与每 token 价格:不同模型 token 不可比

On tokens and prices per token.

原文

On tokens and prices per token.

关于令牌及每个令牌的价格。

I said I’d write more about this, so here goes: an OpenAI token != another model’s token. We compare AI prices in dollars per million tokens as if a token were a standardized unit, like a gram or a kilowatt-hour. It isn’t. Different models use and produce the exact same text using different numbers of tokens, which means a lower price per token does not necessarily mean a lower bill.

我说过我会就此写更多内容,所以现在开始:OpenAI 的令牌并不等同于其他模型的令牌。我们以每百万令牌的美元价格来比较 AI 价格,仿佛令牌是一个标准化单位,就像克或千瓦时一样。但事实并非如此。不同的模型使用不同数量的令牌来生成完全相同的文本,这意味着较低的每令牌价格并不一定意味着较低的账单。

Imagine two identical pizzas. One is cut into 8 slices at $2 each. The other is cut into 16 slices at $1.25 each. The second place advertises cheaper slices, but the whole pizza costs $20 instead of $16. Bummer ... your stomach doesn't actually care about the number of slices you just ate.

想象两个相同的披萨。一个切成 8 片,每片 2 美元。另一个切成 16 片,每片 1.25 美元。第二家宣传更便宜的片价,但整个披萨要 20 美元,而不是 16 美元。真糟糕……你的胃其实并不关心你吃了多少片。

I know you are hungry now, but back to tokens. In one small comparison spanning English, technical, multilingual, and numerical text, the tokenizer we use for GPT-5.6 Sol used 766 tokens versus an estimated 1,170 for Claude Opus 5. That's a very significant difference of about 34.5% fewer tokens. You can get the same exact text, but pay for all those extra tokens. The price per token doesn't really tell this story.

我知道你现在饿了,但回到令牌的话题。在一个涵盖英语、技术、多语言和数字文本的小型比较中,我们用于 GPT-5.6 Sol 的分词器使用了 766 个令牌,而 Claude Opus 5 估计需要 1,170 个令牌。这是一个非常显著的差异,令牌数量减少了约 34.5%。你可以得到完全相同的文本,但需要为那些额外的令牌付费。每令牌的价格并不能真正说明这一点。

Even correcting for tokenizer differences misses the bigger point. What actually matters is price per successful outcome, and for that you can use benchmarks as a starting point, but really you have to try it and measure on your own use cases.

即使纠正了分词器的差异,也忽略了更重要的一点。真正重要的是每次成功结果的价格,为此你可以使用基准测试作为起点,但实际上你必须自己尝试并在自己的用例中进行测量。

That's all. May the tokens flow.

就这样。愿令牌流动。

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

另一事件,读法相近