Google API拦截成人内容标注,Nemotron推理截断踩坑复盘
Labelling ~50k entries (some adult) for $0: Google's API blocked the model, another host didn't
提供了一手的大模型数据标注实战数据,涵盖模型能力对比、API稳定性陷阱及成本控制细节,对从事数据工程的同学极具参考价值。
We're labelling ~50k web catalogue entries into ~12 JSON fields each (~3.8k input tokens per call). Part of the data is adult, which turned out to matter more than model choice.
我们将约5万条网络目录条目标注为每个条目约12个JSON字段(每次调用输入约3.8k token)。部分数据涉及成人内容,结果发现这比模型选择的影响更大。
What we found
我们的发现
- Google AI Studio free tier: Gemma 4 31B worked on our first prompt. After we added two ordinary moderation sections, it returned PROHIBITED_CONTENT on 20/20 entries, including ones that had passed the day before. That block isn't an adjustable safety category, so there's no setting for it.
- Same model, different host: Gemma 4 31B via DeepInfra (through OpenRouter) labelled 30/30 with the identical prompt. OpenRouter's free Gemma is served only by AI Studio, so it has the same filter.
- Quality: Claude Opus labelled 30 entries as a reference. Field match: Nemotron 3.5 Super VL 73–77% (free, NVIDIA API / OpenRouter :free), DeepSeek V4.1 Flash 72% (~$0.002/item), Sonnet 70% (only 11 entries), Gemma 4 31B 57%. Small sample, so we read Nemotron as "on par with Sonnet", not better.
- Gotcha: Nemotron writes ~5k tokens of reasoning before the JSON. At max_tokens 8,192, 5/30 answers were truncated.
- Google AI Studio 免费层级:Gemma 4 31B 在我们的首个提示词下正常工作。但在我们添加了两个常规的内容审核模块后,它在20/20条条目上返回了 PROHIBITED_CONTENT,包括前一天通过审核的条目。该拦截不属于可调整的安全类别,因此没有相关设置选项。
- 同一模型,不同主机:通过 DeepInfra(经由 OpenRouter)运行的 Gemma 4 31B 使用相同的提示词对30/30条条目进行了标注。OpenRouter 的免费 Gemma 仅由 AI Studio 提供服务,因此具有相同的内容过滤机制。
- 质量评估:Claude Opus 对30条条目进行标注作为参考。字段匹配率:Nemotron 3.5 Super VL 73–77%(免费,NVIDIA API / OpenRouter :free)、DeepSeek V4.1 Flash 72%(约$0.002/条)、Sonnet 70%(仅11条条目)、Gemma 4 31B 57%。由于样本量较小,我们将 Nemotron 视为“与 Sonnet 相当”,而非更优。
- 注意事项:Nemotron 在输出 JSON 前会生成约5k token 的思考过程。在 max_tokens 设为8,192的情况下,30个回答中有5个被截断。
Running it
运行过程
One entry per call, alternating NVIDIA and OpenRouter, a 429 pauses that source for all lanes. Going from 4 to 8 lanes gave 300–450/hour for a couple of hours, then NVIDIA started returning 429 about once a minute. Over a full day we average ~250/hour, about 6k labels a day, for $0 plus OpenRouter's one-time $10 credit purchase (which lifts `:free` from 50 to 1,000 requests a day).
每次调用处理一个条目,交替使用 NVIDIA 和 OpenRouter,一旦某个来源触发429错误,所有通道都会暂停使用该来源。从4个通道增加到8个通道后,最初几小时的吞吐量达到每小时300–450条,随后 NVIDIA 开始大约每分钟返回一次429错误。全天平均吞吐量约为每小时250条,即每天约6k条标注,成本为0美元,外加 OpenRouter 一次性购买的10美元额度(这将 `:free` 的每日请求上限从50提升至1,000次)。
Lesson: measure throughput over 24 hours, not over your best two.
经验教训:应测量24小时内的吞吐量,而非仅看最佳两小时的数据。
Has anyone found a way to predict what trips PROHIBITED_CONTENT, or is the answer always "use another host"?
有人找到预测什么会触发 PROHIBITED_CONTENT 的方法了吗?还是答案永远是“换另一个主机”?
Not a native English speaker, so an LLM helped with the wording; all numbers are from our own logs.
非英语母语者,因此借助LLM优化了措辞;所有数据均源自我们自身的日志。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力