用 Jev 整理 2300 篇 AI 论文:成本 14 美分耗时 83 秒
Found a great production use case for Jev.
给做 RAG 或知识库的同学一个极具参考价值的工程实践,低成本快速完成大规模数据清洗与重分类,值得借鉴其 System One/Two 组合思路。
Found a great production use case for Jev.
为 Jev 找到了一个出色的生产环境用例。
I used Jev to organize ~2.3K AI research papers.
我使用 Jev 整理了约 2300 篇 AI 研究论文。
The total cost was $0.14, and it took about 83 seconds.
总成本为 0.14 美元,耗时约 83 秒。
The process:
流程如下:
The papers already had old tags, which I ran through a previous open model (DeepSeek V4 Flash).
这些论文原本已有旧的标签,我通过之前的开源模型(DeepSeek V4 Flash)对其进行了处理。
However, I wasn't confident in the classifications, and I didn't want to spend more on tokens unless I spent time tuning it into a good LLM classifier via few-shot (more expensive). Too tedious, too costly, and unsustainable.
然而,我对分类结果缺乏信心,除非花费时间通过少样本学习将其调优为一个优秀的 LLM 分类器(成本更高),否则我不想再在 token 上投入更多费用。这太繁琐、成本太高且不可持续。
Luckily for us, we now have Jev to help us with organizing papers better.
幸运的是,我们现在有了 Jev 来帮助我们更好地组织论文。
So Jev first went through all the papers and retagged them. It agreed with 75% of the previous tags.
因此,Jev 首先对所有论文进行了重新打标。它与之前 75% 的标签一致。
Jev found about 579 high-confidence topic changes.
Jev 发现了约 579 个高置信度的主题变更。
I evaluated reliability by manually labeling 30 disagreements and accepted all of them. I was astonished by Jev's classification capabilities. We applied and verified all changes in production.
我通过手动标注 30 个分歧案例来评估可靠性,并全部接受了它们。我对 Jev 的分类能力感到惊叹。我们在生产环境中应用并验证了所有更改。
I'm much more satisfied with the classifications, but I think there is still room for improvement.
我对分类结果的满意度大大提高,但我认为仍有改进空间。
Check out the papers here: https://academy.dair.ai/papers
在此查看相关论文:https://academy.dair.ai/papers
I will experiment with Jev more. The takeaway is that pipelines can be significantly improved by carefully combining System One and System Two models.
我会进一步尝试使用 Jev。得出的结论是,通过仔细结合 System One 和 System Two 模型,可以显著提升流水线效率。
Jev clearly unlocks more interesting ways to organize papers and offer a more useful discovery layer for research papers. More updates on that soon.
Jev 显然解锁了更多有趣的论文组织方式,并为研究论文提供了更有用的发现层。稍后将更新更多信息。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力