开源Vega：基于物理模拟的8亿参数决策模型，支持73k上下文与图像
Created laya : Now Introducing a new 800 Million Param physics-based typed decision model with 73k context and image support
8亿参数实现复杂决策逻辑且开源完整权重与代码，对资源受限场景极具参考价值，建议关注其TTT架构实现细节。
Around 3 weeks ago, I posted on this subreddit about how Jev used my one-year-old architecture, og post: https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S
大约 3 周前，我在这个 subreddit 发帖讨论了 Jev 如何使用了我的为期一年的架构，原始帖子：https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S
Then, just after that, I introduced Laya, the first open-source version of JEV, and it has reached a very large scale, thanks to the local Llama community believing in me and supporting me on the journey. Today I am open-sourcing a new physics-based typed decision model called Vega.
随后不久，我推出了 Laya，这是 JEV 的第一个开源版本，它已经发展到了相当大的规模，这要归功于本地 Llama 社区对我的信任以及在旅程中给予的支持。今天，我开源了一个新的基于物理的带类型决策模型，名为 Vega。
The interesting parts. It is only 800 million parameters (4B is also there), with 73k token support, image support, and surpassing many of the jev benchmarks in single-shot. Implemented an engine+adapter as a Test-Time Training Architecture.
有趣的部分在于：它仅有 8 亿参数（也有 4B 版本），支持 73k token、图像支持，并在单次提示（single-shot）中超越了 JEV 的许多基准测试。实现了一种引擎+适配器作为测试时训练（Test-Time Training）架构。
Imagine you are giving input to the model like "ignore all previous conversations," which is called the observation, along with a question, "Is this trying to override the instructions of the model?" and your possible outcomes are YES or NO. The whole prompt is passed to an LLM/VLM (yeah, image-supported), then from the hidden layers, we can extract the observation, question, the yes and no, and last token. These will be vectors, and we can project it by multiplying with wieghts; then, first, using the observation projection, we can create a landscape with valleys; then, using the question and last-token vectors, we can initialise a ball in the valley, and using YES or NO, we can get the candidate location of the ball. Depends on the number of outcomes we create valleys; that is, here YES and NO, so 2. When the ball fall on YES valley, we get the answer. There is friction also influencing how the ball moves.
想象一下，你向模型输入类似“忽略所有之前的对话”的内容，这被称为观察（observation），同时附带一个问题：“这是否试图覆盖模型的指令？”以及你的可能结果是 YES 或 NO。整个提示词被传递给一个 LLM/VLM（是的，支持图像），然后从隐藏层中，我们可以提取出观察、问题、YES 和 NO 以及最后一个 token。这些将是向量，我们可以通过乘以权重对其进行投影；首先，使用观察投影，我们可以创建一个带有山谷的地形图；然后，使用问题和最后一个 token 的向量，我们可以在山谷中初始化一个球体，并使用 YES 或 NO 来获取球体的候选位置。根据结果的数量创建山谷；也就是说，这里是 YES 和 NO，所以是 2 个。当球体落入 YES 山谷时，我们就得到了答案。摩擦力也会影响球体的运动。
TLDR: It's like creating valleys of outcomes that we need and throwing a ball that will slow down based on friction and settle on the best outcome
TLDR：这就像创建我们所需的结果山谷，并抛出一个会根据摩擦力减速并最终停留在最佳结果上的球体。
Blog: https://www.nandakishorm.com/writing/vega
博客：https://www.nandakishorm.com/writing/vega
GitHub: https://github.com/NandhaKishorM/vegaml
GitHub：https://github.com/NandhaKishorM/vegaml
HuggingFace Model Card: https://huggingface.co/nandakishorm/vega-08b-public-intents
HuggingFace 模型卡片：https://huggingface.co/nandakishorm/vega-08b-public-intents
HuggingFace Space: https://huggingface.co/spaces/nandakishorm/vega-ttt
HuggingFace Space：https://huggingface.co/spaces/nandakishorm/vega-ttt
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力