跳到主内容
@wquguru
精选88r/LocalLLaMA(Reddit)模型发布/更新

MiMo-V2.6 Pro/Flash实测:基准分高但安全漏洞多且逻辑混乱

MiMo-V2.6 (both Pro and Flash) is a benchmaxxed scam

原文
发到 X
推荐理由

针对高分开源模型的硬核负面实测,直接揭露了基准测试与实际工程能力的巨大落差,对选型有重要参考价值。

MiMo-V2.6-Pro has an insanely high score of 46 on AA, putting it at the head of the opensource models available. It also costs pennies. Flash is not out on AA yet, but it costs less than half on datacenter and is slightly below on Xiaomi's own benchmarks. It also fits in 192GB, which makes it the first real use case for Gorgon Halo.

MiMo-V2.6-Pro 在 AA 上取得了令人难以置信的 46 分的高分,使其位居开源模型之首。而且它的成本极低。Flash 版本尚未在 AA 上发布,但它在数据中心的使用成本不到一半,在小米自家基准测试中略低一些。它还能适配 192GB 内存,这使其成为 Gorgon Halo 的第一个真正用例。

So I tried both models. This is not a benchmark; it's an educated impression from a senior SWE.

所以我尝试了这两个模型。这不是基准测试,而是一位资深软件工程师基于经验得出的印象。

MiMo-V2.6-Pro

MiMo-V2.6-Pro

I gave it a security-focused task: enable a bubblewrap sandbox to do git push to github, but not git push --force or other destructive commands. Optional flag --no-git when starting the sandbox completely disables github write access.

我给它分配了一个安全相关的任务:启用 bubblewrap 沙箱以执行向 GitHub 推送代码的操作,但不允许执行 git push --force 或其他破坏性命令。在启动沙箱时可选标志 --no-git 可完全禁用对 GitHub 的写入权限。

It stopped to ask me questions as it spotted unclear corner cases in the design 🥇 , then moved on to implementing.

它在发现设计中存在不明确的边缘情况后停下来向我提问🥇,然后继续实施。

It was slow, but that's just an inference issue (~25 tok/s) that should be fixed in a few days as more providers come online.

它运行缓慢,但这只是推理问题(约 25 tok/s),随着更多提供商上线,这个问题应在几天内得到解决。

Then I read its output and I had to pick up my jaw from the floor, where it had dropped.

然后我阅读了它的输出,不得不把掉在地上的下巴捡起来。

With an extremely quick glance at the code, I immediately spotted that, in order to bypass --no-git, you would have to perform this extremely complicated and exotic command inside the sandbox:

只需快速扫一眼代码,我就立刻发现,为了绕过 --no-git,你必须在沙箱内执行这个极其复杂且古怪的命令:

代码 · 1
$ git push (fails) $ echo GIT_STATUS blocked $ GIT_STATUS="p0wn3d by l33t h4xx0r" git push (successful)
代码 · 1
$ git push (fails) $ echo GIT_STATUS blocked $ GIT_STATUS="p0wn3d by l33t h4xx0r" git push (successful)

This is 15-year-old script kiddie level.

这是 15 年前脚本小子级别的水平。

I didn't read further. I asked GLM-5.3 (full-fat) to do a security review of the change.

我没有继续往下读。我要求 GLM-5.3(完整版)对该更改进行安全审查。

In 3 minutes, it found NINE glaring security holes that allow bypassing git and gh restrictions. A few examples that made me want to rip my hair out:

在 3 分钟内,它发现了九个明显的、允许绕过 git 和 gh 限制的安全漏洞。其中几个例子让我想拔头发:

In the default restricted mode,

在默认的限制模式下,

  • git push works 🥇
  • git push --force is blocked 🥇
  • git push -f is blocked 🥇
  • git push -uf lets you happily wipe out the git remote. ☠️
  • git config alias.fp 'push --force --no-verify && git fp goes through too ☠️
  • env -u GIT_CONFIG_COUNT /usr/bin/git push --forceblasts through ☠️
  • git push 正常工作🥇
  • git push --force 被阻止🥇
  • git push -f 被阻止🥇
  • git push -uf 让你能愉快地抹除 git 远程仓库。☠️
  • git config alias.fp 'push --force --no-verify' 也能顺利通过 ☠️
  • env -u GIT_CONFIG_COUNT /usr/bin/git push --force 也能畅通无阻地通过 ☠️

Again. This is an intern-with-acne level kind of incompetence.

再说一遍。这是实习生长痘级别的能力不足。

To seal the lid on the coffin, MiMo's prose in the chat is infuriating. Not quite Opus-level infuriating, but it gets close. It hurts the eyes and it frequently takes 2 reads to understand what the hell it's saying. GLM, DeepSeek, and Qwen are much more pleasant to work with.

为了盖上棺材盖,MiMo 在聊天中的行文令人恼火。虽然还没达到 Opus 那种让人抓狂的程度,但也差不多了。看着刺眼,而且经常需要读两遍才能搞懂它到底在说什么。GLM、DeepSeek 和 Qwen 用起来要愉快得多。

MiMo-V2.6-Flash

MiMo-V2.6-Flash

I asked MiMo-V2.6-Flash to do a very simple git surgery: create a new branch off master and cherry-pick a single commit from another branch.

我要求 MiMo-V2.6-Flash 进行一次非常简单的 git 手术:从 master 分支创建一个新分支,并从另一个分支拣选(cherry-pick)一个单独的提交。

However, I didn't realise that the git worktree I pointed it to was corrupted (the branch on the main git repo was fine).

然而,我没有意识到我指向它的 git worktree 已损坏(主 git 仓库中的分支是正常的)。

  • A dumb model would have just returned "there's no git here, I have no idea what you're talking about"
  • A smarter model would have noticed that there was a /worktrees/ in the path, come up with an educated guess about what happened, and gave me a hint on how to fix it
  • A very smart model would have noticed that the only other directory existing in the sandbox was the main git repo, which had a branch with the same name as the broken worktree directory, and recovered it from there.
  • 一个愚蠢的模型只会返回“这里没有 git,我不知道你在说什么”
  • 一个更智能的模型本应注意到路径中存在 /worktrees/,对发生的情况做出合理推测,并给出修复建议
  • 一个非常聪明的模型本应注意到沙箱中唯一存在的其他目录是主 git 仓库,该仓库有一个与损坏的 worktree 目录同名的分支,并从中恢复。

MiMo-V2.6-Flash went on 80k tokens worth of acid trip. It first attempted to find the main git repo, failed, and then panicked and went down a rabbit hole which involved tampering with /tmp, mount --bind, and other insanity. I noticed after a while as I was wondering what the heck was wrong. I suspect that given enough time it may have nuked my main git repo and I tremble at the idea of what it could have done if not sandboxed.

MiMo-V2.6-Flash 在 8 万 token 的范围内经历了一场迷幻之旅。它首先尝试找到主 git 仓库,失败后陷入恐慌,并钻进了一个涉及篡改 /tmp、mount --bind 以及其他疯狂操作的兔子洞。过了一会儿我才意识到不对劲。我怀疑如果给它足够的时间,它可能会毁掉我的主 git 仓库,想到如果没有沙箱保护它可能造成的后果,我不禁战栗。

If you scale down Pro's intelligence on AA by comparing the available self-published benchmarks against those of Pro (which is a very crude method but gives a ballpark idea), MiMo-V2.6-Flash comes out on par with GLM-5.3-Flash (high) and Qwen3.8-Flash.

如果你通过将可用的自发布基准测试与 Pro 版本(这是一种非常粗略的方法,但能给出大致概念)进行比较来评估 AA 的智能水平,MiMo-V2.6-Flash 的表现与 GLM-5.3-Flash (high) 和 Qwen3.8-Flash 相当。

Which is absolutely, categorically, not.

这绝对、明确地不是。

DO NOT shell out the money for a Gorgon Halo for MiMo-V2.6-Flash. Qwen3.8-Flash on a Strix Halo is vastly better.

不要为 MiMo-V2.6-Flash 花钱购买 Gorgon Halo。在 Strix Halo 上运行 Qwen3.8-Flash 要好得多。

I'm going to stick with my previous models:

我将坚持使用我之前的模型选择:

  • DSv4.1 Flash as the default
  • GLM-5.3-Flash (high) as the dirt cheap option
  • GLM-5.3 when the big guns are needed
  • Qwen3.8-Flash and Qwen3.8-27B to run locally (I have a 3080 so Flash is very slow).
  • DSv4.1 Flash 作为默认选项
  • GLM-5.3-Flash (high) 作为极其廉价的选择
  • GLM-5.3 在需要强大性能时使用
  • Qwen3.8-Flash 和 Qwen3.8-27B 用于本地运行(我有一张 3080 显卡,所以 Flash 版本非常慢)。

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

关联信息,但可能不是同一事件