MiniCPM-V 4.7:35B多模态模型手机端运行演示
A 35B multimodal model running on a phone with ~3GB memory? MiniCPM-V 4.7 demo is pretty wild
端侧多模态落地的重要进展,LES优化方案对移动端部署有参考价值,建议关注其实际性能表现。
Just saw this demo from OpenBMB for MiniCPM-V 4.7, and the on-device capability is pretty impressive.
刚刚看到 OpenBMB 发布的 MiniCPM-V 4.7 演示,其端侧能力相当令人印象深刻。
A 35B-A3B MoE vision-language model running locally on mobile devices is already interesting, but the use cases are what caught my attention:
一个能在移动设备上本地运行的 35B-A3B MoE 视觉语言模型已经很有趣了,但真正吸引我的是它的应用场景:
- 📷 Reading real-world signs
- 📄 Understanding complex documents
- 🧠 Reasoning over visual information
- 🗺️ Planning with real-world constraints
- 📷 阅读现实世界中的标识
- 📄 理解复杂文档
- 🧠 对视觉信息进行推理
- 🗺️ 在现实约束下进行规划
The key piece seems to be LES (Lookahead Expert Scheduling) — instead of just making the model smaller, they optimize MoE inference by predicting and preloading experts to reduce bottlenecks.
关键的技术点似乎是 LES(Lookahead Expert Scheduling,前瞻专家调度)——他们不仅仅是缩小模型规模,而是通过预测和预加载专家来优化 MoE 推理,从而减少瓶颈。
They also released a smaller MiniCPM-V 4.7 1B model for more resource-constrained devices.
他们还发布了一个更小的 MiniCPM-V 4.7 1B 模型,适用于资源受限的设备。
Another interesting addition is Self-Replay Fine-Tuning (SRFT), which aims to let developers customize the model for specific tasks while keeping general capabilities.
另一个有趣的补充是自回放微调(Self-Replay Fine-Tuning, SRFT),旨在让开发者能够针对特定任务定制模型,同时保持通用能力。
The direction is interesting: instead of sending every image/question to the cloud, we're getting closer to having capable multimodal assistants running directly on phones.
这一方向很有意思:我们不再将每张图像/问题都发送到云端,而是正逐渐接近在手机端直接运行强大的多模态助手。
Links:
相关链接:
GitHub: https://github.com/OpenBMB/MiniCPM-V
Hugging Face: https://huggingface.co/collections/openbmb
SRFT details: https://openbmb.cn/en/blog/self-replay-fine-tuning
SRFT 详情: https://openbmb.cn/en/blog/self-replay-fine-tuning
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力