蚂蚁集团开源 LingBot-Vision：1B 参数边界感知视觉基础模型

Robbyant, the embodied-AI company within Ant Group, has open-sourced LingBot-Vision, a family of self-supervised Vision Transformers built for dense spatial perception. The weights ship under Apache-2.0 on Hugging Face in four sizes — ViT-giant, ViT-large, ViT-base, and ViT-small — together with a technical report and inference code. Most vision foundation models are trained for semantic invariance: they learn to answer what is in an image while discarding exactly the fine-grained spatial structure — object boundaries, contours, depth discontinuities — that robots and other physically embodied systems depend on. LingBot-Vision inverts that priority. It treats boundaries as a native pretraining signal rather than a downstream output, and the payoff is a 1B-parameter backbone that matches or surpasses models up to 7× larger on dense spatial tasks, including the 7B DINOv3. What is LingBot-Vision? LingBot-Vision is a self-supervised pretrained encoder for spatially structured downstream tasks. The flagship ViT-g/16 has roughly 1.1B parameters and is trained with a new objective called masked boundary modeling on a curated corpus of about 161M images — selected from a 2B web pool — with no human labels, no external edge detectors, and no pretrained backbone to bootstrap from. The training is also notably economical: the corpus is an order of magnitude smaller than DINOv3’s LVD-1689M, and the model consumes less than a third of DINOv3’s training samples. The encoder outputs dense patch-token features intended for frozen readouts. For deployment at smaller budgets, the flagship is distilled into ViT-L (300M), ViT-B (86M), and ViT-S students that lead dense prediction within their size classes. How Masked Boundary Modeling Works The method builds on the DINO/iBOT self-distillation paradigm: a teacher — an EMA copy of the student — generates online targets, and the student recovers them from masked views. Standard masked image modeling hides patches at random, ignoring what each patch depicts. A flat interior patch is cheap to recover from its neighbors; a patch straddling an object boundary carries structure that context alone cannot supply. Boundaries are the least redundant, most informative regions of an image — and random masking treats them like everything else. LingBot-Vision closes that gap with two ideas. Boundary-forcing. The teacher predicts a dense boundary field online and identifies the boundary-bearing tokens B. These are forced into the student’s masked set on top of the random mask M, giving the combined mask M⁺ = M ∪ B. Masked tokens are then routed by geometry: boundary tokens receive an explicit geometric target in addition to the semantic self-distillation target, while interior masked tokens keep the standard semantic objective alone. This routing matters because a semantic target is inherently ambiguous exactly where two regions meet — the geometric target is well-posed precisely where conventional masked modeling is weakest, which is what lets semantic and geometric representations co-emerge rather than compete. Categorical boundary field. Boundaries are modeled as line segments lifted into a dense field: every nearby pixel stores an attribute vector a(p) = (d, θ, φ¹, φ²) recording its distance to the nearest segment and three angles that locate it. Directly regressing this field in a teacher–student loop collapses. The fix is to discretize each channel into K = 32 bins, recasting boundary prediction as per-pixel classification — which lets the boundary branch inherit the same centering and sharpening machinery that stabilizes modern self-distillation. The categorical form has an elegant side effect. Under the classical a-contrario null hypothesis of “no structure,” boundary orientations are uniformly distributed — and that null is now literally the uniform distribution over bins. Deviation from uniformity is evidence of a real boundary, so a parameter-free Number-of-False-Alarms (NFA) test validates every decoded segment at no extra cost. The teacher exploits this at each iteration: it decodes candidate segments from its own field prediction, keeps only the NFA-validated survivors, and re-renders them into the target field — so unsupported structure never becomes a teaching signal. The full objective sums four terms: L = L_DINO + λᵢ · L_iBOT + λᵦ · L_bnd + λₖ · L_KoLeo Benchmarks and Performance All dense results below use frozen features with a single linear layer, so performance is attributable to the representation rather than a decoder. ModelParamsNYUv2 RMSE ↓KITTI RMSE ↓ADE20K mIoUCityscapes mIoUVOC mIoULingBot-Vision ViT-g1B/160.2962.55253.579.687.5DINOv37B/160.3092.34655.981.186.6V-JEPA 2.1 ViT-G2B/160.3072.46147.973.585.0AM-RADIOv2.51B/140.3402.91853.078.485.4DINOv21B/140.3722.62449.575.683.1SigLIP 21B/160.4943.27342.764.872.7 On NYU-Depth v2, LingBot-Vision posts the best RMSE of the entire comparison (0.296), ahead of the 7B DINOv3 (0.309) with roughly 7× fewer parameters, and ahead of the 2B V-JEPA 2.1 (0.307). On KITTI it is the best model below 2B parameters. On semantic segmentation it is on par with the distilled DINOv3 ViT-H+ — 1.3 mIoU behind on ADE20K, matching on Cityscapes, ahead on VOC12 — while improving over the same-size DINOv2 by 4+ mIoU on all three benchmarks; the only remaining gap is to the DINOv3 family itself (2.4 mIoU on ADE20K to the 7B model), whose dense strength comes from distillation and dedicated dense-feature objectives. Video object segmentation uses training-free label propagation over frozen features. LingBot-Vision reaches 70.0 J&F on DAVIS-2017 and 73.5 on YouTube-VOS — on par with DINOv3 ViT-H+ (71.1 / 74.0) and the 7B DINOv3 (71.1 / 74.1), and the best among all remaining models at any scale. The boundary tokens themselves are stable enough to be tracked through video by plain cosine similarity of frozen features, with no temporal supervision. The trade-off is image-level recognition: ImageNet-1K linear probing reaches 86.32 and k-NN 83.39, trailing DINOv3-7B, which spends its capacity on image-level invariance. The advantages also survive distillation — the 0.3B ViT-L student matches the 7B DINOv3 on NYUv2 depth (0.310 vs. 0.309) with about 23× fewer parameters. Use Cases and How to Load It The frozen patch tokens serve several dense workloads directly: depth estimation reads geometry straight from the features, semantic segmentation benefits from feature transitions that land exactly on object contours, and video object segmentation works through cosine-similarity token matching. The encoder also serves as the initialization for downstream depth-completion training. Loading a backbone follows the official repository: Copy CodeCopiedUse a different Browsergit clone https://github.com/robbyant/lingbot-vision.git cd lingbot-vision conda create -n lingbot-vision python=3.10 -y conda activate lingbot-vision python -m pip install -r requirements.txt python -m pip install -e . Copy CodeCopiedUse a different Browserimport torch from lingbot_vision import load_pretrained_backbone, extract_patch_tokens, load_image device = "cuda" if torch.cuda.is_available() else "cpu" dtype = torch.bfloat16 if device == "cuda" else torch.float32 # Downloads model.pt from Hugging Face on first use. backbone, embed_dim = load_pretrained_backbone( variant="small", # giant | large | base | small; defaults to large device=device, dtype=dtype, ) img_norm, _, _ = load_image( "examples/example.png", size=512, patch_size=backbone.patch_size, mode="square", ) patch_tokens, patch_grid = extract_patch_tokens(backbone, img_norm, device, dtype) print(patch_tokens.shape, patch_grid, embed_dim) # torch.Size([1, 1024, 384]) (32, 32) 384 The variant argument selects the size and defaults to large. Output patch_tokens has shape [B, H*W, C]. Requirements are Python ≥ 3.10 and PyTorch ≥ 2.0, with a GPU recommended for the larger backbones. LingBot-Depth 2.0: The Downstream Payoff To show what a spatial-perception-native encoder bu