Nums AI发布表格基础模型Causilo，TabArena单模型第一
Nums AI Releases Causilo: A Tabular Foundation Model That Tops TabArena Among Single Models
表格领域首个登顶主流基准的开源基础模型，架构创新且性能强劲，值得跟进研究与应用。
Nums AI has released Causilo, a pretrained tabular foundation model for classification and regression. Causilo comes with a scikit-learn interface, Apache-2.0 code and pretrained weights on Hugging Face. On TabArena, it has the highest Elo among single models for both classification and regression.
Nums AI 发布了 Causilo，这是一个用于分类和回归的预训练表格基础模型。Causilo 提供 scikit-learn 接口、Apache-2.0 许可证的代码以及在 Hugging Face 上的预训练权重。在 TabArena 上，它在单模型分类和回归中均拥有最高的 Elo 评分。
Is it deployable? Yes, for research and evaluation today, on CUDA or CPU. Commercial, production and hosted API use need a separate license from Nums AI.
可以部署吗？是的，目前可用于研究和评估，支持 CUDA 或 CPU。商业、生产环境和托管 API 使用需要向 Nums AI 获取单独的许可。
What Causilo Does
Causilo 的功能
Causilo is an in-context learning model. Calling fit does not update the pretrained weights. It stores the training rows as context and predicts query rows in one forward pass. According to its TabArena submission, Nums AI pretrained Causilo only on synthetic data, with no TabArena datasets.
Causilo 是一个上下文学习（in-context learning）模型。调用 fit 不会更新预训练权重。它将训练行存储为上下文，并在一次前向传播中预测查询行。根据其在 TabArena 的提交信息，Nums AI 仅使用合成数据对 Causilo 进行预训练，未使用任何 TabArena 数据集。
Inputs can be NumPy arrays or pandas DataFrames, including categorical features and missing values. Classification supports up to 10 classes. Regression returns mean predictions by default. Version 1.0.1 adds median and quantile outputs, based on 999 native quantiles.
输入可以是 NumPy 数组或 pandas DataFrame，包括分类特征和缺失值。分类任务最多支持 10 个类别。回归任务默认返回均值预测。版本 1.0.1 增加了中位数和分位数输出，基于 999 个原生分位数。
Architecture: Refinement, Compression, In-Context Learning
架构：细化、压缩、上下文学习
Nums AI splits the network into 3 phases: refinement, compression and in-context learning. The released code and configs show how each phase works.
Nums AI 将网络分为三个阶段：细化、压缩和上下文学习。发布的代码和配置文件展示了每个阶段的工作原理。
Features are grouped in sets of 3. Each value is embedded with 16 learned sine and cosine frequencies. Missing values get their own learned vector.
特征被分为每组 3 个。每个值都通过 16 个学习到的正弦和余弦频率进行嵌入。缺失值拥有其专属的学习向量。
2 column stages summarize each feature group. In each, 128 latent slots read only the training rows and pass that summary to every row. Between the 2 column stages, a row stage lets feature groups interact through 4 latent tokens. It uses cross-attention instead of full self-attention, which Nums AI says keeps cost linear in feature count.
两个列阶段（column stages）对每个特征组进行总结。在每个阶段中，128 个潜在槽位（latent slots）仅读取训练行，并将该摘要传递给每一行。在这两个列阶段之间，一个行阶段（row stage）允许特征组通过 4 个潜在标记（latent tokens）进行交互。它使用交叉注意力机制而非完整的自注意力机制，Nums AI 表示这能将成本保持在与特征数量成线性关系的水平。
A pooling block then compresses each row into a fixed 512-dimensional vector. Labels are added to the training rows. A 12-layer prediction block lets query rows attend to those labeled rows. Query rows cannot change the training context or each other.
随后，一个池化块（pooling block）将每行压缩为固定的 512 维向量。标签被添加到训练行中。一个 12 层的预测块让查询行能够关注这些带标签的行。查询行不能改变训练上下文或彼此之间的状态。
By default, 8 ensemble members share the same weights. Each one cycles through none, rank2gaussian, robust or power normalization, with seeded feature and class permutations.
默认情况下，8 个集成成员共享相同的权重。每个成员依次循环执行无归一化、rank2gaussian、robust 或 power 归一化，并采用种子化的特征和类别排列。
TabArena Results
TabArena 结果
Nums AI used the official TabArena pipeline: 51 datasets and 816 Full splits, with 8 estimators and seed 42. A TabArena maintainer re-ran the full evaluation and got the same overall Elo of 1794.
Nums AI 使用了官方的 TabArena 流程：51 个数据集和 816 个 Full 划分，包含 8 个估计器，随机种子为 42。一位 TabArena 维护者重新运行了完整评估，得到了相同的总体 Elo 分数 1794。
| Task | Causilo Elo | Next best single model | Causilo improvability |
|---|---|---|---|
| Overall | 1792.9 | TabFM, 1764.4 | 0.0684 |
| Classification | 1771.8 | EXAONE Tabular, 1758.8 | 0.0875 |
| Regression | 2032.6 | TabFM, 1992.8 | 0.0125 |
| 任务 | Causilo Elo | 最佳单一模型 | Causilo 可改进性 |
|---|---|---|---|
| 总体 | 1792.9 | TabFM, 1764.4 | 0.0684 |
| 分类 | 1771.8 | EXAONE Tabular, 1758.8 | 0.0875 |
| 回归 | 2032.6 | TabFM, 1992.8 | 0.0125 |
The field includes Google Research’s TabFM, LG AI Research’s EXAONE Tabular and Prior Labs’ TabPFN-3 (1636.2 overall).
该领域包括 Google Research 的 TabFM、LG AI Research 的 EXAONE Tabular 和 Prior Labs 的 TabPFN-3（总体得分 1636.2）。
A few points help when reading these numbers:
阅读这些数字时，以下几点有所帮助：
- The #1 positions exclude system entries. With systems included, the maintainer re-run placed Causilo 3rd of 88 overall.
- On improvability, TabFM still leads overall and on classification. Causilo leads on regression.
- The Elo confidence intervals at the top overlap, so the lead over TabFM and EXAONE Tabular is narrow.
- Nums AI also lists Xiaomi-TabLDM and Amazon’s Mitra-v2 behind Causilo. Neither model appears in the benchmark files in Causilo’s repo.
- 第 1 名位置不包括系统条目。包含系统后，维护者重新运行将 Causilo 排在 88 个结果中的第 3 位。
- 在可改进性方面，TabFM 在总体和分类上仍领先。Causilo 在回归上领先。
- 顶部的 Elo 置信区间存在重叠，因此对 TabFM 和 EXAONE Tabular 的领先优势很小。
- Nums AI 还列出了位于 Causilo 之后的 Xiaomi-TabLDM 和 Amazon 的 Mitra-v2。这两个模型均未出现在 Causilo 仓库的基准测试文件中。
ScoringBench Results
ScoringBench 结果
ScoringBench scores regression models with proper scoring rules such as CRPS, alongside RMSE and R². Nums AI submitted Causilo 1.0.1 on 101 datasets, 5 folds each, capped at 3,000 samples. Nums AI reports that Causilo ranks 1st by CRPS, R² and RMSE. The ScoringBench maintainer independently checked the results before committing them.
ScoringBench 使用 CRPS 等proper scoring rules以及 RMSE 和 R² 对回归模型进行评分。Nums AI 在 101 个数据集上提交了 Causilo 1.0.1，每个数据集 5 折，样本上限为 3,000。Nums AI 报告称，Causilo 在 CRPS、R² 和 RMSE 上均排名第 1。ScoringBench 维护者在提交结果前独立验证了这些结果。
Speed and Memory
速度与内存
Nums AI also reran 3 models on 1 H100 80 GB GPU, with 8 CPU cores per job.
Nums AI 还在 1 张 H100 80 GB GPU 上重新运行了 3 个模型，每个作业使用 8 个 CPU 核心。
| Model | Fit (s per 1k rows) | Predict (s per 1k rows) | GPU memory (GiB) |
|---|---|---|---|
| Causilo | 2.504 | 0.251 | 8.15 |
| TabICLv2 | 3.449 | 0.303 | 8.37 |
| TabPFN-3 | 4.18 | 0.686 | 0.88 |
| 模型 | 拟合 (每千行秒数) | 预测 (每千行秒数) | GPU 内存 (GiB) |
|---|---|---|---|
| Causilo | 2.504 | 0.251 | 8.15 |
| TabICLv2 | 3.449 | 0.303 | 8.37 |
| TabPFN-3 | 4.18 | 0.686 | 0.88 |
In this test, Causilo is fastest on both fit and predict. TabPFN-3 uses far less GPU memory. Setting use_kv_cache=True moves context work into fit, using more memory to speed up repeated predictions.
在此测试中，Causilo 在拟合和预测速度上均最快。TabPFN-3 使用的 GPU 内存少得多。设置 use_kv_cache=True 会将上下文工作移至拟合阶段，通过增加内存使用来加速重复预测。
Getting Started
入门指南
Causilo needs Python 3.10 to 3.12 and PyTorch 2.13 or newer. The first fit downloads the checkpoint automatically.
Causilo 需要 Python 3.10 至 3.12 以及 PyTorch 2.13 或更高版本。首次拟合会自动下载检查点。
# pip install causilo
from causilo import CausiloClassifier, CausiloRegressor
clf = CausiloClassifier(n_estimators=8, random_state=42)
clf.fit(X_train, y_train)
proba = clf.predict_proba(X_test)
reg = CausiloRegressor()
reg.fit(X_train, y_train)
bands = reg.predict(X_test, output_type="quantiles", quantiles=[0.05, 0.5, 0.95])# pip install causilo
from causilo import CausiloClassifier, CausiloRegressor
clf = CausiloClassifier(n_estimators=8, random_state=42)
clf.fit(X_train, y_train)
proba = clf.predict_proba(X_test)
reg = CausiloRegressor()
reg.fit(X_train, y_train)
bands = reg.predict(X_test, output_type="quantiles", quantiles=[0.05, 0.5, 0.95])You can also try the Hugging Face demo Space.
你也可以尝试 Hugging Face 演示空间。
Key Takeaways
关键要点
- Causilo has the top TabArena Elo among single models, overall and per task.
- With system ensembles included, a maintainer re-run places it 3rd of 88.
- Row mixing goes through 4 latent tokens, keeping cost linear in feature count.
- The code is Apache-2.0; the weights are research-only without a commercial license.
- Version 1.0.1 adds quantile outputs, so regression intervals work out of the box.
- Causilo 在单一模型中拥有最高的 TabArena Elo 分数，无论是总体还是按任务划分。
- 包含系统集成后，维护者重新运行将其排在 88 个结果中的第 3 位。
- 行混合经过 4 个潜在令牌，使成本与特征数量呈线性关系。
- 代码采用 Apache-2.0 许可证；权重仅限研究用途，无商业许可。
- 版本 1.0.1 增加了分位数输出，因此回归区间开箱即用。
Check out the Repo on GitHub and Model on HF. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
查看 GitHub 上的仓库和 HF 上的模型。所有功劳归于该项目研究人员。此外，欢迎在 Twitter 上关注我们，并别忘了加入我们有 150k+ 成员的 ML SubReddit 以及订阅我们的新闻通讯。等等！你在 Telegram 上吗？现在你也可以加入我们的 Telegram 群组。
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
需要与我们合作推广您的 GitHub 仓库、Hugging Face 页面、产品发布或网络研讨会等？请与我们联系
The post Nums AI Releases Causilo: A Tabular Foundation Model That Tops TabArena Among Single Models appeared first on MarkTechPost.
Nums AI 发布 Causilo：在单模型中登顶 TabArena 的表格基础模型，此文首发于 MarkTechPost。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力