DuckDB v2.0 预览:服务器模式、VARIANT 与触发器
A Preview of DuckDB v2.0
A Preview of DuckDB v2.0
DuckDB v2.0 预览
Mark Raasveldt and Hannes Mühleisen
Mark Raasveldt 和 Hannes Mühleisen
2026-08-17 | 15 min
2026-08-17 | 15 分钟
TL;DR: DuckDB v2.0 is coming this fall. In this post, we preview its headline features: DuckDB as a server, triggers, the VARIANT type, asynchronous I/O, a new SQL parser, a new storage format, and much more.
TL;DR:DuckDB v2.0 将于今年秋季发布。在这篇文章中,我们预览其主打功能:DuckDB 作为服务器、触发器、VARIANT 类型、异步 I/O、新的 SQL 解析器、新的存储格式,以及更多内容。
DuckDB v2.0 will be named “Cyanoptera” after the cinnamon teal (Anas cyanoptera), a strikingly reddish-brown duck found in the western Americas.
DuckDB v2.0 将被命名为“Cyanoptera”,取自肉桂色鸭(Anas cyanoptera),一种在美洲西部发现的、羽毛呈醒目红褐色的鸭子。
A major version bump is not something we do lightly, and it is not just ceremony: v2.0 ships a new SQL parser, a new default storage format, a reworked C API, and a small number of carefully chosen breaking changes. But above all, it is a feature release, built from over 10,000 commits since we released v1.5 in March. Where last year was the year of the lakehouse, this release kicks off the year of DuckDB as a server. We previewed many of these features in the “State of the Duck” talk at DuckCon #7, if you prefer to watch instead of read.
主版本号的提升并非轻率之举,也不仅仅是形式上的:v2.0 带来了新的 SQL 解析器、新的默认存储格式、重新设计的 C API,以及少量精心挑选的破坏性变更。但最重要的是,它是一个功能版本,基于自 3 月发布 v1.5 以来的 10,000 多次提交构建而成。如果说去年是湖仓之年,那么这次发布则开启了 DuckDB 作为服务器的一年。我们在 DuckCon #7 的“State of the Duck”演讲中预览了其中许多功能,如果你更喜欢观看而非阅读的话。
DuckDB is moving rather quickly, and we can only cover a small fraction of the changes here. Condensing all new features down to a shortlist is always a fight over what gets in, and yes, we know that what follows is technically a listicle (Ten Things Coming to DuckDB v2.0, Number Eight Will Shock You). We are not proud of the format, but it works, so here it is, starting with the SQL-level features and working down into the engine.
DuckDB 发展迅速,我们在这里只能涵盖一小部分变更。将所有新功能浓缩成一份清单总是一场关于取舍的较量,是的,我们知道下面这份内容在技术上是一篇列表文章(DuckDB v2.0 即将推出的十件事,第八件会让你震惊)。我们并不为这种格式感到自豪,但它确实有效,所以就在这里呈现,从 SQL 层面的功能开始,逐步深入到引擎内部。
1. DuckDB as a Server: Quack and CONNECT
1. DuckDB 作为服务器:Quack 和 CONNECT
DuckDB has been an in-process database since day one. But people have asked us – very persistently – for a client/server mode, and we have finally caved. The quack extension implements DuckDB's native protocol for talking to other DuckDBs. It was released as a preview shortly before DuckCon #7, graduates to stable in v2.0, and it is a big part of where DuckDB is headed: any DuckDB process can serve its databases over the network, and any other DuckDB can attach to it and route queries there using the new CONNECT statement. For example:
DuckDB 从一开始就是进程内数据库。但人们一直——非常执着地——要求客户端/服务器模式,我们最终妥协了。quack 扩展实现了 DuckDB 的原生协议,用于与其他 DuckDB 实例通信。它在 DuckCon #7 之前作为预览版发布,在 v2.0 中升级为稳定版,并且是 DuckDB 未来发展方向的重要组成部分:任何 DuckDB 进程都可以通过网络提供其数据库服务,任何其他 DuckDB 都可以使用新的 CONNECT 语句附加到它并路由查询。例如:
DuckDB server
DuckDB 服务器
CALL quack_serve(
token = 'my_token'
);CALL quack_serve(
token = 'my_token'
);quack:
quack:
DuckDB client
DuckDB 客户端
ATTACH 'quack:server.example.com'
AS qk (TOKEN 'my_token');
CONNECT qk;
SELECT count(*) FROM events;
-- executes on the server,
-- results stream back
DISCONNECT;ATTACH 'quack:server.example.com'
AS qk (TOKEN 'my_token');
CONNECT qk;
SELECT count(*) FROM events;
-- executes on the server,
-- results stream back
DISCONNECT;CONNECT is the successor to the remote.query($$...$$) workaround we showed when Quack was first revealed – we looked at that syntax and said: no, this cannot be it. And CONNECT is not limited to Quack: it points your session at any remote database that supports it, and the new remote pushdown optimizer (#22914) ships SQL directly to PostgreSQL and MySQL instead of pulling tables over the wire:
CONNECT 是 Quack 首次亮相时我们展示的 remote.query($$...$$) 变通方法的继任者——我们看了那个语法后说:不,这不能是最终方案。而且 CONNECT 不仅限于 Quack:它可以将你的会话指向任何支持它的远程数据库,新的远程下推优化器(#22914)直接将 SQL 发送到 PostgreSQL 和 MySQL,而不是通过网络拉取表:
CONNECT 'postgres://localhost/mydb';
SELECT count(*) FROM orders; -- runs on the PostgreSQL server
DISCONNECT;CONNECT 'postgres://localhost/mydb';
SELECT count(*) FROM orders; -- runs on the PostgreSQL server
DISCONNECT;If you have worked with analytical systems in the past, you may assume that DuckDB cannot handle transactional workloads. But DuckDB has been built as a transactional, multi-connection database with full MVCC and transaction isolation since day one. Most users just never needed that in a single-user scenario. It turns out DuckDB handles transactions well: it's fast enough to compete with general-purpose databases like PostgreSQL on quite a few workloads, and the client/server pattern finally lets that machinery shine in multi-tenant, long-running deployments.
如果你过去使用过分析系统,你可能会认为DuckDB无法处理事务性工作负载。但DuckDB从第一天起就被构建为一个事务性的、多连接的数据库,具备完整的MVCC和事务隔离。大多数用户只是在单用户场景中从未需要过这些功能。事实证明,DuckDB处理事务表现良好:在不少工作负载上,它的速度足以与PostgreSQL等通用数据库竞争,而客户端/服务器模式最终让这些机制在多租户、长期运行的部署中大放异彩。
Running DuckDB long-term also comes with new challenges, which is why v2.0 pushes on better metrics, logs, and observability (see, e.g., the metrics layer rework in #22799) that let you look at a DuckDB instance and see what it is actually doing. People even built standalone clients for the Quack protocol within weeks of the preview. We thought we were extending DuckDB to talk to other DuckDBs; the world said no, no, no, and built their own clients. Who would have thought.
长期运行DuckDB也带来了新的挑战,这就是为什么v2.0在更好的指标、日志和可观测性方面进行了推进(例如,参见#22799中的指标层重构),让你能够查看一个DuckDB实例并了解它实际在做什么。人们甚至在预览版发布后的几周内就为Quack协议构建了独立的客户端。我们以为我们是在扩展DuckDB以与其他DuckDB通信;世界却说,不,不,不,并构建了他们自己的客户端。谁能想到呢。
2. VARIANT Becomes a First-Class Citizen
2. VARIANT成为一等公民
The VARIANT type shipped in DuckDB v1.5, and the way to think about it is JSON on steroids. Basically, imagine if JSON were fast. Like JSON, a VARIANT column can store differently-shaped data in every row. Unlike JSON, it is not a text format: DuckDB automatically detects the common structure hidden in your semi-structured data and “shreds” it, so it compresses well in storage and executes fast in queries, all without you ever declaring a schema. This makes VARIANT a natural fit for real-time log ingestion, where streams of JSON-ish records share structure but evolve over time.
VARIANT类型在DuckDB v1.5中推出,你可以把它想象成增强版的JSON。基本上,想象一下如果JSON很快会怎样。像JSON一样,VARIANT列可以在每一行存储不同形状的数据。与JSON不同,它不是文本格式:DuckDB会自动检测半结构化数据中隐藏的常见结构并将其“切碎”,因此它在存储中压缩良好,在查询中执行快速,而无需你声明模式。这使得VARIANT非常适合实时日志摄取,其中JSON风格记录的流共享结构但随时间演变。
In v2.0, this pipeline works end to end: shredded execution straight from storage (#20912), extraction pushdown into scans (#22478), shredded VARIANT reading and writing for Parquet, and a family of variant_* functions:
在v2.0中,这条管道端到端工作:直接从存储进行切碎执行(#20912)、将提取下推到扫描中(#22478)、为Parquet提供切碎的VARIANT读写,以及一系列variant_*函数:
CREATE TABLE events (payload VARIANT);
INSERT INTO events
VALUES ('{"user": {"id": 42, "tags": ["a", "b"]}}'::JSON::VARIANT);
SELECT variant_type(payload), variant_keys(payload)
FROM events;
SELECT *
FROM events
WHERE variant_contains(payload, {'user': {'id': 42}}::VARIANT);CREATE TABLE events (payload VARIANT);
INSERT INTO events
VALUES ('{"user": {"id": 42, "tags": ["a", "b"]}}'::JSON::VARIANT);
SELECT variant_type(payload), variant_keys(payload)
FROM events;
SELECT *
FROM events
WHERE variant_contains(payload, {'user': {'id': 42}}::VARIANT);Longer term, likely soon after v2.0 (but don't hold us to it), we plan to back the regular JSON type with VARIANT, so existing JSON workloads get all of these benefits without changing a single query.
从长远来看,可能在v2.0之后不久(但别指望我们),我们计划用VARIANT支持常规的JSON类型,这样现有的JSON工作负载无需更改任何查询就能获得所有这些好处。
3. Triggers
3. 触发器
Triggers have been a long-standing feature request, and DuckDB v2.0 delivers them in full: BEFORE and AFTER triggers, FOR EACH ROW and FOR EACH STATEMENT, transition tables via REFERENCING OLD/NEW TABLE, multiple triggers per event, RETURNING on triggered tables, and DROP TRIGGER.
触发器一直是一个长期的功能请求,DuckDB v2.0完整地实现了它们:BEFORE和AFTER触发器、FOR EACH ROW和FOR EACH STATEMENT、通过REFERENCING OLD/NEW TABLE的过渡表、每个事件的多个触发器、触发表上的RETURNING,以及DROP TRIGGER。
The classic use case is audit tables: something happens in the system, and a trigger records what changed. For example:
经典用例是审计表:系统中发生某事,触发器记录发生了什么变化。例如:
CREATE TABLE target (id INTEGER, val INTEGER);
CREATE TABLE audit (id INTEGER, old_val INTEGER, new_val INTEGER);
CREATE TRIGGER trg_audit AFTER UPDATE ON target
REFERENCING OLD TABLE AS o NEW TABLE AS n
FOR EACH STATEMENT
INSERT INTO audit
SELECT n.id, o.val, n.val
FROM o
JOIN n ON o.id = n.id;
INSERT INTO target VALUES (1, 10), (2, 20);
UPDATE target SET val = val * 10 WHERE id <= 2;
SELECT * FROM audit;CREATE TABLE target (id INTEGER, val INTEGER);
CREATE TABLE audit (id INTEGER, old_val INTEGER, new_val INTEGER);
CREATE TRIGGER trg_audit AFTER UPDATE ON target
REFERENCING OLD TABLE AS o NEW TABLE AS n
FOR EACH STATEMENT
INSERT INTO audit
SELECT n.id, o.val, n.val
FROM o
JOIN n ON o.id = n.id;
INSERT INTO target VALUES (1, 10), (2, 20);
UPDATE target SET val = val * 10 WHERE id <= 2;
SELECT * FROM audit;| id | old_val | new_val |
|---|---|---|
| 1 | 10 | 100 |
| 2 | 20 | 200 |
| id | old_val | new_val |
|---|---|---|
| 1 | 10 | 100 |
| 2 | 20 | 200 |
Triggers fit naturally with long-running DuckDB services, and we are also planning to use them internally to build several upcoming features. They are fully exposed at the SQL level too, so you can build your own cool stuff with them.
触发器自然适合长时间运行的DuckDB服务,我们也计划在内部使用它们来构建几个即将推出的功能。它们在SQL层面也完全暴露,因此您可以用它们构建自己的酷炫功能。
4. SQL Dialect Additions
4. SQL方言新增
As always, DuckDB's SQL dialect keeps growing. A few favorites from this release cycle:
一如既往,DuckDB的SQL方言持续增长。本发布周期中的几个亮点:
With NEAREST joins (#24137), top-k similarity search becomes a join clause, handy for vector and embedding workloads:
通过NEAREST连接(#24137),top-k相似性搜索成为连接子句,便于向量和嵌入工作负载:
SELECT q.user_id, t.product_id
FROM users q
INNER JOIN products t APPROX NEAREST 2
BY SIMILARITY array_cosine_similarity(q.embedding, t.embedding);SELECT q.user_id, t.product_id
FROM users q
INNER JOIN products t APPROX NEAREST 2
BY SIMILARITY array_cosine_similarity(q.embedding, t.embedding);DML inside CTEs (#21634, #21997, #24217) lets you use INSERT, UPDATE, DELETE, and COPY as pipeline steps:
CTE内的DML(#21634、#21997、#24217)允许您将INSERT、UPDATE、DELETE和COPY用作管道步骤:
WITH moved AS MATERIALIZED (
DELETE FROM staging RETURNING *
)
INSERT INTO archive SELECT * FROM moved;WITH moved AS MATERIALIZED (
DELETE FROM staging RETURNING *
)
INSERT INTO archive SELECT * FROM moved;Nested schemas (#23492, #24222) allow schemas within schemas:
嵌套模式(#23492、#24222)允许模式内嵌套模式:
CREATE SCHEMA finance;
CREATE SCHEMA finance.reports;
CREATE TABLE finance.reports.q3 (revenue DECIMAL);CREATE SCHEMA finance;
CREATE SCHEMA finance.reports;
CREATE TABLE finance.reports.q3 (revenue DECIMAL);The new variable syntax (#21194) lets you write $x anywhere an expression is allowed, no more getvariable(...) verbiage:
新的变量语法(#21194)允许您在允许表达式的任何位置写$x,不再需要getvariable(...)的冗长表达:
SET VARIABLE threshold = 100;
SELECT * FROM orders WHERE amount > $threshold;SET VARIABLE threshold = 100;
SELECT * FROM orders WHERE amount > $threshold;The JSON mutation functions json_set, json_insert, json_replace, and json_remove (#23786) finally let you modify JSON documents in place:
JSON修改函数json_set、json_insert、json_replace和json_remove(#23786)终于允许您就地修改JSON文档:
SELECT json_set('{"a":1}', '$.b', '2');SELECT json_set('{"a":1}', '$.b', '2');json_set('{"a":1}', '$.b', '2') {"a":1,"b":2}
json_set('{"a":1}', '$.b', '2') {"a":1,"b":2}
And recursive CTEs with USING KEY aggregation (#19481) enable iterative algorithms in pure SQL, backed by the rewritten recursive CTE engine described below:
使用USING KEY聚合的递归CTE(#19481)支持纯SQL中的迭代算法,由下面描述的重写的递归CTE引擎支持:
WITH RECURSIVE tbl(a, b) USING KEY (a, avg(b)) AS (
SELECT 1, 5
UNION
SELECT a, b - 1 FROM tbl WHERE b > 0
)
TABLE tbl;WITH RECURSIVE tbl(a, b) USING KEY (a, avg(b)) AS (
SELECT 1, 5
UNION
SELECT a, b - 1 FROM tbl WHERE b > 0
)
TABLE tbl;| a | b |
|---|---|
| 1 | 2.5 |
| a | b |
|---|---|
| 1 | 2.5 |
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力