Polars Python v2.1.0：禁止字符串与数值混合算术，新增 Iceberg 与递归 CTE

💥 Breaking changes

• Disallow mixed string <=> numeric arithmetic in eager (#30071)

🚀 Performance improvements

• Limit the memory of buckets sorted ahead in the sort flush (#30297)
• Make file readers wait for spills while over the memory budget (#30296)
• Count unspilled frames in the memory estimate right away (#30295)
• Simplify more boolean expressions and drop redundant null checks (#30293)
• Wait for spills while over the out-of-core memory budget (#30294)
• Lower the default out-of-core prefetch fraction to 0.5 (#30292)
• Aggregate unique and n_unique in the streaming group-by itself (#30281)
• Return freed memory to the OS right away while out-of-core data is spilled (#30280)
• Gather List(Array) and Array columns without per-row slices (#30278)
• Evaluate the streaming window node out of core (#30282)
• Avoid deep copies when building a DataFrame from flat dataclasses (#30275)
• Sort the window partitions by each order key instead of their row encoding (#30277)
• Scan the haystack for a single needle in is_in (#30276)
• Prefer fixed-width columns when only the row count is needed (#30274)
• Native gather for List arrays (#30019)
• Evaluate windows in filter predicates in the streaming window node (#30272)
• Don't busy-wait in RAYON.block_on (#30259)
• Encode nested Parquet levels directly from the level buffers (#30251)
• Reduce memory of CSV scans on wide rows (#30165)
• Avoid quadratic JSON serialization of nested lists and maps (#30203)
• Skip the gather when exploding lists without nulls (#30234)
• Speed up writing nested Parquet columns (#30245)
• Don't keep whole parquet string dictionaries alive for a few selected rows (#30248)
• Set default spill compression level to 1 (#30237)
• Reuse zstd contexts in compression and avoid memcpys (#30233)
• Remove repeated and scalar ordering keys before the sort rewrites (#30231)
• Remove repeated keys from by's (#30095)
• Keep only one guessed unique key per join order leaf (#30191)
• Don't let extra join keys shrink an estimate once one key is unique (#30190)
• Reduce binary size of asof join with by keys (#30192)
• Reduce binary size of SQL visitors and parallel group-by collect (#30171)
• Push as-of join key filters to the right input (#30123)
• Drop null as-of right keys with a parallel filter before the join node (#30125)
• Keep as-of nearest join state small (#30126)
• Explode lists in group_by aggregations without a per-group apply (#30020)
• Streaming OOC group-by with shuffles (#29911)
• Drop scalar ordering keys in the optimizer (#30063)
• Merge union inputs that scan files of the same kind into one multi-file scan (#30056)
• Use the streaming scalar window for std, var and combined reductions in over (#30066)
• Limit the bytes of local CSV and NDJSON files read at once in multi-file scans (#30055)
• Check the matched flag of join keys in the table entry before the bitset (#30067)
• Track matched build rows of all join tables in a bitset (#30050)
• Add dedicated list gather kernels (#30049)
• Emit unmatched build rows of single-key joins in row order (#30037)
• Use TopK for sorts sliced at a nonzero offset (#29989)
• Add constant folding for more datetime ops (#30016)
• Rewrite the sum of a column plus a literal to a sum and a count of the column (#30011)
• Push predicates through order-keeping columns (#30007)
• Fix dead early exit in coalesce (#30013)
• Count distinct non-null values in SQL with one group-by in the streaming engine (#30009)
• Skip parquet row groups with the running top-k threshold (#30006)
• Keep decompressed plain string pages as the string data buffer (#29984)
• Fix some in-memory filter regressions (#29988)
• Decompress snappy and zstd parquet pages without zeroing the buffer (#29980)
• Speed up str.len_chars (#29972)
• Speed up str.replace for patterns that end in .*$ (#29971)
• Use the Parquet string prefilter for literal str.contains and SQL LIKE (#29953)
• Keep single-chunk output when filtering wide frames in parallel (#29847)
• Add dedicated list slice kernels (#29938)
• Principled execution order of accumulating streaming nodes (#29933)
• Huge speedup creating wide Struct data (#29813)
• Speed up arg_min and arg_max (especially on data with nulls) (#29849)
• Huge reduction in from_rows and from_dicts memory usage during nested value conversion (#29739)
• Safely push down nested struct field filters to Iceberg (#29665)
• Compare Parquet dictionaries of Categorical and Enum columns as strings for == and is_in (#29885)
• Filter Categorical and Enum Parquet columns while decoding (#29882)
• Rewrite sort().first() to min() and single-key sort_by().first() to min_by() (#29860)
• Decode fixed-length binary Parquet decimals directly into i128 (#29866)
• Look up each dictionary value once when importing Arrow dictionaries as Categorical (#29865)
• Reduce requests and raise concurrency for cloud CSV scans (#29863)
• Reduce requests and raise concurrency for cloud NDJSON scans (#29857)
• Track sortedness through joins (#29852)
• Keep sortedness through widening numeric casts (#29851)
• Rewrite sort_by().first() to min_by and limit sorts read with head() (#29831)
• Cache parsed Iceberg metadata for metadata-path scans (#29791)
• Drop memory when Done in streaming asof_join/is_first_distinct (#29783)
• Drop scalar sort_by keys in the optimizer (#29812)
• Parallel CSV chunk analysis and pread default (#29818)
• Hash inline binview values directly from their view (#29820)
• Use iceberg statistics (#29798)
• Reuse Iceberg catalog and cache REST catalog scans (#29790)
• Preserve sortedness of computed join keys (#29782)
• TopK was holding onto too much data (#29784)
• Write CSV temporal values straight into the buffer (#29778)
• Parse strftime format once and reuse buffer in regex replace (#29777)
• Reduce needless allocations in various kernels (#29775)
• Cache http object store (#29757)
• Fetch each record batch group in its own task (#29755)

✨ Enhancements

• Support SQL CREATE TABLE with column definitions in attached Iceberg catalogs (#30260)
• Support SQL CREATE [OR REPLACE] TEMP TABLE (#30256)
• Support recursive SQL CTEs (#30188)
• Writing encrypted Parquet files (#29905)
• Support SQL DROP TABLE [PURGE] on tables of attached catalogs (#30247)
• Add conversion to/from masked numpy arrays (#30232)
• Add overwrite_filter and dynamic partition overwrite to sink_iceberg (#29909)
• Support SQL JOIN ... ON subqueries that read only one input (#30206)
• Support full joins in join_where (#30169)
• Expose fused and nested loop join predicates in the IR visitor (#30176)
• Support LIMIT 1 in correlated SQL scalar subqueries (#30168)
• Support SQL WITH RECURSIVE when no CTE reads itself (#30166)
• Support correlated IN and EXISTS that read both inputs of a SQL join in ON (#30159)
• Support CURRENT_DATE, CURRENT_TIME and CURRENT_TIMESTAMP in SQL (#30144)
• Allow reinterpret between Int128 and Decimal (#30133)
• Add Expr.truncdiv for division rounding toward zero (#30138)
• Support uncorrelated subqueries in SQL JOIN ... ON (#30141)
• Support correlated subqueries in SQL JOIN ... ON and correlated scalar subqueries without an aggregate (#30131)
• Support sinking to Iceberg table with bucket partitioned field (#30091)
• Add create_namespace_if_not_exists to sink_iceberg (#30087)
• Attach Iceberg catalogs to SQLContext via register_catalog (#30113)
• Support OOC kill fraction (#30104)
• Support qualified table names in SQL (#29963)
• Reverse node for streaming engine (#29259)
• Add pl.delete_iceberg (#30032)
• Support named options in SQL read_ipc table function (#30054)
• Support named options in SQL read_parquet table function (#29998)
• Add experimental geoparquet read/write (#29987)
• Accept a list of paths in SQL read_* table functions (#30047)
• Add SQL read_ndjson table function with named options (#29954)