Python Polars v2.0.0：破坏性变更与弃用 cut/qcut

💥 Breaking changes

• Run SQL window functions over grouped rows on the aggregated rows, and evaluate QUALIFY before the projection (#29746)
• Type exact SQL numeric literals as Decimal and truncate SQL %/DIV (#29536)
• Allow deterministic expression plugins to opt into CSE/CSPE (#29428)
• Read Parquet ENUM type as pl.String (#29331)
• More map operations (#29296)
• Deprecate cut/qcut (#29329)

🚀 Performance improvements

• Fix phases ending too soon due to join sampling (#29752)
• Group record batch fetches of remote IPC scans (#29750)
• Mark group-by node as memory intensive pipeline blocker (#29749)
• Don't inline slow OOC memory drift path (#29745)
• Only prefetch on unspill when there is memory headroom (#29701)
• Pushdown filters to scan_lance (#29538)
• Do not materialize equal ScalarColumns in Column::append/extend (#28989)
• Evaluate scalar windows in the streaming window node (#29698)
• Don't materialize scalar columns in DataFrame::estimated_size (#29711)
• Defer cached pread from prefetch to decode for Parquet (#29702)
• Decode IPC scans out of order when order is not observed (#29675)
• IPC metadata suffix fetch (#29692)
• Speed up null tracking in streaming group-by sum, min and max (#29691)
• Cache Iceberg manifest files across scans (#29623)
• Improve pushdown support for fallible Hive predicates (#29594)
• Streamline BytecodeParser method rewriting (#29632)
• Stream from_dicts records straight into column buffers (#29656)
• Read correlated SQL aggregates from the outer join when the subquery repeats it (#29685)
• Emit files unordered for remote IPC files when order not observed (#29678)
• Don't keep input order for order-insensitive windows in all engines (#29680)
• Fix parallelization in top-k reducer (#29684)
• Share one bloom filter between the build threads of a runtime filter (#29677)
• Add streaming (out-of-core) sort (#29657)
• Insert keys that start a run of equal keys into the hot group-by table right away (#29673)
• Prefetch key-row hash table lookups in blocks (#29663)
• Lower count-guarded sums to a sum that is null on empty input (#29668)
• Use single PUT for small cloud uploads (#29603)
• Don't cache plain scans in the streaming engine (#29654)
• Change single-key group-by hash table layout and add prefetching (#29661)
• Use the integer ranges of sampled parquet row groups in filter estimates (#29655)
• Raise HTTP rate-limiter read init default (#29584)
• Change join hash table layout and add prefetching (#29653)
• Add a fast-path for as_list with only a single input (#29592)
• Use one runtime filter builder per thread for a finished sample (#29646)
• Compute decimal addition, subtraction and multiplication in i64 when values fit (#29638)
• Spread decimal sums and means over lanes and check the sum's 38 digits once (#29637)
• Compute decimal addition, subtraction and multiplication without per-row validity (#29634)
• Skip or reuse UDF disassembly in BytecodeParser (#29625)
• Rescale a single decimal operand once in addition and subtraction (#29631)
• Dynamic hot table size in streaming GroupBy (#29629)
• Lower gathering from literal as elementwise (#29605)
• Execute fixed-size rolling windows on streaming engine (#29602)
• Use the finite rule of equi joins to determine build side of semi/anti (#29617)
• Drop the byte limit on runtime filter build sides (#29618)
• Add eviction-free path to GroupedReduction, with lanes (#29613)
• Improve window functions (#29591)
• Push down is_nan/is_not_nan filters to Iceberg (#29579)
• Compute shared group-by agg input subexpressions once (#29574)
• Emit files unordered for remote Parquet files when order not observed (#29581)
• Filter rows by the runtime join key range (#29576)
• Update few-group sums, means and counts in lanes (#29573)
• Loop over 4k arrays in group-by update (#29572)
• Decode parquet scans out of order when order is not observed (#29546)
• Don't always row-encode multi-column group-by (#29570)
• Push down str.starts_with filters to Iceberg (#29545)
• Trace join runtime filters through unions (#29565)
• Fold SQL temporal literals into plain values (#29562)
• Don't rehash stored key on every TotalIndexMap probe (#29561)
• Fuse drops into all filters on streaming engine (#29550)
• Join probe on Arrays directly (#29558)
• Better build side selection in equi-join sampler (#29524)
• Large optimisation for BytecodeParser rewrite/dispatch (#29548)
• Parallelize expressions in in-memory map fallback (#29535)
• Push down != filters to Iceberg (#29455)
• Improve predicate stats (#29495)
• Scale bloom filters from keys seen (#29477)
• Optimize simple like queries (#29459)
• Insert head slice below select(len() <cmp> n) to enable slice pushdown (#29209)
• Parallelize row encoding in the row_encode expression (#29434)
• Degrade outer join optimization (#29432)
• Optionally reorder semi join (#29443)
• Borrow the cached regex instead of cloning it per page (#29422)
• Lower SQL LIKE '%x%' to a literal substring match (#29431)
• Optimize is_in expression (#29425)
• Pushdown bloom filters in probe side (#29423)
• Narrow the decimal rescale to 64 bits when the value fits (#29396)
• Use a thread-local copy of a pushed-down regex predicate (#29411)
• Sample anti-join build side (#29416)
• Improve performance importing from arrow (#29412)
• Sample table size to determine build side in semi joins (#29375)
• Optimize row-encoding (#29405)
• Order pushed parquet predicate columns by measured selectivity (#29397)
• Coerce float literals to decimal instead of casting the column (#29395)
• Fix OOM on TPCH SQL and fix fuzzing errors (#29389)
• Restrict correlated SQL aggregates to requested keys (#29383)
• Prepare Parquet scans for row-group splitting in Polars Cloud (#29295)
• Skip row groups by join runtime ranges without a statistics frame (#29370)
• Reduce copy in scan_lines (#29310)
• Increase HTTP read rate-limit default (#29363)
• Evaluate a pushed parquet predicate sequentially per conjuct (#29352)
• Disable system certificates for CloudScheme::Http sources (#29284)
• Reduce rechunk in sort_in_place (#29343)
• Dynamic predicates for hash joins (#29312)
• Use HTTP suffix range for Parquet size and footer (#29308)
• Derive predicates from join conditions (#29304)
• Lower uncorrelated subqueries to semi joins and push semi/anti joins below inner joins (#29289)
• Make leaf name iterator unique (#29291)
• Don't clone the full frame per arm in when/then/otherwise (#29258)
• Push inner joins before outer joins and rewrite left-join-is-null to anti join (#29277)
• Use stats to decide cross join buffering side (#29270)
• Improve cache-removal and join-order cost estimates (#29263)
• Improve CSPE cost evaluation (#29250)
• Fuse group-by pre-select into node after partition (#29251)
• Inline hot small functions (#29244)
• Fix plan-time regressions in projection pushdown for wide frames (#28724)
• Rechunk before selecting in group-by pre-select (#29219)
• Reuse Iceberg data file sizes (#29063)

✨ Enhancements

• Python Polars 2.0 (#29723)
• Enable OOC by default with 80% of available RAM as threshold (#29741)
• Support window frames in SQL FIRST_VALUE and LAST_VALUE, and add NTH_VALUE and a LAG/LEAD default (#29740)
• Set default OOC disk budget to 64 GB (#29734)
• Add POLARS_OOMKILL_THRESHOLD_MB (#29735)
• Improve estimated DataFrame memory usage (#29712)
• Extend pipe_with_dtype for multiple expressions (#29676)
• Attribute physical nodes to IR nodes (#29522)
• Make approx_quantile sketch states mergeable across processes (#29660)
• Add unstable scan_lance (#29413)
• New Expression and function: pipe_with_dtype (#29547)
• Resolve is_in and Map lookup coercion from dtypes alone (#29487)
• Cast the needle of is_in and Map lookups exactly or not at all (#29486)
• Support Map in JSON and NDJSON reading and writing (#29478)
• Add scan_external_reader (#29232)