Polars 0.54.4：稳定流式引擎，新增 gather

🏆 Highlights

• Add LazyFrame.gather (#27501)
• Nested common subplan elimination (#27340)
• Stabilize streaming engine (#27497)
• Speed up parquet metadata decode with hand-written Thrift (#27427)
• Add streaming support for grouped AsOf join (#27293)

🚀 Performance improvements

• Eliminate filters with contradictory predicates (#27775)
• Update to new jemalloc (#27797)
• Do not materialize ScalarColumn in Column split_at (#27782)
• Avoid materializing broadcast in array.shift (#27740)
• Avoid materializing broadcast list in list.sample(n) and list.sample(frac) (#27679)
• Adaptive size dispatch to hashset or radix sort + capacity-aware reset in agg_n_unique (#27719)
• Dispatch {list,arr}.{unique,n_unique,reverse} to group_by engine (#27278)
• Improve in-memory grouped non-null count (#27702)
• Factor shared conjuncts out of OR-of-ANDs predicates (#27627)
• Skip downloading IPC batches exceeding slice bounds (#27683)
• Faster Series::is_sorted for logical / non-primitive types (#27567)
• Avoid materializing broadcast list in list.shift (#27628)
• Optimise json_decode Datetime string parsing (#27559)
• Speed up to_numpy C-order via cache-blocked transpose (#27522)
• Optimize select(len()) for non-strict horizontal concat (#27516)
• Pushdown slices to inputs on left/right/full join (#27508)
• Don't infer CSV schema if schema is set (#27507)
• Nested common subplan elimination (#27340)
• Make is_in row-group pruning precise on null-containing haystacks (#27495)
• Don't do fused-multiply-add on scalars (#27479)
• List full fast path (#27477)
• Make is_in row-group pruning precise on multi-value lists (#27475)
• Add streaming GatherNode (#27465)
• Lower non-elementwise FunctionExprIR to ColumnarFunctionNode (#27462)
• Speed up parquet metadata decode with hand-written Thrift (#27427)
• Skip validity mask processing in __array_ufunc__ when no inputs have nulls (#27358)
• Create IR slice from expr slice pushdown (#27200)
• Add streaming support for grouped AsOf join (#27293)
• Avoid unnecessary rechunk when sorting already sorted DataFrame (#27264)
• Lower basic over() to streaming primitives (#27303)
• Lower drop_{nulls,nans} in streaming group_by aggregations (#27296)
• Lower entropy to streaming reductions (#27174)
• Add native streaming interpolate (#27185)
• Streaming strptime with format=None (#27056)
• Lower skew / kurtosis to streaming aggregations (#27176)
• Post apply pyarrow filter in Polars' engine instead of pyarrow (#27192)
• Optimize drop_nulls().{first,last}() to {first,last}(ignore_nulls=True) (#27187)
• Always process pyarrow scan in batches (#27183)
• Make cut output Enum and mark as elementwise (#27173)
• Remove unused expression sorts (#27075)
• Use delta stats for mixed hive and non-hive predicate pushdown (#27102)
• Take into account size per row in join sampling (#27098)
• Streaming is_first_distinct and unique(maintain_order=True) (#27052)
• Streaming cov and corr (#27008)
• Add sorted unique node to streaming engine (#26990)
• Ensure Expr.append is lowered in streaming engine (#27022)
• Collapse consecutive Sort nodes (#26965)
• Drop maintain_order=True requirement in sink_delta (#27007)
• Lower index_of to streaming engine (#26923)
• Streaming native backward_fill (#26967)
• Native streaming forward_fill (#26922)
• Drop unused filter column above cache (#26955)
• Optimize .replace() from a single value (#26948)
• Add a streaming range-join (#26790)
• Lower arg_{min,max} to streaming engine (#26845)
• Additional IR slice pushdown after filter pushdown (#26815)
• Streaming first/last on Enum through physical (#26783)
• Fast filter for scalar predicates (#26745)
• Allow SimpleProjection in streaming engine to rename (#26709)
• Streaming cloud download for scan_csv (#26637)
• Drop columns only needed for predicates after the predicate is applied (#26703)
• Run projection pushdown after predicate pushdown (#26688)
• Comparison literal downcasting (#26663)
• Add dynamic predicates for TopK (#26495)
• Increase minimum default parquet row group prefetch to 8 (#26632)
• Partial predicate conversion to PyArrow (#26567)
• Streaming cloud download for scan_ndjson / scan_lines (#26563)
• Grab GIL fewer times during Object join materialization (#26587)
• Improve CSV and NDJSON cloud sink performance (#26545)
• Tune cloud writer performance (#26518)
• Allow parallel InMemorySinks in streaming engine (#26501)
• Add streaming AsOf join node (#26398)

✨ Enhancements

• Expose fixed-size rolling window expressions in Python visitor (#27108)
• Expose IR::Scan hive parts in the python node visitor (#27829)
• Expose IRFunctionExpr::DynamicPred in the python visitor (#27616)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Add pinning and queuing logic to polars-ooc (#27791)
• Add tiered multi-file parquet metadata resolver (#27720)
• Cache and shuffle DNS for cloud object_store (#27659)
• Update to new jemalloc (#27797)
• Allow deeper expressions (#27768)
• Add is_inherently_nondeterministic helper for AExpr (#27687)
• Use true division for the / operator in Polars SQL (#27391)
• Add Rust backend for Expr.has_nulls (#27590)
• Add block_in_place to Polars' async executor (#27612)
• Stabilize float16 (#27607)
• Add Expr.is_empty (#27583)
• Add support for the SQL FILTER clause for aggregate functions, and STRING_AGG (#27564)
• Make parquet FileMetadata prunable for IR-plan dispatch (#27535)
• Broadcast scalar input for list.slice (#27487)
• Add LazyFrame.gather (#27501)
• Add null_on_oob in {Expr/Series}.gather (#27327)
• Stabilize streaming engine (#27497)
• Process batched arr.eval on overflow boundaries (#27496)
• Process batched list.eval on overflow boundaries (#27483)
• Print SLICED UNION in LazyFrame explain (#27467)
• Cargo deny (#27363)
• Add maintain_order parameter to merge_sorted (#27263)
• Add ignore_nulls to {list,arr}.{any,all} (#27186)
• Lock-free memory manager with spill-to-disk and fully OOC multiplexer (#26774)
• Add is_unique to list/array dtypes (#27290)
• Add pl.merge_sorted operating on multiple frames (#27014)
• Add fast_alloc feature flag, remove default_alloc (#27206)
• Add a GPU slot to OptFlags so we can control CSE (similar to streaming) (#27026)
• Allow group_by() without key exprs (#27141)
• Collapse consecutive Sort nodes (#26965)
• Use UUIDv7 for sink_iceberg directory name generation (#26958)
• Truncate large binary/utf8 Parquet statistics values (#26764)
• Error if PartitionBy path provider returns absolute path that does not begin with base path, or contains '..' (#26894)
• Support Delta deletion vectors in scan_delta (#26867)
• Support Decimal32/64 in scan_parquet (#26941)
• Support casting Duration to String in ISO 8601 format (#26860)
• Add a streaming range-join (#26790)
• Support Expr for holidays in business day calculations (#26193)
• Parameter for pivot to always include value column name (#26730)
• Raise error in .collect_schema() when arr.get() is out-of-bounds (#26866)
• Extend Expr.reinterpret to all numeric types of the same size (#26401)
• Add missing_columns parameter to scan_csv (#26787)
• Clear no-op scan projections (#26858)
• Support nested datatypes for {min,max}_by (#26849)
• Support SQL ARRAY init from typed literals (#26622)
• Accept table identifier string in scan_iceberg() (#26826)
• Add a convenience make fresh command to the Makefile (#26809)
• Add unstable LazyFrame.sink_iceberg (#26799)
• Add maintain order argument on implode (#26782)
• Implement predicate pushdown for aliased groupby keys (#26597)
• Speed up casting primitive to bool by at least 2x (#26823)
• Enable rowgroup skipping for float columns (#26805)
• Add expression context to errors (#26716)
• Add Decimal support for product reduction (#26725)
• Support all Iceberg V2 arrow types in sink_parquet arrow_schema parameter (#26669)