跳到主内容
@wquguru
精选85Polars（GitHub Releases）语言与库

Polars py-1.44.0：弃用 rechunk 参数

Python Polars 1.44.0

原文
推荐理由

本次更新明确弃用了大量代码中高频使用的 rechunk 参数与方法，直接影响现有 DataFrame 处理逻辑。请检查并移除对已弃用 API 的调用，避免后续版本升级导致运行失败。

⚠️ Deprecations

⚠️ 弃用

  • Deprecate rechunk parameter for all read/scan functions (#28063)
  • Deprecate Expr.rechunk() (#28692)
  • Deprecate struct.rename_fields() with an incorrect number of fields (#28672)
  • 弃用所有 read/scan 函数的 rechunk 参数 (#28063)
  • 弃用 Expr.rechunk() (#28692)
  • 弃用字段数量不正确的 struct.rename_fields() (#28672)

🚀 Performance improvements

🚀 性能改进

  • Add private env var toggle for HTTP rate limit (#28882)
  • Relax default CloudRetryConfig for rate limit stability (#28885)
  • Elide generic cross join on subquery decorrelation in equality predicates (#28876)
  • cache CTE's in SQL layer (#28864)
  • Allow CSPE if manual caches are set (#28859)
  • Lower unqualified join predicates to inner joins in SQL (#28854)
  • Rechunk mask for DataFrame filter (#28762)
  • Improve performance of when/then/otherwise by masking out unevaluated elements (#28498)
  • Make async executor scheduling NUMA aware (#28464)
  • Drop unused projections in filter on streaming engine (#28713)
  • Retain Parquet Partial metadata on filter (#28737)
  • 添加用于 HTTP 速率限制的私有环境变量开关 (#28882)
  • 放宽 CloudRetryConfig 的默认设置以稳定速率限制 (#28885)
  • 在等值谓词中省略子查询去相关时的通用交叉连接 (#28876)
  • 在 SQL 层缓存 CTE（公共表表达式）(#28864)
  • 如果设置了手动缓存，则允许 CSPE (#28859)
  • 在 SQL 中将未限定名的连接谓词下推为内连接 (#28854)
  • DataFrame filter 的重分块掩码 (#28762)
  • 通过屏蔽未求值的元素来改善 when/then/otherwise 的性能 (#28498)
  • 使异步执行器的调度感知 NUMA（非统一内存访问）(#28464)
  • 在流式引擎的过滤操作中丢弃未使用的投影 (#28713)
  • 在过滤操作中保留 Parquet 的部分元数据 (#28737)

✨ Enhancements

✨ 增强功能

  • Support integer fixed-array dot products (#28829)
  • SQL correlated in (#28927)
  • Support more join_types in join_where (#28880)
  • Support Iceberg schema evolution (#28794)
  • SQL conformance (#28494)
  • Introduce RemoteEngine and a common base class for all engines (#28800)
  • Support Iceberg table with V3 deletion vectors in native scan_iceberg (#28772)
  • Support Iceberg snapshot properties (#28793)
  • Expose sinked_paths_callback on sink_parquet as unstable parameter (#28814)
  • Add infer_schema_files to CSV inference hint (#28809)
  • Add native fixed-array dot (#28504)
  • Export query metrics to polars_cloud (#28757)
  • Support Iceberg object storage paths (#28634)
  • More CSEE simplification (#28731)
  • Improve error in Series constructor with name in context (#28743)
  • Add adaptive HTTP rate-limiter for cloud IO (#28591)
  • Add struct.drop() (#28666)
  • Improve error message when CSV name de-duplication fails (#28658)
  • Always keep first metadata per source for Parquet (#28661)
  • Improve plan-time row estimates for multi-file parquet scans (#28380)
  • Expose array and plugin function views in Python visitor (#28635)
  • 支持整数固定数组的点积运算 (#28829)
  • SQL 关联 IN 查询 (#28927)
  • 在 join_where 中支持更多 join_types (#28880)
  • 支持 Iceberg 模式演进 (#28794)
  • SQL 合规性 (#28494)
  • 引入 RemoteEngine 以及所有引擎的通用基类 (#28800)
  • 在 native scan_iceberg 中支持带有 V3 删除向量的 Iceberg 表 (#28772)
  • 支持 Iceberg 快照属性 (#28793)
  • 将 sinked_paths_callback 作为不稳定参数暴露给 sink_parquet (#28814)
  • 在 CSV 推断提示中添加 infer_schema_files (#28809)
  • 添加原生固定数组点积运算 (#28504)
  • 将查询指标导出到 polars_cloud (#28757)
  • 支持 Iceberg 对象存储路径 (#28634)
  • 进一步简化 CSEE (#28731)
  • 改进 Series 构造函数中包含名称时的错误信息 (#28743)
  • 为云 IO 添加自适应 HTTP 速率限制器 (#28591)
  • 添加 struct.drop() (#28666)
  • 改进 CSV 名称去重失败时的错误消息 (#28658)
  • 始终为 Parquet 保留每个源的第一条元数据 (#28661)
  • 改进多文件 parquet 扫描的计划时行数估计 (#28380)
  • 在 Python visitor 中暴露数组和插件函数视图 (#28635)

🐞 Bug fixes

🐞 Bug 修复

  • SQL handle quantified comparisons against a subquery (#28929)
  • Fix SQL subquery qualified lowering and cache row-index (#28926)
  • Clippy-stable failures (#28919)
  • Use uuidv7 for cloud observed queries (#28910)
  • Unsoundness in AnyValue::into_static for structs (#28889)
  • Invalidate suborders on projection (#28866)
  • Ensure SQL function registry is available in isolated subqueries (#28881)
  • Handle null in Expr.is_not_nan (#28883)
  • Ensure eager unpivot column selection aligns with the lazy engine (#28846)
  • Fix assertion error / incorrect data load from parquet written by fastparquet (#28656)
  • Fix projection pushdown incorrect select() height (#28751)
  • Restore monitoring as an engine-level parameter (#28821)
  • Bincode serialization of IR hints (#28837)
  • Retain datetime inference across CSV fallback (#28663)
  • Avoid chunk fragmentation on bool agg min and max (#28789)
  • Merge join with coalesce and empty suffix (#28799)
  • Wrong function name call in test (#28801)
  • Preserve row mapping in nested over expressions (#28767)
  • Restore CI compatibility with NumPy 2.5.2 (#28782)
  • Do not deduplicate inherently nondeterministic plan nodes (#28768)
  • Fix OOC spilling of categoricals (#28761)
  • Correct partitionable filter handling for pivot (#28749)
  • Small typo in docstring (#28763)
  • Avoid chunk fragmentation on bool agg any and all (#28754)
  • Ignore nans in (min|max)_horizontal (#28710)
  • Correct function display for str.replace_all (#28742)
  • Exclude debug tests from nodebug streaming run (#28747)
  • Convert predicate series dtype during prefiltered parquet reading (#28711)
  • Do not remove nested caches when performing predicate pushdown past a cache node (#28740)
  • Correct handling of nulls at the start of business day functions (#28703)
  • Do not panic when exceeding Parquet metadata scan cache (#28721)
  • Remove cryptographic hashing from CSPE (#28716)
  • Unsoundness in rayon block_on (#28709)
  • Check for (shallow) IR node equality on top of hashing in CSPE (#28506)
  • Don't hand IO plugins a predicate they cannot evaluate (#28643)
  • Raise on overflow in Decimal sum instead of silently wrapping (#28688)
  • Preserve nulls when importing Arrow maps (#28680)
  • Propagate null by column values in rolling_*_by (#27367)
  • Fix self-referencing field in struct.with_fields with over (#28678)
  • Fix Arrow buffer offset for Utf8 and Binary (#28662)
  • Clamp group-by slice offset (#28579)
  • Release the GIL while resolving the schema in __arrow_c_stream__ (#28642)
  • Fix data corruption on Series from nested Arrow LargeList (#28632)
  • SQL 处理针对子查询的限定比较 (#28929)
  • 修复 SQL 子查询限定符降级并缓存行索引 (#28926)
  • Clippy-stable 失败问题 (#28919)
  • 为云观测查询使用 uuidv7 (#28910)
  • AnyValue::into_static 在处理结构体时的不安全性 (#28889)
  • 在投影时使子排序失效 (#28866)
  • 确保 SQL 函数注册表在隔离的子查询中可用 (#28881)
  • 处理 Expr.is_not_nan 中的空值 (#28883)
  • 确保急切 unpivot 列选择与惰性引擎对齐 (#28846)
  • 修复由 fastparquet 写入的 parquet 文件导致的断言错误/数据加载不正确问题 (#28656)
  • 修复投影下推中 incorrect select() 高度问题 (#28751)
  • 将监控恢复为引擎级参数 (#28821)
  • IR 提示的 Bincode 序列化 (#28837)
  • 在 CSV 回退过程中保留日期时间推断 (#28663)
  • 避免在布尔聚合 min 和 max 上出现块碎片化 (#28789)
  • 合并带有 coalesce 和空后缀的连接 (#28799)
  • 测试中调用了错误的函数名 (#28801)
  • 在嵌套 over 表达式中保留行映射 (#28767)
  • 恢复与 NumPy 2.5.2 的 CI 兼容性 (#28782)
  • 不要去重本质上非确定性的计划节点 (#28768)
  • 修复分类变量的 OOC（Out-Of-Core）溢出问题 (#28761)
  • 修正透视表的分区过滤器处理 (#28749)
  • 文档字符串中的小拼写错误 (#28763)
  • 避免在布尔聚合 any 和 all 上出现块碎片化 (#28754)
  • 忽略 (min|max)_horizontal 中的 NaN 值 (#28710)
  • 修正 str.replace_all 的函数显示 (#28742)
  • 从 nodebug 流式运行中排除调试测试 (#28747)
  • 在预过滤读取 parquet 期间转换谓词序列的数据类型 (#28711)
  • 在执行谓词下推经过缓存节点时，不要移除嵌套缓存 (#28740)
  • 修正业务日函数开头对 null 值的处理 (#28703)
  • 超出 Parquet 元数据扫描缓存时不要 panic (#28721)
  • 从 CSPE 中移除加密哈希 (#28716)
  • rayon block_on 中的不健全性问题 (#28709)
  • 在 CSPE 的哈希基础上检查（浅层）IR 节点相等性 (#28506)
  • 不要将 IO 插件无法评估谓词交给它们 (#28643)
  • Decimal 求和溢出时抛出异常，而不是静默回绕 (#28688)
  • 导入 Arrow map 时保留空值 (#28680)
  • 在 rolling_*_by 中按列值传播空值 (#27367)
  • 修复 struct.with_fields 中使用 over 时的自引用字段问题 (#28678)
  • 修复 Utf8 和 Binary 的 Arrow 缓冲区偏移量 (#28662)
  • 限制 group-by 切片偏移量 (#28579)
  • 在 __arrow_c_stream__ 中解析模式时释放 GIL (#28642)
  • 修复来自嵌套 Arrow LargeList 的 Series 的数据损坏问题 (#28632)

📖 Documentation

📖 文档

  • Interlink join docstrings (#28907)
  • Migration guide for 2.0 release (#28788)
  • Clarify read_database_uri requirements for SQLAlchemy (#28366)
  • Update engine tags (#28852)
  • Clarify "that that side" comment in asof_join (#28847)
  • Update mage.ai documentation link (#28813)
  • Add engine tags (#28708)
  • Document that arg_extremum functions return any index on ties (#28691)
  • Fix Polars Cloud API reference link (#28693)
  • 互连 join 文档字符串 (#28907)
  • 2.0 版本迁移指南 (#28788)
  • 澄清 SQLAlchemy 的 read_database_uri 要求 (#28366)
  • 更新引擎标签 (#28852)
  • 澄清 asof_join 中“that that side”注释的含义 (#28847)
  • 更新 mage.ai 文档链接 (#28813)
  • 添加引擎标签 (#28708)
  • 记录 arg_extremum 函数在并列情况下返回任意索引 (#28691)
  • 修复 Polars Cloud API 引用链接 (#28693)

📦 Build system

📦 构建系统

  • Disable Numa on python release (#28934)
  • 在 Python 发布时禁用 Numa (#28934)

🛠️ Other improvements

🛠️ 其他改进

  • Bump runtime toolchains (#28924)
  • Add min-publish-age for cargo (#28906)
  • Tighten join invariants DSL (#28913)
  • Bump Rust toolchain (#28900)
  • Ensure that predicates are pushed down to the scans (#28857)
  • Clarify "that that side" comment in asof_join (#28847)
  • Restore monitoring as an engine-level parameter (#28821)
  • Remove workaround for inconsistent CSE (#28834)
  • Introduce RemoteEngine and a common base class for all engines (#28800)
  • Add row count and file byte size to sinked paths callback (#28522)
  • Use canonical expression IDs in CSEE (#28786)
  • Put numa behind feature flag (#28785)
  • Enforce 100% type coverage in CI with Pyrefly (#28795)
  • Attribute annotations for CredentialProviderAWS and CredentialProviderAzure (#28790)
  • Attribute annotations for ScanCastOptions (#27906)
  • Disable credentials in upload chunk size tests (#28745)
  • Improve backtrace path formatting (#28732)
  • Exclude debug tests from nodebug streaming run (#28747)
  • Mark test_grouped_agg_parametric as slow (and add param ids) (#28715)
  • Fix first expect in test_group_by_arg_max_boolean_26978 (#28690)
  • Fix the name of struct.drop() in rename_fields() deprecation message (#28687)
  • Add dist=loadgroup to the pytest config (#28676)
  • Enable a lot of may_fail_auto_streaming tests (#28675)
  • Re-enable test_extension() for streaming engine (#28611)
  • Update test_hive_join_rewrite_semi_join test to work with streaming engine (#28610)
  • 升级运行时工具链 (#28924)
  • 为 cargo 添加 min-publish-age (#28906)
  • 收紧 join 不变量 DSL (#28913)
  • 升级 Rust 工具链 (#28900)
  • 确保谓词被下推到扫描阶段 (#28857)
  • 澄清 asof_join 中“that that side”注释 (#28847)
  • 将监控恢复为引擎级参数 (#28821)
  • 移除针对不一致 CSE 的变通方案 (#28834)
  • 引入 RemoteEngine 并为所有引擎提供通用基类 (#28800)
  • 在 sinked paths 回调中添加行数和文件字节大小 (#28522)
  • 在 CSEE 中使用规范表达式 ID (#28786)
  • 将 numa 置于特性标志之后 (#28785)
  • 使用 Pyrefly 在 CI 中强制执行 100% 类型覆盖率 (#28795)
  • 为 CredentialProviderAWS 和 CredentialProviderAzure 添加属性注解 (#28790)
  • 为 ScanCastOptions 添加属性注解 (#27906)
  • 在上传块大小测试中禁用凭据 (#28745)
  • 改进回溯路径格式化 (#28732)
  • 从 nodebug 流式运行中排除调试测试 (#28747)
  • 将 test_grouped_agg_parametric 标记为慢速（并添加 param ids）(#28715)
  • 修复 test_group_by_arg_max_boolean_26978 中的第一个 expect (#28690)
  • 修复 rename_fields() 弃用消息中 struct.drop() 的名称 (#28687)
  • 向 pytest 配置添加 dist=loadgroup (#28676)
  • 启用大量 may_fail_auto_streaming 测试 (#28675)
  • 为流式引擎重新启用 test_extension() (#28611)
  • 更新 test_hive_join_rewrite_semi_join 测试以适配流式引擎 (#28610)

Thank you to all our contributors for making this release possible!

感谢所有贡献者使本次发布成为可能！

@0guban0v, @JakubValtar, @Joosboy, @Kevin-Patyk, @MatteoPossamai, @NicoOhR, @TNieuwdorp, @VedantMadane, @aarushkandukoori, @alexander-beedie, @azimafroozeh, @borchero, @carnarez, @dancsi, @dependabot[bot], @dsprenkels, @gautamvarmadatla, @jonasdedden, @jorenham, @kdn36, @lzcmian, @malhotrashivam, @matthewbayer, @mikhail5555, @nameexhaustion, @orlp, @ritchie46, @subotac and dependabot[bot]

@0guban0v、@JakubValtar、@Joosboy、@Kevin-Patyk、@MatteoPossamai、@NicoOhR、@TNieuwdorp、@VedantMadane、@aarushkandukoori、@alexander-beedie、@azimafroozeh、@borchero、@carnarez、@dancsi、@dependabot[bot]、@dsprenkels、@gautamvarmadatla、@jonasdedden、@jorenham、@kdn36、@lzcmian、@malhotrashivam、@matthewbayer、@mikhail5555、@nameexhaustion、@orlp、@ritchie46、@subotac 以及 dependabot[bot]

更进一步：量化金融体系

看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力

进入量化体系 →
话题#弃用#Polars#Python#Iceberg

相似阅读

另一事件，读法相近