Skip to content

Commit aaa22a2

Browse files
xiedeyantuxiedeyantu
committed
fix
1 parent c3f57de commit aaa22a2

3 files changed

Lines changed: 5 additions & 5 deletions

File tree

‎datafusion/sqllogictest/test_files/information_schema.slt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -308,7 +308,7 @@ datafusion.optimizer.enable_sort_pushdown true
308308
datafusion.optimizer.enable_topk_aggregation true
309309
datafusion.optimizer.enable_topk_dynamic_filter_pushdown true
310310
datafusion.optimizer.enable_topk_repartition true
311-
datafusion.optimizer.enable_unions_to_filter false
311+
datafusion.optimizer.enable_unions_to_filter true
312312
datafusion.optimizer.enable_window_limits true
313313
datafusion.optimizer.enable_window_topn false
314314
datafusion.optimizer.expand_views_at_output false
@@ -456,7 +456,7 @@ datafusion.optimizer.enable_sort_pushdown true Enable sort pushdown optimization
456456
datafusion.optimizer.enable_topk_aggregation true When set to true, the optimizer will attempt to perform limit operations during aggregations, if possible
457457
datafusion.optimizer.enable_topk_dynamic_filter_pushdown true When set to true, the optimizer will attempt to push down TopK dynamic filters into the file scan phase.
458458
datafusion.optimizer.enable_topk_repartition true When set to true, the optimizer will push TopK (Sort with fetch) below hash repartition when the partition key is a prefix of the sort key, reducing data volume before the shuffle.
459-
datafusion.optimizer.enable_unions_to_filter false When set to true, the logical optimizer will rewrite `UNION DISTINCT` branches that read from the same source and differ only by filter predicates into a single branch with a combined filter. This optimization is conservative and only applies when the branches share the same source and compatible wrapper nodes such as identical projections or aliases.
459+
datafusion.optimizer.enable_unions_to_filter true When set to true, the logical optimizer will rewrite `UNION DISTINCT` branches that read from the same source and differ only by filter predicates into a single branch with a combined filter. This optimization is conservative and only applies when the branches share the same source and compatible wrapper nodes such as identical projections or aliases.
460460
datafusion.optimizer.enable_window_limits true When set to true, the optimizer will attempt to push limit operations past window functions, if possible
461461
datafusion.optimizer.enable_window_topn false When set to true, the optimizer will replace Filter(rn<=K) → Window(ROW_NUMBER) → Sort patterns with a PartitionedTopKExec that maintains per-partition heaps, avoiding a full sort of the input. When the window partition key has low cardinality, enabling this optimization can improve performance. However, for high cardinality keys, it may cause regressions in both memory usage and runtime.
462462
datafusion.optimizer.expand_views_at_output false When set to true, if the returned type is a view type then the output will be coerced to a non-view. Coerces `Utf8View` to `LargeUtf8`, and `BinaryView` to `LargeBinary`.

‎datafusion/sqllogictest/test_files/union.slt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -297,10 +297,10 @@ physical_plan
297297
04)--ProjectionExec: expr=[name@0 || _new as name]
298298
05)----DataSourceExec: partitions=1, partition_sizes=[1]
299299

300-
# unions_to_filter is disabled by default
300+
# unions_to_filter is enabled by default
301301

302302
statement ok
303-
set datafusion.optimizer.enable_unions_to_filter = false;
303+
set datafusion.optimizer.enable_unions_to_filter = true;
304304

305305
query TT
306306
EXPLAIN SELECT id, name FROM t1 WHERE id = 1 UNION SELECT id, name FROM t1 WHERE id = 2

‎docs/source/user-guide/configs.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ The following configuration settings are available:
173173
| datafusion.optimizer.expand_views_at_output | false | When set to true, if the returned type is a view type then the output will be coerced to a non-view. Coerces `Utf8View` to `LargeUtf8`, and `BinaryView` to `LargeBinary`. |
174174
| datafusion.optimizer.enable_sort_pushdown | true | Enable sort pushdown optimization. When enabled, attempts to push sort requirements down to data sources that can natively handle them (e.g., by reversing file/row group read order). Returns **inexact ordering**: Sort operator is kept for correctness, but optimized input enables early termination for TopK queries (ORDER BY ... LIMIT N), providing significant speedup. Memory: No additional overhead (only changes read order). Future: Will add option to detect perfectly sorted data and eliminate Sort completely. Default: true |
175175
| datafusion.optimizer.enable_leaf_expression_pushdown | true | When set to true, the optimizer will extract leaf expressions (such as `get_field`) from filter/sort/join nodes into projections closer to the leaf table scans, and push those projections down towards the leaf nodes. |
176-
| datafusion.optimizer.enable_unions_to_filter | false | When set to true, the logical optimizer will rewrite `UNION DISTINCT` branches that read from the same source and differ only by filter predicates into a single branch with a combined filter. This optimization is conservative and only applies when the branches share the same source and compatible wrapper nodes such as identical projections or aliases. |
176+
| datafusion.optimizer.enable_unions_to_filter | true | When set to true, the logical optimizer will rewrite `UNION DISTINCT` branches that read from the same source and differ only by filter predicates into a single branch with a combined filter. This optimization is conservative and only applies when the branches share the same source and compatible wrapper nodes such as identical projections or aliases. |
177177
| datafusion.explain.logical_plan_only | false | When set to true, the explain statement will only print logical plans |
178178
| datafusion.explain.physical_plan_only | false | When set to true, the explain statement will only print physical plans |
179179
| datafusion.explain.show_statistics | false | When set to true, the explain statement will print operator statistics for physical plans |

0 commit comments

Comments
 (0)