Fix PushdownSort dropping LIMIT when eliminating SortExec#21744
Fix PushdownSort dropping LIMIT when eliminating SortExec#21744alamb merged 3 commits intoapache:mainfrom
Conversation
xudong963
left a comment
There was a problem hiding this comment.
LGTM, would be better if there are some slt tests
Also, do you want to include it into new minor release?
alamb
left a comment
There was a problem hiding this comment.
Thanks @sgrebnov and @martin-g and @xudong963
@xudong963 — There's an existing SLT test (sort_pushdown.slt Test 1.3) that already covers the Exact pushdown + LIMIT case where the source supports |
|
Thanks everyone. If it turns out we can make some slt tests perhaps we can add them as a follow on PR |
Which issue does this PR close?
When
PushdownSortremoves aSortExecbecause a source returnsExact(guaranteeing ordering), anyfetch(LIMIT) on theSortExecis silently dropped if the underlying plan does not supportwith_fetch().For example,
ProjectionExecsupportstry_pushdown_sort(delegating to its child) but does not implementwith_fetch(). A plan likeSortExec(fetch=10) → ProjectionExec → sourcethat gets sort-eliminated loses the limit.What changes are included in this PR?
In the
Exactbranch ofPushdownSort, when the eliminatedSortExeccarried afetch:with_fetch()on the pushed-down source firstwith_fetch()returnsNone, fall back to wrapping withGlobalLimitExecAre these changes tested?
Yes. Three new unit tests:
test_sort_pushdown_exact_no_fetch_no_limit— Exact elimination without fetch: no limit wrapper addedtest_sort_pushdown_exact_preserves_fetch_with_global_limit— Exact elimination with fetch, source does NOT supportwith_fetch():GlobalLimitExecwrapper addedtest_sort_pushdown_exact_preserves_fetch_with_source_support— Exact elimination with fetch, source supportswith_fetch(): limit pushed into source directlyAre there any user-facing changes?
No.