[branch-52] SortMergeJoin don't wait for all input before emitting (#20699)

## Which issue does this PR close?

Backport of #20482 to branch-52.
 
## Rationale for this change
                                                          
Cherry-pick fix and prerequisites so that SortMergeJoin emits output
incrementally instead of waiting for all input to
 complete. This resolves OOM issues Comet is seeing with DataFusion 52.

## What changes are included in this PR?

Cherry-picks of the following commits from `main`:

1. #19614 — Extract sort-merge join filter logic into separate module
2. #20463 — Use zero-copy slice instead of take kernel in sort merge
join
3. #20482 — Fix SortMergeJoin to not wait for all input before emitting

## Are these changes tested?

Yes, covered by existing and new tests included in #20482.

## Are there any user-facing changes?

No.

---------

Co-authored-by: Liang-Chi Hsieh <viirya@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Andy Grove <agrove@apache.org>
Co-authored-by: Raz Luvaton <16746759+rluvaton@users.noreply.github.com>

Author: 5 people