feat: make BatchPartitioner::partition_iter public (#21341)

## Which issue does this PR close?

- Closes #21311.

## Rationale for this change

`BatchPartitioner::partition_iter` is already used internally as the
core implementation behind the public `partition` method, and was
intentionally factored out to support both sync and async consumption
patterns. However, since it's private, downstream crates like Ballista
can't use the iterator directly and are forced to run both CPU-bound
partitioning and I/O together in a sync closure.

## What changes are included in this PR?

Changed `partition_iter` visibility from private to public.

## Are these changes tested?

The existing tests for `BatchPartitioner` cover `partition_iter`
indirectly through the `partition` method, which delegates to it. No
behavioral change was made.

## Are there any user-facing changes?

`BatchPartitioner::partition_iter` is now part of the public API. This
is a purely additive change with no breaking impact.

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Author: hcrosse

datafusion/physical-plan/src/repartition/mod.rs

@@ -545,12 +545,21 @@ impl BatchPartitioner {
         })
     }
 
-    /// Actual implementation of [`partition`](Self::partition).
+    /// Returns an iterator of `(partition_index, RecordBatch)` pairs for the given batch.
     ///
-    /// The reason this was pulled out is that we need to have a variant of `partition` that works w/ sync functions,
-    /// and one that works w/ async. Using an iterator as an intermediate representation was the best way to achieve
-    /// this (so we don't need to clone the entire implementation).
-    fn partition_iter(
+    /// This is useful for async consumers that want to separate CPU-bound partitioning
+    /// from I/O. For example, you can iterate results on the async side and send them
+    /// through a channel, while performing file I/O on a blocking task:
+    ///
+    /// ```ignore
+    /// for result in partitioner.partition_iter(batch)? {
+    ///     let (partition, batch) = result?;
+    ///     tx.send((partition, batch)).await?;
+    /// }
+    /// ```
+    ///
+    /// The sync [`partition`](Self::partition) method is implemented on top of this.
+    pub fn partition_iter(
         &mut self,
         batch: RecordBatch,
     ) -> Result<impl Iterator<Item = Result<(usize, RecordBatch)>> + Send + '_> {