File tree Expand file tree Collapse file tree
docs/source/user-guide/dataframe Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -145,10 +145,31 @@ To materialize the results of your DataFrame operations:
145145
146146 # Display results
147147 df.show() # Print tabular format to console
148-
148+
149149 # Count rows
150150 count = df.count()
151151
152+ PyArrow Streaming
153+ -----------------
154+
155+ DataFusion DataFrames implement the ``__arrow_c_stream__ `` protocol, enabling
156+ zero-copy streaming into libraries like `PyArrow <https://arrow.apache.org/ >`_.
157+ Earlier versions eagerly converted the entire DataFrame when exporting to
158+ PyArrow, which could exhaust memory on large datasets. With streaming, batches
159+ are produced lazily so you can process arbitrarily large results without
160+ out-of-memory errors.
161+
162+ .. code-block :: python
163+
164+ import pyarrow as pa
165+
166+ # Create a PyArrow RecordBatchReader without materializing all batches
167+ reader = pa.RecordBatchReader._import_from_c(df.__arrow_c_stream__())
168+ for batch in reader:
169+ ... # process each batch as it is produced
170+
171+ See :doc: `../io/arrow ` for additional details on the Arrow interface.
172+
152173HTML Rendering
153174--------------
154175
You can’t perform that action at this time.
0 commit comments