We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent 2bcbce6 commit 4d6f76cCopy full SHA for 4d6f76c
1 file changed
docs/source/user-guide/dataframe/index.rst
@@ -227,6 +227,20 @@ partition:
227
for batch in stream:
228
... # each stream yields RecordBatches
229
230
+To process partitions concurrently, first collect the streams into a list
231
+and then poll each one in a separate ``asyncio`` task:
232
+
233
+.. code-block:: python
234
235
+ import asyncio
236
237
+ async def consume(stream):
238
+ async for batch in stream:
239
+ ...
240
241
+ streams = list(df.execute_stream_partitioned())
242
+ await asyncio.gather(*(consume(s) for s in streams))
243
244
See :doc:`../io/arrow` for additional details on the Arrow interface.
245
246
HTML Rendering
0 commit comments