Skip to content

Commit df1d264

Browse files
committed
docs: update data sources documentation to include support for Python objects implementing __arrow_c_stream__ protocol
1 parent bdbf8c1 commit df1d264

1 file changed

Lines changed: 6 additions & 1 deletion

File tree

docs/source/user-guide/data-sources.rst

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,12 @@ as Delta Lake. This will require a recent version of
158158
df = ctx.table("my_delta_table")
159159
df.show()
160160
161-
On older versions of ``deltalake`` (prior to 0.22) you can use the
161+
Any Python object that implements the
162+
``__arrow_c_stream__`` protocol can be registered with
163+
``register_dataset``. This includes scanners from libraries such as
164+
``nanoarrow``, ``Polars``, or ``DuckDB``.
165+
166+
On older versions of ``deltalake`` (prior to 0.22) you can use the
162167
`Arrow DataSet <https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html>`_
163168
interface to import to DataFusion, but this does not support features such as filter push down
164169
which can lead to a significant performance difference.

0 commit comments

Comments
 (0)