add first draft of wikipedia article

[gene-bordegaray]

Which issue does this PR close?

• Closes Write a wikipedia article for Apache DataFusion #21076.

1. Go to this page
2. Click Edit source
3. Paste dev/wiki/apache-datafusion.wikitext
4. Click Show preview

[alamb]

Thank you @gene-bordegaray -- this looks great. I left some suggestions on how to make some of this language tighter.

Maybe we can wait a few days more and then submit to the wikipedia editors 🤔

[gene-bordegaray]

also a side note. I wanted to add the DF logo but my account needs to be verified (I think will be in a day or two) 😅

[NNhanptnk]

It provides [[SQL]] and DataFrame interfaces for analytical query execution and is designed to be used as a library by developers building databases, query engines, and analytical tools, rather than as a standalone database server.

I think we can make this a bit better in the sense of introducing DataFusion and its uniqueness. Here's what I think :

Often described as the "LLVM for Databases," [Source 1] Apache DataFusion is a modular, Arrow-native query engine library designed for embedding into custom systems rather than operating as a monolithic standalone server [Source 2 and 3]. This high-performance Rust framework provides a composable foundation, allowing developers to precisely extend query planning and vectorized execution to meet unique architectural requirements. [Source 2 and 3]

Source 1 : https://midas.bu.edu/assets/slides/andrew_lamb_slides.pdf (cc @alamb )

Source 2 and 3 (this is the first two reference) : {{cite journal |last1=Lamb |first1=Andrew |last2=Shen |first2=Yijie |last3=Heres |first3=Daniel |last4=Chakraborty |first4=Jayjeet |last5=Kabak |first5=Mehmet Ozan |last6=Hsieh |first6=Liang-Chi |last7=Sun |first7=Chao |title=Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine |journal=Proceedings of the 2024 International Conference on Management of Data |year=2024 |doi=10.1145/3626246.3653368}}{{cite web |title=Introduction |url=https://datafusion.apache.org/user-guide/introduction.html |website=Apache DataFusion |publisher=Apache Software Foundation |access-date=2026-03-22}}

[gene-bordegaray]

I don't know if we should add the "LLVM for databases". Mostly because the primary source for it is from not the strongest source (slide show) and doesnt appear in the other sources like the SIGMOD paper or other coverage.

I was reviwing the Wikipedia guidelines and they advise anything promotional unless well-cited which this may get flagged for.

https://en.wikipedia.org/wiki/Wikipedia:Verifiability

[NNhanptnk]

I believe the project will continue to grow so we can write at the end :

Apache DataFusion now sees over one million monthly downloads. [cite crate.io source]

source : https://crates.io/search?q=datafusion

[alamb]

We could also say "as of March 2026, DataFusion saw one million monthly downloads" if we wanted to ensure the sstatement remained accurate

[gene-bordegaray]

Ya I think this is great, definitely with the third party source 👍

[ntjohnson1]

I got sick so fell off looking at this. I think this looks great for a first pass and we should push to wikipedia to see what the reviewers say. One note that I don't know if I have time for is that this seems to slightly over emphasize the extensibility perspective.

On a quick read through I would assume this was only for building the infrastructure and could easily miss the SQL/dataframe API bits. At rerun I use datafusion (specifically datafusion-python) quite heavily but don't really know the details about our table provider (since other people build that bit). I suspect our customers will also hit this page since we generate examples for the DataFrame API in python (and are generating more SQL examples). https://rerun.io/docs/howto/query-and-transform/dataframe_operations

Mostly just food for thought that there might be two distinct audiences interested in this page. People who build on datafusion and those who build data products using datafusion top level APIs. (I still think landing the page first makes sense then I or someone else can potentially try to add a section for more SQL/DataFrame API details)

[ntjohnson1]

There isn't a formal page on Dataframes but there is a stub that refers to Spark, pandas, etc. After this page lands we should add a pointer to it from there. https://en.wikipedia.org/wiki/Dataframe

[ntjohnson1]

NIT: I think extensible analytical query engine is clearer than embeddable analytical query engine. Extensible is what is listed on the landing page for datafusion on apache.org

[ntjohnson1]

I'm biased to want to include a link to rerun but we don't have a blog post calling out DataFusion even though it is all over our repo. Will work on that.

[alamb]

Will work on that.

that is the ideal answer!

[AndreaBozzo]

This Is nice Indeed, the expansions on SQL and dataframes can be added later and would be very useful

[alamb]

@gene-bordegaray should we move this to the wikipedia site now (after addressing other comments)?

[gene-bordegaray]

@gene-bordegaray should we move this to the wikipedia site now (after addressing other comments)?

Ya I can address comments tonight and tmrw and take some time this weekend to the wikipedia site 👍

[gene-bordegaray]

I got sick so fell off looking at this. I think this looks great for a first pass and we should push to wikipedia to see what the reviewers say. One note that I don't know if I have time for is that this seems to slightly over emphasize the extensibility perspective.

On a quick read through I would assume this was only for building the infrastructure and could easily miss the SQL/dataframe API bits. At rerun I use datafusion (specifically datafusion-python) quite heavily but don't really know the details about our table provider (since other people build that bit). I suspect our customers will also hit this page since we generate examples for the DataFrame API in python (and are generating more SQL examples). https://rerun.io/docs/howto/query-and-transform/dataframe_operations

Mostly just food for thought that there might be two distinct audiences interested in this page. People who build on datafusion and those who build data products using datafusion top level APIs. (I still think landing the page first makes sense then I or someone else can potentially try to add a section for more SQL/DataFrame API details)

Ya I definitely think it leans toward teh infrastructure sie of things as it stands ( this is what I used DF for so guilty for that 😅 ). I agree that getting something up and someone with more expereinces using DF for the Dataframes / SQL aspects can step in and add what they see fit.

[gene-bordegaray]

I have chnaged the wording to be extensible and left the other comments as be to be addressed after we are first published / reviewed. I will leave this up for a day or so and then submit for review.

Lmk if this is alright with everyone 😄

cc: @alamb @AndreaBozzo @ntjohnson1 @NNhanptnk

[gene-bordegaray]

Also if anyone has a confirmed account and sees this, would greatly appreciate if you could upload the DataFusion logo to Wikipedia, I am not allowed to:

or whichever you think is appropriate 👍

[ntjohnson1]

I have chnaged the wording to be extensible and left the other comments as be to be addressed after we are first published / reviewed. I will leave this up for a day or so and then submit for review.

Lmk if this is alright with everyone 😄

cc: @alamb @AndreaBozzo @ntjohnson1 @NNhanptnk

Works for me! I'd vote ship it now and we/anyone else can edit further after it goes live. Thanks for pushing this forward.

[gene-bordegaray]

went ahead and submitted a draft for review: https://en.wikipedia.org/wiki/Draft:Apache_DataFusion

I can keep an eye on this

[alamb]

went ahead and submitted a draft for review: https://en.wikipedia.org/wiki/Draft:Apache_DataFusion

I can keep an eye on this

4k pending reviews. And I thought the DF review queue was bad 😆