Skip to content

Commit 1416ed4

Browse files
Add benchmarks for Parquet struct leaf-level projection pruning (#21180)
## Rationale for this change This PR adds benchmarks that measure the perf of projecting individual fields from struct columns in Parquet files. #20925 introduced leaf-level projection masking so that `select s['small_int']` on a struct with large string fields only reads the small integer leaf, skipping the expensive string decoding entirely 3 dataset shapes are coevered, each with ~262K rows of 8kb string payloads: a narrow struct (2 leaves), a wide struct (5 leaves), and a nested struct. Each shape benchmarks full-struct reads against single-field projections
1 parent ba399a8 commit 1416ed4

2 files changed

Lines changed: 408 additions & 0 deletions

File tree

datafusion/core/Cargo.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,11 @@ harness = false
247247
name = "parquet_struct_query"
248248
required-features = ["parquet"]
249249

250+
[[bench]]
251+
harness = false
252+
name = "parquet_struct_projection"
253+
required-features = ["parquet"]
254+
250255
[[bench]]
251256
harness = false
252257
name = "range_and_generate_series"

0 commit comments

Comments
 (0)