Commit cb694bf
fix: admit zero-bytes-saved samples so skip fires for filter==projection
Q18 (and several other TPC-DS regressions) had a post-scan
customer_demographics filter — `Optional(DynamicFilter)` on the single
projected column `cd_demo_sk` — that burned 90 ms of CPU per scan but
could never be skipped. The filter was correctly placed at PostScan
(projection ⊆ filter columns ⇒ byte_ratio = 1.0 > threshold) but the
mid-stream skip path never fired.
Root cause: `SelectivityStats::update` only incremented `sample_count`
when `batch_bytes > 0`. When the projection is a subset of the filter
columns, `other_bytes_per_row = 0` and therefore `batch_bytes = 0` on
every call, so the Welford counter stayed at zero, the CI upper bound
stayed `None`, and the skip check short-circuited. Meanwhile the filter
kept running per batch.
Admit samples with `batch_bytes = 0`. The recorded effectiveness for
those samples is legitimately zero (no late-materialization payoff), so
the CI upper bound converges on zero after a few batches and the skip
flag flips for optional filters — exactly what we want: CPU spent, no
byte savings, optional ⇒ drop.
Local TPC-DS sf1 (M-series, pushdown=on) vs main pushdown=off:
| Query | Main(off) | Before | After |
|-------|-----------|--------|-------|
| Q18 | 99 | 182 | 118 |
| Q67 | 312 | 503 | 346 |
| Q26 | 80 | 151 | 94 |
| Q85 | 149 | 246 | 157 |
| Q91 | 64 | 108 | 58 |
| Q53 | 103 | 144 | 99 |
| Q63 | 103 | 148 | 99 |
| Q13 | 399 | 558 | 376 |
| Q72 | 619 | 489 | 277 |
| Q24 | 379 | 70 | 70 |
| Q64 | 28213 | -- | 519 |
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent fee428c commit cb694bf
1 file changed
Lines changed: 13 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
149 | 149 | | |
150 | 150 | | |
151 | 151 | | |
152 | | - | |
153 | | - | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
154 | 160 | | |
155 | | - | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
156 | 166 | | |
157 | 167 | | |
158 | 168 | | |
| |||
0 commit comments