Commit e5966b5
authored
fix: linearized operands in physical binaryexpr protobuf to avoid recursion limit (#21031)
## Which issue does this PR close?
- part of #18602.
## Rationale for this change
When a SQL query contains many filter conditions (e.g., 40+ `AND`/`OR`
clauses in a `WHERE`), serializing the physical plan to protobuf and
deserializing it fails with `DecodeError: recursion limit reached`.
[This is because prost has a default recursion limit of
100](https://docs.rs/prost/latest/src/prost/lib.rs.html#30), and each
`BinaryExpr` nesting consumes ~2 levels of protobuf recursion depth, so
a chain of ~50 AND conditions exceeds the limit.
## What changes are included in this PR?
Applied the same **linearization** approach that [logical expressions
already
use](https://github.com/apache/datafusion/blob/b6b542e87b84f4744096106bea0de755b2e70cc5/datafusion/proto/src/logical_plan/to_proto.rs#L228-L256)
that convert a left-deep tree to linearization list. Instead of encoding
a chain of same-operator binary expressions as a deeply nested tree, we
flatten it into a flat `operands` list:
**Before (nested, O(n) recursion depth):**
```
BinaryExpr(AND) { l: BinaryExpr(AND) { l: BinaryExpr(AND) { l: a, r: b }, r: c }, r: d }
```
**After (flat, O(1) recursion depth for the chain):**
```
BinaryExpr(AND) { operands: [a, b, c, d] }
```
## Are these changes tested?
yes, add some test case
## Are there any user-facing changes?1 parent 5a427cb commit e5966b5
File tree
6 files changed
+394
-26
lines changed- datafusion/proto
- proto
- src
- generated
- physical_plan
- tests/cases
6 files changed
+394
-26
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
981 | 981 | | |
982 | 982 | | |
983 | 983 | | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
984 | 987 | | |
985 | 988 | | |
986 | 989 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
273 | 273 | | |
274 | 274 | | |
275 | 275 | | |
276 | | - | |
277 | | - | |
278 | | - | |
279 | | - | |
280 | | - | |
281 | | - | |
282 | | - | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
288 | | - | |
289 | | - | |
290 | | - | |
291 | | - | |
292 | | - | |
293 | | - | |
294 | | - | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
295 | 329 | | |
296 | 330 | | |
297 | 331 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
306 | 306 | | |
307 | 307 | | |
308 | 308 | | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
309 | 335 | | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
317 | 340 | | |
318 | 341 | | |
319 | 342 | | |
| |||
0 commit comments