[DataFusion] Unify partition leader status and leader query protocol#4643
[DataFusion] Unify partition leader status and leader query protocol#4643AhmedSoliman wants to merge 2 commits intomainfrom
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df46191bd5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if self | ||
| .notified_head | ||
| .is_some_and(|notified| notified == vqueue) | ||
| { | ||
| return Poll::Pending; |
There was a problem hiding this comment.
poll_head now returns Pending whenever notified_head == vqueue, but it never verifies that this head can still be consumed by the scheduler. If a throttled queue is dropped from scheduler state without calling InvokerThrottlingLimiter::remove_from_waiters (for example via DRRScheduler::mark_vqueue_as_dormant on pause/dormancy in crates/vqueues/src/scheduler/drr.rs:204-209 and :268-276), the stale head remains at the front and this branch keeps returning Pending forever, so waiters behind it are never woken and reserved throttling tokens can remain stuck.
Useful? React with 👍 / 👎.
Test Results 7 files 7 suites 2m 37s ⏱️ Results for commit e864d2d. ♻️ This comment has been updated with latest results. |
I'm not loving this, but it works well enough to unblock.
1 participant
Stack created with Sapling. Best reviewed with ReviewStack.