Commit Graph
1 Commits
Author SHA1 Message Date
fcc1aa56d1 Balance controller node selection during capacity ties (#16417)
* Balance controller node selection during capacity ties

When multiple control-plane instances have equal remaining capacity,
the instance selection algorithm always picks the first instance in
iteration order. This causes burst workloads to concentrate all job
management on a single controller pod while others remain idle.

Add a tie-breaking criterion: when would_be_remaining capacity is
equal, prefer the instance with fewer jobs_running. This distributes
the control overhead (event processing, callbacks, output streaming)
more evenly across available controller pods during burst scenarios.

The change is backwards-compatible: when capacity clearly differs,
the existing "most remaining capacity" logic dominates unchanged.

Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com>
Made-with: Cursor

* Track jobs_running on controller node for container-group tasks

Address review feedback: container-group tasks only set controller_node
(execution_node is empty), so the previous consume_capacity call on the
control path never incremented jobs_running. This meant the tie-breaker
could not differentiate between nodes.

Now pass job_impact=True on the control path when the controller is not
also the execution node (avoiding double-counting for hybrid nodes).

Also improve test coverage:
- Fix test case to prove capacity dominates over jobs_running
- Add dedicated tests for container-group controller distribution
- Add test verifying equal-capacity tie-breaking behaviour

Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com>
Made-with: Cursor

* Add docstrings to consume_capacity and fit_task_to_most_remaining_capacity_instance

Address CodeRabbit docstring coverage warning by documenting the two
methods modified in this PR. No functional changes.

Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Liam Allen <lallen@redhat.com>
2026-07-21 15:46:47 +01:00