Hui SongandClaude Haiku 4.5
5499998506
* fix: AAP-92631 Only defer control nodes in mesh stability gate When the mesh routing table is empty, the stability gate was blocking cleanup of ALL lost instances (both control and execution nodes). However, execution nodes don't need the mesh and can be safely reaped immediately. This fix separates lost instances by node type: - Control nodes: defer cleanup (they need mesh consensus) - Execution/hop nodes: reap immediately (they don't need mesh) The previous behavior caused stale execution node instances to accumulate in the database indefinitely, preventing job cleanup and degrading controller performance. Fixes: AAP-92631 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix: AAP-92631 Handle Kubernetes mesh gate differently from cluster deployments The mesh stability gate checks KnownConnectionCosts to detect if receptor routing is established. In cluster/mesh deployments, empty routing means the mesh is re-establishing after a controller restart, so the gate blocks cleanup of lost instances to prevent cluster fragmentation. However, in Kubernetes (IS_K8S=True), controller pods are stateless and independent — they don't need peer connections. Empty routing is the expected steady state, not a sign of instability. The gate was treating both deployments identically, causing lost instance rows to accumulate indefinitely in Kubernetes. This fix adds an IS_K8S check: when Kubernetes mode is enabled, the gate always passes (returns True), allowing normal cleanup. In cluster mode, the gate continues to check KnownConnectionCosts. Fixes: AAP-92631 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix: [AAP-92631] Return execution/hop nodes when mesh gate blocks cleanup ## Issue When the non-K8s mesh gate blocked cleanup (empty routing table), the code returned an empty list, preventing execution and hop node cleanup. This was unintended—these nodes don't depend on mesh consensus and should be reaped immediately. ## Fix - Filter lost_instances to return only execution/hop nodes when mesh gate blocks - Control nodes (which require mesh consensus) continue to be deferred - K8s deployments bypass the gate entirely (empty routing is steady state) ## Test Coverage Added three tests per reviewer request: 1. test_kubernetes_mesh_gate_allows_empty_routing (parametrized) - Unit test verifying gate logic with different mesh_status shapes - Confirms _mesh_all_ready_nodes_visible() returns True in K8s 2. test_non_kubernetes_mesh_gate_blocks_empty_routing - Unit test confirming non-K8s gate blocks with empty routing 3. test_mesh_gate_defers_control_but_cleans_execution_hop - E2E integration test with all three node types - Verifies exec/hop nodes reach cleanup while control nodes defer - Matches production failure scenario from baremetal-a cluster All 18 heartbeat/mesh tests pass. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Caution
The last release of this repository was released on Jul 2, 2024. Releases of this project are now paused during a large scale refactoring. For more information, follow the Forum and - more specifically - see the various communications on the matter:
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
To install AWX, please view the Install guide.
To learn more about using AWX, view the AWX docs site.
The AWX Project Frequently Asked Questions can be found here.
The AWX logos and branding assets are covered by our trademark guidelines.
Contributing
- Refer to the Contributing guide to get started developing, testing, and building AWX.
- All code submissions are made through pull requests against the
develbranch. - All contributors must use
git commit --signofffor any commit to be merged and agree that usage of--signoffconstitutes agreement with the terms of DCO 1.1 - Take care to make sure no merge commits are in the submission, and use
git rebasevs.git mergefor this reason. - If submitting a large code change, it's a good idea to join discuss via the Ansible Forum. This helps everyone know what's going on, and it also helps save time and effort if the community decides some changes are needed.
Reporting Issues
If you're experiencing a problem that you feel is a bug in AWX or have ideas for improving AWX, we encourage you to open an issue and share your feedback. But before opening a new issue, we ask that you please take a look at our Issues guide.
Code of Conduct
We require all of our community members and contributors to adhere to the Ansible code of conduct. If you have questions or need assistance, please reach out to our community team at codeofconduct@ansible.com
Get Involved
We welcome your feedback and ideas via the Ansible Forum.
For a full list of all the ways to talk with the Ansible Community, see the AWX Communication guide.