fix: eliminate race condition in gateway user sync
poll_until confirms the user exists in controller after gateway
creation, but a second GET call could return empty results under
load. Return the user directly from poll_until instead.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: AAP-92631 Only defer control nodes in mesh stability gate
When the mesh routing table is empty, the stability gate was blocking
cleanup of ALL lost instances (both control and execution nodes). However,
execution nodes don't need the mesh and can be safely reaped immediately.
This fix separates lost instances by node type:
- Control nodes: defer cleanup (they need mesh consensus)
- Execution/hop nodes: reap immediately (they don't need mesh)
The previous behavior caused stale execution node instances to accumulate
in the database indefinitely, preventing job cleanup and degrading
controller performance.
Fixes: AAP-92631
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix: AAP-92631 Handle Kubernetes mesh gate differently from cluster deployments
The mesh stability gate checks KnownConnectionCosts to detect if receptor
routing is established. In cluster/mesh deployments, empty routing means
the mesh is re-establishing after a controller restart, so the gate blocks
cleanup of lost instances to prevent cluster fragmentation.
However, in Kubernetes (IS_K8S=True), controller pods are stateless and
independent — they don't need peer connections. Empty routing is the
expected steady state, not a sign of instability. The gate was treating
both deployments identically, causing lost instance rows to accumulate
indefinitely in Kubernetes.
This fix adds an IS_K8S check: when Kubernetes mode is enabled, the gate
always passes (returns True), allowing normal cleanup. In cluster mode, the
gate continues to check KnownConnectionCosts.
Fixes: AAP-92631
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix: [AAP-92631] Return execution/hop nodes when mesh gate blocks cleanup
## Issue
When the non-K8s mesh gate blocked cleanup (empty routing table), the code
returned an empty list, preventing execution and hop node cleanup. This was
unintended—these nodes don't depend on mesh consensus and should be reaped
immediately.
## Fix
- Filter lost_instances to return only execution/hop nodes when mesh gate blocks
- Control nodes (which require mesh consensus) continue to be deferred
- K8s deployments bypass the gate entirely (empty routing is steady state)
## Test Coverage
Added three tests per reviewer request:
1. test_kubernetes_mesh_gate_allows_empty_routing (parametrized)
- Unit test verifying gate logic with different mesh_status shapes
- Confirms _mesh_all_ready_nodes_visible() returns True in K8s
2. test_non_kubernetes_mesh_gate_blocks_empty_routing
- Unit test confirming non-K8s gate blocks with empty routing
3. test_mesh_gate_defers_control_but_cleans_execution_hop
- E2E integration test with all three node types
- Verifies exec/hop nodes reach cleanup while control nodes defer
- Matches production failure scenario from baremetal-a cluster
All 18 heartbeat/mesh tests pass.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Cryptography 50.0.0 added a self._aes attribute to the Fernet class
constructor. Since Fernet256 overrides __init__ without calling
super().__init__(), it needs to manually set this attribute.
Without this attribute, any code using Fernet256 (encryption/decryption
of credentials, secrets, etc.) fails with:
AttributeError: 'Fernet256' object has no attribute '_aes'
This fix:
- Imports algorithms from cryptography.hazmat.primitives.ciphers
- Sets self._aes = algorithms.AES(self._encryption_key) in __init__
The fix is backward compatible with cryptography < 50.0.0 since the
_aes attribute wasn't used by Fernet's methods in older versions.
Fixes: 277 test failures in AWX test suite when using cryptography 50.x
Related: pyca/cryptography#11234 (cryptography 50.0.0 changes)
Co-authored-by: Stevenson Michel <iamstevensonmichel@outlook.com>
* Fix awx-operator molecule test timeout being too tight
The inline `timeout 20m` wrapping the molecule test run in the
"Run test deployment with awx-operator" step (separate from the
never-binding outer `timeout-minutes: 60`) isn't currently causing
failures on this repo's runners (recent successful devel runs take
9.4-10.8m), but it's the same line that downstream (ansible/tower)
had to bump to `timeout 54m` after its runs started legitimately
taking 15-19.5m and getting killed with as little as ~30s of margin
under 20m.
Since downstream syncs this workflow from here, leaving it at 20m
risks a future sync silently reverting the downstream fix. Raise it
to 54m to match, well ahead of any regression in this repo's own
runtime.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Liam Allen <lallen@redhat.com>
* Raise awx-operator job-level timeout to fit the 54m inner timeout
Job-level timeout-minutes covers the whole job cumulatively
(checkout, image build, the deployment step, log collection,
artifact upload), not just the deployment step. With the inner
`timeout 54m` from the previous commit, a worst-case run (setup of
~6-8m plus the full 54m before the inner timeout fires) totals
~60-62m, which can trip the job-level 60m limit first and hard-cancel
the job before "Collect awx-operator logs on failure" or the
timed_out=true write ever execute - losing exactly the debug output
this timeout-marker pattern exists to preserve.
Raise job-level timeout-minutes to 75 to give headroom past the
worst-case inner timeout. No change to the timed_out=true or
conditional log-collection logic.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Liam Allen <lallen@redhat.com>
---------
Signed-off-by: Liam Allen <lallen@redhat.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* AAP-91428 | feat: subsystem metrics for indirect node counting
Expose Prometheus metrics for indirect node counting at
/api/v2/metrics?subsystemonly=1, following the existing DispatcherMetrics
pattern. This is a prerequisite for perf/scale testing (AAP-57413) before
the feature flag can be removed for GA in 2.7.
Metrics added:
- indirect_node_audit_records_created (counter)
- indirect_node_query_execution_seconds (accumulated float)
- indirect_node_jq_query_errors (counter)
- indirect_node_fallback_cleanup_seconds (latest-value float)
Instrumentation points in host_indirect.py:
- jq query execution timing and error counting in build_indirect_host_data
- audit record count after bulk_create in save_indirect_host_entries_of_job
- fallback cleanup job wall-clock timing
Also registers IndirectCountingMetrics in send_subsystem_metrics for
cross-node metric broadcasting.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* AAP-91428 | fix: SonarQube complaints (coverage + complexity)
* AAP-91428 | fix: use jq_str and fix docstring
* AAP-91428 | test: tests to get coverage up
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Previously the ATF tests pipeline only ran on `/run-atf-tests` comment.
Add on-event and on-target-branch annotations so it also triggers
automatically on pull_request events targeting devel.
Signed-off-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-78694 - Add CleanTextMixin to AWX resource serializers
Applies django-ansible-base's CleanTextMixin to the AWX serializers
listed in AAP-78694: Organization, User, Team, Credential(+Create),
CredentialType, Project, JobTemplate, WorkflowJobTemplate, Inventory,
Host, Group, InventorySource, InstanceGroup, Instance,
ExecutionEnvironment, NotificationTemplate, Schedule, AdHocCommand,
WorkflowJobTemplateNode, WorkflowJobTemplateNodeCreateApproval,
JobLaunch, and WorkflowJobLaunch.
Adds a local PromptFieldCleanTextMixin for serializers whose model
derives from LaunchTimeConfigBase, where scm_branch/limit/job_tags/
skip_tags are NullablePromptPseudoField descriptors invisible to the
mixin's normal model-field auto-discovery.
Per-serializer excluded_fields/name_fields overrides are documented
inline where a field's existing behavior would otherwise conflict
with Tier 1/2 validation (Jinja2-templated fields, secret fields,
and existing colon-containing name/hostname syntax).
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Cover CleanTextMixin gaps in Copy, Bulk, and Job Detail paths
Three write paths were found to bypass the Tier 1/2 validation applied
elsewhere for the same resources:
- CopySerializer (used by the Copy button/endpoint for EE, Project,
Credential, JobTemplate, WorkflowJobTemplate, NotificationTemplate)
accepted a user-supplied name with zero validation. Adds a new
PlainSerializerCleanTextMixin for plain serializers.Serializer
subclasses with no backing model.
- BulkJobNodeSerializer's validate() intentionally skips
LaunchConfigurationBaseSerializer.validate() (it assumes
unified_job_template/inventory/execution_environment are model
instances, but bulk nodes pass raw PKs to avoid a per-node DB
lookup), which meant it also skipped CleanTextMixin. Adds
PromptFieldCleanTextMixin._run_clean_text_validation() so the
Tier 1/2 checks still run despite that skip. BulkJobLaunchSerializer
and WorkflowJobNodeSerializer are also covered.
- JobSerializer (backing the narrow JobDetail PUT/PATCH window while
a job's status is "new") had no coverage at all.
Confirmed live against a running controller that a Copy with a
malicious name previously succeeded (201) where the equivalent PATCH
on the same resource was already rejected (400); both are rejected
consistently after this change.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Fix NotificationTemplateSerializer validate() bypass
validate() returned early without calling super().validate() whenever
notification_configuration was absent from the request (e.g. a PATCH
that only changes name/description), which skipped CleanTextMixin
entirely on that path. Same bug class as the BulkJobNodeSerializer
MRO-skip already fixed, caught by code review.
Confirmed live: a PATCH with only a malicious name (no
notification_configuration) previously succeeded; now rejected with
400, while a legitimate name-only PATCH still succeeds.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Add CleanTextMixin to the previously out-of-scope API-only serializers
LabelSerializer, CredentialInputSourceSerializer, and
WorkflowApprovalTemplateSerializer were originally scoped out ("no UI
exposure") but are now in scope per updated ticket guidance.
WorkflowApprovalTemplateSerializer backs the update path for the same
model WorkflowJobTemplateNodeCreateApprovalSerializer already protects
on create -- without this, an approval name rejected at creation could
be rewritten to anything via PATCH.
Confirmed live: Label create and WorkflowApprovalTemplate update both
reject a malicious name (400) and accept a valid one.
CredentialInputSourceSerializer verified at the code level only (plain
CharField input_field_name, no custom validators; metadata is a
JSONSchemaField holding vault paths/keys, not secret values) --
setting up a live external-credential-type chain in this environment
wasn't done.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Correct HostSerializer name_fields demotion comment
The comment claimed the Tier 1 -> Tier 2 demotion was needed for
"hostname:port" syntax, but that colon is stripped from attrs['name']
in validate() before it reaches CleanTextMixin -- that case never
needed the demotion. What actually needs it: names with any colon
count other than exactly 1, which _get_host_port_from_name passes
through untouched -- most notably IPv6 addresses, which its own
comment says it explicitly does not parse. Caught by code review.
Confirmed live: an IPv6-style host name (2 colons) is accepted
unmodified (201).
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Validate non-secret notification_configuration keys
Replaced the blanket notification_configuration exclusion with a
precise excluded_json_keys covering only the actual secret fields.
Unlike Credential.inputs (admin-definable custom types with arbitrary
secret fields), notification_type is a fixed models.CharField(choices)
-- every backend class ships in awx/main/notifications/ and is known
at code-authoring time, so the secret key set can be a static list
rather than resolved per-instance.
Enumerated every init_parameters entry with type "password" across
all 8 backends (email/irc/webhook password, slack/pagerduty token,
twilio account_token, grafana grafana_key, awssns
aws_secret_access_key/aws_session_token) and confirmed none of those
names are used for a non-secret value in a different backend.
Previously-unvalidated, UI-exposed fields like webhook url and slack
channels are now covered. Confirmed live: a javascript: webhook url
and a <script>-tagged slack channel are now rejected (400), while a
weird-looking token/password value is still accepted unchanged.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Exclude extra_data on launch-config serializers
extra_data (WorkflowJobTemplateNode/WorkflowJobNode/Schedule) plays
the same role as extra_vars elsewhere and is submitted in the same
two formats -- a dict, or a raw YAML/JSON string (parse_yaml_or_json
handles both). Its model field is JSONBlob, whose get_internal_type()
reports "TextField" for legacy DB-migration reasons, so
CleanTextMixin silently skipped dict payloads (not a str) while
string payloads hit Tier 2's template-injection blocklist and
incorrectly rejected legitimate Jinja variable references -- a real
regression once enforcement is on, caught by code review.
First attempt put the exclusion on the shared
LaunchConfigurationBaseSerializer base and silently did nothing:
CleanTextMixin's own excluded_fields = frozenset() default sits
earlier in the MRO than that base class for every subclass (mixin is
first in the bases tuple), so the base class's value was always
shadowed. Verified with a standalone MRO test before settling on
setting excluded_fields directly on each leaf serializer
(WorkflowJobTemplateNodeSerializer, WorkflowJobNodeSerializer,
ScheduleSerializer), matching every other exclusion in this file.
Confirmed live on both WorkflowJobTemplateNodeSerializer and
ScheduleSerializer: a YAML-string extra_data with a Jinja reference
is accepted (previously 400), while other Tier 1/2 fields on the same
serializers are still rejected as expected.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Validate non-secret Credential.inputs keys dynamically
Unlike NotificationTemplate.notification_configuration (fixed set of
backend classes, static excluded_json_keys works), CredentialType is
admin-definable via a real "Create a new custom credential type"
endpoint, so the set of secret field names in Credential.inputs can't
be known at class-definition time -- it varies per credential's own
type, via that type's own inputs schema ("secret": true flags,
exposed as CredentialType.secret_fields).
validate() resolves the credential_type from attrs (create) or the
existing instance (update) and sets excluded_json_keys as an instance
attribute before chaining into CleanTextMixin.validate(). Setting an
instance attribute rather than a class attribute sidesteps the MRO
trap found while fixing extra_data on launch-config serializers:
CleanTextMixin's own excluded_fields/excluded_json_keys class-level
defaults sit earlier in the MRO than any shared base class further
down the chain, silently shadowing it -- an instance attribute always
wins regardless of MRO position, and correctly threads through
CredentialSerializerCreate and its User/Team/Organization subclasses
for free, since each of their validate() overrides chains through
super() into this one.
Falls back to excluding the whole inputs blob (prior, safe behavior)
if credential_type can't be resolved.
Confirmed live: malicious username (non-secret, including via PATCH
with no credential_type in the payload) is rejected; a dangerous-
looking password value and a real unencrypted SSH private key
(secret fields) are both still accepted unchanged.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* Increase test coverage
* Cover remaining serializer branches for Sonar gate
Line coverage is already over 90%; Sonar's quality gate uses combined
line+branch coverage. These unit tests exercise previously untested
methods so the PR can clear the 90% new-code threshold.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* style(api): format uncovered serializer tests for ruff
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Assisted-by: Cursor Grok 4.6 (SpaceXAI / Cursor)
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(api): set Mock.name as an attribute, not a constructor kwarg
unittest.mock.Mock(name=...) only sets the mock's repr name, so
getattr(obj, 'name') returned another Mock and four coverage tests
failed in api-test.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Assisted-by: Cursor Grok 4.6 (SpaceXAI / Cursor)
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(api): keep a single call inside pytest.raises blocks
Sonar python:S5918 flags exception tests that construct mocks or
serializers inside the raises context.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Assisted-by: Cursor Grok 4.6 (SpaceXAI / Cursor)
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add more test coverage
---------
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The controller_dir property hardcoded /tmp/awx_{id} as the expected
volume mount prefix. On containerized installs the job execution
directory is at /home/ansible/aap/controller/data/job_execution/awx_{id},
causing a RuntimeError when the prefix didn't match.
Changed to exact path component matching on the awx_{id} segment,
which works for both traditional and containerized path layouts
while preventing false matches (e.g. awx_1 matching awx_10).
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After creating a user on the gateway API, the code immediately queries
the controller API to find the synced user. On containerized installs
(2.6+), the gateway-to-controller sync may not have completed yet,
causing an IndexError when .results.pop() is called on an empty list.
This affected ~32 tests across the tower-qa legacy suite on every
containerized pipeline run (2.6, 2.7, 2.8).
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AAP-90120: add _mesh_all_ready_nodes_visible() to block peer-judgment
while KnownConnectionCosts is empty after a controller restart. Gate
fails open when mesh_status is unavailable. Eliminates double receptor
status call per heartbeat.
AAP-90121: wrap each instance's reap+mark_offline block in
advisory_lock('task_manager_lock', wait=False) so the task manager
cannot dispatch to a node being concurrently marked offline. Extracts
_reap_and_mark_lost_instance() helper.
Tests: rewrite new test classes to use real Django DB (functional/)
instead of MagicMock objects.
---------
Signed-off-by: Hui Song <hsong@redhat.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* Add PostgreSQL utility functions for metrics-utility
Creates two plpgsql functions used by metrics-utility when querying the
AWX controller database via a read-only connection:
- metrics_utility_parse_yaml_field: extracts a named field from YAML-encoded text
- metrics_utility_is_valid_json: validates whether a text value is valid JSON
GRANT EXECUTE TO PUBLIC so any database user (including ms_awx_readonly)
can call the functions without requiring an explicit per-user grant.
The migration is a no-op on SQLite so unit and functional tests are unaffected.
Assisted-by: Claude Code (claude-sonnet-4-6)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ciaran Shiels <cas.ireland@Gmail.com>
* Fix regex injection and broad exception in metrics utility migration
- Escape `field` parameter before interpolating into regex pattern using
regexp_replace with the whole-match `&` reference rather than bare
concatenation; prevents regex metacharacters in the field name from
altering match semantics
- Narrow `EXCEPTION WHEN others` to `WHEN invalid_text_representation`
(SQLSTATE 22P02) in metrics_utility_is_valid_json so only the expected
JSON-cast failure is swallowed and unexpected runtime errors propagate
Assisted-by: Claude Code (claude-sonnet-4-6)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ciaran Shiels <cshiels@redhat.com>
* Renumber migration 0206 to 0210 to avoid conflict
Re-parent create_metrics_utility_functions migration from 0205 to 0209
after other migrations (0206–0209) merged to devel since branch creation.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Renumber migration 0210 to 0211 to avoid conflict
0210_remove_host_last_job_fields arrived on devel after the last rebase,
re-parent create_metrics_utility_functions to depend on it instead.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Signed-off-by: Ciaran Shiels <cas.ireland@Gmail.com>
Signed-off-by: Ciaran Shiels <cshiels@redhat.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* [AAP-89071] Add control plane EE unit/functional tests
Add tests for EE resolution, constructed inventory model,
and ansible-sign signature validation tag injection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [AAP-89071] Fix control plane EE tests
- Test resolve_execution_environment on ProjectUpdate/InventoryUpdate
(job objects) instead of Project/InventorySource (templates), matching
the actual code path in BaseTask.run()
- Consolidate inventory EE resolution into single parametrized test
- Remove trivial kind-check test
- Fix OrderedManyToManyField.add() to add one inventory at a time
- Add asymmetric host counts for better aggregation validation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [AAP-89071] Fix InventoryUpdate missing inventory relation
Add inventory=inv to InventoryUpdate.objects.create() so
resolve_execution_environment() can access self.inventory.kind.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
In particular, this brings two important changes:
- rather than opening a new PR every time it runs, it uses a stable
branch name updating its content instead; this means that at most
there is one open PR per target branch
- a new secret "OPENAPI_SPEC_PR_REVIEW_TEAM" is introduced to specify
the name of the github team to notify when creating new PRs; this
helps to make sure that PRs are reviewed
* Remove dead Host.last_job and Host.last_job_host_summary FK fields
PR #16332 replaced these denormalized ForeignKey columns with dynamic
derivation via Host.latest_summary, but left the columns in place.
Nothing writes to or reads from them anymore — drop the dead weight.
- Remove last_job and last_job_host_summary ForeignKey fields from Host
- Convert HostSerializer to use SerializerMethodField instead of FK-backed fields
- Remove redundant JobHostSummary.latest_for_host/latest_job_for_host classmethods
- Add migration 0206 to drop the columns
- Update tests to use Host.latest_summary instead of removed fields/methods
* Remove dead last_job_host_summary_id UPDATE from cleanup_jobs
The Host.last_job_host_summary FK was removed in the prior commit,
so the raw SQL UPDATE that NULLed it before deleting JHS rows now
references a non-existent column and crashes.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The GitHub Actions runner image update reduced available disk space,
causing the awx-operator molecule test to hang and timeout waiting for
pods that can't start due to disk pressure. Prune BuildKit cache and
dangling images after the AWX image build to reclaim space.
Same fix as ansible/tower PRs #7905 (stable-2.6), #7926 (stable-2.7),
and #7927 (release_4.6).
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Added null guard and restructured transactions
* Flag names that are None and add logging warnings for all modules
* Add test for job-deleted-during-transaction race condition
Covers the Job.DoesNotExist handler inside the transaction.atomic()
block to satisfy SonarQube's 90% new code coverage gate.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-87599 — Re-parent 0206 and update 0207 as merge point for 2.6 backport
Re-parent 0206_jobhostsummary_host_id_idx to depend on 0203 instead
of 0205, and add 0205 as a second dependency on 0207 to converge the
migration graph.
This follows the gateway backport migration pattern (AAP-80663) so that
the same 0206 migration file can be backported to stable-2.6 (where
0204/0205 do not exist) without breaking upgrade paths.
* AAP-87599 — Move merge point to 0209, restore 0207 to original
Reverts the 0205 dependency addition on 0207_alter_skip_tags and
instead adds 0209_merge_0205_0206 at the end of the chain. This
keeps 0207 pristine so it can be backported to stable-2.7 in the
future without a naming conflict.
Migration graph after this change:
0203 → 0204 → 0205 ──────────────────────────────────┐
\ ↓
→ 0206 → 0207_skip_tags → 0208_sys_auditor → 0209_merge
* AAP-87599 — Move merge point to 0207 for devel/2.7 parity
Restructure the migration graph so devel matches stable-2.7 through
the merge point:
- 0207_merge_0205_0206 is now identical on both branches (depends on
0205 + 0206, empty operations)
- Renumber 0207_alter_skip_tags → 0208, 0208_fix_system_auditor → 0209
- Both depend linearly from the merge point forward
Graph:
0205 ──────────────┐
\ ↓
0206 → 0207_merge → 0208_skip_tags → 0209_sys_auditor
* Fix test references after migration rename 0208→0209
Update test_migrations.py to reference 0209_fix_system_auditor_migration
after the merge point restructure moved it from 0208 to 0209.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Replace ResourceMixin/accessible_objects/accessible_pk_qs with DAB RBAC access_qs/access_ids_qs
Remove the ResourceMixin class and migrate all callers to the new
DAB RBAC queryset API: access_qs replaces accessible_objects and
access_ids_qs replaces accessible_pk_qs, using permission strings
(view, change, use, etc.) instead of role field names.
Also updates prefetch_page_capabilities to use permission-based
capability definitions instead of role field references.
Restore UNION pattern in TeamAccess.filtered_queryset
The access_qs migration accidentally collapsed the UNION into an OR,
which can be 48x slower at scale (AAP-83319).
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Unify activity stream format for role assignments across old and new APIs
The old API (/api/v2/users/<pk>/roles/) and new API
(/api/v2/role_user_assignments/) were producing different activity stream
entries for the same logical action. The old API created entries via the
Role.members m2m_changed signal in the legacy format, while the new API
used the new-format handler added in AAP-82996.
This change makes the old API produce the same new-format entry by:
- Skipping the legacy activity_stream_associate for Role_members changes
- Letting the mirrored RoleUserAssignment creation fire the new-format
record_role_assignment_activity_stream handler instead
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Record DAB RBAC role assignment events in the activity stream
* The steam message was showing incorrectly, it was inverted based on the legacy setup
* object_relationship_type was missing, this is set n the legacy handler and emit_activity_stream_change emits it.
* Skip cascade deletes from non-assignment origins, same reasoning as sync_assignments_to_old_rbac_delete
The EE sidecar container creates /etc/receptor/receptor.conf on pod
startup. If the task container's dispatcherd reaches the heartbeat
before the config exists, the instance rejoins the cluster at full
capacity with no receptor — causing fatal job failures with no
recovery path.
Three layers of defense:
1. dispatcherd management command: poll for the receptor config file
before entering the main loop (10s timeout, exits with error to
let k8s restart the container).
2. Heartbeat: if get_receptor_ctl() raises FileNotFoundError (config
or socket missing), mark the instance offline so no jobs are
dispatched to it. Previously this was silently ignored and the
instance kept full capacity.
3. should_update_config(): catch FileNotFoundError and return True so
write_receptor_config() can create the file on first boot instead
of crashing.
Prior art: #12698 (closed without merge) took a broader approach with
custom exception types. See PR comment for overlap analysis.
Closes: AAP-89010
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Migrate Python linting and formatting toolchain from Black (formatter)
and Flake8 (linter) to Ruff, which provides both in a single tool.
- pyproject.toml: remove [tool.black], add [tool.ruff] config sections
- tox.ini: replace black+flake8 deps/commands with ruff in linters env
- awxkit/tox.ini: replace flake8 with ruff in lint env
- Makefile: replace `black` target with `format` + `lint`, keep `black`
as legacy alias, update `check` and `api-lint`
- pre-commit.sh: use `ruff format --check`, honor AWX_IGNORE_RUFF
- requirements_dev.txt: swap black/flake8 for ruff
- Dockerfile.j2: install ruff instead of black
- CONTRIBUTING.md: update tool references
- Fix noqa comments to use # separator instead of ; (ruff requirement)
- Add noqa: F841 for false positive in awxkit/cli/__init__.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use shared lock for project copy when no sync is needed
When a job template runs and no project sync is required (the common
case for slice jobs), multiple jobs were serializing behind an exclusive
lock even though they were only reading the shared project directory to
copy it.
Introduce read/write lock semantics using fcntl LOCK_SH/LOCK_EX:
- acquire_lock gains an exclusive= parameter (default True, preserving
existing behavior for all callers that don't pass it)
- sync_and_copy always acquires LOCK_SH first, then refreshes DB state
and calls get_sync_needs() under the lock — preserving the invariant
that the sync check always happens under a lock (no TOCTOU window)
- If sync is needed, LOCK_SH is released and LOCK_EX is acquired before
proceeding; sync_and_copy_without_lock re-checks internally
- project.refresh_from_db() is guarded by project.pk to handle unsaved
model instances used in unit tests
This allows slice jobs backed by the same project to copy the project
tree concurrently instead of sequentially. Measured improvement on a
200MB/54k-file project with 8 slice jobs: 140.5s total lock wait → 0s,
wall time 42.6s → 16.9s (2.5x).
Assisted-by: Claude Code via Google Vertex AI <cmeyers+claude@redhat.com>
* AAP-87084 — Revert UnifiedJob queryset to DAB APIs, scope actor role filter
The previous patch (7242fe89) inlined all DAB RBAC internals into
UnifiedJobAccess.filtered_queryset to pre-compute role IDs and avoid
repeated subqueries. While it fixed the UNION regression from c84575c,
it created 70 lines of duplicated logic inconsistent with the
UnifiedJobTemplate path and fetched ALL user roles regardless of
relevance.
This commit:
- Reverts filtered_queryset to the clean post-f8fa690 form that
delegates to DAB's access_ids_qs / accessible_pk_qs APIs
- Adds _scoped_actor_role_filter() as a temporary shim that pre-filters
RoleUserAssignment by content_type_id, auto-including Org and Team CTs
for inherited permissions and NULL for global roles
- Wires the scoped filter into UnifiedJobTemplate.accessible_pk_qs
- Adds tests for custom global roles, mixed permissions, and unrelated
role exclusion
The shim duplicates logic planned for DAB's _actor_role_filter
content_type_ids parameter; once that merges, replace and delete.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove unused ContentType and RoleUserAssignment imports to fix flake8
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-87084 — Remove _scoped_actor_role_filter shim, use DAB _actor_role_filter directly
The scoped content_type_ids parameter now lives in DAB's
_actor_role_filter. The AWX shim and its hardcoded Org/Team CT
inclusions are no longer needed. The unified job accessible_pk_qs
uses the unscoped _actor_role_filter for correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fixed migration from old to new rbac for sys auditor
* Added first migration
* Added testing for migrations and fix
* Removed attempts to remove platform auditor assignments
* Fixed migration test failures
When mintmaker bumps tekton resources it is failing on running the
pipeline-migration-tool. This runs as a post upgrade task and fails when
trying to run against tekton bundles. Disabling this to allow CI checks
to pass and merge updated pipeline bundles.
Signed-off-by: Ryan Williams <3375653+ryankwilliams@users.noreply.github.com>
Use SeparateDatabaseAndState with CREATE INDEX IF NOT EXISTS so the
migration succeeds when the index was already created manually (via
awx-manage create_host_summary_index or raw SQL workaround).
Forward-port of tower#7895 (stable-2.7).
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Adds `create_host_summary_index` management command that creates the
composite index (host_id, id DESC) on main_jobhostsummary using
CREATE INDEX CONCURRENTLY.
The command is idempotent: it checks for existing valid/invalid indexes,
drops invalid indexes from failed prior attempts, and skips if the index
already exists. Disables statement_timeout for the session to avoid
killing the index build on large tables where ALTER ROLE timeout is set.
Forward-port of tower#7893 (stable-2.6).
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Adrià SalaandClaude Opus 4.6
Hui SongandClaude Haiku 4.5
Matthew JohnsonandStevenson Michel
red-hat-konflux-kflux-prd-rh02[bot]
Liam AllenandClaude Sonnet 5
Hunter KepleyandClaude Code
Peter Braun
Pratyush BhandariandCursor
Paul Flanagan
John Westcott
John Barker
Pino Toscano
Alan RomingerandClaude Opus 4.6
Chris Meyers
Ryan Williams
Chris SantiagoandClaude Opus 4.6