* [AAP-82668] Skip old RBAC sync on cascade-deleted assignments
When a non-RBAC parent (e.g. Organization) is deleted, Django cascades
to RoleUserAssignment/RoleTeamAssignment and fires post_delete for each.
The sync_assignments_to_old_rbac_delete handler would then do 3-4
FK/GFK queries per assignment to sync removals to the old Role model —
entirely redundant since the old Role M2M tables cascade-delete from the
same content object.
Use Django's post_delete `origin` kwarg (4.1+) to detect this: when
origin is a Model instance whose app_label is not dab_rbac, the delete
is a cascade from a non-RBAC parent and the sync is skipped.
Measured on 150 teams / 321 users / 48K assignments:
Baseline: 78.8s
With fix: 20.4s (via service-index endpoint)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add regression tests for old RBAC sync after bulk claims
Tests that save_user_claims (which uses bulk_create, skipping signals)
correctly syncs old Role.members when run through AwxJWTAuthentication.
Covers add, remove, and multi-org/team scenarios.
* test: call process_permissions instead of internal _sync_old_rbac
Tests now exercise the public AwxJWTAuthentication.process_permissions()
API with the JWT layer mocked, rather than calling the private
_sync_old_rbac method directly.
The test_bulk_claims_skips_old_rbac_signals test asserts that
save_user_claims does NOT populate old Role.members via signals.
On current DAB, save_user_claims uses the serial give_permission
path which fires signals normally, so old Role.members IS populated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-81173 — Replace 4-way OR with UNION in unified job list RBAC query
The UnifiedJobAccess.filtered_queryset() method used a 4-way OR to
determine which unified jobs a user can see. Under load, the OR forces
PostgreSQL to evaluate all four branches in a single plan, preventing
branch-specific index optimization and causing 35 hours of DB time per
30-minute Scale Lab window.
Split each RBAC branch (template read_role, inventory update, ad-hoc
command, org auditor) into separate querysets combined with UNION, giving
the planner an independent optimal plan per branch. The UNION result is
wrapped in pk__in= for compatibility with BaseAccess.get_queryset()
prefetch_related and the workflowapproval filter.
This follows the same pattern proven in AAP-83319 (team list UNION fix).
* AAP-81173 — Add UnifiedJobPagination to prevent COUNT regression
The pk__in UNION pattern used for RBAC filtering forces the large
outer table as the driving table for COUNT(*), requiring PostgreSQL
to materialize all subquery result sets. On large deployments this
produces catastrophic query times (see AAP-83773 for the identical
issue on activity_stream).
Override the paginator count to use an unfiltered
UnifiedJob.objects.count() — the over-count is acceptable for
pagination UI. Also fix test_unified_job_list_rando_sees_nothing
to assert on results length instead of count, since count is now
unfiltered.
* AAP-81173 — Add unit tests for UnifiedJobPagination coverage
Cover the UnifiedJobPaginator.count cached property and the
count_disabled branch in UnifiedJobPagination.paginate_queryset
to satisfy SonarCloud's 90% new-code coverage gate.
* AAP-81173 — Address review feedback: save/restore paginator class, format consistency
- Fix Pagination.paginate_queryset() to save/restore django_paginator_class
instead of hardcoding DjangoPaginator in the finally block. This lets
subclasses set the class attribute without needing to override the method.
- Remove UnifiedJobPagination.paginate_queryset() override — now only needs
to set django_paginator_class = UnifiedJobPaginator as a class attribute.
- Format by_org_auditor consistently with the other three UNION branches.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
awxkit has its own tox.ini and when `tox -e linters` runs, awxkit/.tox/
can get populated with a py3 virtualenv. Since flake8 is configured to
scan the `awxkit` directory, it recurses into
awxkit/.tox/py3/lib/python3.12/site-packages/ and reports hundreds of
false positives (F405, E265, E266) from third-party packages like PyYAML.
Adding .tox to the flake8 exclude list prevents this — matching the
existing exclusion of `env` for virtualenvs.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
dict() with keyword arguments is idiomatic for Ansible module
argument_spec definitions. Suppress the "use literal instead of
constructor" rule (python:S7498) for awx_collection/plugins/modules/.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Jake Jackson <jljacks93@gmail.com>
* fix: prefer scm_revision as the cache_id if available
* add tests
* fix: make sure galaxy requirements are always updated when a new revision is pulled
* make sure existing project cache id is used when available
---------
Co-authored-by: Liam Allen <lallen@redhat.com>
* AAP-83773 — Use unfiltered count for activity stream pagination
Forward-port of tower#7606 (stable-2.6).
The RBAC-filtered COUNT(*) on activity_stream takes ~36 min per call
on large tables (713K rows) due to the pk__in subquery shape introduced
by the AAP-81860 LEFT JOIN fix. Replace with an unfiltered table count
for pagination -- an approximate over-count is harmless for UI page
navigation while the actual page results remain RBAC-filtered.
* AAP-83773 — Add unit tests for ActivityStreamPagination
Cover ActivityStreamPaginator and ActivityStreamPagination to satisfy
SonarCloud coverage gate on new lines in awx/api/pagination.py.
* AAP-83773 — Save/restore django_paginator_class in base Pagination
Address review feedback from lallen92: use a save/restore pattern in
Pagination.paginate_queryset() so subclasses only need to set the class
attribute. Remove the now-redundant paginate_queryset override from
ActivityStreamPagination.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Liam Allen <lallen@redhat.com>
* AAP-84057: Set PostgreSQL statement_timeout on web worker DB connections
When uwsgi's harakiri kills a worker, the PostgreSQL backend continues
running the query indefinitely. These abandoned queries accumulate and
create resource contention for all other queries.
Add a connection_created signal handler that sets statement_timeout on
new DB connections. Under uwsgi, the timeout is auto-derived from the
harakiri value (minus 5s margin so PostgreSQL cancels the query before
uwsgi kills the worker). Outside uwsgi (task workers, migrations),
no timeout is applied. A manual DATABASE_STATEMENT_TIMEOUT setting
is available as a fallback for non-uwsgi deployments.
* Remove timeout value caching because it brings no significant gains
* Use proportional margin between statement_timeout and harakiri timeout
* Fix zero-harakiri test to patch fake uwsgi module instead of None
* Refactor statement_timeout from signal handler to connection string
Move statement_timeout configuration from a connection_created signal
handler (extra SQL round-trip per connection) to a dynaconf merge
function that sets it via the libpq OPTIONS connection string parameter.
This mirrors the existing merge_application_name() pattern and
eliminates the SET statement on every new connection.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Django's cascade collector materializes all JobHostSummary IDs into a
single UPDATE ... IN (...) to SET_NULL on Host.last_job_host_summary.
With many jobs x hosts this exceeds PostgreSQL's max memory allocation
size for a single query (1GB).
Pre-delete JHS rows and clear Host FK references in chunks of 1000 job
IDs using raw SQL before Django's .delete() runs, so the cascade
collector finds nothing to collect.
Assisted-by: Claude Code via Google Vertex AI
The TeamAccess.filtered_queryset() method combined two access paths
(org membership and direct read permission) using Q(…) | Q(…), which
PostgreSQL could not optimize — it scanned all teams for each OR
branch. Replacing the OR with UNION lets each branch use the
RoleEvaluation 3-column index independently. The UNION result is
wrapped in pk__in= so the outer queryset remains compatible with
BaseAccess.get_queryset()'s select_related() call.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
AAP-82221 — Skip reverse sync during post-migrate role definition setup
The dab_post_migrate signal handler added in 64dc097914 calls
setup_managed_role_definitions on every awx-manage migrate run.
During fresh installs this deletes stale managed role definitions
and triggers a reverse sync to the gateway, which returns 423
Locked because migrate_service_data has not finished yet. The
installer retries 5 times but each attempt fires the same signal
and hits the same 423, failing the controller init.
* AAP-76460 — Prioritize REDHAT_CANDLEPIN_HOST over rhsm.conf for subscription validation
When REDHAT_CANDLEPIN_HOST is explicitly configured (Satellite/disconnected
environments), use it as the subscription host instead of falling back to
rhsm.conf. This fixes subscription loading failures in containerized
deployments where rhsm.conf defaults to subscription.rhsm.redhat.com but
the system cannot reach it.
Also guard against double port-appending in get_satellite_subs when the
host URL already includes a port.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Liam Allen <lallen@redhat.com>
* Optimize HostList API: conditional DISTINCT + composite index on JobHostSummary (#16530)
AAP-81517 — Optimize HostList API: conditional DISTINCT + composite index
Make .distinct() conditional on host_filter being set — without it the
RBAC IN subquery on a direct FK cannot produce duplicates, so DISTINCT
is pure overhead. Add composite index (host_id, id DESC) on
main_jobhostsummary so the with_latest_summary_id() correlated subquery
can use an index-only top-1 scan instead of scanning and sorting.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: Increase awx-operator molecule timeout to 20m (#16534)
The awx-operator molecule kind test intermittently times out after
15 minutes on GitHub Actions runners, causing flaky CI failures.
Bump the bash-level timeout from 15m to 20m (step-level
timeout-minutes: 60 is unchanged).
Closes: AAP-81583
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-81860 — Eliminate LEFT OUTER JOINs in ActivityStream RBAC query (#16533)
* AAP-81860 — Eliminate LEFT OUTER JOINs in ActivityStream RBAC query
ActivityStreamAccess.filtered_queryset() used field-traversal Q objects
(e.g. Q(host__inventory__in=...)) across 18 M2M relationships, which
Django translates into LEFT OUTER JOINs. This forces PostgreSQL to join
all intermediary tables before filtering, making it the #2 DB consumer
at 483s in a 10-minute Scale Lab window.
Replace all M2M field traversals with pk__in subqueries using .through
intermediary tables, following the same pattern proven in AAP-81082
(28% improvement for unified job RBAC). This changes the SQL from
LEFT OUTER JOIN to IN (SELECT ...) semi-joins, allowing PostgreSQL to
skip the unconditional joins. Also:
- Remove .distinct() (no longer needed without M2M JOINs)
- Remove unnecessary if-truthy checks that evaluated subqueries as
boolean before including them (5 extra DB roundtrips per request)
- Migrate accessible_pk_qs(user, 'read_role') to access_ids_qs(user,
'view') for direct DAB RBAC API usage
* Use .exists() instead of queryset truth-test for auditing_orgs guard
Avoids evaluating the full queryset when checking whether the user has
auditing orgs — .exists() issues a cheaper SELECT 1 … LIMIT 1 instead.
* Restore .exists() guards for resource-type Q branches
Re-add conditional guards around inventory, credential, project,
job template, workflow job template, and team Q branches. Uses
.exists() (cheap SELECT 1 LIMIT 1) instead of the old queryset
truth-test to avoid unnecessary query evaluation while still
preventing generation of an overly complex query when the user
has no access to a given resource type.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* chore: Upgrade kubernetes package to 36.0.2
* feat: remove extra_vars from jobs and unified_jobs list endpoint. Add… (#16461)
feat: add exclude query parameter to unified_jobs and jobs list endpoints
Allow clients to exclude heavy fields (artifacts, extra_vars) from list
responses via ?exclude=artifacts,extra_vars. These fields are now included
by default instead of being stripped.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-80457 - Fix credential_input_source to handle state: absent when target crede… (#16523)
Fix credential_input_source to handle state: absent when target credential is missing
The credential_input_source module would error when trying to delete a
credential input source (state: absent) if the target credential didn't
exist. This breaks idempotent playbook runs where cleanup tasks assume
resources may already be gone.
The fix only applies to state: absent; state: present still correctly
fails when the target credential doesn't exist.
Co-authored-by: Liam Allen <lallen@redhat.com>
* Add additional test coverage
* get_satellite_subs() reads verify from rhsm.conf, never from REDHAT_CANDLEPIN_VERIFY.
* Address coderabbit feedback
---------
Signed-off-by: Liam Allen <lallen@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Rodrigo Toshiaki Horie <rodrigo.horie@hotmail.com>
Co-authored-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: Nick Meyer <nick.a.meyer@icloud.com>
* Balance controller node selection during capacity ties
When multiple control-plane instances have equal remaining capacity,
the instance selection algorithm always picks the first instance in
iteration order. This causes burst workloads to concentrate all job
management on a single controller pod while others remain idle.
Add a tie-breaking criterion: when would_be_remaining capacity is
equal, prefer the instance with fewer jobs_running. This distributes
the control overhead (event processing, callbacks, output streaming)
more evenly across available controller pods during burst scenarios.
The change is backwards-compatible: when capacity clearly differs,
the existing "most remaining capacity" logic dominates unchanged.
Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com>
Made-with: Cursor
* Track jobs_running on controller node for container-group tasks
Address review feedback: container-group tasks only set controller_node
(execution_node is empty), so the previous consume_capacity call on the
control path never incremented jobs_running. This meant the tie-breaker
could not differentiate between nodes.
Now pass job_impact=True on the control path when the controller is not
also the execution node (avoiding double-counting for hybrid nodes).
Also improve test coverage:
- Fix test case to prove capacity dominates over jobs_running
- Add dedicated tests for container-group controller distribution
- Add test verifying equal-capacity tie-breaking behaviour
Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com>
Made-with: Cursor
* Add docstrings to consume_capacity and fit_task_to_most_remaining_capacity_instance
Address CodeRabbit docstring coverage warning by documenting the two
methods modified in this PR. No functional changes.
Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Signed-off-by: Alexey Masolov <amasolov@redhat.com>
Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Liam Allen <lallen@redhat.com>
* Optimize HostList API: conditional DISTINCT + composite index on JobHostSummary (#16530)
AAP-81517 — Optimize HostList API: conditional DISTINCT + composite index
Make .distinct() conditional on host_filter being set — without it the
RBAC IN subquery on a direct FK cannot produce duplicates, so DISTINCT
is pure overhead. Add composite index (host_id, id DESC) on
main_jobhostsummary so the with_latest_summary_id() correlated subquery
can use an index-only top-1 scan instead of scanning and sorting.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: Increase awx-operator molecule timeout to 20m (#16534)
The awx-operator molecule kind test intermittently times out after
15 minutes on GitHub Actions runners, causing flaky CI failures.
Bump the bash-level timeout from 15m to 20m (step-level
timeout-minutes: 60 is unchanged).
Closes: AAP-81583
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-81860 — Eliminate LEFT OUTER JOINs in ActivityStream RBAC query (#16533)
* AAP-81860 — Eliminate LEFT OUTER JOINs in ActivityStream RBAC query
ActivityStreamAccess.filtered_queryset() used field-traversal Q objects
(e.g. Q(host__inventory__in=...)) across 18 M2M relationships, which
Django translates into LEFT OUTER JOINs. This forces PostgreSQL to join
all intermediary tables before filtering, making it the #2 DB consumer
at 483s in a 10-minute Scale Lab window.
Replace all M2M field traversals with pk__in subqueries using .through
intermediary tables, following the same pattern proven in AAP-81082
(28% improvement for unified job RBAC). This changes the SQL from
LEFT OUTER JOIN to IN (SELECT ...) semi-joins, allowing PostgreSQL to
skip the unconditional joins. Also:
- Remove .distinct() (no longer needed without M2M JOINs)
- Remove unnecessary if-truthy checks that evaluated subqueries as
boolean before including them (5 extra DB roundtrips per request)
- Migrate accessible_pk_qs(user, 'read_role') to access_ids_qs(user,
'view') for direct DAB RBAC API usage
* Use .exists() instead of queryset truth-test for auditing_orgs guard
Avoids evaluating the full queryset when checking whether the user has
auditing orgs — .exists() issues a cheaper SELECT 1 … LIMIT 1 instead.
* Restore .exists() guards for resource-type Q branches
Re-add conditional guards around inventory, credential, project,
job template, workflow job template, and team Q branches. Uses
.exists() (cheap SELECT 1 LIMIT 1) instead of the old queryset
truth-test to avoid unnecessary query evaluation while still
preventing generation of an overly complex query when the user
has no access to a given resource type.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* chore: Upgrade kubernetes package to 36.0.2
* feat: remove extra_vars from jobs and unified_jobs list endpoint. Add… (#16461)
feat: add exclude query parameter to unified_jobs and jobs list endpoints
Allow clients to exclude heavy fields (artifacts, extra_vars) from list
responses via ?exclude=artifacts,extra_vars. These fields are now included
by default instead of being stripped.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-80457 - Fix credential_input_source to handle state: absent when target crede… (#16523)
Fix credential_input_source to handle state: absent when target credential is missing
The credential_input_source module would error when trying to delete a
credential input source (state: absent) if the target credential didn't
exist. This breaks idempotent playbook runs where cleanup tasks assume
resources may already be gone.
The fix only applies to state: absent; state: present still correctly
fails when the target credential doesn't exist.
Co-authored-by: Liam Allen <lallen@redhat.com>
* AAP-83163 — Eliminate LEFT JOIN fan-out in user list RBAC query (#16546)
Replace legacy Role M2M traversal in UserAccess.filtered_queryset()
with DAB's visible_users(), which queries organizational membership
through RoleUserAssignment subqueries instead of LEFT OUTER JOINs
through the objectrole/roleevaluation tables.
Also replace role__in=actor.has_roles.all() in can_admin() with
RoleEvaluation._actor_role_filter() to avoid the objectrole JOIN
in per-object permission checks.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* alter_skip_tags_to_textfield
* Bump migration number
---------
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Rodrigo Toshiaki Horie <rodrigo.horie@hotmail.com>
Co-authored-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: Nick Meyer <nick.a.meyer@icloud.com>
* AAP-82745 — Replace O(N) correlated subqueries in org list with bulk aggregation
The org list endpoint computed per-org user and admin counts using two
correlated subqueries against roleuserassignment — one per org per role
type. At scale this was the #1 worst query in the customer DB (677ms
mean, 336K calls, 3,801 min total DB time).
Replace with a single flat conditional-aggregation query that scans
roleuserassignment once regardless of org count. Also simplify the
detail view to use direct .count() calls and add a query count
regression test.
* Remove dead code: org_counts is always populated after refactor
* Address review feedback on detail view and test
- Detail view: combine two .count() calls into single .aggregate()
with conditional counts, matching the list-view pattern
- Use .update() in else branch for consistency with if branch
- Remove redundant setup_managed_roles fixture (transitive via
organization_resource_creator)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace legacy Role M2M traversal in UserAccess.filtered_queryset()
with DAB's visible_users(), which queries organizational membership
through RoleUserAssignment subqueries instead of LEFT OUTER JOINs
through the objectrole/roleevaluation tables.
Also replace role__in=actor.has_roles.all() in can_admin() with
RoleEvaluation._actor_role_filter() to avoid the objectrole JOIN
in per-object permission checks.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Fix credential_input_source to handle state: absent when target credential is missing
The credential_input_source module would error when trying to delete a
credential input source (state: absent) if the target credential didn't
exist. This breaks idempotent playbook runs where cleanup tasks assume
resources may already be gone.
The fix only applies to state: absent; state: present still correctly
fails when the target credential doesn't exist.
Co-authored-by: Liam Allen <lallen@redhat.com>
feat: add exclude query parameter to unified_jobs and jobs list endpoints
Allow clients to exclude heavy fields (artifacts, extra_vars) from list
responses via ?exclude=artifacts,extra_vars. These fields are now included
by default instead of being stripped.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-81860 — Eliminate LEFT OUTER JOINs in ActivityStream RBAC query
ActivityStreamAccess.filtered_queryset() used field-traversal Q objects
(e.g. Q(host__inventory__in=...)) across 18 M2M relationships, which
Django translates into LEFT OUTER JOINs. This forces PostgreSQL to join
all intermediary tables before filtering, making it the #2 DB consumer
at 483s in a 10-minute Scale Lab window.
Replace all M2M field traversals with pk__in subqueries using .through
intermediary tables, following the same pattern proven in AAP-81082
(28% improvement for unified job RBAC). This changes the SQL from
LEFT OUTER JOIN to IN (SELECT ...) semi-joins, allowing PostgreSQL to
skip the unconditional joins. Also:
- Remove .distinct() (no longer needed without M2M JOINs)
- Remove unnecessary if-truthy checks that evaluated subqueries as
boolean before including them (5 extra DB roundtrips per request)
- Migrate accessible_pk_qs(user, 'read_role') to access_ids_qs(user,
'view') for direct DAB RBAC API usage
* Use .exists() instead of queryset truth-test for auditing_orgs guard
Avoids evaluating the full queryset when checking whether the user has
auditing orgs — .exists() issues a cheaper SELECT 1 … LIMIT 1 instead.
* Restore .exists() guards for resource-type Q branches
Re-add conditional guards around inventory, credential, project,
job template, workflow job template, and team Q branches. Uses
.exists() (cheap SELECT 1 LIMIT 1) instead of the old queryset
truth-test to avoid unnecessary query evaluation while still
preventing generation of an overly complex query when the user
has no access to a given resource type.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The awx-operator molecule kind test intermittently times out after
15 minutes on GitHub Actions runners, causing flaky CI failures.
Bump the bash-level timeout from 15m to 20m (step-level
timeout-minutes: 60 is unchanged).
Closes: AAP-81583
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AAP-81517 — Optimize HostList API: conditional DISTINCT + composite index
Make .distinct() conditional on host_filter being set — without it the
RBAC IN subquery on a direct FK cannot produce duplicates, so DISTINCT
is pure overhead. Add composite index (host_id, id DESC) on
main_jobhostsummary so the with_latest_summary_id() correlated subquery
can use an index-only top-1 scan instead of scanning and sorting.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded credentials in subscriptions module examples
The EXAMPLES block contained values that looked like real client
credentials. Replace them with obvious placeholders and add a
username/password example.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-81082 Eliminate LEFT OUTER JOINs in unified job RBAC query
UnifiedJobAccess.filtered_queryset() used field-traversal Q objects
(inventoryupdate__inventory_source__inventory__id__in=...) which Django
translates into LEFT OUTER JOINs. This forces PostgreSQL to scan all
rows in main_unifiedjob before filtering — at scale, 99.84% are
discarded. Replace with pk__in subqueries that generate IN (SELECT ...)
instead, allowing PostgreSQL to skip the unconditional join. EXPLAIN
ANALYZE shows a 28% reduction in execution time (496ms -> 355ms), with
larger gains expected under concurrent load.
* AAP-81082 Eliminate LEFT OUTER JOINs in RBAC filtered_queryset methods
Replace field-traversal Q objects with pk__in subqueries across all Access
classes that query polymorphic or M2M tables, preventing Django from
generating unconditional LEFT OUTER JOINs. Also migrate legacy
_accessible_pk_qs / accessible_pk_qs calls to DAB RBAC access_ids_qs.
Affected: UnifiedJobAccess, UnifiedJobTemplateAccess, JobAccess,
JobEventAccess, LabelAccess.
* Fix docstring
* Refactor result_stdout_raw_handle to use parameterized COPY SQL.
Replace f-string SQL construction with psycopg.sql composables and bound
parameters so security scans no longer flag formatted raw SQL in the
unified jobs stdout path.
Fix sqlite_copy mock rendering for psycopg3 SQL composables.
* Fix sqlite_copy mock without psycopg SQL internals.
Load stdout from the first populated event table instead of rendering
psycopg composables, which use version-specific private attributes.
* Use sql.Literal in COPY query for Django cursor.copy compatibility.
Django's cursor.copy() does not forward bind parameters to psycopg,
which caused stdout API 500s against real PostgreSQL.
* Fix SonarCloud security rating by removing user-controlled data from sqlite filepath
Replace os.path.basename(sys.argv[0]) with a hardcoded 'unknown' fallback
in RecordedQueryLog.write() to eliminate path injection via CLI arguments.
This resolves SonarCloud rule pythonsecurity:S8706 and helps restore the
AWX security rating from C to A.
Closes: AAP-80006
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove unused sys import from test_db.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Forward-port of ansible/tower#7537 for the devel branch.
When running awxkit's tox tests, pytest picks up the root pytest.ini
which pulls in pytest-django options (--reuse-db, --nomigrations,
DJANGO_SETTINGS_MODULE) and Django-specific filterwarnings. Since the
awxkit tox environment does not install Django or pytest-django, these
cause test collection to fail.
The root cause is that pytest.ini has absolute priority in pytest's
config discovery — it searches all ancestor directories for pytest.ini
before falling back to tox.ini's [pytest] section. A [pytest] section
in awxkit/tox.ini alone cannot prevent the root config from being used.
Fix by:
- Adding awxkit/pytest.ini to act as the primary config boundary
(pytest.ini has the highest priority in config discovery, so its
presence in awxkit/ stops the upward search before reaching root)
- Adding explicit `test` path argument to the pytest command in
awxkit/tox.ini so pytest discovers tests correctly
- Adding `testpaths` and `python_files` to the [pytest] section in
awxkit/tox.ini as a secondary config boundary
- Adding awxkit/conftest.py that registers the Django-specific CLI
options and INI keys as harmless no-ops, as a further safety net
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: handle cursor-paginated API responses in get_one()
DAB PR #1025 switched role_team_assignments and role_user_assignments
endpoints from PageNumberPagination to CursorPagination. Cursor
pagination returns {results, next, previous} without a count field,
causing get_one() to fail with "The endpoint did not provide count
and results".
When the response includes results but no count, infer count from
len(results). Also guard get_all_endpoint() against missing count.
Instances reporting cpu=0 or memory=0 with no errors would be
transitioned to READY state. Treat zero cpu/memory as an error
so the node stays offline until a valid health check is received.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(analytics): ANSTRAT-2268 log request_id, account_number, org_id from ingress API response
Log the ingress API success response fields (request_id, account_number,
org_id) in the controller task log when gather_analytics tarballs are
uploaded. This enables support engineers to trace uploads through to
Kibana without source code modifications.
* style(analytics): use f-strings in _log_shipping_response to match codebase conventions
* style(analytics): fix black formatting in test assertions
* Use _actor_role_filter() in UnifiedJobTemplate.accessible_pk_qs()
Replace `role__in=accessor.has_roles.all()` with the optimized `_actor_role_filter()` subquery pattern from django-ansible-base.
The old pattern causes a 3-table JOIN through RoleUserAssignment -> ObjectRole -> RoleEvaluation on every non-superuser request. _actor_role_filter() skips the ObjectRole table entirely by using a direct subquery on RoleUserAssignment.object_role_id, eliminating the intermediate JOIN and reducing query time for /api/v2/unified_jobs/ requests by non-superusers.
fix: use GPG-signed commits in spec sync workflow
Switch from unsigned GitHub API commits to GPG-signed git commits
using the aap-api-bot GPG key (OPENAPI_SPEC_SYNC_GPG_PRIVATE_KEY).
The aap-openapi-specs repo requires signed commits via org ruleset.
The previous API-based approach didn't sign commits because GitHub
only auto-signs API commits for GitHub App tokens, not user PATs.
This matches the pattern used by EDA and Gateway teams for their
spec sync workflows.
Also fixes template injection risk by using env vars instead of
direct ${{ }} expansion in shell context.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix cartesian product in organization user/admin count queries
The organizations list and detail endpoints annotated each org with user and admin counts using two Count() calls that traverse the Role.members M2M. Django generated two LEFT JOINs on the same through table, crossing every member row with every admin row before COUNT(DISTINCT) reduced the product.
At scale (2,617 members × 46,233 admins) this produced 120M intermediate rows and 96-second query times, causing 504 timeouts.
Replace with independent Subquery expressions that each query main_rbac_roles_members separately - no cross product.
Fixes: AAP-72817
Fixes: AAP-72480
* Fix variable names which do not meet coding standards
* Fix formatting inconsistency in organization detail subquery annotation
Break the long .annotate() line across multiple lines to match the style used in mixin.py.
* Rewrite org count subqueries to use DAB RBAC models
Replace old RBAC Role.members.through subqueries with
RoleUserAssignment-based correlated subqueries, querying
managed RoleDefinitions ('Organization Member' / 'Organization Admin')
directly. This aligns with the DAB RBAC migration direction and
eliminates dependency on the deprecated ImplicitRoleField M2M tables
for these counts.
Update test fixtures to use RoleDefinition.give_permission() and
add setup_managed_roles where needed.
* Fix collection tests: set up managed role definitions
The DAB RBAC migration to use RoleUserAssignment subqueries in
organization views requires managed role definitions (Organization
Member, Organization Admin) to exist in the test database.
Add an autouse fixture to the collection test conftest that calls
setup_managed_role_definitions() before each test.
* Add setup_managed_roles fixture to functional tests hitting org views
Tests that hit organization list/detail views now require the
setup_managed_roles fixture to pre-create the Organization Member
and Organization Admin RoleDefinition objects used by the DAB RBAC
subqueries.
* Revert setup_managed_roles from ext_auditor tests
The setup_managed_roles fixture conflicts with the ext_auditor_rd
fixture by deleting the Alien Auditor role definition. These tests
don't need it — the defensive view code handles missing role
definitions gracefully.
* Handle missing Organization Member/Admin role definitions gracefully
Use filter().first() instead of get() for RoleDefinition lookups in
organization list and detail views. Returns 0 for user/admin counts
when role definitions are not yet created, preventing 500 errors in
environments where post_migrate signals haven't run.
* Cast OuterRef('pk') to TextField for RoleUserAssignment.object_id comparison
RoleUserAssignment.object_id is a TextField, but OuterRef('pk') on
Organization produces an integer. PostgreSQL strictly rejects text = integer
comparisons. Use Cast() to explicitly convert the PK to text.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [AAP-78392] Optimize HostManager.active_count() with cache and functional index
active_count() runs a full sequential scan with LOWER()+DISTINCT on main_host for every license check. At customer scale this consumed 74.5 minutes of DB time over 4 hours (47K calls at 93ms avg).
Add a 60-second Redis-backed cache via the existing memoize decorator to reduce call volume by ~99.5%. Add a functional btree index on LOWER(name) to eliminate the sequential scan for the remaining calls.
* Use AddIndexConcurrently instead of AddIndex in the migration for host name lower index
* Revert AddIndexConcurrently to AddIndex for CI compatibility
The api-migrations CI job runs against SQLite which does not support PostgreSQL-specific AddIndexConcurrently. Standard AddIndex works across all backends and the brief write lock during production upgrades is acceptable for this table size.
* Remove functional index, keep cache-only fix per reviewer feedback
Drop the LOWER(name) functional index and migration to minimize
the change footprint.
----
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The aap-openapi-specs repo requires commit signatures via org ruleset.
Switch from git commit+push to the GitHub Git Data API which
automatically signs commits, satisfying the required_signatures rule.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat: inject x-ai-description from overlay file during schema generation
Many endpoints have human-readable AI descriptions that were added
downstream in aap-mcp-server (PRs #73 and #119) but never backported
as @extend_schema_if_available decorators. This causes 470 out of 631
x-ai-description entries to be lost every time the spec is regenerated.
Add a JSON overlay file (openapi_ai_descriptions.json) containing the
missing descriptions keyed by operationId, and a drf-spectacular
postprocessing hook that merges them into the generated schema for any
operation that doesn't already have x-ai-description from a decorator.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Restore oauth_token backward compatibility for collection token auth
The aap_token rename (c8981e321e) restored module-level token auth but
left two interfaces from earlier collection releases broken:
- The lookup (controller_api) and inventory (controller) plugins
previously declared an oauth_token option. Add oauth_token as an
alias of aap_token in the auth_plugin doc fragment and in
AUTH_ARGSPEC so query(..., oauth_token=...) and inventory YAML keys
keep working.
- tower_cli.cfg-style config files used an oauth_token key under
[general]; it was silently ignored after the rename, quietly
degrading auth. load_config() now also reads the legacy oauth_token
key and maps it to aap_token, with the new aap_token key winning when
both are present. aap_token remains the canonical attribute used by
_parse_aap_token() and the Bearer header logic.
Also make the test helper compatible with ansible-core 2.21+, which
requires a serialization profile alongside _ANSIBLE_ARGS, and extend
the tests to cover the oauth_token alias and legacy config file key.
No changelog fragment added: awx_collection has no changelogs/
directory on devel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Document oauth_token alias in module auth doc fragment
The oauth_token alias was added to aap_token in AUTH_ARGSPEC but not to
the module doc fragment, failing the validate-modules sanity check
(undocumented argument alias).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Generalize version references in compat comments
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The aap_token parameter was added to the collection argspec and docs
in #16025, but nothing consumed it after token auth was removed in
#15623: modules silently ignored the token and fell back to basic
auth, breaking token authentication through the AAP gateway.
Wire it up so requests authenticate with the provided token (e.g. one
issued by the AAP gateway, which validates it and proxies to the
controller):
- Send "Authorization: Bearer <token>" in make_request when aap_token
is set, skipping the basic-auth login probe; basic auth is unchanged
when no token is given
- Accept the token as a string or as the dict set as a fact by the
ansible.platform.token module ({token: ..., id: ...}), which is the
documented cross-collection mint/use/delete workflow
- Restore controller_oauthtoken and tower_oauthtoken as aliases for
back-compat with pre-#15623 playbooks, matching downstream
- Forward aap_token through the controller_api lookup and controller
inventory plugins via short_params, and add the missing
CONTROLLER_OAUTH_TOKEN/TOWER_OAUTH_TOKEN env sources to the plugin
doc fragment (plugins resolve env vars from doc fragments, not
env_fallback); AAP_TOKEN is no longer marked deprecated there
- Support tokens in the awxkit-based export/import modules
- Add unit tests covering the Bearer header for both token forms, the
aliases, the bad-dict failure, and the basic-auth fallback
Verified end-to-end against a live gateway-fronted AAP 2.7 deployment:
modules, the lookup plugin, both aliases, all env sources, dict-form
tokens, job launch/wait, and a clean HTTP 401 on an invalid token.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move plugin loading to lazy-on-first-access, DB sync to dispatcher
Remove credential type and inventory plugin loading from Django's
app.ready() path. In-memory registries (ManagedCredentialType.registry
and InventorySourceOptions.injectors) are now populated lazily on first
access via LazyLoadDict, a dict subclass that calls a loader function
on the first read operation. This ensures web workers, dispatcher
workers, and management commands all get the registries populated
exactly when needed, without eager loading at startup.
The DB sync (CredentialType.setup_tower_managed_defaults) is moved to
the dispatcher's startup task, where it only needs to run once per
deployment rather than in every Django process.
Co-Authored-By: Alan Rominger <arominge@redhat.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* AAP-65883: Refactor clear_setting_cache to use DAB shared utility
Delegate cache invalidation logic to ansible_base.lib.cache.tasks.clear_cache,
passing AWX-specific dependent key resolution (settings_registry) and
post-invalidation hook (LOG_AGGREGATOR_LEVEL reconfiguration) as callbacks.
Requires: ansible/django-ansible-base AAP-65883/dab-cache-invalidation-job
Assisted-by: Claude Code / Opus 4.6 (Anthropic)
* AAP-65883: Extract helper functions to module level
Move _resolve_setting_dependents and _post_setting_invalidation out of
clear_setting_cache for better stack traces and independent testability
per review feedback (John Westcott).
Assisted-by: Claude Code / Opus 4.6 (Anthropic)
* Move PG version check to check_db command
Move to utils, check in pre_migrate signal
* Add back in environment var skip
* Add tests for compliance
tests Assisted-By: claude
[AAP-65882] Switch to DAB Redis cache backend
Replace awx.main.cache.AWXRedisCache with the equivalent
ansible_base.lib.cache.redis_cache.DABRedisCache and remove the
now-redundant AWX driver. The DAB driver was migrated from this
exact code and provides the same fault-tolerant cache behavior.
Signed-off-by: Tomas Z <tznamena@redhat.com>
* Fix bash operator precedence in repo ownership check
The condition had || operators outside proper test block grouping,
which could cause the check to fail with a shell error. Wrap the
OR conditions in parentheses with explicit [[ ]] tests.
Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>
* Replace reusable workflow with direct if conditions for repo ownership check
The reusable workflow with job dependencies had a timing/evaluation issue
where jobs would still execute even when should_run=false. Using direct
if conditions with github context variables (repository, ref_name) is more
reliable and ensures jobs are properly skipped on fork pushes.
Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>
* Remove unused repo-owns-branch reusable workflow
No longer needed after replacing with direct if conditions.
Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>
Django allows passing update_fields=None to model.save() to mean 'update all fields'.
However, kwargs.get('update_fields', []) returns None when the key exists with a None value,
rather than the default empty list.
This caused crashes in mark_field_for_save() and other code that assumes update_fields is
a list when doing membership tests ('field' not in update_fields) or append operations.
Changed pattern from:
update_fields = kwargs.get('update_fields', [])
To:
update_fields = kwargs.get('update_fields') or []
This correctly handles:
- Key missing: get() returns None → or [] gives []
- Key present with None: get() returns None → or [] gives []
- Key present with list: get() returns list → or [] keeps the list
Fixed in 12 locations across 8 model files:
- awx/main/models/base.py (3 instances)
- awx/main/models/unified_jobs.py (2)
- awx/main/models/jobs.py (2)
- awx/main/models/ad_hoc_commands.py
- awx/main/models/inventory.py
- awx/main/models/notifications.py
- awx/main/models/projects.py
- awx/main/models/workflow.py
Fixes task manager crashes with:
TypeError: argument of type 'NoneType' is not iterable
* Address even more pytest warnings, co-authored with Opus 4.6
* Upgrade pyparsing
* Attempt to update smart inventory logic
* Move smart inventory tests here
* Fix some failing dev env tests
Assisted-by: claude
* Use shared fixture for teardown
* Fix test goof
assisted-by: claude Opus 4.6
* [AAP-57274] Fix creator permissions for models without old-style roles
NotificationTemplate has no old-style ImplicitRoleField (like admin_role)
because notification permissions were historically org-level only.
When a non-admin user creates a notification template,
give_creator_permissions tries to sync the DAB RBAC assignment back
to the old role system and hits an AttributeError.
Catch the AttributeError so the DAB RBAC assignment still succeeds.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* INCLUDE_AWX_VAR_PREFIX to USE_TOWER_VAR_PREFIX boolean toggle replace the include legacy prefix boolean with a tower-or-awx toggle USE_TOWER_VAR_PREFIX=True (default) emits only tower_ prefixed variables, false emits only awx_ (deprecated)
* Clean up dead constant and cache get_job_variable_prefixes() calls
* Revise tests to reflect new behavior
* Fix fragile fallback test to actually exercise getattr default
* Fix mock target for settings fallback test
* [AAP-74343] Decouple installed_collections and ansible_version from indirect node counting flag
The indirect_instance_count callback plugin and its artifact processing
were entirely gated behind FEATURE_INDIRECT_NODE_COUNTING_ENABLED. This
caused installed_collections and ansible_version to remain unpopulated
when the flag was off, even though these are baseline analytics fields
unrelated to indirect host counting.
Always run the callback plugin and persist installed_collections and
ansible_version to the database. Only the indirect-counting-specific
parts (EventQuery creation, event_queries_processed flag, and vendor
collections) remain gated behind the feature flag.
* [AAP-74343] Read callbacks_enabled from ansible.cfg so user-configured callbacks are preserved
The check for 'callbacks_enabled' in config_values was dead code because
read_ansible_config was never asked to read that setting. Now that the
callback registration runs unconditionally, fix this by including
'callbacks_enabled' in the variables of interest.
* [AAP-74343] Use comma delimiter for ANSIBLE_CALLBACKS_ENABLED
Ansible's CALLBACKS_ENABLED config is type list and splits on commas.
The colon delimiter would cause combined callback names to be treated
as a single invalid name.
* [AAP-74343] Add tests for ANSIBLE_CALLBACKS_ENABLED configuration
Verify that indirect_instance_count is always set, user-configured
callbacks from ansible.cfg are preserved, and the comma delimiter
is used as ansible-core expects.
* [AAP-74343] Use public API for namespace package path access
Replace library.__path__._path[0] with library.__path__[0] to avoid
relying on a private CPython implementation detail of _NamespacePath.
* [AAP-74343] Skip host query scanning when indirect counting flag is off
The indirect_instance_count callback plugin now checks AWX_COLLECT_HOST_QUERIES
to decide whether to scan for host query files. When the feature flag is off,
the plugin only collects collection metadata (name + version) and ansible_version,
skipping the expensive embedded/external query file discovery.
* [AAP-74343] Set AWX_COLLECT_HOST_QUERIES in query discovery tests
The TestExternalQueryDiscovery tests exercise the host query scanning
path, which now requires AWX_COLLECT_HOST_QUERIES=1 in the environment.
* [AAP-74343] Use Ansible plugin config system for collect_host_queries
Declare collect_host_queries as a formal plugin option in DOCUMENTATION
with env var AWX_COLLECT_HOST_QUERIES, replacing the raw os.getenv() call
with self.get_option(). This follows the standard Ansible plugin
configuration pattern.
* [AAP-74343] Add test for disabled collect_host_queries path
Verify that when collect_host_queries is false, the plugin still
enumerates collections for metadata but skips host query file scanning.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Fix awx-collection tests for ansible-core 2.21.0 compat
ansible-core 2.21.0 introduced _PARSED_MODULE_ARGS in
module_utils/basic.py, used by _return_formatted() to conditionally
include invocation data. The awx_collection test harness bypasses
normal arg parsing via _load_params mock, leaving this variable as
None and causing AttributeError on every exit_json/fail_json call.
Mock _PARSED_MODULE_ARGS with _ansible_inject_invocation=True to
match the pre-2.21 behavior of always including invocation data.
* AAP-12516 [option 2] Handle nested workflow artifacts via root node `ancestor_artifacts` (#16381)
* Add new test for artfact precedence upstream node vs outer workflow
* Fix bugs, upstream artifacts come first for precedence
* Track nested artifacts path through ancestor_artifacts on root nodes
* Fix case where first root node did not get the vars
* touchup comment
* Prevent conflict with sliced jobs hack
* Reorder URLs so that Django debug toolbar can work (#16352)
* Reorder URLs so that Django debug toolbar can work
* Move comment with URL move
* feat: support for oidc credential /test endpoint (#16370)
Adds support for testing external credentials that use OIDC workload identity tokens.
When FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED is enabled, the /test endpoints return
JWT payload details alongside test results.
- Add OIDC credential test endpoints with job template selection
- Return JWT payload and secret value in test response
- Maintain backward compatibility (detail field for errors)
- Add comprehensive unit and functional tests
- Refactor shared error handling logic
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
* Bind the install bundle to the ansible.receptor collection 2.0.8 version (#16396)
* [Devel] Config Endpoint Optimization (#16389)
* Improved performance of the config endpoint by reducing database queries in GET /api/controller/v2/config/
* Fix OIDC workload identity for inventory sync (#16390)
The cloud credential used by inventory updates was not going through
the OIDC workload identity token flow because it lives outside the
normal _credentials list. This overrides populate_workload_identity_tokens
in RunInventoryUpdate to include the cloud credential as an
additional_credentials argument to the base implementation, and
patches get_cloud_credential on the instance so the injector picks up
the credential with OIDC context intact.
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: integrate awx-tui to the awx_devel image (#16399)
* Aap 45980 (#16395)
* support bitbucket_dc webhooks
* add test
* update docs
* fix import for refactored method (#16394)
retrieve_workload_identity_jwt_with_claims is now
in a separate utility file, not in jobs.py
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* AAP-70257 controller collection should retry transient HTTP errors with exponential backoff. (#16415)
controller collection should retry transient HTTP errors with exponential backoff
* AAP-71844 Fix rrule fast-forward across DST boundaries (#16407)
Fix rrule fast-forward producing wrong occurrences across DST boundaries
The UTC round-trip in _fast_forward_rrule shifts the dtstart's local
hour when the original and fast-forwarded times are in different DST
periods. Since dateutil generates HOURLY occurrences by stepping in
local time, the shifted hour changes the set of reachable hours. With
BYHOUR constraints this causes a ValueError crash; without BYHOUR,
occurrences are silently shifted by 1 hour.
Fix by performing all arithmetic in the dtstart's original timezone.
Python aware-datetime subtraction already computes absolute elapsed
time regardless of timezone, so the UTC conversion was unnecessary
for correctness and actively harmful during fall-back ambiguity.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Correctly restrict push actions to ownership repos (#16398)
* Correctly restrict push actions to ownership repos
* Use standard action to see if push actions should run
* Run spec job for 2.6 and higher
* Be even more restrictve, do not push if on a fork
* [Devel] Performance Optimization for Select Hosts Query (#16413)
* Fixed black reformating
* Make test simulate 500k hosts in real world scenario
* feat: improve unauthorized response on aap deployments (#16422)
* fix: do not include secret values in the credentials test endpoint an… (#16425)
fix: do not include secret values in the credentials test endpoint and add a guard to make sure credentials are testable
* [devel backport] AAP-41742: Fix workflow node update failing when JT has unprompted labels (#16426)
* AAP-41742: Fix workflow node update failing when JT has unprompted labels
PATCH extra_data on a workflow node fails with
{"labels":["Field is not configured to prompt on launch."]}
when the node has labels associated but the JT has
ask_labels_on_launch=False.
The serializer was passing all persisted M2M state from prompts_dict()
to _accept_or_ignore_job_kwargs() on every PATCH, re-validating
unchanged fields. Fix scopes validation to only the fields in the
request; full re-validation still occurs when unified_job_template
is being changed.
* Capture attrs keys before _build_mock_obj mutates them
_build_mock_obj() pops pseudo-fields (limit, scm_branch, job_tags,
etc.) from attrs. Computing requested_prompt_fields after the pop
would miss those fields and skip their ask_on_launch validation.
* Include survey_passwords when validating extra_vars prompts
prompts_dict() emits survey_passwords alongside extra_vars.
_accept_or_ignore_job_kwargs uses it to decrypt encrypted survey
values before validation. Without it, encrypted password blobs
are validated as-is against the survey spec.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add test to ensure credential secret values are not returned (#16434)
* AAP-68024 perf: derive last_job_host_summary from query instead of denormalized FK (#16332)
* perf: stop eagerly updating Host.last_job_host_summary on every job completion
The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.
Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.
Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK
The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix .pk AttributeError, add job_template annotations, annotate host sublists
- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs
Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove stale failures_url from deprecated DashboardView
The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to changed files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: replace 10 subquery annotations with bulk prefetch
Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').
This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.
- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary
Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.
- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: scope annotation to views, restore license_error/canceled_on
- Remove with_latest_summary_id() from HostManager.get_queryset() to
avoid applying the correlated subquery to every Host query globally
(count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
host-serving views only
- Restore license_error and canceled_on to last_job summary fields
to avoid breaking API change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard _fetch_all() to skip bulk-attach on non-annotated querysets
Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove name from last_job_host_summary and canceled_on from last_job summary
Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary
Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to test_host_queryset.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Fix flake8 F841: remove unused job1/job2 variables in tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Add comment explaining why Prefetch was not used for host latest summary
Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix null handling to keep old behavior
---------
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
* [AAP-72722] Use url instead of jwt_aud for workload identity audience (#16432)
* [AAP-72722] Use url instead of jwt_aud for workload identity audience
The OIDC credential plugin's jwt_aud field is being removed. Use the
plugin's url field as the audience when requesting workload identity
tokens, since the target service URL is the appropriate audience value.
Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [Devel] Optimize host_list_rbac query (#16408)
* Defer ansible_facts in HostManager to avoid fetching large JSON column in host list queries (AAP-68023)
The host list endpoint (GET /api/v2/hosts/) fetches the ansible_facts
JSON column unnecessarily, contributing to the 7.8s median query time
at scale. This column can be very large and is not used by the list
serializer.
Changes:
- HostManager.get_queryset() now defers ansible_facts
- finish_fact_cache call site uses .only(*HOST_FACTS_FIELDS) to eagerly
load ansible_facts when actually needed, avoiding N+1 queries
- Unit test mocks updated to support .only() queryset chaining
- Points DAB dependency at the RBAC query optimization branch for
combined testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
* fix: constructed inventories no longer increase the host count (#16433)
* Fix version worktree (#16431)
* git worktree friendly precomit install
* worktrees don't have a .git directory. Before, docker-compose would
trigger pre-commit install and fail.
* make docker-compose work in git worktree
* AWX tries to discover the version via info stored in .git/ dir.
setuptools-scm is capable of finding the .git/ dir, starting from a
worktree, but is unable because only the worktree is mapped into the
container, not the .git/ dir itself. Thus, we have to detect and pass
the version into the container from outside. That is why this change
landed in the Makefile.
* fix: as_user() gateway session cookie fallback (#16437)
Add a fallback that checks for `gateway_sessionid` when no cookie
matches `session_cookie_name`, mirroring the existing fallback in
`Connection.login()`. The finally block now cleans up whichever
cookie name was actually used.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Pass setting to dispatcherd so it can be configured (#16438)
* fix: allow blank password field to fix OpenAPI schema validation (#16440)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* first pass porting over metrics
* move settings to defaults
* add more unit tests
* update unit tests
* lint fixes
* more lint fixes
* refactor and address feedback
* remove the api views
* remove model and move helper functions out of licensing
* add settings to API, fix tests, refactoring
* fix circular import
* update tests
* remove duplicate code, handle edge cases, use clearer naming, add test coverage
* update test for changes in ship()
* remove unneeded setting
* _discover_org should account for verify-tls=False
* directly assign settings, detect url, update tests
* log errors close to occurance
* rename function for clarity, focus on critical tests
* rename for clarity, lint fixes
* fix test params, priority for org discovery
* fix test failures and linting
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
Co-authored-by: Tong He <68936428+unnecessary-username@users.noreply.github.com>
Co-authored-by: Stevenson Michel <iamstevensonmichel@outlook.com>
Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: Sean Sullivan <ssulliva@redhat.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>
Co-authored-by: Lila Yasin <lyasin@redhat.com>
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* Consolidate implementation of same-org validation rule
* Update tests for the simplified validation
* Still do validation with deferance to the new callback
* Correctly falsy handling in view logic
AAP-74276: Replace setuptools with packaging in awxkit install_requires
The Python 3.12 upgrade replaced distutils.version.LooseVersion with
packaging.version.Version but did not update awxkit's install_requires.
setuptools is no longer needed at runtime since pkg_resources was also
replaced with importlib.metadata. This causes ModuleNotFoundError on
standalone CLI installs where packaging is not present.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Change fact processing loop to use file listing
* Fix some test
* Address coderabbit comments
* Handle saving facts in batches to keep memory low
* Improve log about mismatch in response to review comment
Add a fallback that checks for `gateway_sessionid` when no cookie
matches `session_cookie_name`, mirroring the existing fallback in
`Connection.login()`. The finally block now cleans up whichever
cookie name was actually used.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* git worktree friendly precomit install
* worktrees don't have a .git directory. Before, docker-compose would
trigger pre-commit install and fail.
* make docker-compose work in git worktree
* AWX tries to discover the version via info stored in .git/ dir.
setuptools-scm is capable of finding the .git/ dir, starting from a
worktree, but is unable because only the worktree is mapped into the
container, not the .git/ dir itself. Thus, we have to detect and pass
the version into the container from outside. That is why this change
landed in the Makefile.
* Defer ansible_facts in HostManager to avoid fetching large JSON column in host list queries (AAP-68023)
The host list endpoint (GET /api/v2/hosts/) fetches the ansible_facts
JSON column unnecessarily, contributing to the 7.8s median query time
at scale. This column can be very large and is not used by the list
serializer.
Changes:
- HostManager.get_queryset() now defers ansible_facts
- finish_fact_cache call site uses .only(*HOST_FACTS_FIELDS) to eagerly
load ansible_facts when actually needed, avoiding N+1 queries
- Unit test mocks updated to support .only() queryset chaining
- Points DAB dependency at the RBAC query optimization branch for
combined testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
* [AAP-72722] Use url instead of jwt_aud for workload identity audience
The OIDC credential plugin's jwt_aud field is being removed. Use the
plugin's url field as the audience when requesting workload identity
tokens, since the target service URL is the appropriate audience value.
Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: stop eagerly updating Host.last_job_host_summary on every job completion
The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.
Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.
Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK
The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix .pk AttributeError, add job_template annotations, annotate host sublists
- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs
Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove stale failures_url from deprecated DashboardView
The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to changed files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: replace 10 subquery annotations with bulk prefetch
Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').
This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.
- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary
Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.
- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: scope annotation to views, restore license_error/canceled_on
- Remove with_latest_summary_id() from HostManager.get_queryset() to
avoid applying the correlated subquery to every Host query globally
(count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
host-serving views only
- Restore license_error and canceled_on to last_job summary fields
to avoid breaking API change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard _fetch_all() to skip bulk-attach on non-annotated querysets
Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove name from last_job_host_summary and canceled_on from last_job summary
Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary
Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to test_host_queryset.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Fix flake8 F841: remove unused job1/job2 variables in tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Add comment explaining why Prefetch was not used for host latest summary
Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix null handling to keep old behavior
---------
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
* AAP-41742: Fix workflow node update failing when JT has unprompted labels
PATCH extra_data on a workflow node fails with
{"labels":["Field is not configured to prompt on launch."]}
when the node has labels associated but the JT has
ask_labels_on_launch=False.
The serializer was passing all persisted M2M state from prompts_dict()
to _accept_or_ignore_job_kwargs() on every PATCH, re-validating
unchanged fields. Fix scopes validation to only the fields in the
request; full re-validation still occurs when unified_job_template
is being changed.
* Capture attrs keys before _build_mock_obj mutates them
_build_mock_obj() pops pseudo-fields (limit, scm_branch, job_tags,
etc.) from attrs. Computing requested_prompt_fields after the pop
would miss those fields and skip their ask_on_launch validation.
* Include survey_passwords when validating extra_vars prompts
prompts_dict() emits survey_passwords alongside extra_vars.
_accept_or_ignore_job_kwargs uses it to decrypt encrypted survey
values before validation. Without it, encrypted password blobs
are validated as-is against the survey spec.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Correctly restrict push actions to ownership repos
* Use standard action to see if push actions should run
* Run spec job for 2.6 and higher
* Be even more restrictve, do not push if on a fork
Fix rrule fast-forward producing wrong occurrences across DST boundaries
The UTC round-trip in _fast_forward_rrule shifts the dtstart's local
hour when the original and fast-forwarded times are in different DST
periods. Since dateutil generates HOURLY occurrences by stepping in
local time, the shifted hour changes the set of reachable hours. With
BYHOUR constraints this causes a ValueError crash; without BYHOUR,
occurrences are silently shifted by 1 hour.
Fix by performing all arithmetic in the dtstart's original timezone.
Python aware-datetime subtraction already computes absolute elapsed
time regardless of timezone, so the UTC conversion was unnecessary
for correctness and actively harmful during fall-back ambiguity.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The cloud credential used by inventory updates was not going through
the OIDC workload identity token flow because it lives outside the
normal _credentials list. This overrides populate_workload_identity_tokens
in RunInventoryUpdate to include the cloud credential as an
additional_credentials argument to the base implementation, and
patches get_cloud_credential on the instance so the injector picks up
the credential with OIDC context intact.
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds support for testing external credentials that use OIDC workload identity tokens.
When FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED is enabled, the /test endpoints return
JWT payload details alongside test results.
- Add OIDC credential test endpoints with job template selection
- Return JWT payload and secret value in test response
- Maintain backward compatibility (detail field for errors)
- Add comprehensive unit and functional tests
- Refactor shared error handling logic
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
* Add new test for artfact precedence upstream node vs outer workflow
* Fix bugs, upstream artifacts come first for precedence
* Track nested artifacts path through ancestor_artifacts on root nodes
* Fix case where first root node did not get the vars
* touchup comment
* Prevent conflict with sliced jobs hack
Fix CI: Pin setuptools_scm<10 to fix api-lint build failure
setuptools-scm 10.0.5 (with its new vcs-versioning dependency) requires
a [tool.setuptools_scm] or [tool.vcs-versioning] section in pyproject.toml.
AWX intentionally omits this section because it uses a custom version
resolution via setup.cfg (version = attr: awx.get_version). The new major
version of setuptools-scm treats the missing section as a fatal error when
building the sdist in tox's isolated build, causing the linters environment
to fail.
Pinning to <10 restores compatibility with the existing version resolution
strategy.
Failing run: https://github.com/ansible/awx/actions/runs/23744310714
Branch: devel
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Pass plugin_description through to CredentialType.description
Propagate the plugin_description field from credential plugins into the
CredentialType description when loading and creating managed credential
types, including updates to existing records.
Assisted-by: Claude
* Add unit tests for plugin_description passthrough to CredentialType
Tests cover load_plugin, get_creation_params, and
_setup_tower_managed_defaults handling of the description field.
Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: PabloHiro <palonso@redhat.com>
Add install-time feature flag for OIDC workload identity credential types
Implements FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED feature flag to gate
HashiCorp Vault OIDC credential types as a Technology Preview feature.
When the feature flag is disabled (default), OIDC credential types are
not loaded into the plugin registry at application startup and do not
exist in the database.
When enabled, OIDC credential types are loaded normally and function
as expected.
Changes:
- Add FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED setting (defaults to False)
- Add OIDC_CREDENTIAL_TYPE_NAMESPACES constant for maintainability
- Modify load_credentials() to skip OIDC types when flag is disabled
- Add test coverage (2 test cases)
This is an install-time flag that requires application restart to take
effect. The flag is checked during application startup when credential
types are loaded from plugins.
Fixes: AAP-64510
Assisted-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: NameError in wsrelay when JSON decode fails with DEBUG logging
run_connection() referenced payload in the JSONDecodeError handler,
but payload was never assigned because json.loads() is what failed.
Use msg.data instead to log the raw message content.
Fixes: AAP-68045
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix other instance of undefined payload
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
* Fix job cancel chain bugs
* Early relief valve for canceled jobs, ATF related changes
* Add test and fix for approval nodes as well
* Revert unwanted change
* Refactor workflow approval nodes to make it more clean
* Revert data structure changes
* Delete local utility file
* Review comment addressing
* Use canceled status in websocket
* Delete slop
* Add agent marker
* Bugbot comment about status websocket mismatch
* Fix SonarCloud Reliability issues: time-dependent class attrs and dict comprehensions
- Move last_stats/last_flush from class body to __init__ in CallbackBrokerWorker
(S8434: time-dependent expressions evaluated at class definition)
- Replace dict comprehensions with dict.fromkeys() in has_create.py
(S7519: constant-value dict should use fromkeys)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix callback receiver tests to use flush(force=True)
Tests were implicitly relying on last_flush being a stale class-level
timestamp. Now that last_flush is set in __init__, the time-based flush
condition isn't met when flush() is called immediately after construction.
Use force=True to explicitly trigger an immediate flush in tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix SonarCloud Reliability Rating issue in Common exception constructor
The constructor had code paths where attributes were not consistently
initialized and super().__init__() was not called, which was flagged
as a Reliability Rating issue by SonarCloud. Ensures all branches
properly set self.status_string and self.msg, and call super().__init__().
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Pass workload TTL to Gateway (minimal changes) assisted-by: Claude
* lint
Assisted-by: Claude
* fix unit tests assisted-by claude
* use existing functions assisted-by: Claude
* fix test assisted-by: Claude
* fixes for sonarcloud assisted-by: Claude
* nit
* nit
* address feedback
* feedback from pr review assisted-by: Claude
* feedback from pr review assisted-by: Claude
* Apply suggestion from @dleehr
Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>
* lint assisted-by: Claude
* fix: narrow vendor_collections_dir fixture teardown scope (#16326)
Only remove the collection directory the fixture created
(redhat/indirect_accounting) instead of the entire
/var/lib/awx/vendor_collections/ root, so we don't accidentally
delete vendor collections that may have been installed by the
build process.
Forward-port of ansible/tower#7350.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-67436 Remove pbr from requirements (#16337)
* Remove pbr from requirements
pbr was temporarily added to support ansible-runner installed from a git
branch. It is no longer needed as a direct dependency.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Retrigger CI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [AAP-64062] Enforce JWT-only authentication for Controller when deployed as part of AAP (#16283)
After all settings are loaded, override DEFAULT_AUTHENTICATION_CLASSES
to only allow Gateway JWT authentication when RESOURCE_SERVER__URL is
set. This makes the lockdown immutable — no configuration file or
environment variable can re-enable legacy auth methods (Basic, Session,
OAuth2, Token).
This is the same pattern used by Hub (galaxy_ng) and EDA (eda-server)
for ANSTRAT-1840.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Re-trigger CI
Made-with: Cursor
* Re-trigger CI
Made-with: Cursor
* [AAP-63314] Pass job timeout as workload_ttl_seconds to Gateway Assisted-by: Claude
* Additional unit test requested at review Assisted-by: Claude
* Revert profiled_pg/base.py rebase error, unrelated to AAP-63314
* revert requirements changes introduced by testing
* revert
* revert
* docstring nit from coderabbit
---------
Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
After all settings are loaded, override DEFAULT_AUTHENTICATION_CLASSES
to only allow Gateway JWT authentication when RESOURCE_SERVER__URL is
set. This makes the lockdown immutable — no configuration file or
environment variable can re-enable legacy auth methods (Basic, Session,
OAuth2, Token).
This is the same pattern used by Hub (galaxy_ng) and EDA (eda-server)
for ANSTRAT-1840.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove pbr from requirements
pbr was temporarily added to support ansible-runner installed from a git
branch. It is no longer needed as a direct dependency.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Retrigger CI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Only remove the collection directory the fixture created
(redhat/indirect_accounting) instead of the entire
/var/lib/awx/vendor_collections/ root, so we don't accidentally
delete vendor collections that may have been installed by the
build process.
Forward-port of ansible/tower#7350.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "AAP-58452 Add version fallback for external query files (#16309)"
This reverts commit 0f2692b504.
* AAP-58441: Add runtime integration for external query collection (#7208)
Extend build_private_data_files() to copy vendor collections from
/var/lib/awx/vendor_collections/ to the job's private_data_dir,
making external query files available to the indirect node counting
callback plugin in execution environments.
Changes:
- Copy vendor_collections to private_data_dir during job preparation
- Add vendor_collections path to ANSIBLE_COLLECTIONS_PATH in build_env()
- Gracefully handle missing source directory with warning log
- Feature gated by FEATURE_INDIRECT_NODE_COUNTING_ENABLED flag
This enables external query file discovery for indirect node counting
across all deployment types (RPM, Podman, OpenShift, Kubernetes) using
the existing private_data_dir mechanism.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* [stable-2.6] AAP-58451: Add callback plugin discovery for external query files (#7223)
* AAP-58451: Add callback plugin discovery for external query files
Extend the indirect_instance_count callback plugin to discover and load
external query files from the bundled redhat.indirect_accounting collection
when embedded queries are not present in the target collection.
Changes:
- Add external query discovery with precedence (embedded queries first)
- External query path: redhat.indirect_accounting/extensions/audit/
external_queries/{namespace}.{name}.{version}.yml
- Use self._display.v() for external query messages (visible with -v)
- Use self._display.vv() for embedded query messages (visible with -vv)
- Fix: Change .exists() to .is_file() per Traversable ABC
- Handle missing external query collection gracefully (ModuleNotFoundError)
Note: This implements exact version match only. Version fallback logic
is covered in AAP-58452.
* fix CI error when using Traversable.is_file
* Add minimal implementation for AAP-58451
* Fix formatting
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* AAP-58452 Add version fallback for external query files (#7254)
* AAP-58456 unit test suite for external query handling (#7283)
* Add unit tests for external query handling
* Refactor unit tests for external query handling
* Refactor indirect node counting callback code to improve testing code
* Refactor unit tests for external query handling for improved callback code
* Fix test for majore version boundary check
* Fix weaknesses in some unit tests
* Make callback plugin module self contained, independent from awx
* AAP-58470 integration tests (core) for external queries (#7278)
* Add collection for testing external queries
* Add query files for testing external query file runtime integration
* Add live tests for external query file runtime integration
* Remove redundant wait for events and refactor test data folders
* Fix unit tests: mock flag_enabled to avoid DB access
The AAP-58441 cherry-pick added a flag_enabled() call in
BaseTask.build_private_data_files(), which is called by all task types.
Tests for RunInventoryUpdate and RunJob credentials now hit this code
path and need the flag mocked to avoid database access in unit tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: attempt exact query file match before Version parsing (#7345)
The exact-version filename check does not require PEP440 parsing, but
Version() was called first, causing early return on non-PEP440 version
strings even when an exact file exists on disk. Move the exact file
check before Version parsing so fallback logic only parses when needed.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Do no longer mutate global sys.modules (#7337)
* [stable-2.6] AAP-58452 fix: Add queries_dir guard (#7338)
* Add queries_dir guard
* fix: update unit tests to mock _get_query_file_dir instead of files
The TestVersionFallback tests mocked `files()` with chainable path
mocks, but `find_external_query_with_fallback` now uses
`_get_query_file_dir()` which returns the queries directory directly.
Mock the helper instead for simpler, correct tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove unused EXTERNAL_QUERY_PATH constant (#7336)
The constant was defined but never referenced — the path is constructed
inline via Traversable's `/` operator which requires individual segments,
not a slash-separated string.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: restore original feature flag state in test fixture (#7347)
The enable_indirect_host_counting fixture unconditionally disabled the
FEATURE_INDIRECT_NODE_COUNTING_ENABLED flag on teardown, even when it
was already enabled before the test (as is the case in development via
development_defaults.py). This caused test_indirect_host_counting to
fail when run after the external query tests, because the callback
plugin was no longer enabled.
Save and restore the original flag state instead.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add new tests for bug saving concurrent facts
* Fix first bug and improve tests
* Fix new bug where concurrent job clears facts from other job in unwanted way
* minor test fixes
* Add in missing playbook
* Fix host reference for constructed inventory
* Increase speed for concurrent fact tests
* Make test a bit faster
* Fix linters
* Add some functional tests
* Remove the sanity test
* Agent markers added
* Address SonarCloud
* Do backdating method, resolving stricter assertions
* Address coderabbit comments
* Address review comment with qs only method
* Delete missed sleep statement
* Add more coverage
Django 5.2 restricts LogoutView to POST only (deprecated in 4.1,
removed in 5.0+). Without this fix, GET requests to /api/logout/
return 405 Method Not Allowed.
Add http_method_names override and a get() method that delegates
to post() where auth_logout() actually runs
* feat: workload identity credentials integration
* feat: cache credentials and add context property to Credential
Assisted-by: Claude
* feat: include safeguard in case feature flag is disabled
* feat: tests to validate workload identity credentials integration
* fix: affected tests by the credential cache mechanism
* feat: remove word cache from variables and comments, use standard library decorators
* fix: reorder tests in correct files
* Use better error catching mechanisms
* Adjust logic to support multiple credential input sources and use internal field
* Remove hardcoded credential type names
* Add tests for the internal field
Assited-by: Claude
* Added worflow dispatch to trigger ci on release_4.6 and stable-2.6
* fix error cascading to subsequent jobs for cron
* made compose_tag resolve to the correct branch name
Remove SELECT FOR UPDATE from job dispatch to reduce transaction rollbacks
Move status transition from BaseTask.transition_status (which used
SELECT FOR UPDATE inside transaction.atomic()) into
dispatch_waiting_jobs. The new approach uses filter().update() which
is atomic at the database level without requiring explicit row locks,
reducing transaction contention and rollbacks observed in perfscale
testing.
The transition_status method was an artifact of the feature flag era
where we needed to support both old and new code paths. Since
dispatch_waiting_jobs is already a singleton
(on_duplicate='queue_one') scoped to the local node, the
de-duplication logic is unnecessary.
Status is updated after task submission to dispatcherd, so the job's
UUID is in the dispatch pipeline before being marked running —
preventing the reaper from incorrectly reaping jobs during the
handoff window. RunJob.run() handles the race where a worker picks
up the task before the status update lands by accepting waiting and
transitioning it to running itself.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add retrieve_workload_identity_jwt to jobs.py and tests
* Apply linting
* Add precondition to client retrieval
* Add test case for client not configured
* Remove trailing period in match string
The schema-swagger-ui URL was removed from awx/api/urls/urls.py in
d7eb714859 when docs endpoints moved to DAB's api_documentation app,
but the reverse call in ApiRootView was not removed, causing a
NoReverseMatch error in development mode.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The awx CLI derives available fields for the `modify` command from
the list endpoint's POST action schema. Users with object-level
admin permissions (e.g., Project Admin) but no list-level POST
permission see no field flags, making modify unusable despite having
PUT access on the detail endpoint.
Fall back to the detail endpoint's action schema when POST is not
available on the list endpoint, and prefer PUT over POST when
building modify arguments.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Introduced in PR https://github.com/ansible/awx/pull/16058/changes
then a later large merge from AAP back into devel removed the changes
* This PR re-introduces the github app lookup migration rename tests
with the migration names updated and the kind to namespace correction
* Multiple credentialtype's have the same kind and kind values look
like: cloud, network, machine, etc.
* namespace is the field that we want to rename
fix: align pip version constraint in requirements_dev.txt with requirements.txt (fixes#16272)
requirements.txt pins pip==25.3 while requirements_dev.txt specified
pip>=21.3,<=24.0, causing ResolutionImpossible when installing both.
Updated requirements_dev.txt to use pip>=25.3 to maintain compatibility.
* AAP-62657 Add populate_claims_for_workload function and unit tests
* Update safe_get helper function
* Trigger CI rebuild to pick up latest django-ansible-base
* Trigger CI after org visibility update
* Retrigger CI
* Rename workload to job, refine safe_get helper function
* Update test_jobs to use job fixture
* Retrigger CI
* Create fresh job, removed launched_by since this is read-only property
* Retrigger CI after runner issues
* Retrigger CI after runner issues
* Add unit tests for other workload types
* Update CLAIM_LAUNCHED_BY_USER_NAME and CLAIM_LAUNCHED_BY_USER_ID, with CLAIM_LAUNCHED_BY_NAME and CLAIM_LAUNCHED_BY_ID
* Generate claims with a more static schema
try to operate directly on object when possible
For cases where field is valid for the type, but null value
still add the field, so blank and null values appear
* Allow unified related items to be omittied
---------
Co-authored-by: AlanCoding <arominge@redhat.com>
* Enable new fancy asyncio metrics for dispatcherd
Remove old dispatcher metrics and patch in new data from local whatever
Update test fixture to new dispatcherd version
* Update dispatcherd again
* Handle node filter in URL, and catch more errors
* Add test for metric filter
* Split module for dispatcherd metrics
The OpenAPI schema incorrectly showed all 12 credential type kinds as
valid for POST/PUT/PATCH operations, when only 'cloud' and 'net' are
allowed for custom credential types. This caused API clients and LLM
agents to receive HTTP 400 errors when attempting to create credential
types with invalid kind values.
Add postprocessing hook to filter CredentialTypeRequest and
PatchedCredentialTypeRequest schemas to only show 'cloud', 'net',
and null as valid enum values, matching the existing validation logic.
No API behavior changes - this is purely a documentation fix.
Co-authored-by: Claude <noreply@anthropic.com>
Changed two instances of 'cancelled' to 'canceled' in awx/main/wsrelay.py
to match AWX's standardized American English spelling convention.
- Updated log message in WebsocketRelayConnection.connect()
- Updated comment in WebSocketRelayManager.cleanup_offline_host()
Fixes#15177
Signed-off-by: Joey Washburn <joey@joeywashburn.com>
Strip leading and trailing whitespace from SSH keys in validate_ssh_private_key()
to handle common copy-paste scenarios where hidden newlines cause base64 decoding
failures.
Changes:
- Added data.strip() in validate_ssh_private_key() before calling validate_pem()
- Added test_ssh_key_with_whitespace() to verify keys with leading/trailing
newlines are properly sanitized and validated
This prevents the confusing "HTTP 500: Internal Server Error" and
"binascii.Error: Incorrect padding" errors when users paste SSH keys with
accidental whitespace.
Fixes#14219
Signed-off-by: Joey Washburn <joey@joeywashburn.com>
* Add dispatcherctl command
* Add tests for dispatcherctl command
* Exit early if sqlite3
* Switch to dispatcherd mgmt cmd
* Move unwanted command options to run_dispatcher
* Add test for new stuff
* Update the SOS report status command
* make docs always reference new command
* Consistently error if given config file
This setting is set in defaults.py, but
currently not being used. More technically,
project_update.yml is not passing this value to
the insights.py action plugin. Therefore, we
can safely remove references to it.
insights.py already has a default oidc endpoint
defined for authentication.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Additional dispatcher removal simplifications and waiting repear updates
* Fix double call and logging message
* Implement bugbot comment, should reap running on lost instances
* Add test case for new pending behavior
* Added link and ref to openAPI spec for community
* Update docs/docsite/rst/contributor/openapi_link.rst
Co-authored-by: Don Naro <dnaro@redhat.com>
* add sphinxcontrib-redoc to requirements
* sphinxcontrib.redoc configuration
* create openapi directory and files
* update download script for both schema files
* suppress warning for redoc
* update labels
* fix extra closing parenthesis
* update schema url
* exclude doc config and download script
The Sphinx configuration (conf.py) and schema download script
(download-json.py) are not application logic and used only for building
documentation. Coverage requirements for these files are overkill.
* exclude only the sphinx config file
---------
Co-authored-by: Don Naro <dnaro@redhat.com>
* WIP First pass
* started removing feature flags and adjusting logic
* Add decorator
* moved to dispatcher decorator
* updated as many as I could find
* Keep callback receiver working
* remove any code that is not used by the call back receiver
* add back auto_max_workers
* added back get_auto_max_workers into common utils
* Remove control and hazmat (squash this not done)
* moved status out and deleted control as no longer needed
* removed unused imports
* adjusted test import to pull correct method
* fixed imports and addressed clusternode heartbeat test
* Update function comments
* Add back hazmat for config and remove baseworker
* added back hazmat per @alancoding feedback around config
* removed baseworker completely and refactored it into the callback
worker
* Fix dispatcher run call and remove dispatch setting
* remove dispatcher mock publish setting
* Adjust heartbeat arg and more formatting
* fixed the call to cluster_node_heartbeat missing binder
* Fix attribute error in server logs
* Enhance OpenAPI schema with AI descriptions and fix method names
Add x-ai-description extensions to API endpoints for better AI agent
comprehension. Fix view method names to
ensure proper drf-spectacular schema generation.
* Enhance OpenAPI schema with AI descriptions and fix method names
Add x-ai-description extensions to API endpoints for better AI agent
comprehension. Fix view method names to
ensure proper drf-spectacular schema generation.
Remove transitive dependencies no longer needed by kubernetes 35.0.0
Removes google-auth and rsa which were transitive dependencies of the older
kubernetes client but are no longer required in v35.0.0.
Adds cachetools as a direct dependency since it's used by awx/conf/settings.py
for TTLCache (was previously a transitive dep of google-auth).
- Move kubernetes from git-based install to PyPI (v35.0.0 now available)
- Remove urllib3 cap comment since kubernetes 35.0.0 no longer restricts it
- Update README.md upgrade blocker documentation
* docs: update readthedocs.io URLs to docs.ansible.com equivalents
🤖 Generated with Claude Code
https://claude.ai/code
Co-Authored-By: Claude <noreply@anthropic.com>
* Update Bullhorn newsletter link in communication docs
---------
Co-authored-by: Claude <noreply@anthropic.com>
Refactored code to use Python's built-in datetime.timezone and zoneinfo instead of pytz for timezone handling. This modernizes the codebase and removes the dependency on pytz, aligning with current best practices for timezone-aware datetime objects.
Introduces new Makefile targets to update and upgrade requirements files using pip-compile, both directly and via docker-runner. These additions streamline dependency management for development and CI workflows.
Switch to git-based installation of kubernetes python client from
github.com/kubernetes-client/python at commit df31d90d6c910d6b5c883b98011c93421cac067d
(release-34.0 branch). This also allows removing the urllib3<2.4.0 upper bound
constraint that was previously required by kubernetes 34.1.0 from PyPI.
Use dnf module for Node.js 18 instead of n version manager
The n version manager fails to extract Node.js archives due to very long
file paths in include/node/openssl/archs/ directories when running in
Docker BuildKit's overlay filesystem. This causes CI build failures with
tar "Cannot open: Invalid argument" errors.
Switch to installing Node.js 18 directly from CentOS Stream 9's module
stream which avoids the archive extraction issue entirely.
* Fix ARM64 build failure by upgrading dev container Node.js to 18
Node.js 16.13.1 fails to extract on ARM64 in Docker BuildKit's
overlay filesystem during multi-arch builds. Upgrade to Node 18
which is already used by the UI builder stage and has proper
ARM64 support.
* Fix collectstatic failure by setting AWX_MODE=default
AWX_MODE=defaults is an intentionally "invalid" environment name that:
1. Loads only defaults.py - the base settings file without any environment-specific overrides (development_defaults.py, production_defaults.py, etc.)
2. Bypasses production checks - since "production" not in "defaults", it skips the assertion that requires /etc/tower/settings.py to exist
3. Bypasses development mode - since is_development_mode would be false
This is perfect for collectstatic during container build because:
- No database connection needed
- No secret key needed (hence SKIP_SECRET_KEY_CHECK)
- No PostgreSQL version check (hence SKIP_PG_VERSION_CHECK)
- Just need minimal Django settings to collect static files
* Fix pip version constraint for Python 3.12 compatibility
Remove outdated pip<22.0 constraint that was a workaround for
pip-tools#1558. This issue was fixed in pip-tools 6.5.0+ and
the old constraint breaks Python 3.12 where pkgutil.ImpImporter
was removed.
* Update requirements.txt
* Fix license file inconsistencies with requirements
- Rename awx-plugins.interfaces.txt to awx-plugins-interfaces.txt
to match the package name in requirements
- Remove backports-tarfile.txt and importlib-resources.txt as these
packages are no longer in requirements
* Fix updater.sh for pip 25.3 normalized output format
Changes to requirements_git.txt:
- Update to PEP 440 format (name @ git+url) to match pip-compile output
- Normalize package names (hyphens instead of dots/underscores)
- Sort extras alphabetically with hyphens (e.g., jwt-consumer not jwt_consumer)
- Add documentation explaining format requirements
Changes to updater.sh:
- Escape BRE regex metacharacters in sed pattern to handle brackets in extras
- Change sed delimiter from ! to | to avoid conflict with comment text
- Add explicit return statements to functions
- Assign positional parameters to local variables
- Redirect error messages to stderr
- Replace backticks with $() for command substitution
- Pin pip to version 25.3
requirements.txt regenerated via updater.sh
* Normalize package names in requirements.in to match pip output
- prometheus_client -> prometheus-client
- setuptools_scm -> setuptools-scm
- dispatcherd[pg_notify] -> dispatcherd[pg-notify]
PEP 503 specifies that package names should use hyphens.
* Fix license files to match normalized package names
- Remove awx_plugins.interfaces.txt (duplicate of awx-plugins-interfaces.txt)
- Rename system-certifi.txt to certifi.txt to match package name
Deleted the awx/main/management/commands/graph_jobs.py file and removed the asciichartpy package from requirements. This cleans up unused code and dependencies related to terminal job status graphing.
* update to Python 3.12
* remove use of utcnow
* switch to timezone.utc
datetime.UTC is an alias of datetime.timezone.utc. if we're doing the double import for datetime it's more straightforward to just import timezone as well and get it directly
* debug python env version issue
* change python version
* pin to SHA and remove debug portion
* Remove the dynamic filter on dispatcher startup
Configure the dynamic logging level only on startup
* Special case for log level on settings change
* Add unit test for new behavior
* Add test for initial config
* Mark test django DB
* Do necessary requirement bump
* Delete cache in live test fixture
* Add test to recreate the error
* Also begin to add detection for empty event
* Remove breakpoint
* fix: ignore events with missing event types
* run linter and apply changes
---------
Co-authored-by: AlanCoding <arominge@redhat.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
docker buildx build fails with
"Error: Unable to find a match: rsyslog-8.2102.0-106.el9"
unpinning builds successfully for both arm64 and x86_64
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Upgrade to Django 5.2 LTS with compatibility fixes across fields, migrations, dispatch config, tests, and dev deps.
Dependencies:
- Upgrade django to 5.2.8 and relax requirements.in to >=5.2,<5.3.
- Bump django-debug-toolbar to >=6.0 for compatibility.
Backend:
- awx/conf/fields.py: switch URL TLD regex to use DomainNameValidator.ul in custom URLField.
- awx/main/management/commands/gather_analytics.py: use datetime.timezone.utc for naïve datetime handling.
- awx/main/dispatch/config.py: add mock_publish option; avoid DB access for test runs, set default max_workers, and support a noop broker.
Migrations (SQLite/Postgres compatibility):
- Add awx/main/migrations/_sqlite_helper.py with db-aware AlterIndexTogether/RenameIndex wrappers; consume in 0144_event_partitions.py and 0184_django_indexes.py.
- Update 0187_hop_nodes.py to use CheckConstraint(condition=...).
- Add 0205_alter_instance_peers_alter_job_hosts_and_more.py adjusting through_fields/relations on instance.peers, job.hosts, and role.ancestors.
- _dab_rbac.py: iterate roles with chunk_size=1000 for migration performance.
Tests:
Include hcp_terraform in default credential types in test_credential.py.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
Adding ansible_base.api_documentation
to the INSTALL_APPS which extends the schema
to include an LLM-friendly description
to each endpoint
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
* AAP-57817 Add Redis connection retry using redis-py 7.0+ built-in mechanism
* Refactor Redis client helpers to use settings and eliminate code duplication
* Create awx/main/utils/redis.py and move Redis client functions to avoid circular imports
* Fix subsystem_metrics to share Redis connection pool between
client and pipeline
* Cache Redis clients in RelayConsumer and RelayWebsocketStatsManager to avoid creating new connection pools on every call
* Add cap and base config
* Add Redis retry logic with exponential backoff to handle connection failures during long-running operations
* Add REDIS_BACKOFF_CAP and REDIS_BACKOFF_BASE settings to allow
adjustment of retry timing in worst-case scenarios without code changes
* Simplify Redis retry tests by removing unnecessary reload logic
Update schema upload workflows to organize S3 files by product name:
- Upload schemas to s3://awx-public-ci-files/{product}/{branch}/schema.json
- Update Makefile to download from product-specific paths for schema diff
- Update feature branch deletion to clean up from correct product path
This separates AWX and Tower schemas into distinct S3 folders.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
* add force flag to refspec
* Development of git --amend test
* Update awx/main/tests/live/tests/conftest.py
Co-authored-by: Alan Rominger <arominge@redhat.com>
---------
Co-authored-by: AlanCoding <arominge@redhat.com>
Modify the invocation of @task_awx to accept timeout and
on_duplicate keyword arguments. These arguments are
only used in the new dispatcher implementation.
Add decorator params:
- timeout
- on_duplicate
to tasks to ensure better recovery for
stuck or long-running processes.
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Change Swagger UI endpoint from /api/swagger/ to /api/docs/
- Update URL pattern to use /docs/ instead of /swagger/
- Update API root response to show 'docs' key instead of 'swagger'
- Add authentication requirement for schema documentation endpoints
- Update contact email to controller-eng@redhat.com
The schema endpoints (/api/docs/, /api/schema/, /api/redoc/) now
require authentication to prevent unauthorized access to API
documentation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Require authentication for all schema endpoints including /api/schema/
Create custom view classes that enforce authentication for all schema
endpoints to prevent inconsistent access control where UI views required
authentication but the raw schema endpoint remained publicly accessible.
This ensures all schema endpoints (/api/schema/, /api/docs/, /api/redoc/)
consistently require authentication.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add unit tests for authenticated schema view classes
Add test coverage for the new AuthenticatedSpectacular* view classes
to ensure they properly require authentication.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* remove unused import
---------
Co-authored-by: Claude <noreply@anthropic.com>
* AAP-45927 Add drf-spectacular
- Remove drf-yasg
- Add drf-spectacular
* move SPECTACULAR_SETTINGS from development_defaults.py to defaults.py
* move SPECTACULAR_SETTINGS from development_defaults.py to defaults.py
* Fix swagger tests: enable schema endpoints in all modes
Schema endpoints were restricted to development mode, causing
test_swagger_generation.py to fail. Made schema URLs available in
all modes and fixed deprecated Django warning filters in pytest.ini.
* remove swagger from Makefile
* remove swagger from Makefile
* change docker-compose-build-swagger to docker-compose-build-schema
* remove MODE
* remove unused import
* Update genschema to use drf-spectacular with awx-link dependency
- Add awx-link as dependency for genschema targets to ensure package metadata exists
- Remove --validate --fail-on-warn flags (schema needs improvements first)
- Add genschema-yaml target for YAML output
- Add schema.yaml to .gitignore
* Fix detect-schema-change to not fail on schema differences
Add '-' prefix to diff command so Make ignores its exit status.
diff returns exit code 1 when files differ, which is expected behavior
for schema change detection, not an error.
* Truncate schema diff summary to stay under GitHub's 1MB limit
Limit schema diff output in job summary to first 1000 lines to avoid
exceeding GitHub's 1MB step summary size limit. Add message indicating
when diff is truncated and direct users to job logs or artifacts for
full output.
* readd MODE
* add drf-spectacular to requirements.in and the requirements.txt generated from the script
* Add drf-spectacular BSD license file
Required for test_python_licenses test to pass now that drf-spectacular
is in requirements.txt.
* add licenses
* Add comprehensive unit tests for CustomAutoSchema
Adds 15 unit tests for awx/api/schema.py to improve SonarCloud test
coverage. Tests cover all code paths in CustomAutoSchema including:
- get_tags() method with various scenarios (swagger_topic, serializer
Meta.model, view.model, exception handling, fallbacks, warnings)
- is_deprecated() method with different view configurations
- Edge cases and priority ordering
All tests passing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* remove unused imports
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix CI: Use Python 3.13 for ansible-test compatibility
ansible-test only supports Python 3.11, 3.12, and 3.13.
Changed collection-integration jobs from '3.x' to '3.13'
to avoid using Python 3.14 which is not supported.
* Fix ansible-test Python version for CI integration tests
ansible-test only supports Python 3.11, 3.12, and 3.13.
Added ANSIBLE_TEST_PYTHON_VERSION variable to explicitly pass
--python 3.13 flag to ansible-test integration command.
This prevents ansible-test from auto-detecting and using
Python 3.14.0, which is not supported.
* Fix CI: Execute ansible-test with Python 3.13 to avoid
unsupported Python 3.14
* Fix CI: Use Python 3.13 across all jobs to avoid Python
3.14 compatibility issues
* Fix CI: Use 'python' and 'ansible.test' module for Python
3.13 compatibility
* Fix CI: Use 'python' instead of 'python3' for Python 3.13
compatibility
* Fix CI: Ensure ansible-test uses Python 3.13 environment
explicitly
* Fix: Remove silent failure check for ansible-core in test suite
* Fix CI: Export PYTHONPATH to make awxkit available to ansible-test
* Fix CI: Use 'python' in run_awx_devel to maintain Python
3.13 environment
* Fix CI: Remove setup-python from awx_devel_image that was resetting Python 3.13 to 3.14
* actually upload PR coverage reports and inject PR number if report is
generated from a PR
* upload general report of devel on merge and make things kinda pretty
* add new file to separate out the schema check so that it is no longer
part of CI check and won't cacuse the whole workflow to fail
* remove old API schema check from ci.yml
Moved the AddField operation before the RunPython operations for 'rename_jts' and 'rename_projects' in migration 0200_template_name_constraint.py. This ensures the new 'org_unique' field exists before related data migrations are executed.
Fix
```
django.db.utils.ProgrammingError: column main_unifiedjobtemplate.org_unique does not exist
```
while applying migration 0200_template_name_constraint.py
when there's a job template or poject with duplicate name in the same org
* Requirements POC docs from Claude Code eval
* Removed unnecessary reference.
* Excluded custom DRF configurations per @AlanCoding
* Implement review changes from @chrismeyersfsu
---------
Co-authored-by: Peter Braun <pbraun@redhat.com>
* prometheus-client returns an additional value as of v.0.22.0
* add license, remove outdated ones, add new embedded sources
* update requirements and UPGRADE BLOCKERs in README
* added sonar config file and started cleaning up
* we do not place the report at the root of the repo
* limit scope to only the awx directory and its contents
* update exclusions for things in awx/ that we don't want covered
settings.SUBSCRIPTIONS_USERNAME and
settings.SUBSCRIPTIONS_CLIENT_ID
should be mutually exclusive. This is because
the POST to api/v2/config/attach/ accepts only
a subscription_id, and infers which credentials to
use based on settings. If both are set, it is ambiguous
and can lead to unexpected 400s when attempting
to attach a license.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Previously, we would error out because we assumed that when we got a
metrics payload from redis, that there was data in it and it was for
the current host.
* Now, we do not assume that since we got a metrics payload, that is
well formed and for the current hostname because the hostname could
have changed and we could have not yet collected metrics for the new
host.
Separate out operation subsystem metrics to fix duplicate error
Remove unnecessary comments
Revert to single subsystem_metrics_* metric with labels
Format via black
* We had race conditions with the system_administrator role being
created just-in-time. Instead of fixing the race condition(s), dodge
them by ensuring the role always exists
* Disconnect logic to fill in role parents
Get tests passing hopefully
Whatever SonarCloud
* remove role parents/children endpoints and related views
* remove duplicate get_queryset method from RoleTeamsList
---------
Co-authored-by: Peter Braun <pbraun@redhat.com>
Allow users to do subscription management using
Red Hat username and password.
In basic auth case, the candlepin API
at subscriptions.rhsm.redhat.com will be used instead
of console.redhat.com.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Added tests for cross org sharing of credentials
* added negative testing for sharing of credentials
* added conditions and tests for roleteamslist regarding cross org credentials
* removed redundant codes
* made error message more articulated and specific
2025-09-30 10:44:27 -04:00
495 changed files with 22862 additions and 5967 deletions
@@ -70,10 +70,10 @@ Thank you for your submission and for supporting AWX!
- Hello, we'd love to help, but we need a little more information about the problem you're having. Screenshots, log outputs, or any reproducers would be very helpful.
### Code of Conduct
- Hello. Please keep in mind that Ansible adheres to a Code of Conduct in its community spaces. The spirit of the code of conduct is to be kind, and this is your friendly reminder to be so. Please see the full code of conduct here if you have questions: https://docs.ansible.com/ansible/latest/community/code_of_conduct.html
- Hello. Please keep in mind that Ansible adheres to a Code of Conduct in its community spaces. The spirit of the code of conduct is to be kind, and this is your friendly reminder to be so. Please see the full code of conduct here if you have questions: https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html
### EE Contents / Community General
- Hello. The awx-ee contains the collections and dependencies needed for supported AWX features to function. Anything beyond that (like the community.general package) will require you to build your own EE. For information on how to do that, see https://ansible-builder.readthedocs.io/en/stable/ \
- Hello. The awx-ee contains the collections and dependencies needed for supported AWX features to function. Anything beyond that (like the community.general package) will require you to build your own EE. For information on how to do that, see https://docs.ansible.com/projects/builder/en/stable/ \
\
The Ansible Community is looking at building an EE that corresponds to all of the collections inside the ansible package. That may help you if and when it happens; see https://github.com/ansible-community/community-topics/issues/31 for details.
@@ -88,7 +88,7 @@ The Ansible Community is looking at building an EE that corresponds to all of th
- Hello, we think your idea is good! Please consider contributing a PR for this following our contributing guidelines: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md
### Receptor
- You can find the receptor docs here: https://receptor.readthedocs.io/en/latest/
- You can find the receptor docs here: https://docs.ansible.com/projects/receptor/en/latest/
- Hello, your issue seems related to receptor. Could you please open an issue in the receptor repository? https://github.com/ansible/receptor. Thanks!
@@ -31,7 +31,7 @@ Have questions about this document or anything not covered here? Create a topic
- Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason.
- If collaborating with someone else on the same branch, consider using `--force-with-lease` instead of `--force`. This will prevent you from accidentally overwriting commits pushed by someone else. For more information, see [git push docs](https://git-scm.com/docs/git-push#git-push---force-with-leaseltrefnamegt).
- If submitting a large code change, it's a good idea to create a [forum topic tagged with 'awx'](https://forum.ansible.com/tag/awx), and talk about what you would like to do or add first. This not only helps everyone know what's going on, it also helps save time and effort, if the community decides some changes are needed.
- We ask all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions, or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
- We ask all of our community members and contributors to adhere to the [Ansible code of conduct](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html). If you have questions, or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
## Setting up your development environment
@@ -103,6 +103,12 @@ When necessary, remove any AWX containers and images by running the following:
### Pre commit hooks
Install the pre-commit hook before contributing:
```
make pre-commit
```
When you attempt to perform a `git commit` there will be a pre-commit hook that gets run before the commit is allowed to your local repository. For example, python's [black](https://pypi.org/project/black/) will be run to test the formatting of any python files.
While you can use environment variables to skip the pre-commit hooks GitHub will run similar tests and prevent merging of PRs if the tests do not pass.
@command -v black >/dev/null 2>&1||{echo"could not find black on your PATH, you may need to \`pip install black\`, or set AWX_IGNORE_BLACK=1"&&exit 1;}
@(set -o pipefail &&$@$(BLACK_ARGS) awx awxkit awx_collection | tee reports/$@.report)
[](https://github.com/ansible/awx/actions/workflows/ci.yml) [](https://codecov.io/github/ansible/awx) [](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html) [](https://github.com/ansible/awx/blob/devel/LICENSE.md) [](https://forum.ansible.com/tag/awx)
[](https://github.com/ansible/awx/actions/workflows/ci.yml) [](https://codecov.io/github/ansible/awx) [](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html) [](https://github.com/ansible/awx/blob/devel/LICENSE.md) [](https://forum.ansible.com/tag/awx)
@@ -18,7 +18,7 @@ AWX provides a web-based user interface, REST API, and task engine built on top
To install AWX, please view the [Install guide](./INSTALL.md).
To learn more about using AWX, view the [AWX docs site](https://ansible.readthedocs.io/projects/awx/en/latest/).
To learn more about using AWX, view the [AWX docs site](https://docs.ansible.com/projects/awx/en/latest/).
The AWX Project Frequently Asked Questions can be found [here](https://www.ansible.com/awx-project-faq).
@@ -41,11 +41,11 @@ If you're experiencing a problem that you feel is a bug in AWX or have ideas for
Code of Conduct
---------------
We require all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
We require all of our community members and contributors to adhere to the [Ansible code of conduct](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html). If you have questions or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
Get Involved
------------
We welcome your feedback and ideas via the [Ansible Forum](https://forum.ansible.com/tag/awx).
For a full list of all the ways to talk with the Ansible Community, see the [AWX Communication guide](https://ansible.readthedocs.io/projects/awx/en/latest/contributor/communication.html).
For a full list of all the ways to talk with the Ansible Community, see the [AWX Communication guide](https://docs.ansible.com/projects/awx/en/latest/contributor/communication.html).
'When enabled (default), auto-generated job variables are emitted '
'with both the tower_ prefix and the deprecated awx_ prefix for '
'backward compatibility. Disable to emit only tower_ prefixed '
'variables and eliminate duplicates. The awx_ prefix is deprecated '
'and this setting will default to False in a future release.'
),
category=_('Jobs'),
category_slug='jobs',
)
register(
'AWX_ISOLATION_BASE_PATH',
field_class=fields.CharField,
@@ -793,6 +874,58 @@ register(
unit=_('seconds'),
)
register(
'CANDLEPIN_CONSUMER_UUID',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=False,
label=_('Candlepin Consumer UUID'),
help_text=_('UUID of the registered Candlepin consumer for this AAP instance.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'CANDLEPIN_CERT_PEM',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=True,
label=_('Candlepin Identity Certificate'),
help_text=_('PEM-encoded Candlepin identity certificate for mTLS authentication.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'CANDLEPIN_KEY_PEM',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=True,
label=_('Candlepin Identity Key'),
help_text=_('PEM-encoded private key for Candlepin identity certificate.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'CANDLEPIN_SERIAL_NUMBER',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=False,
label=_('Candlepin Certificate Serial Number'),
help_text=_('Serial number of the Candlepin identity certificate for tracking.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'IS_K8S',
field_class=fields.BooleanField,
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.