Compare commits

...

201 Commits

Author SHA1 Message Date
Dirk Julich
1d2a82308b [AAP-74343] Use public API for namespace package path access
Replace library.__path__._path[0] with library.__path__[0] to avoid
relying on a private CPython implementation detail of _NamespacePath.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-19 17:37:10 +02:00
Dirk Julich
ea2c278355 [AAP-74343] Add tests for ANSIBLE_CALLBACKS_ENABLED configuration
Verify that indirect_instance_count is always set, user-configured
callbacks from ansible.cfg are preserved, and the comma delimiter
is used as ansible-core expects.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-19 17:27:17 +02:00
Dirk Julich
f00c28ee20 [AAP-74343] Use comma delimiter for ANSIBLE_CALLBACKS_ENABLED
Ansible's CALLBACKS_ENABLED config is type list and splits on commas.
The colon delimiter would cause combined callback names to be treated
as a single invalid name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-19 17:07:52 +02:00
Dirk Julich
c2e9424044 [AAP-74343] Read callbacks_enabled from ansible.cfg so user-configured callbacks are preserved
The check for 'callbacks_enabled' in config_values was dead code because
read_ansible_config was never asked to read that setting. Now that the
callback registration runs unconditionally, fix this by including
'callbacks_enabled' in the variables of interest.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-19 16:57:55 +02:00
Dirk Julich
91d8755576 [AAP-74343] Decouple installed_collections and ansible_version from indirect node counting flag
The indirect_instance_count callback plugin and its artifact processing
were entirely gated behind FEATURE_INDIRECT_NODE_COUNTING_ENABLED. This
caused installed_collections and ansible_version to remain unpopulated
when the flag was off, even though these are baseline analytics fields
unrelated to indirect host counting.

Always run the callback plugin and persist installed_collections and
ansible_version to the database. Only the indirect-counting-specific
parts (EventQuery creation, event_queries_processed flag, and vendor
collections) remain gated behind the feature flag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-19 14:50:50 +02:00
Adrià Sala
5eeb854620 feat: increase parallel count for atf runs (#16450)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-18 15:32:15 +02:00
Lila Yasin
45480941f8 [AAP-53283] Fix analytics API requests to respect proxy environment variables (#16451)
Fix analytics API requests to respect proxy environment variables 
Assisted-by: Claude
2026-05-15 13:13:15 -04:00
jessicamack
90b7d35554 Implement Candlepin certificate integration (#16388)
* AAP-12516 [option 2] Handle nested workflow artifacts via root node `ancestor_artifacts` (#16381)

* Add new test for artfact precedence upstream node vs outer workflow

* Fix bugs, upstream artifacts come first for precedence

* Track nested artifacts path through ancestor_artifacts on root nodes

* Fix case where first root node did not get the vars

* touchup comment

* Prevent conflict with sliced jobs hack

* Reorder URLs so that Django debug toolbar can work (#16352)

* Reorder URLs so that Django debug toolbar can work

* Move comment with URL move

* feat: support for oidc credential /test endpoint (#16370)

Adds support for testing external credentials that use OIDC workload identity tokens.
When FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED is enabled, the /test endpoints return
JWT payload details alongside test results.

- Add OIDC credential test endpoints with job template selection
- Return JWT payload and secret value in test response
- Maintain backward compatibility (detail field for errors)
- Add comprehensive unit and functional tests
- Refactor shared error handling logic

Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>

* Bind the install bundle to the ansible.receptor collection 2.0.8 version (#16396)

* [Devel] Config Endpoint Optimization (#16389)

* Improved performance of the config endpoint by reducing database queries in GET /api/controller/v2/config/

* Fix OIDC workload identity for inventory sync (#16390)

The cloud credential used by inventory updates was not going through
the OIDC workload identity token flow because it lives outside the
normal _credentials list. This overrides populate_workload_identity_tokens
in RunInventoryUpdate to include the cloud credential as an
additional_credentials argument to the base implementation, and
patches get_cloud_credential on the instance so the injector picks up
the credential with OIDC context intact.

Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: integrate awx-tui to the awx_devel image (#16399)

* Aap 45980 (#16395)

* support bitbucket_dc webhooks

* add test

* update docs

* fix import for refactored method (#16394)

retrieve_workload_identity_jwt_with_claims is now
in a separate utility file, not in jobs.py

Signed-off-by: Seth Foster <fosterbseth@gmail.com>

* AAP-70257 controller collection should retry transient HTTP errors with exponential backoff. (#16415)

controller collection should retry transient HTTP errors with exponential backoff

* AAP-71844 Fix rrule fast-forward across DST boundaries (#16407)

Fix rrule fast-forward producing wrong occurrences across DST boundaries

The UTC round-trip in _fast_forward_rrule shifts the dtstart's local
hour when the original and fast-forwarded times are in different DST
periods. Since dateutil generates HOURLY occurrences by stepping in
local time, the shifted hour changes the set of reachable hours. With
BYHOUR constraints this causes a ValueError crash; without BYHOUR,
occurrences are silently shifted by 1 hour.

Fix by performing all arithmetic in the dtstart's original timezone.
Python aware-datetime subtraction already computes absolute elapsed
time regardless of timezone, so the UTC conversion was unnecessary
for correctness and actively harmful during fall-back ambiguity.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Correctly restrict push actions to ownership repos (#16398)

* Correctly restrict push actions to ownership repos

* Use standard action to see if push actions should run

* Run spec job for 2.6 and higher

* Be even more restrictve, do not push if on a fork

* [Devel] Performance Optimization for Select Hosts Query (#16413)

* Fixed black reformating

* Make test simulate 500k hosts in real world scenario

* feat: improve unauthorized response on aap deployments (#16422)

* fix: do not include secret values in the credentials test endpoint an… (#16425)

fix: do not include secret values in the credentials test endpoint and add a guard to make sure credentials are testable

* [devel backport] AAP-41742: Fix workflow node update failing when JT has unprompted labels (#16426)

* AAP-41742: Fix workflow node update failing when JT has unprompted labels

PATCH extra_data on a workflow node fails with
{"labels":["Field is not configured to prompt on launch."]}
when the node has labels associated but the JT has
ask_labels_on_launch=False.

The serializer was passing all persisted M2M state from prompts_dict()
to _accept_or_ignore_job_kwargs() on every PATCH, re-validating
unchanged fields. Fix scopes validation to only the fields in the
request; full re-validation still occurs when unified_job_template
is being changed.

* Capture attrs keys before _build_mock_obj mutates them

_build_mock_obj() pops pseudo-fields (limit, scm_branch, job_tags,
etc.) from attrs. Computing requested_prompt_fields after the pop
would miss those fields and skip their ask_on_launch validation.

* Include survey_passwords when validating extra_vars prompts

prompts_dict() emits survey_passwords alongside extra_vars.
_accept_or_ignore_job_kwargs uses it to decrypt encrypted survey
values before validation. Without it, encrypted password blobs
are validated as-is against the survey spec.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add test to ensure credential secret values are not returned (#16434)

* AAP-68024 perf: derive last_job_host_summary from query instead of denormalized FK (#16332)

* perf: stop eagerly updating Host.last_job_host_summary on every job completion

The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.

Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.

Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK

The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix .pk AttributeError, add job_template annotations, annotate host sublists

- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
  views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
  subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
  GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs

Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove stale failures_url from deprecated DashboardView

The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Apply black formatting to changed files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Refactor: replace 10 subquery annotations with bulk prefetch

Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').

This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.

- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary

Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.

- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
  JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
  individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
  HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix: scope annotation to views, restore license_error/canceled_on

- Remove with_latest_summary_id() from HostManager.get_queryset() to
  avoid applying the correlated subquery to every Host query globally
  (count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
  host-serving views only
- Restore license_error and canceled_on to last_job summary fields
  to avoid breaking API change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Guard _fetch_all() to skip bulk-attach on non-annotated querysets

Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove name from last_job_host_summary and canceled_on from last_job summary

Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary

Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Apply black formatting to test_host_queryset.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>

* Fix flake8 F841: remove unused job1/job2 variables in tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>

* Add comment explaining why Prefetch was not used for host latest summary

Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix null handling to keep old behavior

---------

Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>

* [AAP-72722] Use url instead of jwt_aud for workload identity audience (#16432)

* [AAP-72722] Use url instead of jwt_aud for workload identity audience

The OIDC credential plugin's jwt_aud field is being removed. Use the
plugin's url field as the audience when requesting workload identity
tokens, since the target service URL is the appropriate audience value.

Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [Devel] Optimize host_list_rbac query  (#16408)

* Defer ansible_facts in HostManager to avoid fetching large JSON column in host list queries (AAP-68023)

The host list endpoint (GET /api/v2/hosts/) fetches the ansible_facts
JSON column unnecessarily, contributing to the 7.8s median query time
at scale. This column can be very large and is not used by the list
serializer.

Changes:
- HostManager.get_queryset() now defers ansible_facts
- finish_fact_cache call site uses .only(*HOST_FACTS_FIELDS) to eagerly
  load ansible_facts when actually needed, avoiding N+1 queries
- Unit test mocks updated to support .only() queryset chaining
- Points DAB dependency at the RBAC query optimization branch for
  combined testing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------

* fix: constructed inventories no longer increase the host count (#16433)

* Fix version worktree (#16431)

* git worktree friendly precomit install

* worktrees don't have a .git directory. Before, docker-compose would
  trigger pre-commit install and fail.

* make docker-compose work in git worktree

* AWX tries to discover the version via info stored in .git/ dir.
  setuptools-scm is capable of finding the .git/ dir, starting from a
  worktree, but is unable because only the worktree is mapped into the
  container, not the .git/ dir itself. Thus, we have to detect and pass
  the version into the container from outside. That is why this change
  landed in the Makefile.

* fix: as_user() gateway session cookie fallback (#16437)

Add a fallback that checks for `gateway_sessionid` when no cookie
matches `session_cookie_name`, mirroring the existing fallback in
`Connection.login()`. The finally block now cleans up whichever
cookie name was actually used.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Pass setting to dispatcherd so it can be configured (#16438)

* fix: allow blank password field to fix OpenAPI schema validation (#16440)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* first pass porting over metrics

* move settings to defaults

* add more unit tests

* update unit tests

* lint fixes

* more lint fixes

* refactor and address feedback

* remove the api views

* remove model and move helper functions out of licensing

* add settings to API, fix tests, refactoring

* fix circular import

* update tests

* remove duplicate code, handle edge cases, use clearer naming, add test coverage

* update test for changes in ship()

* remove unneeded setting

* _discover_org should account for verify-tls=False

* directly assign settings, detect url, update tests

* log errors close to occurance

* rename function for clarity, focus on critical tests

* rename for clarity, lint fixes

* fix test params, priority for org discovery

* fix test failures and linting

---------

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
Co-authored-by: Tong He <68936428+unnecessary-username@users.noreply.github.com>
Co-authored-by: Stevenson Michel <iamstevensonmichel@outlook.com>
Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: Sean Sullivan <ssulliva@redhat.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>
Co-authored-by: Lila Yasin <lyasin@redhat.com>
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
2026-05-14 09:33:48 -04:00
Alan Rominger
9606366625 Consolidate validation rules for same-org restrictions (#16427)
* Consolidate implementation of same-org validation rule

* Update tests for the simplified validation

* Still do validation with deferance to the new callback

* Correctly falsy handling in view logic
2026-05-12 08:59:45 -04:00
Alan Rominger
188c10c7d6 Only request read permission for PR scan (#16295) 2026-05-11 18:09:55 +00:00
Ryan Williams
2d02a72218 feat: add tekton pipeline run atf tests on pull request (#16443)
Signed-off-by: Ryan Williams <3375653+ryankwilliams@users.noreply.github.com>
2026-05-11 12:50:38 +02:00
Lila Yasin
d3b40cb57e [devel]AAP-74276: Replace setuptools with packaging in awxkit install_requires (#16444)
AAP-74276: Replace setuptools with packaging in awxkit install_requires

The Python 3.12 upgrade replaced distutils.version.LooseVersion with
packaging.version.Version but did not update awxkit's install_requires.
setuptools is no longer needed at runtime since pkg_resources was also
replaced with importlib.metadata. This causes ModuleNotFoundError on
standalone CLI installs where packaging is not present.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-06 11:25:52 -04:00
Alan Rominger
6179b16987 AAP-72269 Change fact processing loop to use file listing (#16403)
* Change fact processing loop to use file listing

* Fix some test

* Address coderabbit comments

* Handle saving facts in batches to keep memory low

* Improve log about mismatch in response to review comment
2026-05-05 15:35:46 +02:00
jessicamack
cbbd683720 AAP-70294: Migrate Unit Test Candidates from ATF to Upstream (#16385)
* add converted atf tests

* fix bulk settings test
2026-05-04 15:07:46 +00:00
Adrià Sala
2451156fc6 fix: allow blank password field to fix OpenAPI schema validation (#16440)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 16:03:36 +02:00
Alan Rominger
83f60cddc2 Pass setting to dispatcherd so it can be configured (#16438) 2026-04-30 15:58:52 -04:00
Adrià Sala
c67d93218f fix: as_user() gateway session cookie fallback (#16437)
Add a fallback that checks for `gateway_sessionid` when no cookie
matches `session_cookie_name`, mirroring the existing fallback in
`Connection.login()`. The finally block now cleans up whichever
cookie name was actually used.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-30 10:43:08 +02:00
Chris Meyers
eac8968217 Fix version worktree (#16431)
* git worktree friendly precomit install

* worktrees don't have a .git directory. Before, docker-compose would
  trigger pre-commit install and fail.

* make docker-compose work in git worktree

* AWX tries to discover the version via info stored in .git/ dir.
  setuptools-scm is capable of finding the .git/ dir, starting from a
  worktree, but is unable because only the worktree is mapped into the
  container, not the .git/ dir itself. Thus, we have to detect and pass
  the version into the container from outside. That is why this change
  landed in the Makefile.
2026-04-28 16:07:14 -04:00
Peter Braun
df771d0e9d fix: constructed inventories no longer increase the host count (#16433) 2026-04-28 20:01:21 +00:00
Lila Yasin
1213ea6f62 [Devel] Optimize host_list_rbac query (#16408)
* Defer ansible_facts in HostManager to avoid fetching large JSON column in host list queries (AAP-68023)

The host list endpoint (GET /api/v2/hosts/) fetches the ansible_facts
JSON column unnecessarily, contributing to the 7.8s median query time
at scale. This column can be very large and is not used by the list
serializer.

Changes:
- HostManager.get_queryset() now defers ansible_facts
- finish_fact_cache call site uses .only(*HOST_FACTS_FIELDS) to eagerly
  load ansible_facts when actually needed, avoiding N+1 queries
- Unit test mocks updated to support .only() queryset chaining
- Points DAB dependency at the RBAC query optimization branch for
  combined testing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
2026-04-28 14:00:13 -04:00
Dan Leehr
b66c0105ae [AAP-72722] Use url instead of jwt_aud for workload identity audience (#16432)
* [AAP-72722] Use url instead of jwt_aud for workload identity audience

The OIDC credential plugin's jwt_aud field is being removed. Use the
plugin's url field as the audience when requesting workload identity
tokens, since the target service URL is the appropriate audience value.

Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-28 10:53:09 -04:00
Ben Thomasson
d1b3ae53ae AAP-68024 perf: derive last_job_host_summary from query instead of denormalized FK (#16332)
* perf: stop eagerly updating Host.last_job_host_summary on every job completion

The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.

Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.

Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK

The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix .pk AttributeError, add job_template annotations, annotate host sublists

- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
  views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
  subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
  GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs

Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove stale failures_url from deprecated DashboardView

The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Apply black formatting to changed files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Refactor: replace 10 subquery annotations with bulk prefetch

Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').

This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.

- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary

Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.

- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
  JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
  individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
  HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix: scope annotation to views, restore license_error/canceled_on

- Remove with_latest_summary_id() from HostManager.get_queryset() to
  avoid applying the correlated subquery to every Host query globally
  (count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
  host-serving views only
- Restore license_error and canceled_on to last_job summary fields
  to avoid breaking API change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Guard _fetch_all() to skip bulk-attach on non-annotated querysets

Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove name from last_job_host_summary and canceled_on from last_job summary

Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary

Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Apply black formatting to test_host_queryset.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>

* Fix flake8 F841: remove unused job1/job2 variables in tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>

* Add comment explaining why Prefetch was not used for host latest summary

Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix null handling to keep old behavior

---------

Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
2026-04-28 10:47:22 -04:00
Peter Braun
f3b7d442c3 feat: add test to ensure credential secret values are not returned (#16434) 2026-04-27 12:50:51 +00:00
Dirk Julich
376f964a40 [devel backport] AAP-41742: Fix workflow node update failing when JT has unprompted labels (#16426)
* AAP-41742: Fix workflow node update failing when JT has unprompted labels

PATCH extra_data on a workflow node fails with
{"labels":["Field is not configured to prompt on launch."]}
when the node has labels associated but the JT has
ask_labels_on_launch=False.

The serializer was passing all persisted M2M state from prompts_dict()
to _accept_or_ignore_job_kwargs() on every PATCH, re-validating
unchanged fields. Fix scopes validation to only the fields in the
request; full re-validation still occurs when unified_job_template
is being changed.

* Capture attrs keys before _build_mock_obj mutates them

_build_mock_obj() pops pseudo-fields (limit, scm_branch, job_tags,
etc.) from attrs. Computing requested_prompt_fields after the pop
would miss those fields and skip their ask_on_launch validation.

* Include survey_passwords when validating extra_vars prompts

prompts_dict() emits survey_passwords alongside extra_vars.
_accept_or_ignore_job_kwargs uses it to decrypt encrypted survey
values before validation. Without it, encrypted password blobs
are validated as-is against the survey spec.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 16:17:04 +02:00
Peter Braun
c71a49e044 fix: do not include secret values in the credentials test endpoint an… (#16425)
fix: do not include secret values in the credentials test endpoint and add a guard to make sure credentials are testable
2026-04-24 12:35:12 +00:00
Adrià Sala
99ac0d39dc feat: improve unauthorized response on aap deployments (#16422) 2026-04-23 14:35:45 +00:00
Stevenson Michel
55ad29ac68 [Devel] Performance Optimization for Select Hosts Query (#16413)
* Fixed black reformating

* Make test simulate 500k hosts in real world scenario
2026-04-22 12:05:36 -04:00
Alan Rominger
3fd3b741b6 Correctly restrict push actions to ownership repos (#16398)
* Correctly restrict push actions to ownership repos

* Use standard action to see if push actions should run

* Run spec job for 2.6 and higher

* Be even more restrictve, do not push if on a fork
2026-04-21 11:26:04 -04:00
Seth Foster
1636abd669 AAP-71844 Fix rrule fast-forward across DST boundaries (#16407)
Fix rrule fast-forward producing wrong occurrences across DST boundaries

The UTC round-trip in _fast_forward_rrule shifts the dtstart's local
hour when the original and fast-forwarded times are in different DST
periods. Since dateutil generates HOURLY occurrences by stepping in
local time, the shifted hour changes the set of reachable hours. With
BYHOUR constraints this causes a ValueError crash; without BYHOUR,
occurrences are silently shifted by 1 hour.

Fix by performing all arithmetic in the dtstart's original timezone.
Python aware-datetime subtraction already computes absolute elapsed
time regardless of timezone, so the UTC conversion was unnecessary
for correctness and actively harmful during fall-back ambiguity.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 10:54:42 -04:00
Sean Sullivan
d21e0141ce AAP-70257 controller collection should retry transient HTTP errors with exponential backoff. (#16415)
controller collection should retry transient HTTP errors with exponential backoff
2026-04-21 08:12:08 -06:00
Seth Foster
e5bae59f5a fix import for refactored method (#16394)
retrieve_workload_identity_jwt_with_claims is now
in a separate utility file, not in jobs.py

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2026-04-15 10:47:17 -04:00
Peter Braun
a8afbd1ca3 Aap 45980 (#16395)
* support bitbucket_dc webhooks

* add test

* update docs
2026-04-14 14:00:59 +00:00
Adrià Sala
da996c01a0 feat: integrate awx-tui to the awx_devel image (#16399) 2026-04-14 10:08:19 +02:00
Seth Foster
b8c9ae73cd Fix OIDC workload identity for inventory sync (#16390)
The cloud credential used by inventory updates was not going through
the OIDC workload identity token flow because it lives outside the
normal _credentials list. This overrides populate_workload_identity_tokens
in RunInventoryUpdate to include the cloud credential as an
additional_credentials argument to the base implementation, and
patches get_cloud_credential on the instance so the injector picks up
the credential with OIDC context intact.

Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 16:26:18 -04:00
Stevenson Michel
d71f18fa44 [Devel] Config Endpoint Optimization (#16389)
* Improved performance of the config endpoint by reducing database queries in GET /api/controller/v2/config/
2026-04-09 16:24:03 -04:00
Tong He
e82a4246f3 Bind the install bundle to the ansible.receptor collection 2.0.8 version (#16396) 2026-04-09 17:09:26 +02:00
Daniel Finca
b83019bde6 feat: support for oidc credential /test endpoint (#16370)
Adds support for testing external credentials that use OIDC workload identity tokens.
When FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED is enabled, the /test endpoints return
JWT payload details alongside test results.

- Add OIDC credential test endpoints with job template selection
- Return JWT payload and secret value in test response
- Maintain backward compatibility (detail field for errors)
- Add comprehensive unit and functional tests
- Refactor shared error handling logic

Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
2026-04-06 15:56:11 -04:00
Alan Rominger
6d94aa84e7 Reorder URLs so that Django debug toolbar can work (#16352)
* Reorder URLs so that Django debug toolbar can work

* Move comment with URL move
2026-04-03 10:22:21 -04:00
Alan Rominger
7155400efc AAP-12516 [option 2] Handle nested workflow artifacts via root node ancestor_artifacts (#16381)
* Add new test for artfact precedence upstream node vs outer workflow

* Fix bugs, upstream artifacts come first for precedence

* Track nested artifacts path through ancestor_artifacts on root nodes

* Fix case where first root node did not get the vars

* touchup comment

* Prevent conflict with sliced jobs hack
2026-04-02 15:18:11 -04:00
melissalkelly
e80ce43f87 Fix workload identity project updates (#16373)
* fix: enable workload identity credentials for project updates

* Add explanatory comment for credential context handling

* Revert build_passwords
2026-03-31 14:48:27 +02:00
Stevenson Michel
595e093bbf [CI-Fix] Pin setuptools_scm<10 to fix api-lint failure (#16376)
Fix CI: Pin setuptools_scm<10 to fix api-lint build failure

setuptools-scm 10.0.5 (with its new vcs-versioning dependency) requires
a [tool.setuptools_scm] or [tool.vcs-versioning] section in pyproject.toml.
AWX intentionally omits this section because it uses a custom version
resolution via setup.cfg (version = attr: awx.get_version). The new major
version of setuptools-scm treats the missing section as a fatal error when
building the sdist in tox's isolated build, causing the linters environment
to fail.

Pinning to <10 restores compatibility with the existing version resolution
strategy.

Failing run: https://github.com/ansible/awx/actions/runs/23744310714
Branch: devel

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 14:37:16 -04:00
TVo
cd7f6f602f Fix OpenAPI schema validation message mismatch (#16372) 2026-03-25 12:36:10 -06:00
Chris Meyers
310dd3e18f Update dispatcherd to version 2026.3.25 (#16369)
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 10:57:01 -04:00
Matthew Sandoval
7c75788b0a AAP-67740 Pass plugin_description through to CredentialType.description (#16364)
* Pass plugin_description through to CredentialType.description

Propagate the plugin_description field from credential plugins into the
CredentialType description when loading and creating managed credential
types, including updates to existing records.

Assisted-by: Claude

* Add unit tests for plugin_description passthrough to CredentialType

Tests cover load_plugin, get_creation_params, and
_setup_tower_managed_defaults handling of the description field.

Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: PabloHiro <palonso@redhat.com>
2026-03-25 11:03:11 +01:00
Peter Braun
ab294385ad fix: avoid delete in loop in inventory import (#16366) 2026-03-24 15:37:59 +00:00
Alan Rominger
377dfce197 Record whether a file was written for fact cache (#16361) 2026-03-20 12:53:34 -04:00
Matthew Sandoval
ff68d6196d Add feature flag for OIDC workload identity credential types (#16348)
Add install-time feature flag for OIDC workload identity credential types

Implements FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED feature flag to gate
HashiCorp Vault OIDC credential types as a Technology Preview feature.

When the feature flag is disabled (default), OIDC credential types are
not loaded into the plugin registry at application startup and do not
exist in the database.

When enabled, OIDC credential types are loaded normally and function
as expected.

Changes:
- Add FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED setting (defaults to False)
- Add OIDC_CREDENTIAL_TYPE_NAMESPACES constant for maintainability
- Modify load_credentials() to skip OIDC types when flag is disabled
- Add test coverage (2 test cases)

This is an install-time flag that requires application restart to take
effect. The flag is checked during application startup when credential
types are loaded from plugins.

Fixes: AAP-64510

Assisted-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-18 22:40:33 -04:00
Ben Thomasson
bfefee5aef fix: NameError in wsrelay when JSON decode fails with DEBUG logging (#16340)
* fix: NameError in wsrelay when JSON decode fails with DEBUG logging

run_connection() referenced payload in the JSONDecodeError handler,
but payload was never assigned because json.loads() is what failed.
Use msg.data instead to log the raw message content.

Fixes: AAP-68045

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix other instance of undefined payload

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
2026-03-18 17:17:25 +00:00
Alan Rominger
0aaca1bffd Fix job cancel chain bugs (#16325)
* Fix job cancel chain bugs

* Early relief valve for canceled jobs, ATF related changes

* Add test and fix for approval nodes as well

* Revert unwanted change

* Refactor workflow approval nodes to make it more clean

* Revert data structure changes

* Delete local utility file

* Review comment addressing

* Use canceled status in websocket

* Delete slop

* Add agent marker

* Bugbot comment about status websocket mismatch
2026-03-18 12:08:27 -04:00
Rodrigo Toshiaki Horie
679e48cbe8 [AAP-68258] Fix SonarCloud Reliability issues (#16354)
* Fix SonarCloud Reliability issues: time-dependent class attrs and dict comprehensions

- Move last_stats/last_flush from class body to __init__ in CallbackBrokerWorker
  (S8434: time-dependent expressions evaluated at class definition)
- Replace dict comprehensions with dict.fromkeys() in has_create.py
  (S7519: constant-value dict should use fromkeys)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix callback receiver tests to use flush(force=True)

Tests were implicitly relying on last_flush being a stale class-level
timestamp. Now that last_flush is set in __init__, the time-based flush
condition isn't met when flush() is called immediately after construction.
Use force=True to explicitly trigger an immediate flush in tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 14:21:39 +00:00
Alan Rominger
c591eb4a7a Do not ignore errors on activity stream connection (#16344) 2026-03-17 17:04:13 -04:00
Alan Rominger
cc2fbf332c Stop writing tmp test files that are not cleaned up (#16358)
* Stop writing tmp test files that are not cleaned up
2026-03-17 17:02:47 -04:00
Alan Rominger
1646694258 Fix jinja2 error from newer ansible versions (#16356)
* Fix error from newer ansible versions

* Include fix for setting cachable

* Revert "Include fix for setting cachable"

This reverts commit 477293c258.
2026-03-17 13:36:33 -04:00
Jake Jackson
643a9849df update editable_dependencies readme (#15471)
fixed typo/project naming to match example.
2026-03-12 19:36:18 +00:00
Rodrigo Toshiaki Horie
8bd8bcda94 [AAP-68258] Fix SonarCloud Reliability Rating issue in Common exception constructor (#16351)
Fix SonarCloud Reliability Rating issue in Common exception constructor

The constructor had code paths where attributes were not consistently
initialized and super().__init__() was not called, which was flagged
as a Reliability Rating issue by SonarCloud. Ensures all branches
properly set self.status_string and self.msg, and call super().__init__().

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-12 15:21:01 -03:00
Alan Rominger
63f3c735ea Delete unused contains method (#16346) 2026-03-12 12:00:29 -04:00
Tong He
7e29f9e3f2 Enrich tests against is_ha_environment() 2026-03-11 11:43:12 +01:00
Pablo H.
c115e0168a Align log message of workload identity tokens with other task messages (#16345) 2026-03-10 14:05:31 -04:00
Andrea Restle-Lay
619d8c67a9 [AAP-63314] P4.4: Controller - Pass Workload TTL to Gateway (#16303)
* Pass workload TTL to Gateway (minimal changes) assisted-by: Claude

* lint
Assisted-by: Claude

* fix unit tests assisted-by claude

* use existing functions assisted-by: Claude

* fix test assisted-by: Claude

* fixes for sonarcloud assisted-by: Claude

* nit

* nit

* address feedback

* feedback from pr review assisted-by: Claude

* feedback from pr review assisted-by: Claude

* Apply suggestion from @dleehr

Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>

* lint assisted-by: Claude

* fix: narrow vendor_collections_dir fixture teardown scope (#16326)

Only remove the collection directory the fixture created
(redhat/indirect_accounting) instead of the entire
/var/lib/awx/vendor_collections/ root, so we don't accidentally
delete vendor collections that may have been installed by the
build process.

Forward-port of ansible/tower#7350.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* AAP-67436 Remove pbr from requirements (#16337)

* Remove pbr from requirements

pbr was temporarily added to support ansible-runner installed from a git
branch. It is no longer needed as a direct dependency.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Retrigger CI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [AAP-64062] Enforce JWT-only authentication for Controller when deployed as part of AAP (#16283)

After all settings are loaded, override DEFAULT_AUTHENTICATION_CLASSES
to only allow Gateway JWT authentication when RESOURCE_SERVER__URL is
set. This makes the lockdown immutable — no configuration file or
environment variable can re-enable legacy auth methods (Basic, Session,
OAuth2, Token).

This is the same pattern used by Hub (galaxy_ng) and EDA (eda-server)
for ANSTRAT-1840.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Re-trigger CI

Made-with: Cursor

* Re-trigger CI

Made-with: Cursor

* [AAP-63314] Pass job timeout as workload_ttl_seconds to Gateway    Assisted-by: Claude

* Additional unit test requested at review  Assisted-by: Claude

* Revert profiled_pg/base.py rebase error, unrelated to AAP-63314

* revert requirements changes introduced by testing

* revert

* revert

* docstring nit from coderabbit

---------

Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
2026-03-10 08:54:28 -04:00
Hao Liu
0d08a4da60 [AAP-64062] Enforce JWT-only authentication for Controller when deployed as part of AAP (#16283)
After all settings are loaded, override DEFAULT_AUTHENTICATION_CLASSES
to only allow Gateway JWT authentication when RESOURCE_SERVER__URL is
set. This makes the lockdown immutable — no configuration file or
environment variable can re-enable legacy auth methods (Basic, Session,
OAuth2, Token).

This is the same pattern used by Hub (galaxy_ng) and EDA (eda-server)
for ANSTRAT-1840.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-09 17:46:31 +01:00
Hao Liu
36a1121cd8 AAP-67436 Remove pbr from requirements (#16337)
* Remove pbr from requirements

pbr was temporarily added to support ansible-runner installed from a git
branch. It is no longer needed as a direct dependency.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Retrigger CI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-09 15:53:38 +00:00
Dirk Julich
212546f92b fix: narrow vendor_collections_dir fixture teardown scope (#16326)
Only remove the collection directory the fixture created
(redhat/indirect_accounting) instead of the entire
/var/lib/awx/vendor_collections/ root, so we don't accidentally
delete vendor collections that may have been installed by the
build process.

Forward-port of ansible/tower#7350.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 14:19:24 +01:00
Adrià Sala
fad4881280 fix: include awxkit coverage in sonarqube 2026-03-06 09:41:38 +01:00
Adrià Sala
65b1867114 fix: help-context detection to ignore option values 2026-03-06 09:41:38 +01:00
Adrià Sala
1a3085ff40 fix: nitpick and upload awxkit coverage to ci 2026-03-06 09:41:38 +01:00
Adrià Sala
51ed59c506 fix: awxkit help flags for detailed & general help
Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
2026-03-06 09:41:38 +01:00
Alan Rominger
670dfeed25 Address more ignored pytest warnings, co-authored with Opus 4.6 (#16298)
* Address more ignored pytest warnings

* Fix what we can with CI results

* Add new migration file
2026-03-05 13:51:19 -05:00
Alan Rominger
7384c73c9a Update hash for related action update (#16328) 2026-03-05 18:34:34 +00:00
Alan Rominger
25b43deec0 Address unused variables issue (#16327) 2026-03-05 12:39:18 -05:00
Seth Foster
f74f82e30c Forward port external query files from stable-2.6 (#16312)
* Revert "AAP-58452 Add version fallback for external query files (#16309)"

This reverts commit 0f2692b504.

* AAP-58441: Add runtime integration for external query collection (#7208)

Extend build_private_data_files() to copy vendor collections from
/var/lib/awx/vendor_collections/ to the job's private_data_dir,
making external query files available to the indirect node counting
callback plugin in execution environments.

Changes:
- Copy vendor_collections to private_data_dir during job preparation
- Add vendor_collections path to ANSIBLE_COLLECTIONS_PATH in build_env()
- Gracefully handle missing source directory with warning log
- Feature gated by FEATURE_INDIRECT_NODE_COUNTING_ENABLED flag

This enables external query file discovery for indirect node counting
across all deployment types (RPM, Podman, OpenShift, Kubernetes) using
the existing private_data_dir mechanism.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* [stable-2.6] AAP-58451: Add callback plugin discovery for external query files (#7223)

* AAP-58451: Add callback plugin discovery for external query files

Extend the indirect_instance_count callback plugin to discover and load
external query files from the bundled redhat.indirect_accounting collection
when embedded queries are not present in the target collection.

Changes:
- Add external query discovery with precedence (embedded queries first)
- External query path: redhat.indirect_accounting/extensions/audit/
  external_queries/{namespace}.{name}.{version}.yml
- Use self._display.v() for external query messages (visible with -v)
- Use self._display.vv() for embedded query messages (visible with -vv)
- Fix: Change .exists() to .is_file() per Traversable ABC
- Handle missing external query collection gracefully (ModuleNotFoundError)

Note: This implements exact version match only. Version fallback logic
is covered in AAP-58452.

* fix CI error when using Traversable.is_file

* Add minimal implementation for AAP-58451

* Fix formatting

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* AAP-58452 Add version fallback for external query files (#7254)

* AAP-58456 unit test suite for external query handling (#7283)

* Add unit tests for external query handling

* Refactor unit tests for external query handling

* Refactor indirect node counting callback code to improve testing code

* Refactor unit tests for external query handling for improved callback code

* Fix test for majore version boundary check

* Fix weaknesses in some unit tests

* Make callback plugin module self contained, independent from awx

* AAP-58470 integration tests (core) for external queries (#7278)

* Add collection for testing external queries

* Add query files for testing external query file runtime integration

* Add live tests for external query file runtime integration

* Remove redundant wait for events and refactor test data folders

* Fix unit tests: mock flag_enabled to avoid DB access

The AAP-58441 cherry-pick added a flag_enabled() call in
BaseTask.build_private_data_files(), which is called by all task types.
Tests for RunInventoryUpdate and RunJob credentials now hit this code
path and need the flag mocked to avoid database access in unit tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: attempt exact query file match before Version parsing (#7345)

The exact-version filename check does not require PEP440 parsing, but
Version() was called first, causing early return on non-PEP440 version
strings even when an exact file exists on disk. Move the exact file
check before Version parsing so fallback logic only parses when needed.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Do no longer mutate global sys.modules (#7337)

* [stable-2.6] AAP-58452 fix: Add queries_dir guard (#7338)

* Add queries_dir guard

* fix: update unit tests to mock _get_query_file_dir instead of files

The TestVersionFallback tests mocked `files()` with chainable path
mocks, but `find_external_query_with_fallback` now uses
`_get_query_file_dir()` which returns the queries directory directly.
Mock the helper instead for simpler, correct tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused EXTERNAL_QUERY_PATH constant (#7336)

The constant was defined but never referenced — the path is constructed
inline via Traversable's `/` operator which requires individual segments,
not a slash-separated string.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore original feature flag state in test fixture (#7347)

The enable_indirect_host_counting fixture unconditionally disabled the
FEATURE_INDIRECT_NODE_COUNTING_ENABLED flag on teardown, even when it
was already enabled before the test (as is the case in development via
development_defaults.py). This caused test_indirect_host_counting to
fail when run after the external query tests, because the callback
plugin was no longer enabled.

Save and restore the original flag state instead.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-05 14:05:58 +01:00
Alan Rominger
be5fbf365e AAP-65054 Fix bugs where concurrent jobs would clear facts of unrelated hosts (#16318)
* Add new tests for bug saving concurrent facts

* Fix first bug and improve tests

* Fix new bug where concurrent job clears facts from other job in unwanted way

* minor test fixes

* Add in missing playbook

* Fix host reference for constructed inventory

* Increase speed for concurrent fact tests

* Make test a bit faster

* Fix linters

* Add some functional tests

* Remove the sanity test

* Agent markers added

* Address SonarCloud

* Do backdating method, resolving stricter assertions

* Address coderabbit comments

* Address review comment with qs only method

* Delete missed sleep statement

* Add more coverage
2026-03-05 07:58:32 -05:00
Peter Braun
0995f7c5fe update bindep.txt to versionles dependencies (#16316) 2026-03-04 16:25:09 -07:00
Alan Rominger
3fbc71e6c8 Null value handling, discovered in production logs (#16323) 2026-03-04 14:48:21 -05:00
Alan Rominger
143d4cee34 Pin container versions (#16322)
* Pin container versions

* downgrades from coderabbit

* Add notes for future upgrades
2026-03-04 14:47:50 -05:00
Lila Yasin
af7fbea854 Fix LoggedLogoutView for Django 5.2 GET removal (#16317)
Django 5.2 restricts LogoutView to POST only (deprecated in 4.1,
  removed in 5.0+). Without this fix, GET requests to /api/logout/
  return 405 Method Not Allowed.

  Add http_method_names override and a get() method that delegates
  to post() where auth_logout() actually runs
2026-03-04 14:05:32 -05:00
Pablo H.
57f9eb093a feat: workload identity credentials integration (#16286)
* feat: workload identity credentials integration

* feat: cache credentials and add context property to Credential

Assisted-by: Claude

* feat: include safeguard in case feature flag is disabled

* feat: tests to validate workload identity credentials integration

* fix: affected tests by the credential cache mechanism

* feat: remove word cache from variables and comments, use standard library decorators

* fix: reorder tests in correct files

* Use better error catching mechanisms

* Adjust logic to support multiple credential input sources and use internal field

* Remove hardcoded credential type names

* Add tests for the internal field

Assited-by: Claude
2026-03-04 10:22:27 -05:00
Dirk Julich
8d191046b5 AAP-63318: Internal/developer documentation for indirect query files (#16319)
Add documentation for indirect query files
2026-03-04 15:45:44 +01:00
Stevenson Michel
7a5f0998d2 [Devel] Added HTTP_X_FORWARDED_FOR in Devel for production (#16314)
Added HTTP_X_FORWARDED_FOR in Devel for production
2026-03-03 17:08:38 -05:00
Stevenson Michel
d1f4fc3e97 [Devel] Addition of logic to trigger worflow dispatch on release_4.6 and stable-2.6 (#16315)
* Added worflow dispatch to trigger ci on release_4.6 and stable-2.6

* fix error cascading to subsequent jobs for cron

* made compose_tag resolve to the correct branch name
2026-03-02 18:48:42 -05:00
Lila Yasin
0f2692b504 AAP-58452 Add version fallback for external query files (#16309)
Forward port of 5c4653fbbca4b542fea05f070efea0396b034839 from stable-2.6.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 15:08:27 -05:00
Alan Rominger
e1e2c60f2e AAP-66379 Include scaledown fix from dispatcherd (#16305)
Include scaledown fix from dispatcherd
2026-02-27 14:45:57 -05:00
Alan Rominger
d8a2aa1dc3 Do not write to dispatcher.log from AWX application code (#16302) 2026-02-27 14:45:27 -05:00
Alan Rominger
9d61e42ede Bump action version (#16306) 2026-02-27 09:52:38 -05:00
Seth Foster
2c71bcda32 Improve transactional integrity for starting controller jobs in dispatcherd (#16300)
Remove SELECT FOR UPDATE from job dispatch to reduce transaction rollbacks
                                                                                                                                                                                                                                                                                           
  Move status transition from BaseTask.transition_status (which used
  SELECT FOR UPDATE inside transaction.atomic()) into                                                                                                                                                                                                                                      
  dispatch_waiting_jobs. The new approach uses filter().update() which                                                                                                                                                                                                                     
  is atomic at the database level without requiring explicit row locks,
  reducing transaction contention and rollbacks observed in perfscale
  testing.

  The transition_status method was an artifact of the feature flag era
  where we needed to support both old and new code paths. Since
  dispatch_waiting_jobs is already a singleton
  (on_duplicate='queue_one') scoped to the local node, the
  de-duplication logic is unnecessary.

  Status is updated after task submission to dispatcherd, so the job's
  UUID is in the dispatch pipeline before being marked running —
  preventing the reaper from incorrectly reaping jobs during the
  handoff window. RunJob.run() handles the race where a worker picks
  up the task before the status update lands by accepting waiting and
  transitioning it to running itself.

  Signed-off-by: Seth Foster <fosterbseth@gmail.com>
  Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-26 14:16:36 -05:00
Stevenson Michel
a21f9fbdb8 Addition of Cron Schedule (#16301)
Added cron schedule for ci jobs
2026-02-23 16:31:34 -05:00
Daniel Finca Martínez
2a35ce5524 AAP-62693 Integrate workload identity client to request JWTs (#16296)
* Add retrieve_workload_identity_jwt to jobs.py and tests

* Apply linting

* Add precondition to client retrieval

* Add test case for client not configured

* Remove trailing period in match string
2026-02-19 09:13:32 -05:00
Alan Rominger
567a980a03 Give error details of sliced jobs if they error in live tests (#16273) 2026-02-18 15:12:12 -05:00
Alan Rominger
9059cfbda6 Fix some pytest warnings using Opus 4.6 (#16269)
* Fix some pytest warnings using Opus 4.6

* Fix review comments

* Use raw-strings and regex markers for matching exception pattern

Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>

* Make regex work

* Undo always true assertion edit

---------

Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
2026-02-18 15:11:41 -05:00
Jake Jackson
d8fd953732 Update PR template to add steps to repro (#16294)
* Update PR template to add steps to repro

* updated bottom section to include steps to reproduce to help with
  debugging and triage
2026-02-18 15:01:36 -05:00
Stevenson Michel
39851c392a [Devel] Removal of Token Auth From Devel (#16293)
* Revert "[Devel][AAP-65384]Restoration of Token Authentication for AWX CLI (#16281)"

This reverts commit 994a2b3c04.
2026-02-18 14:32:44 -05:00
Chris Meyers
aeba4a1a3f Revert "Change remote host finding logic"
This reverts commit 08f1507f70.
2026-02-17 14:46:45 -05:00
Adrià Sala
915deca78c fix: awxkit user creation through gw 2026-02-17 15:38:20 +01:00
Peter Braun
1a79e853fe do not add optional survey fields with empty strings that are not bac… (#16289)
* do not add optional survey fields with empty strings that are not backed by extra_vars

* exclude password fields from skipping if not defined
2026-02-17 12:59:38 +01:00
Chris Meyers
08f1507f70 Change remote host finding logic
* When the remote host header values contains a comma separated list,
  only consider the first entry. Previously we considered every item in
  the list.
2026-02-16 15:46:47 -05:00
Stevenson Michel
994a2b3c04 [Devel][AAP-65384]Restoration of Token Authentication for AWX CLI (#16281)
* Added token authentication in logic, arguments, and test
2026-02-16 10:14:31 -05:00
Seth Foster
7ccc14daeb Remove stale api:schema-swagger-ui reference from API root (#16282)
The schema-swagger-ui URL was removed from awx/api/urls/urls.py in
d7eb714859 when docs endpoints moved to DAB's api_documentation app,
but the reverse call in ApiRootView was not removed, causing a
NoReverseMatch error in development mode.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-12 20:12:34 +00:00
Seth Foster
9700fb01f2 Fix awx CLI modify command for users with object-level permissions (#16276)
The awx CLI derives available fields for the `modify` command from
the list endpoint's POST action schema. Users with object-level
admin permissions (e.g., Project Admin) but no list-level POST
permission see no field flags, making modify unusable despite having
PUT access on the detail endpoint.

Fall back to the detail endpoint's action schema when POST is not
available on the list endpoint, and prefer PUT over POST when
building modify arguments.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-12 13:26:15 -05:00
Seth Foster
c515b86fa6 Bump wheel to address CVE-2026-24049 (#16253)
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2026-02-12 13:09:25 -05:00
Chris Meyers
01293f1b45 Restore github app lookup tests
* Introduced in PR https://github.com/ansible/awx/pull/16058/changes
  then a later large merge from AAP back into devel removed the changes
* This PR re-introduces the github app lookup migration rename tests
  with the migration names updated and the kind to namespace correction
2026-02-12 11:45:10 -05:00
Chris Meyers
fd847862a7 Fix wrong field rename
* Multiple credentialtype's have the same kind and kind values look
  like: cloud, network, machine, etc.
* namespace is the field that we want to rename
2026-02-12 11:45:10 -05:00
Dave
980d9db192 fix: align pip version constraint in requirements_dev.txt (#16275)
fix: align pip version constraint in requirements_dev.txt with requirements.txt (fixes #16272)

requirements.txt pins pip==25.3 while requirements_dev.txt specified
pip>=21.3,<=24.0, causing ResolutionImpossible when installing both.
Updated requirements_dev.txt to use pip>=25.3 to maintain compatibility.
2026-02-12 11:35:01 -05:00
Alan Rominger
f2438a0e86 Fix server error from PATCH to inventory source (#16274)
Fix server error from PATCH to inventory source, co-authored with Claude opus 4.6
2026-02-11 15:10:32 -05:00
Rodrigo Toshiaki Horie
707f2fa5da Add OpenAPI spec sync workflow (#16267) 2026-02-10 19:13:47 -03:00
Rodrigo Toshiaki Horie
1f18396438 Add CI Checks for syntactically valid OpenAPI Specification (#16266) 2026-02-10 19:13:34 -03:00
melissalkelly
6f0cfb5ace AAP-62657 Implement logic to extract and populate JWT claims from Controller Jobs (#16259)
* AAP-62657 Add populate_claims_for_workload function and unit tests

* Update safe_get helper function

* Trigger CI rebuild to pick up latest django-ansible-base

* Trigger CI after org visibility update

* Retrigger CI

* Rename workload to job, refine safe_get helper function

* Update test_jobs to use job fixture

* Retrigger CI

* Create fresh job, removed launched_by since this is read-only property

* Retrigger CI after runner issues

* Retrigger CI after runner issues

* Add unit tests for other workload types

* Update CLAIM_LAUNCHED_BY_USER_NAME and CLAIM_LAUNCHED_BY_USER_ID, with CLAIM_LAUNCHED_BY_NAME and CLAIM_LAUNCHED_BY_ID

* Generate claims with a more static schema

try to operate directly on object when possible

For cases where field is valid for the type, but null value
  still add the field, so blank and null values appear

* Allow unified related items to be omittied

---------

Co-authored-by: AlanCoding <arominge@redhat.com>
2026-02-09 20:58:49 +00:00
🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко)
fc0a4cddce 🧪 Use the unified test reporting action (#16168) 2026-02-09 09:54:04 -05:00
Seth Foster
99511efe81 bump pyasn1 (#16249)
Bump pyasn1 for CVE-2026-2349

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2026-02-05 14:12:59 -05:00
Rodrigo Toshiaki Horie
30bf910bd5 fix schema generator (#16265) 2026-02-04 20:54:28 -03:00
jessicamack
c9085e4b7f Update OpenAPI spec to improve descriptions and messages (#16260)
* Update OpenAPI spec

* lint fixes

* fix decorator for retrieve endpoints

* change decorator method

* fix import

* lint fix
2026-02-04 22:32:57 +00:00
Alan Rominger
5e93f60b9e AAP-41776 Enable new fancy asyncio metrics for dispatcherd (#16233)
* Enable new fancy asyncio metrics for dispatcherd

Remove old dispatcher metrics and patch in new data from local whatever

Update test fixture to new dispatcherd version

* Update dispatcherd again

* Handle node filter in URL, and catch more errors

* Add test for metric filter

* Split module for dispatcherd metrics
2026-02-04 15:28:34 -05:00
Rodrigo Toshiaki Horie
6a031158ce Fix OpenAPI schema enum values for CredentialType kind field (#16262)
The OpenAPI schema incorrectly showed all 12 credential type kinds as
valid for POST/PUT/PATCH operations, when only 'cloud' and 'net' are
allowed for custom credential types. This caused API clients and LLM
agents to receive HTTP 400 errors when attempting to create credential
types with invalid kind values.

Add postprocessing hook to filter CredentialTypeRequest and
PatchedCredentialTypeRequest schemas to only show 'cloud', 'net',
and null as valid enum values, matching the existing validation logic.

No API behavior changes - this is purely a documentation fix.

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-04 16:39:03 -03:00
joeywashburn
749735b941 Standardize spelling of 'canceled' in wsrelay.py (#16178)
Changed two instances of 'cancelled' to 'canceled' in awx/main/wsrelay.py
to match AWX's standardized American English spelling convention.

- Updated log message in WebsocketRelayConnection.connect()
- Updated comment in WebSocketRelayManager.cleanup_offline_host()

Fixes #15177

Signed-off-by: Joey Washburn <joey@joeywashburn.com>
2026-02-04 12:29:45 -05:00
Chris Meyers
315f9c7eef Rename args var
* https://sonarcloud.io/project/issues?open=AZDmRbV12PiUXMD3dYmh&id=ansible_awx
2026-02-04 08:17:51 -05:00
Chris Meyers
00c0f7e8db add test 2026-02-03 16:12:22 -05:00
Chris Meyers
37ccbc28bd Harden log message output containing user input
* base64 encode user inputed url when logging so that newlines or other
  malicious payloads can't be injected into the log stream
2026-02-03 16:12:22 -05:00
Chris Meyers
63fafec76f Remove init return value
* https://sonarcloud.io/project/issues?open=AZDmRaaJ2PiUXMD3dXly&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
cba01339a1 Remove redunant role attribute
* https://sonarcloud.io/project/issues?open=AZDmRbYN2PiUXMD3dYo5&id=ansible_awx&tab=code
2026-02-03 16:12:00 -05:00
Chris Meyers
2622e9d295 Add alt text for awx logo
* https://sonarcloud.io/project/issues?open=AZDmRbYR2PiUXMD3dYo6&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
a6afec6ebb Add generic font family
* https://sonarcloud.io/project/issues?open=AZDmRbX42PiUXMD3dYo1&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
f406a377f7 Add generic font family
* https://sonarcloud.io/project/issues?open=AZDmRbX42PiUXMD3dYo0&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
adc3e35978 Add generic font family
* remove quotes so that it's treated as a generic font family
* https://sonarcloud.io/project/issues?open=AZDmRbX42PiUXMD3dYoz&id=ansible_awx&tab=code
2026-02-03 16:12:00 -05:00
Chris Meyers
838e67005c Remove duplicate css property
* Last one wins so remove the first one.
* https://sonarcloud.io/project/issues?open=AZpWSq7yO74rjWmAOcwf&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
e13fcfe29f Add alt text to 504 image
* https://sonarcloud.io/project/issues?open=AZDmRbXt2PiUXMD3dYor&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
0f4e91419a Add lang english tag to 504 page
* https://sonarcloud.io/project/issues?open=AZDmRbXt2PiUXMD3dYop&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
cca70b242a Add alt text to 502 image
* https://sonarcloud.io/project/issues?open=AZDmRbXx2PiUXMD3dYou&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
edf459f8ec Add language english to 502
* https://sonarcloud.io/project/issues?open=AZDmRbXx2PiUXMD3dYos&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
f4286216d6 Add doctype and lang
* https://sonarcloud.io/project/issues?open=AZDmRbX02PiUXMD3dYov&id=ansible_awx
* https://sonarcloud.io/project/issues?open=AZDmRbX02PiUXMD3dYow&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
0ab1fea731 Use replaceAll() for global regex
* https://sonarcloud.io/project/issues?open=AZlyqQeaRtfhwxlTsfP0&id=ansible_awx
* https://sonarcloud.io/project/issues?open=AZlyqQeaRtfhwxlTsfP1&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
e3ac581fdf Always use a tz aware timestamp
* https://sonarcloud.io/project/issues?open=AZDmRade2PiUXMD3dXnx&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
5aa3e8cf3b Make tz aware
* Note, this doesn't change the logic or output, but maybe makes
  someones life easier later.
* https://sonarcloud.io/project/issues?open=AZDmRade2PiUXMD3dXnx&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
8289003c0d Remove unreachable code path
* https://sonarcloud.io/project/issues?open=AZDmRaXd2PiUXMD3dXkN&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
125083538a Compare float to float
* https://sonarcloud.io/project/issues?open=AZDmRaXd2PiUXMD3dXkF&id=ansible_awx&tab=code
2026-02-03 16:12:00 -05:00
Chris Meyers
ed5ab8becd Remove unused variable
* https://sonarcloud.io/project/issues?open=AZL9AcQZ0bcYsK7qDrTo&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
fc0087f1b2 Add language to api stdout for translation helping
* https://sonarcloud.io/project/issues?open=AZDmRbV82PiUXMD3dYmx&id=ansible_awx&tab=code
2026-02-03 16:12:00 -05:00
Chris Meyers
cfc5ad9d91 Remove return value from __init__
* https://sonarcloud.io/project/issues?open=AZDmRaZX2PiUXMD3dXle&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
d929b767b6 Rename kwargs
* https://sonarcloud.io/project/issues?open=AZDmRbVW2PiUXMD3dYmW&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
5f434ac348 Rename exception args variable
* https://sonarcloud.io/project/issues?open=AZDmRbV12PiUXMD3dYmg&id=ansible_awx
* https://sonarcloud.io/project/issues?open=AZDmRaZX2PiUXMD3dXle&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
4de9c8356b Use fromkeys for constant
* https://sonarcloud.io/project/issues?open=AZeD0GsJyrLLb-kZUOoF&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
91118adbd3 Fix summary_dict None check
* https://sonarcloud.io/project/issues?open=AZDmRbWy2PiUXMD3dYoJ&id=ansible_awx
2026-02-03 16:12:00 -05:00
Chris Meyers
25f538277a Fix init return
* __init__ shouldn't return
2026-02-03 16:12:00 -05:00
joeywashburn
82cb52d648 Sanitize SSH key whitespace to prevent validation errors (#16179)
Strip leading and trailing whitespace from SSH keys in validate_ssh_private_key()
to handle common copy-paste scenarios where hidden newlines cause base64 decoding
failures.

Changes:
- Added data.strip() in validate_ssh_private_key() before calling validate_pem()
- Added test_ssh_key_with_whitespace() to verify keys with leading/trailing
  newlines are properly sanitized and validated

This prevents the confusing "HTTP 500: Internal Server Error" and
"binascii.Error: Incorrect padding" errors when users paste SSH keys with
accidental whitespace.

Fixes #14219

Signed-off-by: Joey Washburn <joey@joeywashburn.com>
2026-02-02 11:16:28 -05:00
Peter Braun
f7958b93bd add deprecated fields to x-ai-description for credential post (#16255) 2026-01-29 18:17:31 +01:00
Alan Rominger
3d68ca848e Fix race condition of un-expired cache in local workers (#16256) 2026-01-29 11:31:06 -05:00
Adrià Sala
99dce79078 fix: add py311 to make version detection 2026-01-28 14:20:48 +01:00
Alan Rominger
271383d018 AAP-60470 Add dispatcherctl and dispatcherd commands as updated interface to dispatcherd lib (#16206)
* Add dispatcherctl command

* Add tests for dispatcherctl command

* Exit early if sqlite3

* Switch to dispatcherd mgmt cmd

* Move unwanted command options to run_dispatcher

* Add test for new stuff

* Update the SOS report status command

* make docs always reference new command

* Consistently error if given config file
2026-01-27 15:57:23 -05:00
Alan Rominger
1128ad5a57 AAP-64221 Fix broken cancel logic with dispatcherd (#16247)
* Fix broken cancel logic with dispatcherd

Update tests for UnifiedJob

Update test assertion

* Further simply cancel path
2026-01-27 14:39:08 -05:00
Seth Foster
823b736afe Remove unused INSIGHTS_OIDC_ENDPOINT (#16235)
This setting is set in defaults.py, but
currently not being used. More technically,
project_update.yml is not passing this value to
the insights.py action plugin. Therefore, we
can safely remove references to it.

insights.py already has a default oidc endpoint
defined for authentication.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2026-01-27 10:13:37 -05:00
Alan Rominger
f80bbc57d8 AAP-43117 Additional dispatcher removal simplifications and waiting reaper updates (#16243)
* Additional dispatcher removal simplifications and waiting repear updates

* Fix double call and logging message

* Implement bugbot comment, should reap running on lost instances

* Add test case for new pending behavior
2026-01-26 13:55:37 -05:00
TVo
12a7229ee9 Publish open api spec on AWX for community use (#16221)
* Added link and ref to openAPI spec for community

* Update docs/docsite/rst/contributor/openapi_link.rst

Co-authored-by: Don Naro <dnaro@redhat.com>

* add sphinxcontrib-redoc to requirements

* sphinxcontrib.redoc configuration

* create openapi directory and files

* update download script for both schema files

* suppress warning for redoc

* update labels

* fix extra closing parenthesis

* update schema url

* exclude doc config and download script

The Sphinx configuration (conf.py) and schema download script
(download-json.py) are not application logic and used only for building
documentation. Coverage requirements for these files are overkill.

* exclude only the sphinx config file

---------

Co-authored-by: Don Naro <dnaro@redhat.com>
2026-01-26 18:16:49 +00:00
Don Naro
ceed692354 change contact email address (#16245)
Use the ansible-community@redhat.com alias as the contact email address.
2026-01-26 17:28:00 +00:00
Jake Jackson
36a00ec46b AAP-58539 Move to dispatcherd (#16209)
* WIP First pass
* started removing feature flags and adjusting logic
* Add decorator
* moved to dispatcher decorator
* updated as many as I could find
* Keep callback receiver working
* remove any code that is not used by the call back receiver
* add back auto_max_workers
* added back get_auto_max_workers into common utils
* Remove control and hazmat (squash this not done)
* moved status out and deleted control as no longer needed
* removed unused imports
* adjusted test import to pull correct method
* fixed imports and addressed clusternode heartbeat test
* Update function comments
* Add back hazmat for config and remove baseworker
* added back hazmat per @alancoding feedback around config
* removed baseworker completely and refactored it into the callback
  worker
* Fix dispatcher run call and remove dispatch setting
* remove dispatcher mock publish setting
* Adjust heartbeat arg and more formatting
* fixed the call to cluster_node_heartbeat missing binder
* Fix attribute error in server logs
2026-01-23 20:49:32 +00:00
Alan Rominger
94d5769f32 Fix extremely flaky failure (#16161) 2026-01-23 10:05:44 -05:00
Alan Rominger
98430db58f Collect operator logs on timeout (#16239)
* Collect operator logs on timeout

* Set timeout back to prod value

* Add e to the bash with timeout block
2026-01-23 10:05:32 -05:00
Rodrigo Toshiaki Horie
acf8721a09 Enhance OpenAPI schema with AI descriptions and fix method names (#16228)
* Enhance OpenAPI schema with AI descriptions and fix method names

Add x-ai-description extensions to API endpoints for better AI agent
comprehension. Fix view method names to
ensure proper drf-spectacular schema generation.

* Enhance OpenAPI schema with AI descriptions and fix method names

Add x-ai-description extensions to API endpoints for better AI agent
comprehension. Fix view method names to
ensure proper drf-spectacular schema generation.
2026-01-21 16:53:19 -03:00
Hao Liu
a839ce8cb1 Update kubernetes python client to 35.0.0 from PyPI (#16237)
Remove transitive dependencies no longer needed by kubernetes 35.0.0

Removes google-auth and rsa which were transitive dependencies of the older
kubernetes client but are no longer required in v35.0.0.

Adds cachetools as a direct dependency since it's used by awx/conf/settings.py
for TTLCache (was previously a transitive dep of google-auth).
2026-01-20 17:31:18 -05:00
Hao Liu
543b2a66a3 Update kubernetes python client to 35.0.0 from PyPI (#16236)
- Move kubernetes from git-based install to PyPI (v35.0.0 now available)
- Remove urllib3 cap comment since kubernetes 35.0.0 no longer restricts it
- Update README.md upgrade blocker documentation
2026-01-20 16:20:41 -05:00
Alan Rominger
8c5cf49c23 Avoid errors installing with python 3.11 (#16231) 2026-01-20 10:24:41 -05:00
Peter Braun
80bb0c9862 remove artifacts from list endpoint (#16230) 2026-01-20 10:58:01 +01:00
Alan Rominger
b34ee01fb3 Slightly alter history to avoid having a Django 5 related migration (#16214)
* Slightly alter history to avoid having a Django 5 related migration

* Revert prior field states to be slightly more clear
2026-01-19 13:33:46 -05:00
Alan Rominger
dce5ac73c5 Apply new rules from black update (#16232) 2026-01-19 12:58:07 -05:00
PabloHiro
43a3a620e3 [AAP-43413] Removing hardcoded number of flags from feature flag test
Assited-by: Claude
2026-01-19 09:37:20 +01:00
PascalKont
051357e573 fixed description for option notification_templates_approvals in module organizations (#16170)
fixed module organizations description for option notification_templates_approvals

Co-authored-by: Pascal Kontschan <pascal.kontschan.extern@atruvia.de>
2026-01-15 11:48:27 -05:00
John Barker
75aba0f62d docs: migrate RTD URLs to docs.ansible.com (#16189)
* docs: update readthedocs.io URLs to docs.ansible.com equivalents

🤖 Generated with Claude Code
https://claude.ai/code

Co-Authored-By: Claude <noreply@anthropic.com>

* Update Bullhorn newsletter link in communication docs

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-15 12:27:47 +00:00
Hao Liu
fee71b8917 Replace pytz with standard library timezone (#16197)
Refactored code to use Python's built-in datetime.timezone and zoneinfo instead of pytz for timezone handling. This modernizes the codebase and removes the dependency on pytz, aligning with current best practices for timezone-aware datetime objects.
2026-01-09 16:05:08 -05:00
Hao Liu
dbe979b425 Add make targets for updating requirements (#16195)
Introduces new Makefile targets to update and upgrade requirements files using pip-compile, both directly and via docker-runner. These additions streamline dependency management for development and CI workflows.
2026-01-09 16:04:27 -05:00
Hao Liu
03d0ed882c Add kubernetes python client from git at release-34.0 (#16226)
Switch to git-based installation of kubernetes python client from
github.com/kubernetes-client/python at commit df31d90d6c910d6b5c883b98011c93421cac067d
(release-34.0 branch). This also allows removing the urllib3<2.4.0 upper bound
constraint that was previously required by kubernetes 34.1.0 from PyPI.
2026-01-09 20:16:34 +00:00
Hao Liu
b0debf93a0 Use dnf module for Node.js 18 instead of n version manager - damn it aarch64 (#16225)
Use dnf module for Node.js 18 instead of n version manager

The n version manager fails to extract Node.js archives due to very long
file paths in include/node/openssl/archs/ directories when running in
Docker BuildKit's overlay filesystem. This causes CI build failures with
tar "Cannot open: Invalid argument" errors.

Switch to installing Node.js 18 directly from CentOS Stream 9's module
stream which avoids the archive extraction issue entirely.
2026-01-09 18:37:34 +00:00
Hao Liu
d018096cae Fix devel awx, awx_devel, awx_kube_devel build (#16219)
* Fix ARM64 build failure by upgrading dev container Node.js to 18

Node.js 16.13.1 fails to extract on ARM64 in Docker BuildKit's
overlay filesystem during multi-arch builds. Upgrade to Node 18
which is already used by the UI builder stage and has proper
ARM64 support.

* Fix collectstatic failure by setting AWX_MODE=default

  AWX_MODE=defaults is an intentionally "invalid" environment name that:

  1. Loads only defaults.py - the base settings file without any environment-specific overrides (development_defaults.py, production_defaults.py, etc.)
  2. Bypasses production checks - since "production" not in "defaults", it skips the assertion that requires /etc/tower/settings.py to exist
  3. Bypasses development mode - since is_development_mode would be false

  This is perfect for collectstatic during container build because:
  - No database connection needed
  - No secret key needed (hence SKIP_SECRET_KEY_CHECK)
  - No PostgreSQL version check (hence SKIP_PG_VERSION_CHECK)
  - Just need minimal Django settings to collect static files
2026-01-09 12:07:23 -05:00
Alan Rominger
cfe0b367b5 Do not eat errors building images (#16216) 2026-01-09 02:48:34 +00:00
Alan Rominger
7d24bdbf13 Clear in-memory cache, suggested by bugbot (#16218)
* Clear in-memory cache, suggested by bugbot

* Clear the cache even harder than we were before

* Syntax bugbot
2026-01-08 16:03:29 -05:00
Alan Rominger
3cba5e1744 Cache juggling to help address test flake (#16217) 2026-01-08 14:23:01 -05:00
Hao Liu
10a2946f9f Fix requirement for python3.12 (#16215)
* Fix pip version constraint for Python 3.12 compatibility

Remove outdated pip<22.0 constraint that was a workaround for
pip-tools#1558. This issue was fixed in pip-tools 6.5.0+ and
the old constraint breaks Python 3.12 where pkgutil.ImpImporter
was removed.

* Update requirements.txt

* Fix license file inconsistencies with requirements

- Rename awx-plugins.interfaces.txt to awx-plugins-interfaces.txt
  to match the package name in requirements
- Remove backports-tarfile.txt and importlib-resources.txt as these
  packages are no longer in requirements

* Fix updater.sh for pip 25.3 normalized output format

Changes to requirements_git.txt:
- Update to PEP 440 format (name @ git+url) to match pip-compile output
- Normalize package names (hyphens instead of dots/underscores)
- Sort extras alphabetically with hyphens (e.g., jwt-consumer not jwt_consumer)
- Add documentation explaining format requirements

Changes to updater.sh:
- Escape BRE regex metacharacters in sed pattern to handle brackets in extras
- Change sed delimiter from ! to | to avoid conflict with comment text
- Add explicit return statements to functions
- Assign positional parameters to local variables
- Redirect error messages to stderr
- Replace backticks with $() for command substitution
- Pin pip to version 25.3

requirements.txt regenerated via updater.sh

* Normalize package names in requirements.in to match pip output

- prometheus_client -> prometheus-client
- setuptools_scm -> setuptools-scm
- dispatcherd[pg_notify] -> dispatcherd[pg-notify]

PEP 503 specifies that package names should use hyphens.

* Fix license files to match normalized package names

- Remove awx_plugins.interfaces.txt (duplicate of awx-plugins-interfaces.txt)
- Rename system-certifi.txt to certifi.txt to match package name
2026-01-08 14:21:11 -05:00
Hao Liu
049a4b6438 Remove graph_jobs management command and asciichartpy dependency (#16078)
Deleted the awx/main/management/commands/graph_jobs.py file and removed the asciichartpy package from requirements. This cleans up unused code and dependencies related to terminal job status graphing.
2026-01-07 20:53:30 -05:00
jessicamack
de86b93690 AAP-59874: Update to Python 3.12 (#16208)
* update to Python 3.12

* remove use of utcnow

* switch to timezone.utc

datetime.UTC is an alias of datetime.timezone.utc. if we're doing the double import for datetime it's more straightforward to just import timezone as well and get it directly

* debug python env version issue

* change python version

* pin to SHA and remove debug portion
2026-01-07 11:57:24 -05:00
Alan Rominger
48c7534b57 AAP-60452 Remove the dynamic log level filter for the dispatcherd main process (#16200)
* Remove the dynamic filter on dispatcher startup

Configure the dynamic logging level only on startup

* Special case for log level on settings change

* Add unit test for new behavior

* Add test for initial config

* Mark test django DB

* Do necessary requirement bump

* Delete cache in live test fixture
2026-01-02 15:45:06 -05:00
Alan Rominger
40059512d8 Bump requirement because version was yanked from PyPI (#16212) 2026-01-02 11:14:00 -05:00
Bryan Havenstein
e2c1c5116d AAP-58457 Update UT for removed IPv6 feature flag 2026-01-02 09:39:04 -05:00
Chris Meyers
41f1ffc1dd AAP-45541 Add test to recreate jobs/4075584/job_events/children_summary/ error (#16163)
* Add test to recreate the error

* Also begin to add detection for empty event

* Remove breakpoint

* fix: ignore events with missing event types

* run linter and apply changes

---------

Co-authored-by: AlanCoding <arominge@redhat.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
2025-12-17 21:34:53 +01:00
Alan Rominger
d7eb714859 Remove custom docs endpoint in DAB now (#16204)
* Remove custom docs endpoint in DAB now
2025-12-16 13:57:27 -05:00
Alan Rominger
7a58377aff Update ENV pattern in Dockerfile (#16202) 2025-12-16 09:57:54 -05:00
Hao Liu
2fbfe4ca73 Fix __pycache__ directory removal in clean target (#16196)
Replaces the use of 'find -delete' with 'find -exec rm -rf {} +' to ensure all __pycache__ directories are properly removed during the clean process.
2025-12-16 09:00:38 -05:00
Hao Liu
04fadab253 Remove unused ANSIBLE_BASE_PERMISSION_MODEL setting (#16198)
Deleted the ANSIBLE_BASE_PERMISSION_MODEL configuration from defaults.py as it is no longer needed.
2025-12-16 08:57:14 -05:00
Alan Rominger
054f6032fd AAP-47956 Use pg_notify for cancel and debugging, abandon socket approach (#16199)
* Use pg_notify for cancel and debugging, abandon socket approach

* Bump dispatcherd for pg_notify chunking
2025-12-10 14:38:39 -05:00
Seth Foster
f935134a19 Unpin rsyslog in container (#16203)
docker buildx build fails with
"Error: Unable to find a match: rsyslog-8.2102.0-106.el9"

unpinning builds successfully for both arm64 and x86_64

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2025-12-10 14:32:01 -05:00
Hao Liu
b24156805a Upgrade to Django 5.2 LTS (#16185)
Upgrade to Django 5.2 LTS with compatibility fixes across fields, migrations, dispatch config, tests, and dev deps.

Dependencies:
- Upgrade django to 5.2.8 and relax requirements.in to >=5.2,<5.3.
- Bump django-debug-toolbar to >=6.0 for compatibility.

Backend:
- awx/conf/fields.py: switch URL TLD regex to use DomainNameValidator.ul in custom URLField.
- awx/main/management/commands/gather_analytics.py: use datetime.timezone.utc for naïve datetime handling.
- awx/main/dispatch/config.py: add mock_publish option; avoid DB access for test runs, set default max_workers, and support a noop broker.

Migrations (SQLite/Postgres compatibility):
- Add awx/main/migrations/_sqlite_helper.py with db-aware AlterIndexTogether/RenameIndex wrappers; consume in 0144_event_partitions.py and 0184_django_indexes.py.
- Update 0187_hop_nodes.py to use CheckConstraint(condition=...).
- Add 0205_alter_instance_peers_alter_job_hosts_and_more.py adjusting through_fields/relations on instance.peers, job.hosts, and role.ancestors.
- _dab_rbac.py: iterate roles with chunk_size=1000 for migration performance.

Tests:
Include hcp_terraform in default credential types in test_credential.py.
---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
2025-12-03 14:22:52 -05:00
Elijah DeLee
711b018ae7 cache dashboard query (#16165)
This causes an expensive query and the view sometimes called excessively
by the UI.  Memoize per unique user and params (time period) for 15s.
2025-12-03 13:03:39 -05:00
Stevenson Michel
be30a75c4f Removal of Warning for Distro Deprecation (#16193)
remove warning for distro deprecation
2025-12-02 14:28:12 -05:00
Seth Foster
a20f299cd6 Add x-ai-description to schema (#16186)
Adding ansible_base.api_documentation
to the INSTALL_APPS which extends the schema
to include an LLM-friendly description
to each endpoint

---------

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
2025-12-01 19:45:28 -05:00
Lila Yasin
4f41b50a09 AAP-57817 Add Redis connection retry using redis-py 7.0+ built-in (#16176)
* AAP-57817 Add Redis connection retry using redis-py 7.0+ built-in mechanism

* Refactor Redis client helpers to use settings and eliminate code duplication

* Create awx/main/utils/redis.py and move Redis client functions to avoid circular imports

* Fix subsystem_metrics to share Redis connection pool between
  client and pipeline

* Cache Redis clients in RelayConsumer and RelayWebsocketStatsManager to avoid creating new connection pools on every call

* Add cap and base config

* Add Redis retry logic with exponential backoff to handle connection failures during long-running operations

* Add REDIS_BACKOFF_CAP and REDIS_BACKOFF_BASE settings to allow
  adjustment of retry timing in worst-case scenarios without code changes

* Simplify Redis retry tests by removing unnecessary reload logic
2025-12-01 09:08:47 -05:00
Rodrigo Toshiaki Horie
0d86874d5d Organize S3 schema uploads by product (awx/tower) (#16190)
Update schema upload workflows to organize S3 files by product name:
- Upload schemas to s3://awx-public-ci-files/{product}/{branch}/schema.json
- Update Makefile to download from product-specific paths for schema diff
- Update feature branch deletion to clean up from correct product path

This separates AWX and Tower schemas into distinct S3 folders.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-27 10:43:38 -03:00
Fabricio Aguiar
2b2f2b73ac Move to Runtime Platform Flags (#16148)
* move to platform flags

Signed-off-by: Fabricio Aguiar <fabricio.aguiar@gmail.com>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

* SonarCloud analyzes files without coverage data.
2025-11-25 10:20:04 -05:00
Lila Yasin
e03beb4d54 Add hcp_terraform to list of expected cred types to fix failing api test CI Check (#16188)
* Add hcp_terraform to list of expected cred types to fix failing api test ci check
2025-11-24 13:09:04 -05:00
Chris Meyers
4db52e074b Fix collectstatic (#16162) 2025-11-21 15:19:48 -05:00
Lila Yasin
4e1911f7c4 Bump Django to 4.2.26 to agree with DAB changes (#16183) 2025-11-19 14:56:29 -05:00
Peter Braun
b02117979d AAP-29938 add force flag to refspec (#16173)
* add force flag to refspec

* Development of git --amend test

* Update awx/main/tests/live/tests/conftest.py

Co-authored-by: Alan Rominger <arominge@redhat.com>

---------

Co-authored-by: AlanCoding <arominge@redhat.com>
2025-11-13 14:51:23 +01:00
Seth Foster
2fa2cd8beb Add timeout and on duplicate to system tasks (#16169)
Modify the invocation of @task_awx to accept timeout and
on_duplicate keyword arguments. These arguments are
only used in the new dispatcher implementation.

Add decorator params:
- timeout
- on_duplicate

to tasks to ensure better recovery for
stuck or long-running processes.

---------

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2025-11-12 23:18:57 -05:00
Rodrigo Toshiaki Horie
f81859510c Change Swagger UI endpoint from /api/swagger/ to /api/docs/ (#16172)
* Change Swagger UI endpoint from /api/swagger/ to /api/docs/

- Update URL pattern to use /docs/ instead of /swagger/
- Update API root response to show 'docs' key instead of 'swagger'
- Add authentication requirement for schema documentation endpoints
- Update contact email to controller-eng@redhat.com

The schema endpoints (/api/docs/, /api/schema/, /api/redoc/) now
require authentication to prevent unauthorized access to API
documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Require authentication for all schema endpoints including /api/schema/

Create custom view classes that enforce authentication for all schema
endpoints to prevent inconsistent access control where UI views required
authentication but the raw schema endpoint remained publicly accessible.

This ensures all schema endpoints (/api/schema/, /api/docs/, /api/redoc/)
consistently require authentication.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add unit tests for authenticated schema view classes

Add test coverage for the new AuthenticatedSpectacular* view classes
to ensure they properly require authentication.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove unused import

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-12 09:14:54 -03:00
Rodrigo Toshiaki Horie
335a4bbbc6 AAP-45927 Add drf-spectacular (#16154)
* AAP-45927 Add drf-spectacular

- Remove drf-yasg
- Add drf-spectacular

* move SPECTACULAR_SETTINGS from development_defaults.py to defaults.py

* move SPECTACULAR_SETTINGS from development_defaults.py to defaults.py

* Fix swagger tests: enable schema endpoints in all modes

Schema endpoints were restricted to development mode, causing
test_swagger_generation.py to fail. Made schema URLs available in
all modes and fixed deprecated Django warning filters in pytest.ini.

* remove swagger from Makefile

* remove swagger from Makefile

* change docker-compose-build-swagger to docker-compose-build-schema

* remove MODE

* remove unused import

* Update genschema to use drf-spectacular with awx-link dependency

- Add awx-link as dependency for genschema targets to ensure package metadata exists
- Remove --validate --fail-on-warn flags (schema needs improvements first)
- Add genschema-yaml target for YAML output
- Add schema.yaml to .gitignore

* Fix detect-schema-change to not fail on schema differences

Add '-' prefix to diff command so Make ignores its exit status.
diff returns exit code 1 when files differ, which is expected behavior
for schema change detection, not an error.

* Truncate schema diff summary to stay under GitHub's 1MB limit

Limit schema diff output in job summary to first 1000 lines to avoid
exceeding GitHub's 1MB step summary size limit. Add message indicating
when diff is truncated and direct users to job logs or artifacts for
full output.

* readd MODE

* add drf-spectacular to requirements.in and the requirements.txt generated from the script

* Add drf-spectacular BSD license file

Required for test_python_licenses test to pass now that drf-spectacular
is in requirements.txt.

* add licenses

* Add comprehensive unit tests for CustomAutoSchema

Adds 15 unit tests for awx/api/schema.py to improve SonarCloud test
coverage. Tests cover all code paths in CustomAutoSchema including:
- get_tags() method with various scenarios (swagger_topic, serializer
  Meta.model, view.model, exception handling, fallbacks, warnings)
- is_deprecated() method with different view configurations
- Edge cases and priority ordering

All tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove unused imports

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-10 12:35:22 -03:00
Lila Yasin
5ea2fe65b0 Fix failing Collection CI Checks (#16157)
* Fix CI: Use Python 3.13 for ansible-test compatibility

  ansible-test only supports Python 3.11, 3.12, and 3.13.
  Changed collection-integration jobs from '3.x' to '3.13'
  to avoid using Python 3.14 which is not supported.

* Fix ansible-test Python version for CI integration tests

  ansible-test only supports Python 3.11, 3.12, and 3.13.
  Added ANSIBLE_TEST_PYTHON_VERSION variable to explicitly pass
  --python 3.13 flag to ansible-test integration command.

  This prevents ansible-test from auto-detecting and using
  Python 3.14.0, which is not supported.

* Fix CI: Execute ansible-test with Python 3.13 to avoid
  unsupported Python 3.14

* Fix CI: Use Python 3.13 across all jobs to avoid Python
  3.14 compatibility issues

* Fix CI: Use 'python' and 'ansible.test' module for Python
   3.13 compatibility

* Fix CI: Use 'python' instead of 'python3' for Python 3.13
   compatibility

* Fix CI: Ensure ansible-test uses Python 3.13 environment
  explicitly

* Fix: Remove silent failure check for ansible-core in test suite

* Fix CI: Export PYTHONPATH to make awxkit available to ansible-test

* Fix CI: Use 'python' in run_awx_devel to maintain Python
  3.13 environment

* Fix CI: Remove setup-python from awx_devel_image that was resetting Python 3.13 to 3.14
2025-11-06 09:22:44 -05:00
Daniel Finca Martínez
f3f10ae9ce [AAP-42616] Bump receptor collection version to 2.0.6 (#16156)
Bump receptor collection version to 2.0.6
2025-11-05 15:49:43 -07:00
Jake Jackson
5be4462395 Update sonar and CI (#16153)
* actually upload PR coverage reports and inject PR number if report is
  generated from a PR
* upload general report of devel on merge and make things kinda pretty
2025-11-03 14:42:55 +00:00
424 changed files with 15135 additions and 4778 deletions

View File

@@ -1,3 +1,3 @@
# Community Code of Conduct # Community Code of Conduct
Please see the official [Ansible Community Code of Conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html). Please see the official [Ansible Community Code of Conduct](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html).

View File

@@ -13,7 +13,7 @@ body:
attributes: attributes:
label: Please confirm the following label: Please confirm the following
options: options:
- label: I agree to follow this project's [code of conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html). - label: I agree to follow this project's [code of conduct](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html).
required: true required: true
- label: I have checked the [current issues](https://github.com/ansible/awx/issues) for duplicates. - label: I have checked the [current issues](https://github.com/ansible/awx/issues) for duplicates.
required: true required: true

View File

@@ -5,7 +5,7 @@ contact_links:
url: https://github.com/ansible/awx#get-involved url: https://github.com/ansible/awx#get-involved
about: For general debugging or technical support please see the Get Involved section of our readme. about: For general debugging or technical support please see the Get Involved section of our readme.
- name: 📝 Ansible Code of Conduct - name: 📝 Ansible Code of Conduct
url: https://docs.ansible.com/ansible/latest/community/code_of_conduct.html?utm_medium=github&utm_source=issue_template_chooser url: https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html?utm_medium=github&utm_source=issue_template_chooser
about: AWX uses the Ansible Code of Conduct; ❤ Be nice to other members of the community. ☮ Behave. about: AWX uses the Ansible Code of Conduct; ❤ Be nice to other members of the community. ☮ Behave.
- name: 💼 For Enterprise - name: 💼 For Enterprise
url: https://www.ansible.com/products/engine?utm_medium=github&utm_source=issue_template_chooser url: https://www.ansible.com/products/engine?utm_medium=github&utm_source=issue_template_chooser

View File

@@ -13,7 +13,7 @@ body:
attributes: attributes:
label: Please confirm the following label: Please confirm the following
options: options:
- label: I agree to follow this project's [code of conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html). - label: I agree to follow this project's [code of conduct](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html).
required: true required: true
- label: I have checked the [current issues](https://github.com/ansible/awx/issues) for duplicates. - label: I have checked the [current issues](https://github.com/ansible/awx/issues) for duplicates.
required: true required: true

View File

@@ -24,7 +24,7 @@ in as the first entry for your PR title.
##### ADDITIONAL INFORMATION ##### STEPS TO REPRODUCE AND EXTRA INFO
<!--- <!---
Include additional information to help people understand the change here. Include additional information to help people understand the change here.
For bugs that don't have a linked bug report, a step-by-step reproduction For bugs that don't have a linked bug report, a step-by-step reproduction

View File

@@ -11,8 +11,6 @@ inputs:
runs: runs:
using: composite using: composite
steps: steps:
- uses: ./.github/actions/setup-python
- name: Set lower case owner name - name: Set lower case owner name
shell: bash shell: bash
run: echo "OWNER_LC=${OWNER,,}" >> $GITHUB_ENV run: echo "OWNER_LC=${OWNER,,}" >> $GITHUB_ENV

View File

@@ -36,7 +36,7 @@ runs:
- name: Upgrade ansible-core - name: Upgrade ansible-core
shell: bash shell: bash
run: python3 -m pip install --upgrade ansible-core run: python -m pip install --upgrade ansible-core
- name: Install system deps - name: Install system deps
shell: bash shell: bash

View File

@@ -70,10 +70,10 @@ Thank you for your submission and for supporting AWX!
- Hello, we'd love to help, but we need a little more information about the problem you're having. Screenshots, log outputs, or any reproducers would be very helpful. - Hello, we'd love to help, but we need a little more information about the problem you're having. Screenshots, log outputs, or any reproducers would be very helpful.
### Code of Conduct ### Code of Conduct
- Hello. Please keep in mind that Ansible adheres to a Code of Conduct in its community spaces. The spirit of the code of conduct is to be kind, and this is your friendly reminder to be so. Please see the full code of conduct here if you have questions: https://docs.ansible.com/ansible/latest/community/code_of_conduct.html - Hello. Please keep in mind that Ansible adheres to a Code of Conduct in its community spaces. The spirit of the code of conduct is to be kind, and this is your friendly reminder to be so. Please see the full code of conduct here if you have questions: https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html
### EE Contents / Community General ### EE Contents / Community General
- Hello. The awx-ee contains the collections and dependencies needed for supported AWX features to function. Anything beyond that (like the community.general package) will require you to build your own EE. For information on how to do that, see https://ansible-builder.readthedocs.io/en/stable/ \ - Hello. The awx-ee contains the collections and dependencies needed for supported AWX features to function. Anything beyond that (like the community.general package) will require you to build your own EE. For information on how to do that, see https://docs.ansible.com/projects/builder/en/stable/ \
\ \
The Ansible Community is looking at building an EE that corresponds to all of the collections inside the ansible package. That may help you if and when it happens; see https://github.com/ansible-community/community-topics/issues/31 for details. The Ansible Community is looking at building an EE that corresponds to all of the collections inside the ansible package. That may help you if and when it happens; see https://github.com/ansible-community/community-topics/issues/31 for details.
@@ -88,7 +88,7 @@ The Ansible Community is looking at building an EE that corresponds to all of th
- Hello, we think your idea is good! Please consider contributing a PR for this following our contributing guidelines: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md - Hello, we think your idea is good! Please consider contributing a PR for this following our contributing guidelines: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md
### Receptor ### Receptor
- You can find the receptor docs here: https://receptor.readthedocs.io/en/latest/ - You can find the receptor docs here: https://docs.ansible.com/projects/receptor/en/latest/
- Hello, your issue seems related to receptor. Could you please open an issue in the receptor repository? https://github.com/ansible/receptor. Thanks! - Hello, your issue seems related to receptor. Could you please open an issue in the receptor repository? https://github.com/ansible/receptor. Thanks!
### Ansible Engine not AWX ### Ansible Engine not AWX

55
.github/workflows/_repo-owns-branch.yml vendored Normal file
View File

@@ -0,0 +1,55 @@
---
name: Repo Owns Branch
# Reusable workflow that determines whether the current repository
# owns the current branch for push operations.
#
# Ownership rules:
# - ansible/awx owns: devel, feature_*
# - ansible/tower owns: stable-*, release_*
# - workflow_dispatch is always allowed
#
# All other repo/branch combinations are skipped.
on:
workflow_call:
outputs:
should_run:
description: Whether this repo owns the current branch
value: ${{ jobs.check.outputs.should_run }}
jobs:
check:
runs-on: ubuntu-latest
outputs:
should_run: ${{ steps.check.outputs.should_run }}
steps:
- name: Check branch ownership
id: check
run: |
REPO="${{ github.repository }}"
BRANCH="${{ github.ref_name }}"
EVENT="${{ github.event_name }}"
if [[ "$EVENT" == "workflow_dispatch" ]]; then
echo "should_run=true" >> $GITHUB_OUTPUT
echo "Manual trigger — allowed"
exit 0
fi
# ansible/awx owns devel and feature_* branches
if [[ "$REPO" == "ansible/awx" ]] && [[ "$BRANCH" == "devel" || "$BRANCH" == feature_* ]]; then
echo "should_run=true" >> $GITHUB_OUTPUT
echo "Repository '$REPO' owns branch '$BRANCH'"
exit 0
fi
# ansible/tower owns stable-* and release_* branches
if [[ "$REPO" == "ansible/tower" ]] && [[ "$BRANCH" == stable-* || "$BRANCH" == release_* ]]; then
echo "should_run=true" >> $GITHUB_OUTPUT
echo "Repository '$REPO' owns branch '$BRANCH'"
exit 0
fi
echo "should_run=false" >> $GITHUB_OUTPUT
echo "Repository '$REPO' does not own branch '$BRANCH' — skipping"

View File

@@ -45,22 +45,58 @@ jobs:
make docker-runner 2>&1 | tee schema-diff.txt make docker-runner 2>&1 | tee schema-diff.txt
exit ${PIPESTATUS[0]} exit ${PIPESTATUS[0]}
- name: Add schema diff to job summary - name: Validate OpenAPI schema
if: always() id: schema-validation
# show text and if for some reason, it can't be generated, state that it can't be. continue-on-error: true
run: | run: |
echo "## API Schema Change Detection Results" >> $GITHUB_STEP_SUMMARY AWX_DOCKER_ARGS='-e GITHUB_ACTIONS' \
AWX_DOCKER_CMD='make validate-openapi-schema' \
make docker-runner 2>&1 | tee schema-validation.txt
exit ${PIPESTATUS[0]}
- name: Add schema validation and diff to job summary
if: always()
# show text and if for some reason, it can't be generated, state that it can't be.
run: |
echo "## API Schema Check Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY
# Show validation status
echo "### OpenAPI Validation" >> $GITHUB_STEP_SUMMARY
if [ -f schema-validation.txt ] && grep -q "✓ Schema is valid" schema-validation.txt; then
echo "✅ **Status:** PASSED - Schema is valid OpenAPI 3.0.3" >> $GITHUB_STEP_SUMMARY
else
echo "❌ **Status:** FAILED - Schema validation failed" >> $GITHUB_STEP_SUMMARY
if [ -f schema-validation.txt ]; then
echo "" >> $GITHUB_STEP_SUMMARY
echo "<details><summary>Validation errors</summary>" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
cat schema-validation.txt >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
echo "</details>" >> $GITHUB_STEP_SUMMARY
fi
fi
echo "" >> $GITHUB_STEP_SUMMARY
# Show schema changes
echo "### Schema Changes" >> $GITHUB_STEP_SUMMARY
if [ -f schema-diff.txt ]; then if [ -f schema-diff.txt ]; then
if grep -q "^+" schema-diff.txt || grep -q "^-" schema-diff.txt; then if grep -q "^+" schema-diff.txt || grep -q "^-" schema-diff.txt; then
echo "### Schema changes detected" >> $GITHUB_STEP_SUMMARY echo "**Changes detected** between this PR and the base branch" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY
# Truncate to first 1000 lines to stay under GitHub's 1MB summary limit
TOTAL_LINES=$(wc -l < schema-diff.txt)
if [ $TOTAL_LINES -gt 1000 ]; then
echo "_Showing first 1000 of ${TOTAL_LINES} lines. See job logs or download artifact for full diff._" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
fi
echo '```diff' >> $GITHUB_STEP_SUMMARY echo '```diff' >> $GITHUB_STEP_SUMMARY
cat schema-diff.txt >> $GITHUB_STEP_SUMMARY head -n 1000 schema-diff.txt >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY echo '```' >> $GITHUB_STEP_SUMMARY
else else
echo "### No schema changes detected" >> $GITHUB_STEP_SUMMARY echo "No schema changes detected" >> $GITHUB_STEP_SUMMARY
fi fi
else else
echo "### Unable to generate schema diff" >> $GITHUB_STEP_SUMMARY echo "Unable to generate schema diff" >> $GITHUB_STEP_SUMMARY
fi fi

View File

@@ -4,14 +4,46 @@ env:
LC_ALL: "C.UTF-8" # prevent ERROR: Ansible could not initialize the preferred locale: unsupported locale setting LC_ALL: "C.UTF-8" # prevent ERROR: Ansible could not initialize the preferred locale: unsupported locale setting
CI_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} CI_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DEV_DOCKER_OWNER: ${{ github.repository_owner }} DEV_DOCKER_OWNER: ${{ github.repository_owner }}
COMPOSE_TAG: ${{ github.base_ref || 'devel' }} COMPOSE_TAG: ${{ github.base_ref || github.ref_name || 'devel' }}
UPSTREAM_REPOSITORY_ID: 91594105 UPSTREAM_REPOSITORY_ID: 91594105
on: on:
pull_request: pull_request:
push: push:
branches: branches:
- devel # needed to publish code coverage post-merge - devel # needed to publish code coverage post-merge
schedule:
- cron: '0 12,18 * * 1-5'
workflow_dispatch: {}
jobs: jobs:
trigger-release-branches:
name: "Dispatch CI to release branches"
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
permissions:
actions: write
steps:
- name: Trigger CI on release_4.6
id: dispatch_release_46
continue-on-error: true
run: gh workflow run ci.yml --ref release_4.6
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
- name: Trigger CI on stable-2.6
id: dispatch_stable_26
continue-on-error: true
run: gh workflow run ci.yml --ref stable-2.6
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
- name: Check dispatch results
if: steps.dispatch_release_46.outcome == 'failure' || steps.dispatch_stable_26.outcome == 'failure'
run: |
echo "One or more dispatches failed:"
echo " release_4.6: ${{ steps.dispatch_release_46.outcome }}"
echo " stable-2.6: ${{ steps.dispatch_stable_26.outcome }}"
exit 1
common-tests: common-tests:
name: ${{ matrix.tests.name }} name: ${{ matrix.tests.name }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -32,9 +64,6 @@ jobs:
- name: api-lint - name: api-lint
command: /var/lib/awx/venv/awx/bin/tox -e linters command: /var/lib/awx/venv/awx/bin/tox -e linters
coverage-upload-name: "" coverage-upload-name: ""
- name: api-swagger
command: /start_tests.sh swagger
coverage-upload-name: ""
- name: awx-collection - name: awx-collection
command: /start_tests.sh test_collection_all command: /start_tests.sh test_collection_all
coverage-upload-name: "awx-collection" coverage-upload-name: "awx-collection"
@@ -57,6 +86,21 @@ jobs:
AWX_DOCKER_CMD='${{ matrix.tests.command }}' AWX_DOCKER_CMD='${{ matrix.tests.command }}'
make docker-runner make docker-runner
- name: Inject PR number into coverage.xml
if: >-
!cancelled()
&& github.event_name == 'pull_request'
&& steps.make-run.outputs.cov-report-files != ''
run: |
if [ -f "reports/coverage.xml" ]; then
sed -i '2i<!-- PR ${{ github.event.pull_request.number }} -->' reports/coverage.xml
echo "Injected PR number ${{ github.event.pull_request.number }} into reports/coverage.xml"
fi
if [ -f "awxkit/coverage.xml" ]; then
sed -i '2i<!-- PR ${{ github.event.pull_request.number }} -->' awxkit/coverage.xml
echo "Injected PR number ${{ github.event.pull_request.number }} into awxkit/coverage.xml"
fi
- name: Upload test coverage to Codecov - name: Upload test coverage to Codecov
if: >- if: >-
!cancelled() !cancelled()
@@ -96,25 +140,37 @@ jobs:
}} }}
token: ${{ secrets.CODECOV_TOKEN }} token: ${{ secrets.CODECOV_TOKEN }}
- name: Upload awx jUnit test reports - name: Upload test artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.tests.name }}-artifacts
path: |
reports/coverage.xml
awxkit/coverage.xml
retention-days: 5
- name: >-
Upload ${{
matrix.tests.coverage-upload-name || 'awx'
}} jUnit test reports to the unified dashboard
if: >- if: >-
!cancelled() !cancelled()
&& steps.make-run.outputs.test-result-files != '' && steps.make-run.outputs.test-result-files != ''
&& github.event_name == 'push' && github.event_name == 'push'
&& env.UPSTREAM_REPOSITORY_ID == github.repository_id && env.UPSTREAM_REPOSITORY_ID == github.repository_id
&& github.ref_name == github.event.repository.default_branch && github.ref_name == github.event.repository.default_branch
run: | uses: ansible/gh-action-record-test-results@3784db66a1b7fb3809999a7251c8a7203a7ffbe8
for junit_file in $(echo '${{ steps.make-run.outputs.test-result-files }}' | sed 's/,/ /') with:
do aggregation-server-url: ${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_URL }}
curl \ http-auth-password: >-
-v \ ${{ secrets.PDE_ORG_RESULTS_UPLOAD_PASSWORD }}
--user "${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_USER }}:${{ secrets.PDE_ORG_RESULTS_UPLOAD_PASSWORD }}" \ http-auth-username: >-
--form "xunit_xml=@${junit_file}" \ ${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_USER }}
--form "component_name=${{ matrix.tests.coverage-upload-name || 'awx' }}" \ project-component-name: >-
--form "git_commit_sha=${{ github.sha }}" \ ${{ matrix.tests.coverage-upload-name || 'awx' }}
--form "git_repository_url=https://github.com/${{ github.repository }}" \ test-result-files: >-
"${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_URL }}/api/results/upload/" ${{ steps.make-run.outputs.test-result-files }}
done
dev-env: dev-env:
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -126,7 +182,7 @@ jobs:
- uses: ./.github/actions/setup-python - uses: ./.github/actions/setup-python
with: with:
python-version: '3.x' python-version: '3.13'
- uses: ./.github/actions/run_awx_devel - uses: ./.github/actions/run_awx_devel
id: awx id: awx
@@ -167,14 +223,19 @@ jobs:
path: awx-operator path: awx-operator
- name: Setup python, referencing action at awx relative path - name: Setup python, referencing action at awx relative path
uses: ./awx/.github/actions/setup-python uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with: with:
python-version: '3.x' python-version: '3.12'
- name: Install playbook dependencies - name: Install playbook dependencies
run: | run: |
python3 -m pip install docker python -m pip install docker
- name: Check Python version
working-directory: awx
run: |
make print-PYTHON
- name: Build AWX image - name: Build AWX image
working-directory: awx working-directory: awx
run: | run: |
@@ -186,27 +247,59 @@ jobs:
- name: Run test deployment with awx-operator - name: Run test deployment with awx-operator
working-directory: awx-operator working-directory: awx-operator
id: awx_operator_test
timeout-minutes: 60
continue-on-error: true
run: | run: |
python3 -m pip install -r molecule/requirements.txt set +e
python3 -m pip install PyYAML # for awx/tools/scripts/rewrite-awx-operator-requirements.py timeout 15m bash -elc '
$(realpath ../awx/tools/scripts/rewrite-awx-operator-requirements.py) molecule/requirements.yml $(realpath ../awx) python -m pip install -r molecule/requirements.txt
ansible-galaxy collection install -r molecule/requirements.yml python -m pip install PyYAML # for awx/tools/scripts/rewrite-awx-operator-requirements.py
sudo rm -f $(which kustomize) $(realpath ../awx/tools/scripts/rewrite-awx-operator-requirements.py) molecule/requirements.yml $(realpath ../awx)
make kustomize ansible-galaxy collection install -r molecule/requirements.yml
KUSTOMIZE_PATH=$(readlink -f bin/kustomize) molecule -v test -s kind -- --skip-tags=replicas sudo rm -f $(which kustomize)
make kustomize
KUSTOMIZE_PATH=$(readlink -f bin/kustomize) molecule -v test -s kind -- --skip-tags=replicas
'
rc=$?
if [ $rc -eq 124 ]; then
echo "timed_out=true" >> "$GITHUB_OUTPUT"
fi
exit $rc
env: env:
AWX_TEST_IMAGE: local/awx AWX_TEST_IMAGE: local/awx
AWX_TEST_VERSION: ci AWX_TEST_VERSION: ci
AWX_EE_TEST_IMAGE: quay.io/ansible/awx-ee:latest AWX_EE_TEST_IMAGE: quay.io/ansible/awx-ee:latest
STORE_DEBUG_OUTPUT: true STORE_DEBUG_OUTPUT: true
- name: Collect awx-operator logs on timeout
# Only run on timeout; normal failures should use molecule's built-in log collection.
if: steps.awx_operator_test.outputs.timed_out == 'true'
run: |
mkdir -p "$DEBUG_OUTPUT_DIR"
if command -v kind >/dev/null 2>&1; then
for cluster in $(kind get clusters 2>/dev/null); do
kind export logs "$DEBUG_OUTPUT_DIR/$cluster" --name "$cluster" || true
done
fi
if command -v kubectl >/dev/null 2>&1; then
kubectl get all -A -o wide > "$DEBUG_OUTPUT_DIR/kubectl-get-all.txt" || true
kubectl get pods -A -o wide > "$DEBUG_OUTPUT_DIR/kubectl-get-pods.txt" || true
kubectl describe pods -A > "$DEBUG_OUTPUT_DIR/kubectl-describe-pods.txt" || true
fi
docker ps -a > "$DEBUG_OUTPUT_DIR/docker-ps.txt" || true
- name: Upload debug output - name: Upload debug output
if: failure() if: always()
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: awx-operator-debug-output name: awx-operator-debug-output
path: ${{ env.DEBUG_OUTPUT_DIR }} path: ${{ env.DEBUG_OUTPUT_DIR }}
- name: Fail awx-operator check if test deployment failed
if: steps.awx_operator_test.outcome != 'success'
run: exit 1
collection-sanity: collection-sanity:
name: awx_collection sanity name: awx_collection sanity
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -241,18 +334,16 @@ jobs:
&& github.event_name == 'push' && github.event_name == 'push'
&& env.UPSTREAM_REPOSITORY_ID == github.repository_id && env.UPSTREAM_REPOSITORY_ID == github.repository_id
&& github.ref_name == github.event.repository.default_branch && github.ref_name == github.event.repository.default_branch
run: | uses: ansible/gh-action-record-test-results@3784db66a1b7fb3809999a7251c8a7203a7ffbe8
for junit_file in $(echo '${{ steps.make-run.outputs.test-result-files }}' | sed 's/,/ /') with:
do aggregation-server-url: ${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_URL }}
curl \ http-auth-password: >-
-v \ ${{ secrets.PDE_ORG_RESULTS_UPLOAD_PASSWORD }}
--user "${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_USER }}:${{ secrets.PDE_ORG_RESULTS_UPLOAD_PASSWORD }}" \ http-auth-username: >-
--form "xunit_xml=@${junit_file}" \ ${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_USER }}
--form "component_name=awx" \ project-component-name: awx
--form "git_commit_sha=${{ github.sha }}" \ test-result-files: >-
--form "git_repository_url=https://github.com/${{ github.repository }}" \ ${{ steps.make-run.outputs.test-result-files }}
"${{ vars.PDE_ORG_RESULTS_AGGREGATOR_UPLOAD_URL }}/api/results/upload/"
done
collection-integration: collection-integration:
name: awx_collection integration name: awx_collection integration
@@ -275,7 +366,11 @@ jobs:
- uses: ./.github/actions/setup-python - uses: ./.github/actions/setup-python
with: with:
python-version: '3.x' python-version: '3.13'
- name: Remove system ansible to avoid conflicts
run: |
python -m pip uninstall -y ansible ansible-core || true
- uses: ./.github/actions/run_awx_devel - uses: ./.github/actions/run_awx_devel
id: awx id: awx
@@ -286,8 +381,9 @@ jobs:
- name: Install dependencies for running tests - name: Install dependencies for running tests
run: | run: |
python3 -m pip install -e ./awxkit/ python -m pip install -e ./awxkit/
python3 -m pip install -r awx_collection/requirements.txt python -m pip install -r awx_collection/requirements.txt
hash -r # Rehash to pick up newly installed scripts
- name: Run integration tests - name: Run integration tests
id: make-run id: make-run
@@ -299,6 +395,7 @@ jobs:
echo 'password = password' >> ~/.tower_cli.cfg echo 'password = password' >> ~/.tower_cli.cfg
echo 'verify_ssl = false' >> ~/.tower_cli.cfg echo 'verify_ssl = false' >> ~/.tower_cli.cfg
TARGETS="$(ls awx_collection/tests/integration/targets | grep '${{ matrix.target-regex.regex }}' | tr '\n' ' ')" TARGETS="$(ls awx_collection/tests/integration/targets | grep '${{ matrix.target-regex.regex }}' | tr '\n' ' ')"
export PYTHONPATH="$(python -c 'import site; print(":".join(site.getsitepackages()))')${PYTHONPATH:+:$PYTHONPATH}"
make COLLECTION_VERSION=100.100.100-git COLLECTION_TEST_TARGET="--requirements $TARGETS" test_collection_integration make COLLECTION_VERSION=100.100.100-git COLLECTION_TEST_TARGET="--requirements $TARGETS" test_collection_integration
env: env:
ANSIBLE_TEST_PREFER_PODMAN: 1 ANSIBLE_TEST_PREFER_PODMAN: 1
@@ -353,10 +450,14 @@ jobs:
- uses: ./.github/actions/setup-python - uses: ./.github/actions/setup-python
with: with:
python-version: '3.x' python-version: '3.13'
- name: Remove system ansible to avoid conflicts
run: |
python -m pip uninstall -y ansible ansible-core || true
- name: Upgrade ansible-core - name: Upgrade ansible-core
run: python3 -m pip install --upgrade ansible-core run: python -m pip install --upgrade ansible-core
- name: Download coverage artifacts - name: Download coverage artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
@@ -371,11 +472,12 @@ jobs:
mkdir -p ~/.ansible/collections/ansible_collections/awx/awx/tests/output/coverage mkdir -p ~/.ansible/collections/ansible_collections/awx/awx/tests/output/coverage
cp -rv coverage/* ~/.ansible/collections/ansible_collections/awx/awx/tests/output/coverage/ cp -rv coverage/* ~/.ansible/collections/ansible_collections/awx/awx/tests/output/coverage/
cd ~/.ansible/collections/ansible_collections/awx/awx cd ~/.ansible/collections/ansible_collections/awx/awx
ansible-test coverage combine --requirements hash -r # Rehash to pick up newly installed scripts
ansible-test coverage html PATH="$(python -c 'import sys; import os; print(os.path.dirname(sys.executable))'):$PATH" ansible-test coverage combine --requirements
PATH="$(python -c 'import sys; import os; print(os.path.dirname(sys.executable))'):$PATH" ansible-test coverage html
echo '## AWX Collection Integration Coverage' >> $GITHUB_STEP_SUMMARY echo '## AWX Collection Integration Coverage' >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY echo '```' >> $GITHUB_STEP_SUMMARY
ansible-test coverage report >> $GITHUB_STEP_SUMMARY PATH="$(python -c 'import sys; import os; print(os.path.dirname(sys.executable))'):$PATH" ansible-test coverage report >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY echo '```' >> $GITHUB_STEP_SUMMARY
echo >> $GITHUB_STEP_SUMMARY echo >> $GITHUB_STEP_SUMMARY
echo '## AWX Collection Integration Coverage HTML' >> $GITHUB_STEP_SUMMARY echo '## AWX Collection Integration Coverage HTML' >> $GITHUB_STEP_SUMMARY

View File

@@ -12,7 +12,12 @@ on:
- feature_* - feature_*
- stable-* - stable-*
jobs: jobs:
check-ownership:
uses: ./.github/workflows/_repo-owns-branch.yml
push-development-images: push-development-images:
needs: check-ownership
if: needs.check-ownership.outputs.should_run == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 120 timeout-minutes: 120
permissions: permissions:
@@ -30,12 +35,6 @@ jobs:
make-target: awx-kube-buildx make-target: awx-kube-buildx
steps: steps:
- name: Skipping build of awx image for non-awx repository
run: |
echo "Skipping build of awx image for non-awx repository"
exit 0
if: matrix.build-targets.image-name == 'awx' && !endsWith(github.repository, '/awx')
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
show-progress: false show-progress: false

View File

@@ -20,4 +20,4 @@ jobs:
run: | run: |
ansible localhost -c local, -m command -a "{{ ansible_python_interpreter + ' -m pip install boto3'}}" ansible localhost -c local, -m command -a "{{ ansible_python_interpreter + ' -m pip install boto3'}}"
ansible localhost -c local -m aws_s3 \ ansible localhost -c local -m aws_s3 \
-a "bucket=awx-public-ci-files object=${GITHUB_REF##*/}/schema.json mode=delobj permission=public-read" -a "bucket=awx-public-ci-files object=${{ github.event.repository.name }}/${GITHUB_REF##*/}/schema.json mode=delobj permission=public-read"

View File

@@ -12,7 +12,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 20 timeout-minutes: 20
permissions: permissions:
packages: write packages: read
contents: read contents: read
steps: steps:
- name: Check for each of the lines - name: Check for each of the lines

View File

@@ -1,85 +1,248 @@
--- # SonarCloud Analysis Workflow for awx
name: SonarQube #
# This workflow runs SonarCloud analysis triggered by CI workflow completion.
# It is split into two separate jobs for clarity and maintainability:
#
# FLOW: CI completes → workflow_run triggers this workflow → appropriate job runs
#
# JOB 1: sonar-pr-analysis (for PRs)
# - Triggered by: workflow_run (CI on pull_request)
# - Steps: Download coverage → Get PR info → Get changed files → Run SonarCloud PR analysis
# - Scans: All changed files in the PR (Python, YAML, JSON, etc.)
# - Quality gate: Focuses on new/changed code in PR only
#
# JOB 2: sonar-branch-analysis (for long-lived branches)
# - Triggered by: workflow_run (CI on push to devel)
# - Steps: Download coverage → Run SonarCloud branch analysis
# - Scans: Full codebase
# - Quality gate: Focuses on overall project health
#
# This ensures coverage data is always available from CI before analysis runs.
#
# What files are scanned:
# - All files in the repository that SonarCloud can analyze
# - Excludes: tests, scripts, dev environments, external collections (see sonar-project.properties)
# With much help from:
# https://community.sonarsource.com/t/how-to-use-sonarcloud-with-a-forked-repository-on-github/7363/30
# https://community.sonarsource.com/t/how-to-use-sonarcloud-with-a-forked-repository-on-github/7363/32
name: SonarCloud
on: on:
workflow_run: workflow_run: # This is triggered by CI being completed.
workflows: workflows:
- CI - CI
types: types:
- completed - completed
permissions: read-all permissions: read-all
jobs: jobs:
sonarqube: sonar-pr-analysis:
name: SonarCloud PR Analysis
runs-on: ubuntu-latest runs-on: ubuntu-latest
if: github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.event == 'pull_request' if: |
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'pull_request' &&
github.repository == 'ansible/awx'
steps: steps:
- name: Checkout Code - uses: actions/checkout@v4
uses: actions/checkout@v4
with:
fetch-depth: 0
show-progress: false
- name: Download coverage report artifact # Download all individual coverage artifacts from CI workflow
uses: actions/download-artifact@v4 - name: Download coverage artifacts
uses: dawidd6/action-download-artifact@246dbf436b23d7c49e21a7ab8204ca9ecd1fe615
with: with:
name: coverage-report github_token: ${{ secrets.GITHUB_TOKEN }}
path: reports/ workflow: CI
github-token: ${{ secrets.GITHUB_TOKEN }} run_id: ${{ github.event.workflow_run.id }}
run-id: ${{ github.event.workflow_run.id }} pattern: api-test-artifacts
- name: Download PR number artifact # Extract PR metadata from workflow_run event
uses: actions/download-artifact@v4 - name: Set PR metadata and prepare files for analysis
with:
name: pr-number
path: .
github-token: ${{ secrets.GITHUB_TOKEN }}
run-id: ${{ github.event.workflow_run.id }}
- name: Extract PR number
run: |
cat pr-number.txt
echo "PR_NUMBER=$(cat pr-number.txt)" >> $GITHUB_ENV
- name: Get PR info
uses: octokit/request-action@v2.x
id: pr_info
with:
route: GET /repos/{repo}/pulls/{number}
repo: ${{ github.event.repository.full_name }}
number: ${{ env.PR_NUMBER }}
env: env:
COMMIT_SHA: ${{ github.event.workflow_run.head_sha }}
REPO_NAME: ${{ github.event.repository.full_name }}
HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Set PR info into env
run: | run: |
echo "PR_BASE=${{ fromJson(steps.pr_info.outputs.data).base.ref }}" >> $GITHUB_ENV # Find all downloaded coverage XML files
echo "PR_HEAD=${{ fromJson(steps.pr_info.outputs.data).head.ref }}" >> $GITHUB_ENV coverage_files=$(find . -name "coverage.xml" -type f | tr '\n' ',' | sed 's/,$//')
echo "Found coverage files: $coverage_files"
echo "COVERAGE_PATHS=$coverage_files" >> $GITHUB_ENV
# Extract PR number from first coverage.xml file found
first_coverage=$(find . -name "coverage.xml" -type f | head -1)
if [ -f "$first_coverage" ]; then
PR_NUMBER=$(grep -m 1 '<!-- PR' "$first_coverage" | awk '{print $3}' || echo "")
else
PR_NUMBER=""
fi
echo "🔍 SonarCloud Analysis Decision Summary"
echo "========================================"
echo "├── CI Event: ✅ Pull Request"
echo "├── PR Number from coverage.xml: #${PR_NUMBER:-<not found>}"
if [ -z "$PR_NUMBER" ]; then
echo "##[error]❌ FATAL: PR number not found in coverage.xml"
echo "##[error]This job requires a PR number to run PR analysis."
echo "##[error]The ci workflow should have injected the PR number into coverage.xml."
exit 1
fi
# Get PR metadata from GitHub API
PR_DATA=$(gh api "repos/$REPO_NAME/pulls/$PR_NUMBER")
PR_BASE=$(echo "$PR_DATA" | jq -r '.base.ref')
PR_HEAD=$(echo "$PR_DATA" | jq -r '.head.ref')
# Print summary
echo "🔍 SonarCloud Analysis Decision Summary"
echo "========================================"
echo "├── CI Event: ✅ Pull Request"
echo "├── PR Number: #$PR_NUMBER"
echo "├── Base Branch: $PR_BASE"
echo "├── Head Branch: $PR_HEAD"
echo "├── Repo: $REPO_NAME"
# Export to GitHub env for later steps
echo "PR_NUMBER=$PR_NUMBER" >> $GITHUB_ENV
echo "PR_BASE=$PR_BASE" >> $GITHUB_ENV
echo "PR_HEAD=$PR_HEAD" >> $GITHUB_ENV
echo "COMMIT_SHA=$COMMIT_SHA" >> $GITHUB_ENV
echo "REPO_NAME=$REPO_NAME" >> $GITHUB_ENV
# Get all changed files from PR (with error handling)
files=""
if [ -n "$PR_NUMBER" ]; then
if gh api repos/$REPO_NAME/pulls/$PR_NUMBER/files --jq '.[].filename' > /tmp/pr_files.txt 2>/tmp/pr_error.txt; then
files=$(cat /tmp/pr_files.txt)
else
echo "├── Changed Files: ⚠️ Could not fetch (likely test repo or PR not found)"
if [ -f coverage.xml ] && [ -s coverage.xml ]; then
echo "├── Coverage Data: ✅ Available"
else
echo "├── Coverage Data: ⚠️ Not available"
fi
echo "└── Result: ✅ Running SonarCloud analysis (full scan)"
# No files = no inclusions filter = full scan
exit 0
fi
else
echo "├── PR Number: ⚠️ Not available"
if [ -f coverage.xml ] && [ -s coverage.xml ]; then
echo "├── Coverage Data: ✅ Available"
else
echo "├── Coverage Data: ⚠️ Not available"
fi
echo "└── Result: ✅ Running SonarCloud analysis (full scan)"
exit 0
fi
# Get file extensions and count for summary
extensions=$(echo "$files" | sed 's/.*\.//' | sort | uniq | tr '\n' ',' | sed 's/,$//')
file_count=$(echo "$files" | wc -l)
echo "├── Changed Files: $file_count file(s) (.${extensions})"
# Check if coverage.xml exists and has content
if [ -f coverage.xml ] && [ -s coverage.xml ]; then
echo "├── Coverage Data: ✅ Available"
else
echo "├── Coverage Data: ⚠️ Not available (analysis will proceed without coverage)"
fi
# Prepare file list for Sonar
echo "All changed files in PR:"
echo "$files"
# Filter out files that are excluded by .coveragerc to avoid coverage conflicts
# This prevents SonarCloud from analyzing files that have no coverage data
if [ -n "$files" ]; then
# Filter out files matching .coveragerc omit patterns
filtered_files=$(echo "$files" | grep -v "settings/.*_defaults\.py$" | grep -v "settings/defaults\.py$" | grep -v "main/migrations/")
# Show which files were filtered out for transparency
excluded_files=$(echo "$files" | grep -E "(settings/.*_defaults\.py$|settings/defaults\.py$|main/migrations/)" || true)
if [ -n "$excluded_files" ]; then
echo "├── Filtered out (coverage-excluded): $(echo "$excluded_files" | wc -l) file(s)"
echo "$excluded_files" | sed 's/^/│ - /'
fi
if [ -n "$filtered_files" ]; then
inclusions=$(echo "$filtered_files" | tr '\n' ',' | sed 's/,$//')
echo "SONAR_INCLUSIONS=$inclusions" >> $GITHUB_ENV
echo "└── Result: ✅ Will scan these files (excluding coverage-omitted files): $inclusions"
else
echo "└── Result: ✅ All changed files are excluded by coverage config, running full SonarCloud analysis"
# Don't set SONAR_INCLUSIONS, let it scan everything per sonar-project.properties
fi
else
echo "└── Result: ✅ Running SonarCloud analysis"
fi
- name: Add base branch - name: Add base branch
if: env.PR_NUMBER != ''
run: | run: |
gh pr checkout ${{ env.PR_NUMBER }} gh pr checkout ${{ env.PR_NUMBER }}
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Extract and export repo owner/name - name: SonarCloud Scan
run: | uses: SonarSource/sonarqube-scan-action@fd88b7d7ccbaefd23d8f36f73b59db7a3d246602 # v6
REPO_SLUG="${GITHUB_REPOSITORY}"
IFS="/" read -r REPO_OWNER REPO_NAME <<< "$REPO_SLUG"
echo "REPO_OWNER=$REPO_OWNER" >> $GITHUB_ENV
echo "REPO_NAME=$REPO_NAME" >> $GITHUB_ENV
- name: SonarQube scan
uses: SonarSource/sonarqube-scan-action@v5
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets[format('{0}', vars.SONAR_TOKEN_SECRET_NAME)] }} SONAR_TOKEN: ${{ secrets.CICD_ORG_SONAR_TOKEN_CICD_BOT }}
with: with:
args: > args: >
-Dsonar.organization=${{ env.REPO_OWNER }} -Dsonar.scm.revision=${{ env.COMMIT_SHA }}
-Dsonar.projectKey=${{ env.REPO_OWNER }}_${{ env.REPO_NAME }}
-Dsonar.pullrequest.key=${{ env.PR_NUMBER }} -Dsonar.pullrequest.key=${{ env.PR_NUMBER }}
-Dsonar.pullrequest.branch=${{ env.PR_HEAD }} -Dsonar.pullrequest.branch=${{ env.PR_HEAD }}
-Dsonar.pullrequest.base=${{ env.PR_BASE }} -Dsonar.pullrequest.base=${{ env.PR_BASE }}
-Dsonar.python.coverage.reportPaths=${{ env.COVERAGE_PATHS }}
${{ env.SONAR_INCLUSIONS && format('-Dsonar.inclusions={0}', env.SONAR_INCLUSIONS) || '' }}
sonar-branch-analysis:
name: SonarCloud Branch Analysis
runs-on: ubuntu-latest
if: |
github.event_name == 'workflow_run' &&
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
github.repository == 'ansible/awx'
steps:
- uses: actions/checkout@v4
# Download all individual coverage artifacts from CI workflow (optional for branch pushes)
- name: Download coverage artifacts
continue-on-error: true
uses: dawidd6/action-download-artifact@246dbf436b23d7c49e21a7ab8204ca9ecd1fe615
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
workflow: CI
run_id: ${{ github.event.workflow_run.id }}
pattern: api-test-artifacts
- name: Print SonarCloud Analysis Summary
env:
BRANCH_NAME: ${{ github.event.workflow_run.head_branch }}
run: |
# Find all downloaded coverage XML files
coverage_files=$(find . -name "coverage.xml" -type f | tr '\n' ',' | sed 's/,$//')
echo "Found coverage files: $coverage_files"
echo "COVERAGE_PATHS=$coverage_files" >> $GITHUB_ENV
echo "🔍 SonarCloud Analysis Summary"
echo "=============================="
echo "├── CI Event: ✅ Push (via workflow_run)"
echo "├── Branch: $BRANCH_NAME"
echo "├── Coverage Files: ${coverage_files:-none}"
echo "├── Python Changes: N/A (Full codebase scan)"
echo "└── Result: ✅ Proceed - \"Running SonarCloud analysis\""
- name: SonarCloud Scan
uses: SonarSource/sonarqube-scan-action@fd88b7d7ccbaefd23d8f36f73b59db7a3d246602 # v6
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.CICD_ORG_SONAR_TOKEN_CICD_BOT }}
with:
args: >
-Dsonar.scm.revision=${{ github.event.workflow_run.head_sha }} -Dsonar.scm.revision=${{ github.event.workflow_run.head_sha }}
-Dsonar.branch.name=${{ github.event.workflow_run.head_branch }}
${{ env.COVERAGE_PATHS && format('-Dsonar.python.coverage.reportPaths={0}', env.COVERAGE_PATHS) || '' }}

183
.github/workflows/spec-sync-on-merge.yml vendored Normal file
View File

@@ -0,0 +1,183 @@
# Sync OpenAPI Spec on Merge
#
# This workflow runs when code is merged to the devel branch.
# It runs the dev environment to generate the OpenAPI spec, then syncs it to
# the central spec repository.
#
# FLOW: PR merged → push to branch → dev environment runs → spec synced to central repo
#
# NOTE: This is an inlined version for testing with private forks.
# Production version will use a reusable workflow from the org repos.
name: Sync OpenAPI Spec on Merge
env:
LC_ALL: "C.UTF-8"
DEV_DOCKER_OWNER: ${{ github.repository_owner }}
on:
push:
branches:
- devel
- 'stable-2.[6-9]'
- 'stable-2.[1-9][0-9]'
workflow_dispatch: # Allow manual triggering for testing
jobs:
check-ownership:
uses: ./.github/workflows/_repo-owns-branch.yml
sync-openapi-spec:
needs: check-ownership
if: needs.check-ownership.outputs.should_run == 'true'
name: Sync OpenAPI spec to central repo
runs-on: ubuntu-latest
permissions:
packages: write
contents: read
steps:
- name: Checkout Controller repository
uses: actions/checkout@v4
with:
show-progress: false
- name: Build awx_devel image to use for schema gen
uses: ./.github/actions/awx_devel_image
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
private-github-key: ${{ secrets.PRIVATE_GITHUB_KEY }}
- name: Generate API Schema
run: |
DEV_DOCKER_TAG_BASE=ghcr.io/${OWNER_LC} \
COMPOSE_TAG=${{ github.base_ref || github.ref_name }} \
docker run -u $(id -u) --rm -v ${{ github.workspace }}:/awx_devel/:Z \
--workdir=/awx_devel `make print-DEVEL_IMAGE_NAME` /start_tests.sh genschema
- name: Verify spec file exists
run: |
SPEC_FILE="./schema.json"
if [ ! -f "$SPEC_FILE" ]; then
echo "❌ Spec file not found at $SPEC_FILE"
echo "Contents of workspace:"
ls -la .
exit 1
fi
echo "✅ Found spec file at $SPEC_FILE"
- name: Checkout spec repo
id: checkout_spec_repo
continue-on-error: true
uses: actions/checkout@v4
with:
repository: ansible-automation-platform/aap-openapi-specs
ref: ${{ github.ref_name }}
path: spec-repo
token: ${{ secrets.OPENAPI_SPEC_SYNC_TOKEN }}
- name: Fail if branch doesn't exist
if: steps.checkout_spec_repo.outcome == 'failure'
run: |
echo "##[error]❌ Branch '${{ github.ref_name }}' does not exist in the central spec repository."
echo "##[error]Expected branch: ${{ github.ref_name }}"
echo "##[error]This branch must be created in the spec repo before specs can be synced."
exit 1
- name: Compare specs
id: compare
run: |
COMPONENT_SPEC="./schema.json"
SPEC_REPO_FILE="spec-repo/controller.json"
# Check if spec file exists in spec repo
if [ ! -f "$SPEC_REPO_FILE" ]; then
echo "Spec file doesn't exist in spec repo - will create new file"
echo "has_diff=true" >> $GITHUB_OUTPUT
echo "is_new_file=true" >> $GITHUB_OUTPUT
else
# Compare files
if diff -q "$COMPONENT_SPEC" "$SPEC_REPO_FILE" > /dev/null; then
echo "✅ No differences found - specs are identical"
echo "has_diff=false" >> $GITHUB_OUTPUT
else
echo "📝 Differences found - spec has changed"
echo "has_diff=true" >> $GITHUB_OUTPUT
echo "is_new_file=false" >> $GITHUB_OUTPUT
fi
fi
- name: Update spec file
if: steps.compare.outputs.has_diff == 'true'
run: |
cp "./schema.json" "spec-repo/controller.json"
echo "✅ Updated spec-repo/controller.json"
- name: Create PR in spec repo
if: steps.compare.outputs.has_diff == 'true'
working-directory: spec-repo
env:
GH_TOKEN: ${{ secrets.OPENAPI_SPEC_SYNC_TOKEN }}
COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
run: |
# Configure git
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
# Create branch for PR
SHORT_SHA="${{ github.sha }}"
SHORT_SHA="${SHORT_SHA:0:7}"
BRANCH_NAME="update-Controller-${{ github.ref_name }}-${SHORT_SHA}"
git checkout -b "$BRANCH_NAME"
# Add and commit changes
git add "controller.json"
if [ "${{ steps.compare.outputs.is_new_file }}" == "true" ]; then
COMMIT_MSG="Add Controller OpenAPI spec for ${{ github.ref_name }}"
else
COMMIT_MSG="Update Controller OpenAPI spec for ${{ github.ref_name }}"
fi
git commit -m "$COMMIT_MSG
Synced from ${{ github.repository }}@${{ github.sha }}
Source branch: ${{ github.ref_name }}
Co-Authored-By: github-actions[bot] <github-actions[bot]@users.noreply.github.com>"
# Push branch
git push origin "$BRANCH_NAME"
# Create PR
PR_TITLE="[${{ github.ref_name }}] Update Controller spec from merged commit"
PR_BODY="## Summary
Automated OpenAPI spec sync from component repository merge.
**Source:** ${{ github.repository }}@${{ github.sha }}
**Branch:** \`${{ github.ref_name }}\`
**Component:** \`Controller\`
**Spec File:** \`controller.json\`
## Changes
$(if [ "${{ steps.compare.outputs.is_new_file }}" == "true" ]; then echo "- 🆕 New spec file created"; else echo "- 📝 Spec file updated with latest changes"; fi)
## Source Commit
\`\`\`
${COMMIT_MESSAGE}
\`\`\`
---
🤖 This PR was automatically generated by the OpenAPI spec sync workflow."
gh pr create \
--title "$PR_TITLE" \
--body "$PR_BODY" \
--base "${{ github.ref_name }}" \
--head "$BRANCH_NAME"
echo "✅ Created PR in spec repo"
- name: Report results
if: always()
run: |
if [ "${{ steps.compare.outputs.has_diff }}" == "true" ]; then
echo "📝 Spec sync completed - PR created in spec repo"
else
echo "✅ Spec sync completed - no changes needed"
fi

View File

@@ -13,7 +13,12 @@ on:
- feature_** - feature_**
- stable-** - stable-**
jobs: jobs:
check-ownership:
uses: ./.github/workflows/_repo-owns-branch.yml
push: push:
needs: check-ownership
if: needs.check-ownership.outputs.should_run == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 60 timeout-minutes: 60
permissions: permissions:
@@ -42,7 +47,7 @@ jobs:
with: with:
command: cp command: cp
source: ${{ github.workspace }}/schema.json source: ${{ github.workspace }}/schema.json
destination: s3://awx-public-ci-files/${{ github.ref_name }}/schema.json destination: s3://awx-public-ci-files/${{ github.event.repository.name }}/${{ github.ref_name }}/schema.json
aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY }} aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY }}
aws_secret_access_key: ${{ secrets.AWS_SECRET_KEY }} aws_secret_access_key: ${{ secrets.AWS_SECRET_KEY }}
aws_region: us-east-1 aws_region: us-east-1

1
.gitignore vendored
View File

@@ -1,6 +1,7 @@
# Ignore generated schema # Ignore generated schema
swagger.json swagger.json
schema.json schema.json
schema.yaml
reference-schema.json reference-schema.json
# Tags # Tags

View File

@@ -7,7 +7,7 @@ build:
os: ubuntu-22.04 os: ubuntu-22.04
tools: tools:
python: >- python: >-
3.11 3.12
commands: commands:
- pip install --user tox - pip install --user tox
- python3 -m tox -e docs --notest -v - python3 -m tox -e docs --notest -v

View File

@@ -0,0 +1,65 @@
---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: awx-atf-tests-pull-request
annotations:
build.appstudio.openshift.io/repo: https://github.com/{{repo_owner}}/{{repo_name}}?rev={{revision}}
build.appstudio.redhat.com/commit_sha: '{{revision}}'
build.appstudio.redhat.com/pull_request_number: '{{pull_request_number}}'
build.appstudio.redhat.com/target_branch: '{{target_branch}}'
pipelinesascode.tekton.dev/cancel-in-progress: 'true'
pipelinesascode.tekton.dev/max-keep-runs: "3"
pipelinesascode.tekton.dev/on-comment: "^/run-atf-tests$"
pipelinesascode.tekton.dev/target-namespace: ansible-ci-tenant
labels:
appstudio.openshift.io/application: '{{repo_owner}}'
appstudio.openshift.io/component: '{{repo_owner}}-{{repo_name}}'
pipelines.appstudio.openshift.io/type: build
spec:
timeouts:
pipeline: "8h"
tasks: "7h"
finally: "1h"
pipelineRef:
resolver: bundles
params:
- name: name
value: aap-api-tests
- name: bundle
value: quay.io/aap-ci/tekton-catalog/pipeline/test/aap-api-tests:0.1@sha256:54d9e941748bae94b2154b3b253a985e628751dfa4508a138d9b05f74a3c1ddf
- name: kind
value: pipeline
- name: secret
value: quay-aap-ci-viewer
taskRunTemplate:
serviceAccountName: konflux-integration-runner
params:
- name: git-url
value: "{{source_url}}"
- name: pipeline-github-org
value: "{{repo_owner}}"
- name: pipeline-github-repo
value: "{{repo_name}}"
- name: pipeline-github-target-branch
value: '{{target_branch}}'
- name: pipeline-github-pr-revision
value: "{{revision}}"
- name: pipeline-github-pr-number
value: "{{pull_request_number}}"
- name: aap-dev-component-source-name
value: "controller"
- name: pytest-number-of-parallel-processes
value: "6"
workspaces:
- name: workspace
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi

View File

@@ -31,7 +31,7 @@ Have questions about this document or anything not covered here? Create a topic
- Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason. - Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason.
- If collaborating with someone else on the same branch, consider using `--force-with-lease` instead of `--force`. This will prevent you from accidentally overwriting commits pushed by someone else. For more information, see [git push docs](https://git-scm.com/docs/git-push#git-push---force-with-leaseltrefnamegt). - If collaborating with someone else on the same branch, consider using `--force-with-lease` instead of `--force`. This will prevent you from accidentally overwriting commits pushed by someone else. For more information, see [git push docs](https://git-scm.com/docs/git-push#git-push---force-with-leaseltrefnamegt).
- If submitting a large code change, it's a good idea to create a [forum topic tagged with 'awx'](https://forum.ansible.com/tag/awx), and talk about what you would like to do or add first. This not only helps everyone know what's going on, it also helps save time and effort, if the community decides some changes are needed. - If submitting a large code change, it's a good idea to create a [forum topic tagged with 'awx'](https://forum.ansible.com/tag/awx), and talk about what you would like to do or add first. This not only helps everyone know what's going on, it also helps save time and effort, if the community decides some changes are needed.
- We ask all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions, or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com) - We ask all of our community members and contributors to adhere to the [Ansible code of conduct](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html). If you have questions, or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
## Setting up your development environment ## Setting up your development environment
@@ -103,6 +103,12 @@ When necessary, remove any AWX containers and images by running the following:
### Pre commit hooks ### Pre commit hooks
Install the pre-commit hook before contributing:
```
make pre-commit
```
When you attempt to perform a `git commit` there will be a pre-commit hook that gets run before the commit is allowed to your local repository. For example, python's [black](https://pypi.org/project/black/) will be run to test the formatting of any python files. When you attempt to perform a `git commit` there will be a pre-commit hook that gets run before the commit is allowed to your local repository. For example, python's [black](https://pypi.org/project/black/) will be run to test the formatting of any python files.
While you can use environment variables to skip the pre-commit hooks GitHub will run similar tests and prevent merging of PRs if the tests do not pass. While you can use environment variables to skip the pre-commit hooks GitHub will run similar tests and prevent merging of PRs if the tests do not pass.

122
Makefile
View File

@@ -1,6 +1,6 @@
-include awx/ui/Makefile -include awx/ui/Makefile
PYTHON := $(notdir $(shell for i in python3.11 python3; do command -v $$i; done|sed 1q)) PYTHON := $(notdir $(shell for i in python3.12 python3.11 python3; do command -v $$i; done|sed 1q))
SHELL := bash SHELL := bash
DOCKER_COMPOSE ?= docker compose DOCKER_COMPOSE ?= docker compose
OFFICIAL ?= no OFFICIAL ?= no
@@ -10,6 +10,7 @@ KIND_BIN ?= $(shell which kind)
CHROMIUM_BIN=/tmp/chrome-linux/chrome CHROMIUM_BIN=/tmp/chrome-linux/chrome
GIT_REPO_NAME ?= $(shell basename `git rev-parse --show-toplevel`) GIT_REPO_NAME ?= $(shell basename `git rev-parse --show-toplevel`)
GIT_BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD) GIT_BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD)
GIT_IS_WORKTREE := $(shell test -f .git && echo yes)
MANAGEMENT_COMMAND ?= awx-manage MANAGEMENT_COMMAND ?= awx-manage
VERSION ?= $(shell $(PYTHON) tools/scripts/scm_version.py 2> /dev/null) VERSION ?= $(shell $(PYTHON) tools/scripts/scm_version.py 2> /dev/null)
@@ -27,6 +28,8 @@ TEST_DIRS ?= awx/main/tests/unit awx/main/tests/functional awx/conf/tests
PARALLEL_TESTS ?= -n auto PARALLEL_TESTS ?= -n auto
# collection integration test directories (defaults to all) # collection integration test directories (defaults to all)
COLLECTION_TEST_TARGET ?= COLLECTION_TEST_TARGET ?=
# Python version for ansible-test (must be 3.11, 3.12, or 3.13)
ANSIBLE_TEST_PYTHON_VERSION ?= 3.13
# args for collection install # args for collection install
COLLECTION_PACKAGE ?= awx COLLECTION_PACKAGE ?= awx
COLLECTION_NAMESPACE ?= awx COLLECTION_NAMESPACE ?= awx
@@ -77,7 +80,7 @@ RECEPTOR_IMAGE ?= quay.io/ansible/receptor:devel
SRC_ONLY_PKGS ?= cffi,pycparser,psycopg,twilio SRC_ONLY_PKGS ?= cffi,pycparser,psycopg,twilio
# These should be upgraded in the AWX and Ansible venv before attempting # These should be upgraded in the AWX and Ansible venv before attempting
# to install the actual requirements # to install the actual requirements
VENV_BOOTSTRAP ?= pip==21.2.4 setuptools==80.9.0 setuptools_scm[toml]==8.0.4 wheel==0.42.0 cython==3.1.3 VENV_BOOTSTRAP ?= pip==25.3 setuptools==80.9.0 setuptools_scm[toml]==9.2.2 wheel==0.46.3 cython==3.1.3
NAME ?= awx NAME ?= awx
@@ -104,12 +107,23 @@ else
DOCKER_KUBE_CACHE_FLAG=$(DOCKER_CACHE) DOCKER_KUBE_CACHE_FLAG=$(DOCKER_CACHE)
endif endif
# AWX TUI variables
AWX_HOST ?= https://localhost:8043
AWX_USER ?= admin
AWX_PASSWORD ?= $$(awk -F"'" '/^admin_password:/{print $$2}' tools/docker-compose/_sources/secrets/admin_password.yml 2>/dev/null || echo "admin")
AWX_VERIFY_SSL ?= false
# For git worktree to find the referenced git dir
GIT_COMMON_DIR := $(shell git rev-parse --git-common-dir 2>/dev/null || echo .git)
.PHONY: awx-link clean clean-tmp clean-venv requirements requirements_dev \ .PHONY: awx-link clean clean-tmp clean-venv requirements requirements_dev \
update_requirements upgrade_requirements update_requirements_dev \
docker_update_requirements docker_upgrade_requirements docker_update_requirements_dev \
develop refresh adduser migrate dbchange \ develop refresh adduser migrate dbchange \
receiver test test_unit test_coverage coverage_html \ receiver test test_unit test_coverage coverage_html \
sdist \ sdist \
VERSION PYTHON_VERSION docker-compose-sources \ VERSION PYTHON_VERSION docker-compose-sources \
.git/hooks/pre-commit pre-commit
clean-tmp: clean-tmp:
rm -rf tmp/ rm -rf tmp/
@@ -144,7 +158,7 @@ clean-api:
rm -rf build $(NAME)-$(VERSION) *.egg-info rm -rf build $(NAME)-$(VERSION) *.egg-info
rm -rf .tox rm -rf .tox
find . -type f -regex ".*\.py[co]$$" -delete find . -type f -regex ".*\.py[co]$$" -delete
find . -type d -name "__pycache__" -delete find . -type d -name "__pycache__" -exec rm -rf {} +
rm -f awx/awx_test.sqlite3* rm -f awx/awx_test.sqlite3*
rm -rf requirements/vendor rm -rf requirements/vendor
rm -rf awx/projects rm -rf awx/projects
@@ -194,6 +208,36 @@ requirements_dev: requirements_awx requirements_awx_dev
requirements_test: requirements requirements_test: requirements
## Update requirements files using pip-compile (run inside container)
update_requirements:
cd requirements && ./updater.sh run
## Upgrade all requirements to latest versions (run inside container)
upgrade_requirements:
cd requirements && ./updater.sh upgrade
## Update development requirements (run inside container)
update_requirements_dev:
cd requirements && ./updater.sh dev
## Update requirements using docker-runner
docker_update_requirements:
@echo "Running requirements updater..."
AWX_DOCKER_CMD='make update_requirements' $(MAKE) docker-runner
@echo "Requirements update complete!"
## Upgrade requirements using docker-runner
docker_upgrade_requirements:
@echo "Running requirements upgrader..."
AWX_DOCKER_CMD='make upgrade_requirements' $(MAKE) docker-runner
@echo "Requirements upgrade complete!"
## Update dev requirements using docker-runner
docker_update_requirements_dev:
@echo "Running dev requirements updater..."
AWX_DOCKER_CMD='make update_requirements_dev' $(MAKE) docker-runner
@echo "Dev requirements update complete!"
## "Install" awx package in development mode. ## "Install" awx package in development mode.
develop: develop:
@if [ "$(VIRTUAL_ENV)" ]; then \ @if [ "$(VIRTUAL_ENV)" ]; then \
@@ -255,7 +299,7 @@ dispatcher:
@if [ "$(VENV_BASE)" ]; then \ @if [ "$(VENV_BASE)" ]; then \
. $(VENV_BASE)/awx/bin/activate; \ . $(VENV_BASE)/awx/bin/activate; \
fi; \ fi; \
$(PYTHON) manage.py run_dispatcher $(PYTHON) manage.py dispatcherd
## Run to start the zeromq callback receiver ## Run to start the zeromq callback receiver
receiver: receiver:
@@ -308,26 +352,22 @@ black: reports
@command -v black >/dev/null 2>&1 || { echo "could not find black on your PATH, you may need to \`pip install black\`, or set AWX_IGNORE_BLACK=1" && exit 1; } @command -v black >/dev/null 2>&1 || { echo "could not find black on your PATH, you may need to \`pip install black\`, or set AWX_IGNORE_BLACK=1" && exit 1; }
@(set -o pipefail && $@ $(BLACK_ARGS) awx awxkit awx_collection | tee reports/$@.report) @(set -o pipefail && $@ $(BLACK_ARGS) awx awxkit awx_collection | tee reports/$@.report)
.git/hooks/pre-commit: $(GIT_COMMON_DIR)/hooks/pre-commit:
@echo "if [ -x pre-commit.sh ]; then" > .git/hooks/pre-commit ln -sf ../../pre-commit.sh $(GIT_COMMON_DIR)/hooks/pre-commit
@echo " ./pre-commit.sh;" >> .git/hooks/pre-commit
@echo "fi" >> .git/hooks/pre-commit
@chmod +x .git/hooks/pre-commit
genschema: reports pre-commit: $(GIT_COMMON_DIR)/hooks/pre-commit
$(MAKE) swagger PYTEST_ADDOPTS="--genschema --create-db "
mv swagger.json schema.json
swagger: reports genschema: awx-link reports
@if [ "$(VENV_BASE)" ]; then \ @if [ "$(VENV_BASE)" ]; then \
. $(VENV_BASE)/awx/bin/activate; \ . $(VENV_BASE)/awx/bin/activate; \
fi; \ fi; \
(set -o pipefail && py.test $(COVERAGE_ARGS) $(PARALLEL_TESTS) awx/conf/tests/functional awx/main/tests/functional/api awx/main/tests/docs | tee reports/$@.report) $(MANAGEMENT_COMMAND) spectacular --format openapi-json --file schema.json
@if [ "${GITHUB_ACTIONS}" = "true" ]; \
then \ genschema-yaml: awx-link reports
echo 'cov-report-files=reports/coverage.xml' >> "${GITHUB_OUTPUT}"; \ @if [ "$(VENV_BASE)" ]; then \
echo 'test-result-files=reports/junit.xml' >> "${GITHUB_OUTPUT}"; \ . $(VENV_BASE)/awx/bin/activate; \
fi fi; \
$(MANAGEMENT_COMMAND) spectacular --format openapi --file schema.yaml
check: black check: black
@@ -431,8 +471,8 @@ test_collection_sanity:
test_collection_integration: install_collection test_collection_integration: install_collection
cd $(COLLECTION_INSTALL) && \ cd $(COLLECTION_INSTALL) && \
ansible-test integration --coverage -vvv $(COLLECTION_TEST_TARGET) && \ PATH="$$($(PYTHON) -c 'import sys; import os; print(os.path.dirname(sys.executable))'):$$PATH" ansible-test integration --python $(ANSIBLE_TEST_PYTHON_VERSION) --coverage -vvv $(COLLECTION_TEST_TARGET) && \
ansible-test coverage xml --requirements --group-by command --group-by version PATH="$$($(PYTHON) -c 'import sys; import os; print(os.path.dirname(sys.executable))'):$$PATH" ansible-test coverage xml --requirements --group-by command --group-by version
@if [ "${GITHUB_ACTIONS}" = "true" ]; \ @if [ "${GITHUB_ACTIONS}" = "true" ]; \
then \ then \
echo cov-report-files="$$(find "$(COLLECTION_INSTALL)/tests/output/reports/" -type f -name 'coverage=integration*.xml' -print0 | tr '\0' ',' | sed 's#,$$##')" >> "${GITHUB_OUTPUT}"; \ echo cov-report-files="$$(find "$(COLLECTION_INSTALL)/tests/output/reports/" -type f -name 'coverage=integration*.xml' -print0 | tr '\0' ',' | sed 's#,$$##')" >> "${GITHUB_OUTPUT}"; \
@@ -490,7 +530,7 @@ ifneq ($(ADMIN_PASSWORD),)
EXTRA_SOURCES_ANSIBLE_OPTS := -e admin_password=$(ADMIN_PASSWORD) $(EXTRA_SOURCES_ANSIBLE_OPTS) EXTRA_SOURCES_ANSIBLE_OPTS := -e admin_password=$(ADMIN_PASSWORD) $(EXTRA_SOURCES_ANSIBLE_OPTS)
endif endif
docker-compose-sources: .git/hooks/pre-commit docker-compose-sources:
@if [ $(MINIKUBE_CONTAINER_GROUP) = true ]; then\ @if [ $(MINIKUBE_CONTAINER_GROUP) = true ]; then\
$(ANSIBLE_PLAYBOOK) -i tools/docker-compose/inventory -e minikube_setup=$(MINIKUBE_SETUP) tools/docker-compose-minikube/deploy.yml; \ $(ANSIBLE_PLAYBOOK) -i tools/docker-compose/inventory -e minikube_setup=$(MINIKUBE_SETUP) tools/docker-compose-minikube/deploy.yml; \
fi; fi;
@@ -522,7 +562,7 @@ docker-compose: awx/projects docker-compose-sources
$(MAKE) docker-compose-up $(MAKE) docker-compose-up
docker-compose-up: docker-compose-up:
$(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml $(COMPOSE_OPTS) up $(COMPOSE_UP_OPTS) --remove-orphans $(if $(GIT_IS_WORKTREE),SETUPTOOLS_SCM_PRETEND_VERSION="$(VERSION)") $(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml $(COMPOSE_OPTS) up $(COMPOSE_UP_OPTS) --remove-orphans
docker-compose-down: docker-compose-down:
$(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml $(COMPOSE_OPTS) down --remove-orphans $(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml $(COMPOSE_OPTS) down --remove-orphans
@@ -537,14 +577,34 @@ docker-compose-test: awx/projects docker-compose-sources
docker-compose-runtest: awx/projects docker-compose-sources docker-compose-runtest: awx/projects docker-compose-sources
$(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports awx_1 /start_tests.sh $(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports awx_1 /start_tests.sh
docker-compose-build-swagger: awx/projects docker-compose-sources docker-compose-build-schema: awx/projects docker-compose-sources
$(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports --no-deps awx_1 /start_tests.sh swagger $(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports --no-deps awx_1 make genschema
awx-tui:
@if ! command -v awx-tui > /dev/null 2>&1; then \
$(PYTHON) -m pip install awx-tui; \
fi
@if [ -f "$(HOME)/.config/awx-tui/config.yaml" ]; then \
$(PYTHON) -m awx_tui.main; \
else \
AWX_HOST=$(AWX_HOST) \
AWX_USER=$(AWX_USER) \
AWX_PASSWORD=$(AWX_PASSWORD) \
AWX_VERIFY_SSL=$(AWX_VERIFY_SSL) \
$(PYTHON) -m awx_tui.main --host $(AWX_HOST); \
fi
SCHEMA_DIFF_BASE_FOLDER ?= awx
SCHEMA_DIFF_BASE_BRANCH ?= devel SCHEMA_DIFF_BASE_BRANCH ?= devel
detect-schema-change: genschema detect-schema-change: genschema
curl https://s3.amazonaws.com/awx-public-ci-files/$(SCHEMA_DIFF_BASE_BRANCH)/schema.json -o reference-schema.json curl https://s3.amazonaws.com/awx-public-ci-files/$(SCHEMA_DIFF_BASE_FOLDER)/$(SCHEMA_DIFF_BASE_BRANCH)/schema.json -o reference-schema.json
# Ignore differences in whitespace with -b # Ignore differences in whitespace with -b
diff -u -b reference-schema.json schema.json # diff exits with 1 when files differ - capture but don't fail
-diff -u -b reference-schema.json schema.json
validate-openapi-schema: genschema
@echo "Validating OpenAPI schema from schema.json..."
@python3 -c "from openapi_spec_validator import validate; import json; spec = json.load(open('schema.json')); validate(spec); print('✓ Schema is valid')"
docker-compose-clean: awx/projects docker-compose-clean: awx/projects
$(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml rm -sf $(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml rm -sf
@@ -577,7 +637,7 @@ docker-compose-build: Dockerfile.dev
docker-compose-buildx: Dockerfile.dev docker-compose-buildx: Dockerfile.dev
- docker buildx create --name docker-compose-buildx - docker buildx create --name docker-compose-buildx
docker buildx use docker-compose-buildx docker buildx use docker-compose-buildx
- docker buildx build \ docker buildx build \
--ssh default=$(SSH_AUTH_SOCK) \ --ssh default=$(SSH_AUTH_SOCK) \
--push \ --push \
--build-arg BUILDKIT_INLINE_CACHE=1 \ --build-arg BUILDKIT_INLINE_CACHE=1 \
@@ -637,7 +697,7 @@ awx-kube-build: Dockerfile
awx-kube-buildx: Dockerfile awx-kube-buildx: Dockerfile
- docker buildx create --name awx-kube-buildx - docker buildx create --name awx-kube-buildx
docker buildx use awx-kube-buildx docker buildx use awx-kube-buildx
- docker buildx build \ docker buildx build \
--ssh default=$(SSH_AUTH_SOCK) \ --ssh default=$(SSH_AUTH_SOCK) \
--push \ --push \
--build-arg VERSION=$(VERSION) \ --build-arg VERSION=$(VERSION) \
@@ -671,7 +731,7 @@ awx-kube-dev-build: Dockerfile.kube-dev
awx-kube-dev-buildx: Dockerfile.kube-dev awx-kube-dev-buildx: Dockerfile.kube-dev
- docker buildx create --name awx-kube-dev-buildx - docker buildx create --name awx-kube-dev-buildx
docker buildx use awx-kube-dev-buildx docker buildx use awx-kube-dev-buildx
- docker buildx build \ docker buildx build \
--ssh default=$(SSH_AUTH_SOCK) \ --ssh default=$(SSH_AUTH_SOCK) \
--push \ --push \
--build-arg BUILDKIT_INLINE_CACHE=1 \ --build-arg BUILDKIT_INLINE_CACHE=1 \

View File

@@ -1,4 +1,4 @@
[![CI](https://github.com/ansible/awx/actions/workflows/ci.yml/badge.svg?branch=devel)](https://github.com/ansible/awx/actions/workflows/ci.yml) [![codecov](https://codecov.io/github/ansible/awx/graph/badge.svg?token=4L4GSP9IAR)](https://codecov.io/github/ansible/awx) [![Code of Conduct](https://img.shields.io/badge/code%20of%20conduct-Ansible-yellow.svg)](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html) [![Apache v2 License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](https://github.com/ansible/awx/blob/devel/LICENSE.md) [![AWX on the Ansible Forum](https://img.shields.io/badge/mailing%20list-AWX-orange.svg)](https://forum.ansible.com/tag/awx) [![CI](https://github.com/ansible/awx/actions/workflows/ci.yml/badge.svg?branch=devel)](https://github.com/ansible/awx/actions/workflows/ci.yml) [![codecov](https://codecov.io/github/ansible/awx/graph/badge.svg?token=4L4GSP9IAR)](https://codecov.io/github/ansible/awx) [![Code of Conduct](https://img.shields.io/badge/code%20of%20conduct-Ansible-yellow.svg)](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html) [![Apache v2 License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](https://github.com/ansible/awx/blob/devel/LICENSE.md) [![AWX on the Ansible Forum](https://img.shields.io/badge/mailing%20list-AWX-orange.svg)](https://forum.ansible.com/tag/awx)
[![Ansible Matrix](https://img.shields.io/badge/matrix-Ansible%20Community-blueviolet.svg?logo=matrix)](https://chat.ansible.im/#/welcome) [![Ansible Discourse](https://img.shields.io/badge/discourse-Ansible%20Community-yellowgreen.svg?logo=discourse)](https://forum.ansible.com) [![Ansible Matrix](https://img.shields.io/badge/matrix-Ansible%20Community-blueviolet.svg?logo=matrix)](https://chat.ansible.im/#/welcome) [![Ansible Discourse](https://img.shields.io/badge/discourse-Ansible%20Community-yellowgreen.svg?logo=discourse)](https://forum.ansible.com)
<img src="https://raw.githubusercontent.com/ansible/awx-logos/master/awx/ui/client/assets/logo-login.svg?sanitize=true" width=200 alt="AWX" /> <img src="https://raw.githubusercontent.com/ansible/awx-logos/master/awx/ui/client/assets/logo-login.svg?sanitize=true" width=200 alt="AWX" />
@@ -18,7 +18,7 @@ AWX provides a web-based user interface, REST API, and task engine built on top
To install AWX, please view the [Install guide](./INSTALL.md). To install AWX, please view the [Install guide](./INSTALL.md).
To learn more about using AWX, view the [AWX docs site](https://ansible.readthedocs.io/projects/awx/en/latest/). To learn more about using AWX, view the [AWX docs site](https://docs.ansible.com/projects/awx/en/latest/).
The AWX Project Frequently Asked Questions can be found [here](https://www.ansible.com/awx-project-faq). The AWX Project Frequently Asked Questions can be found [here](https://www.ansible.com/awx-project-faq).
@@ -41,11 +41,11 @@ If you're experiencing a problem that you feel is a bug in AWX or have ideas for
Code of Conduct Code of Conduct
--------------- ---------------
We require all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com) We require all of our community members and contributors to adhere to the [Ansible code of conduct](https://docs.ansible.com/projects/ansible/latest/community/code_of_conduct.html). If you have questions or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
Get Involved Get Involved
------------ ------------
We welcome your feedback and ideas via the [Ansible Forum](https://forum.ansible.com/tag/awx). We welcome your feedback and ideas via the [Ansible Forum](https://forum.ansible.com/tag/awx).
For a full list of all the ways to talk with the Ansible Community, see the [AWX Communication guide](https://ansible.readthedocs.io/projects/awx/en/latest/contributor/communication.html). For a full list of all the ways to talk with the Ansible Community, see the [AWX Communication guide](https://docs.ansible.com/projects/awx/en/latest/contributor/communication.html).

View File

@@ -7,7 +7,6 @@ from rest_framework import serializers
# AWX # AWX
from awx.conf import fields, register, register_validate from awx.conf import fields, register, register_validate
register( register(
'SESSION_COOKIE_AGE', 'SESSION_COOKIE_AGE',
field_class=fields.IntegerField, field_class=fields.IntegerField,

View File

@@ -21,7 +21,7 @@ class NullFieldMixin(object):
""" """
def validate_empty_values(self, data): def validate_empty_values(self, data):
(is_empty_value, data) = super(NullFieldMixin, self).validate_empty_values(data) is_empty_value, data = super(NullFieldMixin, self).validate_empty_values(data)
if is_empty_value and data is None: if is_empty_value and data is None:
return (False, data) return (False, data)
return (is_empty_value, data) return (is_empty_value, data)
@@ -89,7 +89,7 @@ class DeprecatedCredentialField(serializers.IntegerField):
def to_internal_value(self, pk): def to_internal_value(self, pk):
try: try:
pk = int(pk) pk = int(pk)
except ValueError: except (ValueError, TypeError):
self.fail('invalid') self.fail('invalid')
try: try:
Credential.objects.get(pk=pk) Credential.objects.get(pk=pk)

View File

@@ -131,8 +131,14 @@ class LoggedLoginView(auth_views.LoginView):
class LoggedLogoutView(auth_views.LogoutView): class LoggedLogoutView(auth_views.LogoutView):
# Override http_method_names to allow GET requests (Django 5.2+ defaults to POST only)
http_method_names = ["get", "post", "options"]
success_url_allowed_hosts = set(settings.LOGOUT_ALLOWED_HOSTS.split(",")) if settings.LOGOUT_ALLOWED_HOSTS else set() success_url_allowed_hosts = set(settings.LOGOUT_ALLOWED_HOSTS.split(",")) if settings.LOGOUT_ALLOWED_HOSTS else set()
def get(self, request, *args, **kwargs):
"""Handle GET requests for logout (for backward compatibility)."""
return self.post(request, *args, **kwargs)
def dispatch(self, request, *args, **kwargs): def dispatch(self, request, *args, **kwargs):
if is_proxied_request(): if is_proxied_request():
# 1) We intentionally don't obey ?next= here, just always redirect to platform login # 1) We intentionally don't obey ?next= here, just always redirect to platform login
@@ -161,16 +167,14 @@ def get_view_description(view, html=False):
def get_default_schema(): def get_default_schema():
if settings.DYNACONF.is_development_mode: # drf-spectacular is configured via REST_FRAMEWORK['DEFAULT_SCHEMA_CLASS']
from awx.api.swagger import schema_view # Just use the DRF default, which will pick up our CustomAutoSchema
return views.APIView.schema
return schema_view
else:
return views.APIView.schema
class APIView(views.APIView): class APIView(views.APIView):
schema = get_default_schema() # Schema is inherited from DRF's APIView, which uses DEFAULT_SCHEMA_CLASS
# No need to override it here - drf-spectacular will handle it
versioning_class = URLPathVersioning versioning_class = URLPathVersioning
def initialize_request(self, request, *args, **kwargs): def initialize_request(self, request, *args, **kwargs):
@@ -268,7 +272,10 @@ class APIView(views.APIView):
response = self.handle_exception(self.__init_request_error__) response = self.handle_exception(self.__init_request_error__)
if response.status_code == 401: if response.status_code == 401:
if response.data and 'detail' in response.data: if response.data and 'detail' in response.data:
response.data['detail'] += _(' To establish a login session, visit') + ' /api/login/.' if getattr(settings, 'RESOURCE_SERVER__URL', None):
response.data['detail'] += _(' Direct access is not allowed, authenticate via the platform gateway.')
else:
response.data['detail'] += _(' To establish a login session, visit') + ' /api/login/.'
logger.info(status_msg) logger.info(status_msg)
else: else:
logger.warning(status_msg) logger.warning(status_msg)
@@ -766,7 +773,7 @@ class SubListCreateAttachDetachAPIView(SubListCreateAPIView):
return Response(status=status.HTTP_204_NO_CONTENT) return Response(status=status.HTTP_204_NO_CONTENT)
def unattach(self, request, *args, **kwargs): def unattach(self, request, *args, **kwargs):
(sub_id, res) = self.unattach_validate(request) sub_id, res = self.unattach_validate(request)
if res: if res:
return res return res
return self.unattach_by_id(request, sub_id) return self.unattach_by_id(request, sub_id)
@@ -1025,6 +1032,9 @@ class GenericCancelView(RetrieveAPIView):
# In subclass set model, serializer_class # In subclass set model, serializer_class
obj_permission_type = 'cancel' obj_permission_type = 'cancel'
def get(self, request, *args, **kwargs):
return super(GenericCancelView, self).get(request, *args, **kwargs)
@transaction.non_atomic_requests @transaction.non_atomic_requests
def dispatch(self, *args, **kwargs): def dispatch(self, *args, **kwargs):
return super(GenericCancelView, self).dispatch(*args, **kwargs) return super(GenericCancelView, self).dispatch(*args, **kwargs)

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import MetricsView from awx.api.views import MetricsView
urls = [re_path(r'^$', MetricsView.as_view(), name='metrics_view')] urls = [re_path(r'^$', MetricsView.as_view(), name='metrics_view')]
__all__ = ['urls'] __all__ = ['urls']

View File

@@ -111,7 +111,7 @@ class UnifiedJobEventPagination(Pagination):
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
self.use_limit_paginator = False self.use_limit_paginator = False
self.limit_pagination = LimitPagination() self.limit_pagination = LimitPagination()
return super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
def paginate_queryset(self, queryset, request, view=None): def paginate_queryset(self, queryset, request, view=None):
if 'limit' in request.query_params: if 'limit' in request.query_params:

119
awx/api/schema.py Normal file
View File

@@ -0,0 +1,119 @@
import warnings
from rest_framework.permissions import IsAuthenticated
from drf_spectacular.openapi import AutoSchema
from drf_spectacular.views import (
SpectacularAPIView,
SpectacularSwaggerView,
SpectacularRedocView,
)
def filter_credential_type_schema(
result,
generator, # NOSONAR
request, # NOSONAR
public, # NOSONAR
):
"""
Postprocessing hook to filter CredentialType kind enum values.
For CredentialTypeRequest and PatchedCredentialTypeRequest schemas (POST/PUT/PATCH),
filter the 'kind' enum to only show 'cloud' and 'net' values.
This ensures the OpenAPI schema accurately reflects that only 'cloud' and 'net'
credential types can be created or modified via the API, matching the validation
in CredentialTypeSerializer.validate().
Args:
result: The OpenAPI schema dict to be modified
generator, request, public: Required by drf-spectacular interface (unused)
Returns:
The modified OpenAPI schema dict
"""
schemas = result.get('components', {}).get('schemas', {})
# Filter CredentialTypeRequest (POST/PUT) - field is required
if 'CredentialTypeRequest' in schemas:
kind_prop = schemas['CredentialTypeRequest'].get('properties', {}).get('kind', {})
if 'enum' in kind_prop:
# Filter to only cloud and net (no None - field is required)
kind_prop['enum'] = ['cloud', 'net']
kind_prop['description'] = "* `cloud` - Cloud\\n* `net` - Network"
# Filter PatchedCredentialTypeRequest (PATCH) - field is optional
if 'PatchedCredentialTypeRequest' in schemas:
kind_prop = schemas['PatchedCredentialTypeRequest'].get('properties', {}).get('kind', {})
if 'enum' in kind_prop:
# Filter to only cloud and net (None allowed - field can be omitted in PATCH)
kind_prop['enum'] = ['cloud', 'net', None]
kind_prop['description'] = "* `cloud` - Cloud\\n* `net` - Network"
return result
class CustomAutoSchema(AutoSchema):
"""Custom AutoSchema to add swagger_topic to tags and handle deprecated endpoints."""
def get_tags(self):
tags = []
try:
if hasattr(self.view, 'get_serializer'):
serializer = self.view.get_serializer()
else:
serializer = None
except Exception:
serializer = None
warnings.warn(
'{}.get_serializer() raised an exception during '
'schema generation. Serializer fields will not be '
'generated for this view.'.format(self.view.__class__.__name__)
)
if hasattr(self.view, 'swagger_topic'):
tags.append(str(self.view.swagger_topic).title())
elif serializer and hasattr(serializer, 'Meta') and hasattr(serializer.Meta, 'model'):
tags.append(str(serializer.Meta.model._meta.verbose_name_plural).title())
elif hasattr(self.view, 'model'):
tags.append(str(self.view.model._meta.verbose_name_plural).title())
else:
tags = super().get_tags() # Use default drf-spectacular behavior
if not tags:
warnings.warn(f'Could not determine tags for {self.view.__class__.__name__}')
tags = ['api'] # Fallback to default value
return tags
def is_deprecated(self):
"""Return `True` if this operation is to be marked as deprecated."""
return getattr(self.view, 'deprecated', False)
class AuthenticatedSpectacularAPIView(SpectacularAPIView):
"""SpectacularAPIView that requires authentication."""
permission_classes = [IsAuthenticated]
class AuthenticatedSpectacularSwaggerView(SpectacularSwaggerView):
"""SpectacularSwaggerView that requires authentication."""
permission_classes = [IsAuthenticated]
class AuthenticatedSpectacularRedocView(SpectacularRedocView):
"""SpectacularRedocView that requires authentication."""
permission_classes = [IsAuthenticated]
# Schema view (returns OpenAPI schema JSON/YAML)
schema_view = AuthenticatedSpectacularAPIView.as_view()
# Swagger UI view
swagger_ui_view = AuthenticatedSpectacularSwaggerView.as_view(url_name='api:schema-json')
# ReDoc UI view
redoc_view = AuthenticatedSpectacularRedocView.as_view(url_name='api:schema-json')

View File

@@ -122,7 +122,6 @@ from awx.main.scheduler.task_manager_models import TaskManagerModels
from awx.main.redact import UriCleaner, REPLACE_STR from awx.main.redact import UriCleaner, REPLACE_STR
from awx.main.signals import update_inventory_computed_fields from awx.main.signals import update_inventory_computed_fields
from awx.main.validators import vars_validate_or_raise from awx.main.validators import vars_validate_or_raise
from awx.api.versioning import reverse from awx.api.versioning import reverse
@@ -175,8 +174,8 @@ SUMMARIZABLE_FK_FIELDS = {
'workflow_approval': DEFAULT_SUMMARY_FIELDS + ('timeout',), 'workflow_approval': DEFAULT_SUMMARY_FIELDS + ('timeout',),
'schedule': DEFAULT_SUMMARY_FIELDS + ('next_run',), 'schedule': DEFAULT_SUMMARY_FIELDS + ('next_run',),
'unified_job_template': DEFAULT_SUMMARY_FIELDS + ('unified_job_type',), 'unified_job_template': DEFAULT_SUMMARY_FIELDS + ('unified_job_type',),
'last_job': DEFAULT_SUMMARY_FIELDS + ('finished', 'status', 'failed', 'license_error', 'canceled_on'), # last_job and last_job_host_summary are derived from JobHostSummary in HostSerializer,
'last_job_host_summary': DEFAULT_SUMMARY_FIELDS + ('failed',), # not from the stale FK fields on Host.
'last_update': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'), 'last_update': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'),
'current_update': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'), 'current_update': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'),
'current_job': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'), 'current_job': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'),
@@ -963,13 +962,13 @@ class UnifiedJobSerializer(BaseSerializer):
class UnifiedJobListSerializer(UnifiedJobSerializer): class UnifiedJobListSerializer(UnifiedJobSerializer):
class Meta: class Meta:
fields = ('*', '-job_args', '-job_cwd', '-job_env', '-result_traceback', '-event_processing_finished') fields = ('*', '-job_args', '-job_cwd', '-job_env', '-result_traceback', '-event_processing_finished', '-artifacts')
def get_field_names(self, declared_fields, info): def get_field_names(self, declared_fields, info):
field_names = super(UnifiedJobListSerializer, self).get_field_names(declared_fields, info) field_names = super(UnifiedJobListSerializer, self).get_field_names(declared_fields, info)
# Meta multiple inheritance and -field_name options don't seem to be # Meta multiple inheritance and -field_name options don't seem to be
# taking effect above, so remove the undesired fields here. # taking effect above, so remove the undesired fields here.
return tuple(x for x in field_names if x not in ('job_args', 'job_cwd', 'job_env', 'result_traceback', 'event_processing_finished')) return tuple(x for x in field_names if x not in ('job_args', 'job_cwd', 'job_env', 'result_traceback', 'event_processing_finished', 'artifacts'))
def get_types(self): def get_types(self):
if type(self) is UnifiedJobListSerializer: if type(self) is UnifiedJobListSerializer:
@@ -1022,7 +1021,7 @@ class UnifiedJobStdoutSerializer(UnifiedJobSerializer):
class UserSerializer(BaseSerializer): class UserSerializer(BaseSerializer):
password = serializers.CharField(required=False, default='', help_text=_('Field used to change the password.')) password = serializers.CharField(required=False, default='', allow_blank=True, help_text=_('Field used to change the password.'))
is_system_auditor = serializers.BooleanField(default=False) is_system_auditor = serializers.BooleanField(default=False)
show_capabilities = ['edit', 'delete'] show_capabilities = ['edit', 'delete']
@@ -1230,7 +1229,7 @@ class OrganizationSerializer(BaseSerializer, OpaQueryPathMixin):
# to a team. This provides a hint to the ui so it can know to not # to a team. This provides a hint to the ui so it can know to not
# display these roles for team role selection. # display these roles for team role selection.
for key in ('admin_role', 'member_role'): for key in ('admin_role', 'member_role'):
if key in summary_dict.get('object_roles', {}): if summary_dict and key in summary_dict.get('object_roles', {}):
summary_dict['object_roles'][key]['user_only'] = True summary_dict['object_roles'][key]['user_only'] = True
return summary_dict return summary_dict
@@ -1838,19 +1837,35 @@ class HostSerializer(BaseSerializerWithVariables):
res['ansible_facts'] = self.reverse('api:host_ansible_facts_detail', kwargs={'pk': obj.instance_id}) res['ansible_facts'] = self.reverse('api:host_ansible_facts_detail', kwargs={'pk': obj.instance_id})
if obj.inventory: if obj.inventory:
res['inventory'] = self.reverse('api:inventory_detail', kwargs={'pk': obj.inventory.pk}) res['inventory'] = self.reverse('api:inventory_detail', kwargs={'pk': obj.inventory.pk})
if obj.last_job: last_summary = obj.latest_summary
res['last_job'] = self.reverse('api:job_detail', kwargs={'pk': obj.last_job.pk}) if last_summary:
if obj.last_job_host_summary: res['last_job_host_summary'] = self.reverse('api:job_host_summary_detail', kwargs={'pk': last_summary.pk})
res['last_job_host_summary'] = self.reverse('api:job_host_summary_detail', kwargs={'pk': obj.last_job_host_summary.pk}) if last_summary.job_id:
res['last_job'] = self.reverse('api:job_detail', kwargs={'pk': last_summary.job_id})
return res return res
def get_summary_fields(self, obj): def get_summary_fields(self, obj):
d = super(HostSerializer, self).get_summary_fields(obj) d = super(HostSerializer, self).get_summary_fields(obj)
try: last_summary = obj.latest_summary
d['last_job']['job_template_id'] = obj.last_job.job_template.id if last_summary:
d['last_job']['job_template_name'] = obj.last_job.job_template.name d['last_job_host_summary'] = OrderedDict()
except (KeyError, AttributeError): d['last_job_host_summary']['id'] = last_summary.id
pass d['last_job_host_summary']['failed'] = last_summary.failed
try:
last_job = last_summary.job
d['last_job'] = OrderedDict()
for field in DEFAULT_SUMMARY_FIELDS + ('finished', 'status', 'failed', 'canceled_on'):
fval = getattr(last_job, field, None)
if fval is not None:
d['last_job'][field] = fval
if last_job.job_template:
d['last_job']['job_template_id'] = last_job.job_template.id
d['last_job']['job_template_name'] = last_job.job_template.name
except ObjectDoesNotExist:
pass
else:
d.pop('last_job', None)
d.pop('last_job_host_summary', None)
if has_model_field_prefetched(obj, 'groups'): if has_model_field_prefetched(obj, 'groups'):
group_list = sorted([{'id': g.id, 'name': g.name} for g in obj.groups.all()], key=lambda x: x['id'])[:5] group_list = sorted([{'id': g.id, 'name': g.name} for g in obj.groups.all()], key=lambda x: x['id'])[:5]
else: else:
@@ -1925,14 +1940,16 @@ class HostSerializer(BaseSerializerWithVariables):
return ret return ret
if 'inventory' in ret and not obj.inventory: if 'inventory' in ret and not obj.inventory:
ret['inventory'] = None ret['inventory'] = None
if 'last_job' in ret and not obj.last_job: last_summary = obj.latest_summary
ret['last_job'] = None if 'last_job' in ret:
if 'last_job_host_summary' in ret and not obj.last_job_host_summary: ret['last_job'] = last_summary.job_id if last_summary else None
ret['last_job_host_summary'] = None if 'last_job_host_summary' in ret:
ret['last_job_host_summary'] = last_summary.pk if last_summary else None
return ret return ret
def get_has_active_failures(self, obj): def get_has_active_failures(self, obj):
return bool(obj.last_job_host_summary and obj.last_job_host_summary.failed) last_summary = obj.latest_summary
return bool(last_summary and last_summary.failed)
def get_has_inventory_sources(self, obj): def get_has_inventory_sources(self, obj):
return obj.inventory_sources.exists() return obj.inventory_sources.exists()
@@ -2079,9 +2096,17 @@ class BulkHostCreateSerializer(serializers.Serializer):
if request and not request.user.is_superuser: if request and not request.user.is_superuser:
if request.user not in inv.admin_role: if request.user not in inv.admin_role:
raise serializers.ValidationError(_(f'Inventory with id {inv.id} not found or lack permissions to add hosts.')) raise serializers.ValidationError(_(f'Inventory with id {inv.id} not found or lack permissions to add hosts.'))
current_hostnames = set(inv.hosts.values_list('name', flat=True))
# Performance optimization (AAP-67978): Instead of loading ALL host names from
# the inventory, only check if the specific new names already exist in the database.
new_names = [host['name'] for host in attrs['hosts']] new_names = [host['name'] for host in attrs['hosts']]
duplicate_new_names = [n for n in new_names if n in current_hostnames or new_names.count(n) > 1]
new_name_counts = Counter(new_names)
duplicates_in_new = [name for name, count in new_name_counts.items() if count > 1]
unique_new_names = list(new_name_counts.keys())
existing_duplicates = list(Host.objects.filter(inventory=inv, name__in=unique_new_names).values_list('name', flat=True))
duplicate_new_names = list(set(duplicates_in_new + existing_duplicates))
if duplicate_new_names: if duplicate_new_names:
raise serializers.ValidationError(_(f'Hostnames must be unique in an inventory. Duplicates found: {duplicate_new_names}')) raise serializers.ValidationError(_(f'Hostnames must be unique in an inventory. Duplicates found: {duplicate_new_names}'))
@@ -2165,13 +2190,13 @@ class BulkHostDeleteSerializer(serializers.Serializer):
attrs['hosts_data'] = attrs['host_qs'].values() attrs['hosts_data'] = attrs['host_qs'].values()
if len(attrs['host_qs']) == 0: if len(attrs['host_qs']) == 0:
error_hosts = {host: "Hosts do not exist or you lack permission to delete it" for host in attrs['hosts']} error_hosts = dict.fromkeys(attrs['hosts'], "Hosts do not exist or you lack permission to delete it")
raise serializers.ValidationError({'hosts': error_hosts}) raise serializers.ValidationError({'hosts': error_hosts})
if len(attrs['host_qs']) < len(attrs['hosts']): if len(attrs['host_qs']) < len(attrs['hosts']):
hosts_exists = [host['id'] for host in attrs['hosts_data']] hosts_exists = [host['id'] for host in attrs['hosts_data']]
failed_hosts = list(set(attrs['hosts']).difference(hosts_exists)) failed_hosts = list(set(attrs['hosts']).difference(hosts_exists))
error_hosts = {host: "Hosts do not exist or you lack permission to delete it" for host in failed_hosts} error_hosts = dict.fromkeys(failed_hosts, "Hosts do not exist or you lack permission to delete it")
raise serializers.ValidationError({'hosts': error_hosts}) raise serializers.ValidationError({'hosts': error_hosts})
# Getting all inventories that the hosts can be in # Getting all inventories that the hosts can be in
@@ -2932,6 +2957,19 @@ class CredentialTypeSerializer(BaseSerializer):
field['label'] = _(field['label']) field['label'] = _(field['label'])
if 'help_text' in field: if 'help_text' in field:
field['help_text'] = _(field['help_text']) field['help_text'] = _(field['help_text'])
# Deep copy inputs to avoid modifying the original model data
inputs = value.get('inputs')
if not isinstance(inputs, dict):
inputs = {}
value['inputs'] = copy.deepcopy(inputs)
fields = value['inputs'].get('fields', [])
if not isinstance(fields, list):
fields = []
# Normalize fields and filter out internal fields
value['inputs']['fields'] = [f for f in fields if not f.get('internal')]
return value return value
def filter_field_metadata(self, fields, method): def filter_field_metadata(self, fields, method):
@@ -3527,7 +3565,7 @@ class JobRelaunchSerializer(BaseSerializer):
choices=NEW_JOB_TYPE_CHOICES, choices=NEW_JOB_TYPE_CHOICES,
write_only=True, write_only=True,
) )
credential_passwords = VerbatimField(required=True, write_only=True) credential_passwords = VerbatimField(required=False, write_only=True)
class Meta: class Meta:
model = Job model = Job
@@ -4122,9 +4160,28 @@ class LaunchConfigurationBaseSerializer(BaseSerializer):
attrs['extra_data'][key] = db_extra_data[key] attrs['extra_data'][key] = db_extra_data[key]
# Build unsaved version of this config, use it to detect prompts errors # Build unsaved version of this config, use it to detect prompts errors
# Capture keys before _build_mock_obj pops pseudo-fields from attrs
incoming_attr_keys = set(attrs.keys())
mock_obj = self._build_mock_obj(attrs) mock_obj = self._build_mock_obj(attrs)
if set(list(ujt.get_ask_mapping().keys()) + ['extra_data']) & set(attrs.keys()): ask_mapping_keys = set(ujt.get_ask_mapping().keys())
accepted, rejected, errors = ujt._accept_or_ignore_job_kwargs(_exclude_errors=self.exclude_errors, **mock_obj.prompts_dict()) requested_prompt_fields = incoming_attr_keys & ask_mapping_keys
if 'extra_data' in incoming_attr_keys:
requested_prompt_fields.add('extra_vars')
requested_prompt_fields.add('survey_passwords')
# prompts_dict() pulls persisted M2M state (labels, credentials,
# instance_groups) via the instance pk. Only re-validate the full prompt
# state when the caller is switching the underlying template; otherwise
# restrict validation to the fields the request explicitly provided.
if 'unified_job_template' in attrs:
prompts_to_validate = mock_obj.prompts_dict()
elif requested_prompt_fields:
prompts_to_validate = {k: v for k, v in mock_obj.prompts_dict().items() if k in requested_prompt_fields}
else:
prompts_to_validate = None
if prompts_to_validate is not None:
accepted, rejected, errors = ujt._accept_or_ignore_job_kwargs(_exclude_errors=self.exclude_errors, **prompts_to_validate)
else: else:
# Only perform validation of prompts if prompts fields are provided # Only perform validation of prompts if prompts fields are provided
errors = {} errors = {}

View File

@@ -1,55 +0,0 @@
import warnings
from rest_framework.permissions import AllowAny
from drf_yasg import openapi
from drf_yasg.inspectors import SwaggerAutoSchema
from drf_yasg.views import get_schema_view
class CustomSwaggerAutoSchema(SwaggerAutoSchema):
"""Custom SwaggerAutoSchema to add swagger_topic to tags."""
def get_tags(self, operation_keys=None):
tags = []
try:
if hasattr(self.view, 'get_serializer'):
serializer = self.view.get_serializer()
else:
serializer = None
except Exception:
serializer = None
warnings.warn(
'{}.get_serializer() raised an exception during '
'schema generation. Serializer fields will not be '
'generated for {}.'.format(self.view.__class__.__name__, operation_keys)
)
if hasattr(self.view, 'swagger_topic'):
tags.append(str(self.view.swagger_topic).title())
elif serializer and hasattr(serializer, 'Meta'):
tags.append(str(serializer.Meta.model._meta.verbose_name_plural).title())
elif hasattr(self.view, 'model'):
tags.append(str(self.view.model._meta.verbose_name_plural).title())
else:
tags = ['api'] # Fallback to default value
if not tags:
warnings.warn(f'Could not determine tags for {self.view.__class__.__name__}')
return tags
def is_deprecated(self):
"""Return `True` if this operation is to be marked as deprecated."""
return getattr(self.view, 'deprecated', False)
schema_view = get_schema_view(
openapi.Info(
title='AWX API',
default_version='v2',
description='AWX API Documentation',
terms_of_service='https://www.google.com/policies/terms/',
contact=openapi.Contact(email='contact@snippets.local'),
license=openapi.License(name='Apache License'),
),
public=True,
permission_classes=[AllowAny],
)

View File

@@ -1,6 +1,6 @@
{% if content_only %}<div class="nocode ansi_fore ansi_back{% if dark %} ansi_dark{% endif %}">{% else %} {% if content_only %}<div class="nocode ansi_fore ansi_back{% if dark %} ansi_dark{% endif %}">{% else %}
<!DOCTYPE HTML> <!DOCTYPE HTML>
<html> <html lang="en">
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>{{ title }}</title> <title>{{ title }}</title>

View File

@@ -1,4 +1,4 @@
--- ---
collections: collections:
- name: ansible.receptor - name: ansible.receptor
version: 2.0.3 version: 2.0.8

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import ActivityStreamList, ActivityStreamDetail from awx.api.views import ActivityStreamList, ActivityStreamDetail
urls = [ urls = [
re_path(r'^$', ActivityStreamList.as_view(), name='activity_stream_list'), re_path(r'^$', ActivityStreamList.as_view(), name='activity_stream_list'),
re_path(r'^(?P<pk>[0-9]+)/$', ActivityStreamDetail.as_view(), name='activity_stream_detail'), re_path(r'^(?P<pk>[0-9]+)/$', ActivityStreamDetail.as_view(), name='activity_stream_detail'),

View File

@@ -14,7 +14,6 @@ from awx.api.views import (
AdHocCommandStdout, AdHocCommandStdout,
) )
urls = [ urls = [
re_path(r'^$', AdHocCommandList.as_view(), name='ad_hoc_command_list'), re_path(r'^$', AdHocCommandList.as_view(), name='ad_hoc_command_list'),
re_path(r'^(?P<pk>[0-9]+)/$', AdHocCommandDetail.as_view(), name='ad_hoc_command_detail'), re_path(r'^(?P<pk>[0-9]+)/$', AdHocCommandDetail.as_view(), name='ad_hoc_command_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import AdHocCommandEventDetail from awx.api.views import AdHocCommandEventDetail
urls = [ urls = [
re_path(r'^(?P<pk>[0-9]+)/$', AdHocCommandEventDetail.as_view(), name='ad_hoc_command_event_detail'), re_path(r'^(?P<pk>[0-9]+)/$', AdHocCommandEventDetail.as_view(), name='ad_hoc_command_event_detail'),
] ]

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
import awx.api.views.analytics as analytics import awx.api.views.analytics as analytics
urls = [ urls = [
re_path(r'^$', analytics.AnalyticsRootView.as_view(), name='analytics_root_view'), re_path(r'^$', analytics.AnalyticsRootView.as_view(), name='analytics_root_view'),
re_path(r'^authorized/$', analytics.AnalyticsAuthorizedView.as_view(), name='analytics_authorized'), re_path(r'^authorized/$', analytics.AnalyticsAuthorizedView.as_view(), name='analytics_authorized'),

View File

@@ -16,7 +16,6 @@ from awx.api.views import (
CredentialExternalTest, CredentialExternalTest,
) )
urls = [ urls = [
re_path(r'^$', CredentialList.as_view(), name='credential_list'), re_path(r'^$', CredentialList.as_view(), name='credential_list'),
re_path(r'^(?P<pk>[0-9]+)/activity_stream/$', CredentialActivityStreamList.as_view(), name='credential_activity_stream_list'), re_path(r'^(?P<pk>[0-9]+)/activity_stream/$', CredentialActivityStreamList.as_view(), name='credential_activity_stream_list'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import CredentialInputSourceDetail, CredentialInputSourceList from awx.api.views import CredentialInputSourceDetail, CredentialInputSourceList
urls = [ urls = [
re_path(r'^$', CredentialInputSourceList.as_view(), name='credential_input_source_list'), re_path(r'^$', CredentialInputSourceList.as_view(), name='credential_input_source_list'),
re_path(r'^(?P<pk>[0-9]+)/$', CredentialInputSourceDetail.as_view(), name='credential_input_source_detail'), re_path(r'^(?P<pk>[0-9]+)/$', CredentialInputSourceDetail.as_view(), name='credential_input_source_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import CredentialTypeList, CredentialTypeDetail, CredentialTypeCredentialList, CredentialTypeActivityStreamList, CredentialTypeExternalTest from awx.api.views import CredentialTypeList, CredentialTypeDetail, CredentialTypeCredentialList, CredentialTypeActivityStreamList, CredentialTypeExternalTest
urls = [ urls = [
re_path(r'^$', CredentialTypeList.as_view(), name='credential_type_list'), re_path(r'^$', CredentialTypeList.as_view(), name='credential_type_list'),
re_path(r'^(?P<pk>[0-9]+)/$', CredentialTypeDetail.as_view(), name='credential_type_detail'), re_path(r'^(?P<pk>[0-9]+)/$', CredentialTypeDetail.as_view(), name='credential_type_detail'),

View File

@@ -8,7 +8,6 @@ from awx.api.views import (
ExecutionEnvironmentActivityStreamList, ExecutionEnvironmentActivityStreamList,
) )
urls = [ urls = [
re_path(r'^$', ExecutionEnvironmentList.as_view(), name='execution_environment_list'), re_path(r'^$', ExecutionEnvironmentList.as_view(), name='execution_environment_list'),
re_path(r'^(?P<pk>[0-9]+)/$', ExecutionEnvironmentDetail.as_view(), name='execution_environment_detail'), re_path(r'^(?P<pk>[0-9]+)/$', ExecutionEnvironmentDetail.as_view(), name='execution_environment_detail'),

View File

@@ -18,7 +18,6 @@ from awx.api.views import (
GroupAdHocCommandsList, GroupAdHocCommandsList,
) )
urls = [ urls = [
re_path(r'^$', GroupList.as_view(), name='group_list'), re_path(r'^$', GroupList.as_view(), name='group_list'),
re_path(r'^(?P<pk>[0-9]+)/$', GroupDetail.as_view(), name='group_detail'), re_path(r'^(?P<pk>[0-9]+)/$', GroupDetail.as_view(), name='group_detail'),

View File

@@ -18,7 +18,6 @@ from awx.api.views import (
HostAdHocCommandEventsList, HostAdHocCommandEventsList,
) )
urls = [ urls = [
re_path(r'^$', HostList.as_view(), name='host_list'), re_path(r'^$', HostList.as_view(), name='host_list'),
re_path(r'^(?P<pk>[0-9]+)/$', HostDetail.as_view(), name='host_detail'), re_path(r'^(?P<pk>[0-9]+)/$', HostDetail.as_view(), name='host_detail'),

View File

@@ -14,7 +14,6 @@ from awx.api.views import (
) )
from awx.api.views.instance_install_bundle import InstanceInstallBundle from awx.api.views.instance_install_bundle import InstanceInstallBundle
urls = [ urls = [
re_path(r'^$', InstanceList.as_view(), name='instance_list'), re_path(r'^$', InstanceList.as_view(), name='instance_list'),
re_path(r'^(?P<pk>[0-9]+)/$', InstanceDetail.as_view(), name='instance_detail'), re_path(r'^(?P<pk>[0-9]+)/$', InstanceDetail.as_view(), name='instance_detail'),

View File

@@ -12,7 +12,6 @@ from awx.api.views import (
InstanceGroupObjectRolesList, InstanceGroupObjectRolesList,
) )
urls = [ urls = [
re_path(r'^$', InstanceGroupList.as_view(), name='instance_group_list'), re_path(r'^$', InstanceGroupList.as_view(), name='instance_group_list'),
re_path(r'^(?P<pk>[0-9]+)/$', InstanceGroupDetail.as_view(), name='instance_group_detail'), re_path(r'^(?P<pk>[0-9]+)/$', InstanceGroupDetail.as_view(), name='instance_group_detail'),

View File

@@ -29,7 +29,6 @@ from awx.api.views import (
InventoryVariableData, InventoryVariableData,
) )
urls = [ urls = [
re_path(r'^$', InventoryList.as_view(), name='inventory_list'), re_path(r'^$', InventoryList.as_view(), name='inventory_list'),
re_path(r'^(?P<pk>[0-9]+)/$', InventoryDetail.as_view(), name='inventory_detail'), re_path(r'^(?P<pk>[0-9]+)/$', InventoryDetail.as_view(), name='inventory_detail'),

View File

@@ -18,7 +18,6 @@ from awx.api.views import (
InventorySourceNotificationTemplatesSuccessList, InventorySourceNotificationTemplatesSuccessList,
) )
urls = [ urls = [
re_path(r'^$', InventorySourceList.as_view(), name='inventory_source_list'), re_path(r'^$', InventorySourceList.as_view(), name='inventory_source_list'),
re_path(r'^(?P<pk>[0-9]+)/$', InventorySourceDetail.as_view(), name='inventory_source_detail'), re_path(r'^(?P<pk>[0-9]+)/$', InventorySourceDetail.as_view(), name='inventory_source_detail'),

View File

@@ -15,7 +15,6 @@ from awx.api.views import (
InventoryUpdateCredentialsList, InventoryUpdateCredentialsList,
) )
urls = [ urls = [
re_path(r'^$', InventoryUpdateList.as_view(), name='inventory_update_list'), re_path(r'^$', InventoryUpdateList.as_view(), name='inventory_update_list'),
re_path(r'^(?P<pk>[0-9]+)/$', InventoryUpdateDetail.as_view(), name='inventory_update_detail'), re_path(r'^(?P<pk>[0-9]+)/$', InventoryUpdateDetail.as_view(), name='inventory_update_detail'),

View File

@@ -19,7 +19,6 @@ from awx.api.views import (
JobHostSummaryDetail, JobHostSummaryDetail,
) )
urls = [ urls = [
re_path(r'^$', JobList.as_view(), name='job_list'), re_path(r'^$', JobList.as_view(), name='job_list'),
re_path(r'^(?P<pk>[0-9]+)/$', JobDetail.as_view(), name='job_detail'), re_path(r'^(?P<pk>[0-9]+)/$', JobDetail.as_view(), name='job_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import JobHostSummaryDetail from awx.api.views import JobHostSummaryDetail
urls = [re_path(r'^(?P<pk>[0-9]+)/$', JobHostSummaryDetail.as_view(), name='job_host_summary_detail')] urls = [re_path(r'^(?P<pk>[0-9]+)/$', JobHostSummaryDetail.as_view(), name='job_host_summary_detail')]
__all__ = ['urls'] __all__ = ['urls']

View File

@@ -23,7 +23,6 @@ from awx.api.views import (
JobTemplateCopy, JobTemplateCopy,
) )
urls = [ urls = [
re_path(r'^$', JobTemplateList.as_view(), name='job_template_list'), re_path(r'^$', JobTemplateList.as_view(), name='job_template_list'),
re_path(r'^(?P<pk>[0-9]+)/$', JobTemplateDetail.as_view(), name='job_template_detail'), re_path(r'^(?P<pk>[0-9]+)/$', JobTemplateDetail.as_view(), name='job_template_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views.labels import LabelList, LabelDetail from awx.api.views.labels import LabelList, LabelDetail
urls = [re_path(r'^$', LabelList.as_view(), name='label_list'), re_path(r'^(?P<pk>[0-9]+)/$', LabelDetail.as_view(), name='label_detail')] urls = [re_path(r'^$', LabelList.as_view(), name='label_list'), re_path(r'^(?P<pk>[0-9]+)/$', LabelDetail.as_view(), name='label_detail')]
__all__ = ['urls'] __all__ = ['urls']

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import NotificationList, NotificationDetail from awx.api.views import NotificationList, NotificationDetail
urls = [ urls = [
re_path(r'^$', NotificationList.as_view(), name='notification_list'), re_path(r'^$', NotificationList.as_view(), name='notification_list'),
re_path(r'^(?P<pk>[0-9]+)/$', NotificationDetail.as_view(), name='notification_detail'), re_path(r'^(?P<pk>[0-9]+)/$', NotificationDetail.as_view(), name='notification_detail'),

View File

@@ -11,7 +11,6 @@ from awx.api.views import (
NotificationTemplateCopy, NotificationTemplateCopy,
) )
urls = [ urls = [
re_path(r'^$', NotificationTemplateList.as_view(), name='notification_template_list'), re_path(r'^$', NotificationTemplateList.as_view(), name='notification_template_list'),
re_path(r'^(?P<pk>[0-9]+)/$', NotificationTemplateDetail.as_view(), name='notification_template_detail'), re_path(r'^(?P<pk>[0-9]+)/$', NotificationTemplateDetail.as_view(), name='notification_template_detail'),

View File

@@ -27,7 +27,6 @@ from awx.api.views.organization import (
) )
from awx.api.views import OrganizationCredentialList from awx.api.views import OrganizationCredentialList
urls = [ urls = [
re_path(r'^$', OrganizationList.as_view(), name='organization_list'), re_path(r'^$', OrganizationList.as_view(), name='organization_list'),
re_path(r'^(?P<pk>[0-9]+)/$', OrganizationDetail.as_view(), name='organization_detail'), re_path(r'^(?P<pk>[0-9]+)/$', OrganizationDetail.as_view(), name='organization_detail'),

View File

@@ -22,7 +22,6 @@ from awx.api.views import (
ProjectCopy, ProjectCopy,
) )
urls = [ urls = [
re_path(r'^$', ProjectList.as_view(), name='project_list'), re_path(r'^$', ProjectList.as_view(), name='project_list'),
re_path(r'^(?P<pk>[0-9]+)/$', ProjectDetail.as_view(), name='project_detail'), re_path(r'^(?P<pk>[0-9]+)/$', ProjectDetail.as_view(), name='project_detail'),

View File

@@ -13,7 +13,6 @@ from awx.api.views import (
ProjectUpdateEventsList, ProjectUpdateEventsList,
) )
urls = [ urls = [
re_path(r'^$', ProjectUpdateList.as_view(), name='project_update_list'), re_path(r'^$', ProjectUpdateList.as_view(), name='project_update_list'),
re_path(r'^(?P<pk>[0-9]+)/$', ProjectUpdateDetail.as_view(), name='project_update_detail'), re_path(r'^(?P<pk>[0-9]+)/$', ProjectUpdateDetail.as_view(), name='project_update_detail'),

View File

@@ -8,7 +8,6 @@ from awx.api.views import (
ReceptorAddressDetail, ReceptorAddressDetail,
) )
urls = [ urls = [
re_path(r'^$', ReceptorAddressesList.as_view(), name='receptor_addresses_list'), re_path(r'^$', ReceptorAddressesList.as_view(), name='receptor_addresses_list'),
re_path(r'^(?P<pk>[0-9]+)/$', ReceptorAddressDetail.as_view(), name='receptor_address_detail'), re_path(r'^(?P<pk>[0-9]+)/$', ReceptorAddressDetail.as_view(), name='receptor_address_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import RoleList, RoleDetail, RoleUsersList, RoleTeamsList from awx.api.views import RoleList, RoleDetail, RoleUsersList, RoleTeamsList
urls = [ urls = [
re_path(r'^$', RoleList.as_view(), name='role_list'), re_path(r'^$', RoleList.as_view(), name='role_list'),
re_path(r'^(?P<pk>[0-9]+)/$', RoleDetail.as_view(), name='role_detail'), re_path(r'^(?P<pk>[0-9]+)/$', RoleDetail.as_view(), name='role_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import ScheduleList, ScheduleDetail, ScheduleUnifiedJobsList, ScheduleCredentialsList, ScheduleLabelsList, ScheduleInstanceGroupList from awx.api.views import ScheduleList, ScheduleDetail, ScheduleUnifiedJobsList, ScheduleCredentialsList, ScheduleLabelsList, ScheduleInstanceGroupList
urls = [ urls = [
re_path(r'^$', ScheduleList.as_view(), name='schedule_list'), re_path(r'^$', ScheduleList.as_view(), name='schedule_list'),
re_path(r'^(?P<pk>[0-9]+)/$', ScheduleDetail.as_view(), name='schedule_detail'), re_path(r'^(?P<pk>[0-9]+)/$', ScheduleDetail.as_view(), name='schedule_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import SystemJobList, SystemJobDetail, SystemJobCancel, SystemJobNotificationsList, SystemJobEventsList from awx.api.views import SystemJobList, SystemJobDetail, SystemJobCancel, SystemJobNotificationsList, SystemJobEventsList
urls = [ urls = [
re_path(r'^$', SystemJobList.as_view(), name='system_job_list'), re_path(r'^$', SystemJobList.as_view(), name='system_job_list'),
re_path(r'^(?P<pk>[0-9]+)/$', SystemJobDetail.as_view(), name='system_job_detail'), re_path(r'^(?P<pk>[0-9]+)/$', SystemJobDetail.as_view(), name='system_job_detail'),

View File

@@ -14,7 +14,6 @@ from awx.api.views import (
SystemJobTemplateNotificationTemplatesSuccessList, SystemJobTemplateNotificationTemplatesSuccessList,
) )
urls = [ urls = [
re_path(r'^$', SystemJobTemplateList.as_view(), name='system_job_template_list'), re_path(r'^$', SystemJobTemplateList.as_view(), name='system_job_template_list'),
re_path(r'^(?P<pk>[0-9]+)/$', SystemJobTemplateDetail.as_view(), name='system_job_template_detail'), re_path(r'^(?P<pk>[0-9]+)/$', SystemJobTemplateDetail.as_view(), name='system_job_template_detail'),

View File

@@ -15,7 +15,6 @@ from awx.api.views import (
TeamAccessList, TeamAccessList,
) )
urls = [ urls = [
re_path(r'^$', TeamList.as_view(), name='team_list'), re_path(r'^$', TeamList.as_view(), name='team_list'),
re_path(r'^(?P<pk>[0-9]+)/$', TeamDetail.as_view(), name='team_detail'), re_path(r'^(?P<pk>[0-9]+)/$', TeamDetail.as_view(), name='team_detail'),

View File

@@ -4,7 +4,6 @@
from __future__ import absolute_import, unicode_literals from __future__ import absolute_import, unicode_literals
from django.urls import include, re_path from django.urls import include, re_path
from awx import MODE
from awx.api.generics import LoggedLoginView, LoggedLogoutView from awx.api.generics import LoggedLoginView, LoggedLogoutView
from awx.api.views.root import ( from awx.api.views.root import (
ApiRootView, ApiRootView,
@@ -148,21 +147,15 @@ v2_urls = [
app_name = 'api' app_name = 'api'
urlpatterns = [ urlpatterns = [
re_path(r'^$', ApiRootView.as_view(), name='api_root_view'), re_path(r'^$', ApiRootView.as_view(), name='api_root_view'),
re_path(r'^(?P<version>(v2))/', include(v2_urls)), re_path(r'^(?P<version>(v2))/', include(v2_urls)),
re_path(r'^login/$', LoggedLoginView.as_view(template_name='rest_framework/login.html', extra_context={'inside_login_context': True}), name='login'), re_path(r'^login/$', LoggedLoginView.as_view(template_name='rest_framework/login.html', extra_context={'inside_login_context': True}), name='login'),
re_path(r'^logout/$', LoggedLogoutView.as_view(next_page='/api/', redirect_field_name='next'), name='logout'), re_path(r'^logout/$', LoggedLogoutView.as_view(next_page='/api/', redirect_field_name='next'), name='logout'),
# the docs/, schema-related endpoints used to be listed here but now exposed by DAB api_documentation app
] ]
if MODE == 'development':
# Only include these if we are in the development environment
from awx.api.swagger import schema_view
from awx.api.urls.debug import urls as debug_urls from awx.api.urls.debug import urls as debug_urls
urlpatterns += [re_path(r'^debug/', include(debug_urls))] urlpatterns += [re_path(r'^debug/', include(debug_urls))]
urlpatterns += [
re_path(r'^swagger(?P<format>\.json|\.yaml)/$', schema_view.without_ui(cache_timeout=0), name='schema-json'),
re_path(r'^swagger/$', schema_view.with_ui('swagger', cache_timeout=0), name='schema-swagger-ui'),
re_path(r'^redoc/$', schema_view.with_ui('redoc', cache_timeout=0), name='schema-redoc'),
]

View File

@@ -2,7 +2,6 @@ from django.urls import re_path
from awx.api.views.webhooks import WebhookKeyView, GithubWebhookReceiver, GitlabWebhookReceiver, BitbucketDcWebhookReceiver from awx.api.views.webhooks import WebhookKeyView, GithubWebhookReceiver, GitlabWebhookReceiver, BitbucketDcWebhookReceiver
urlpatterns = [ urlpatterns = [
re_path(r'^webhook_key/$', WebhookKeyView.as_view(), name='webhook_key'), re_path(r'^webhook_key/$', WebhookKeyView.as_view(), name='webhook_key'),
re_path(r'^github/$', GithubWebhookReceiver.as_view(), name='webhook_receiver_github'), re_path(r'^github/$', GithubWebhookReceiver.as_view(), name='webhook_receiver_github'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import WorkflowApprovalList, WorkflowApprovalDetail, WorkflowApprovalApprove, WorkflowApprovalDeny from awx.api.views import WorkflowApprovalList, WorkflowApprovalDetail, WorkflowApprovalApprove, WorkflowApprovalDeny
urls = [ urls = [
re_path(r'^$', WorkflowApprovalList.as_view(), name='workflow_approval_list'), re_path(r'^$', WorkflowApprovalList.as_view(), name='workflow_approval_list'),
re_path(r'^(?P<pk>[0-9]+)/$', WorkflowApprovalDetail.as_view(), name='workflow_approval_detail'), re_path(r'^(?P<pk>[0-9]+)/$', WorkflowApprovalDetail.as_view(), name='workflow_approval_detail'),

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.api.views import WorkflowApprovalTemplateDetail, WorkflowApprovalTemplateJobsList from awx.api.views import WorkflowApprovalTemplateDetail, WorkflowApprovalTemplateJobsList
urls = [ urls = [
re_path(r'^(?P<pk>[0-9]+)/$', WorkflowApprovalTemplateDetail.as_view(), name='workflow_approval_template_detail'), re_path(r'^(?P<pk>[0-9]+)/$', WorkflowApprovalTemplateDetail.as_view(), name='workflow_approval_template_detail'),
re_path(r'^(?P<pk>[0-9]+)/approvals/$', WorkflowApprovalTemplateJobsList.as_view(), name='workflow_approval_template_jobs_list'), re_path(r'^(?P<pk>[0-9]+)/approvals/$', WorkflowApprovalTemplateJobsList.as_view(), name='workflow_approval_template_jobs_list'),

View File

@@ -14,7 +14,6 @@ from awx.api.views import (
WorkflowJobActivityStreamList, WorkflowJobActivityStreamList,
) )
urls = [ urls = [
re_path(r'^$', WorkflowJobList.as_view(), name='workflow_job_list'), re_path(r'^$', WorkflowJobList.as_view(), name='workflow_job_list'),
re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobDetail.as_view(), name='workflow_job_detail'), re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobDetail.as_view(), name='workflow_job_detail'),

View File

@@ -14,7 +14,6 @@ from awx.api.views import (
WorkflowJobNodeInstanceGroupsList, WorkflowJobNodeInstanceGroupsList,
) )
urls = [ urls = [
re_path(r'^$', WorkflowJobNodeList.as_view(), name='workflow_job_node_list'), re_path(r'^$', WorkflowJobNodeList.as_view(), name='workflow_job_node_list'),
re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobNodeDetail.as_view(), name='workflow_job_node_detail'), re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobNodeDetail.as_view(), name='workflow_job_node_detail'),

View File

@@ -22,7 +22,6 @@ from awx.api.views import (
WorkflowJobTemplateLabelList, WorkflowJobTemplateLabelList,
) )
urls = [ urls = [
re_path(r'^$', WorkflowJobTemplateList.as_view(), name='workflow_job_template_list'), re_path(r'^$', WorkflowJobTemplateList.as_view(), name='workflow_job_template_list'),
re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobTemplateDetail.as_view(), name='workflow_job_template_detail'), re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobTemplateDetail.as_view(), name='workflow_job_template_detail'),

View File

@@ -15,7 +15,6 @@ from awx.api.views import (
WorkflowJobTemplateNodeInstanceGroupsList, WorkflowJobTemplateNodeInstanceGroupsList,
) )
urls = [ urls = [
re_path(r'^$', WorkflowJobTemplateNodeList.as_view(), name='workflow_job_template_node_list'), re_path(r'^$', WorkflowJobTemplateNodeList.as_view(), name='workflow_job_template_node_list'),
re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobTemplateNodeDetail.as_view(), name='workflow_job_template_node_detail'), re_path(r'^(?P<pk>[0-9]+)/$', WorkflowJobTemplateNodeDetail.as_view(), name='workflow_job_template_node_detail'),

File diff suppressed because it is too large Load Diff

View File

@@ -9,12 +9,14 @@ from django.utils import translation
from awx.api.generics import APIView, Response from awx.api.generics import APIView, Response
from awx.api.permissions import AnalyticsPermission from awx.api.permissions import AnalyticsPermission
from awx.api.versioning import reverse from awx.api.versioning import reverse
from awx.main.utils import get_awx_version from awx.main.utils import get_awx_version, set_environ
from awx.main.utils.analytics_proxy import OIDCClient from awx.main.utils.analytics_proxy import OIDCClient
from rest_framework import status from rest_framework import status
from collections import OrderedDict from collections import OrderedDict
from ansible_base.lib.utils.schema import extend_schema_if_available
AUTOMATION_ANALYTICS_API_URL_PATH = "/api/tower-analytics/v1" AUTOMATION_ANALYTICS_API_URL_PATH = "/api/tower-analytics/v1"
AWX_ANALYTICS_API_PREFIX = 'analytics' AWX_ANALYTICS_API_PREFIX = 'analytics'
@@ -38,6 +40,8 @@ class MissingSettings(Exception):
class GetNotAllowedMixin(object): class GetNotAllowedMixin(object):
skip_ai_description = True
def get(self, request, format=None): def get(self, request, format=None):
return Response(status=status.HTTP_405_METHOD_NOT_ALLOWED) return Response(status=status.HTTP_405_METHOD_NOT_ALLOWED)
@@ -45,8 +49,9 @@ class GetNotAllowedMixin(object):
class AnalyticsRootView(APIView): class AnalyticsRootView(APIView):
permission_classes = (AnalyticsPermission,) permission_classes = (AnalyticsPermission,)
name = _('Automation Analytics') name = _('Automation Analytics')
swagger_topic = 'Automation Analytics' resource_purpose = 'automation analytics endpoints'
@extend_schema_if_available(extensions={"x-ai-description": "A list of additional API endpoints related to analytics"})
def get(self, request, format=None): def get(self, request, format=None):
data = OrderedDict() data = OrderedDict()
data['authorized'] = reverse('api:analytics_authorized', request=request) data['authorized'] = reverse('api:analytics_authorized', request=request)
@@ -99,6 +104,8 @@ class AnalyticsGenericView(APIView):
return Response(response.json(), status=response.status_code) return Response(response.json(), status=response.status_code)
""" """
resource_purpose = 'base view for analytics api proxy'
permission_classes = (AnalyticsPermission,) permission_classes = (AnalyticsPermission,)
@staticmethod @staticmethod
@@ -203,31 +210,32 @@ class AnalyticsGenericView(APIView):
return self._error_response(ERROR_UNSUPPORTED_METHOD, method, remote=False, status_code=status.HTTP_500_INTERNAL_SERVER_ERROR) return self._error_response(ERROR_UNSUPPORTED_METHOD, method, remote=False, status_code=status.HTTP_500_INTERNAL_SERVER_ERROR)
url = self._get_analytics_url(request.path) url = self._get_analytics_url(request.path)
using_subscriptions_credentials = False using_subscriptions_credentials = False
try: with set_environ(**settings.AWX_TASK_ENV):
rh_user = getattr(settings, 'REDHAT_USERNAME', None) try:
rh_password = getattr(settings, 'REDHAT_PASSWORD', None) rh_user = getattr(settings, 'REDHAT_USERNAME', None)
if not (rh_user and rh_password): rh_password = getattr(settings, 'REDHAT_PASSWORD', None)
rh_user = self._get_setting('SUBSCRIPTIONS_CLIENT_ID', None, ERROR_MISSING_USER) if not (rh_user and rh_password):
rh_password = self._get_setting('SUBSCRIPTIONS_CLIENT_SECRET', None, ERROR_MISSING_PASSWORD) rh_user = self._get_setting('SUBSCRIPTIONS_CLIENT_ID', None, ERROR_MISSING_USER)
using_subscriptions_credentials = True rh_password = self._get_setting('SUBSCRIPTIONS_CLIENT_SECRET', None, ERROR_MISSING_PASSWORD)
using_subscriptions_credentials = True
client = OIDCClient(rh_user, rh_password) client = OIDCClient(rh_user, rh_password)
response = client.make_request( response = client.make_request(
method, method,
url, url,
headers=headers, headers=headers,
verify=settings.INSIGHTS_CERT_PATH, verify=settings.INSIGHTS_CERT_PATH,
params=getattr(request, 'query_params', {}), params=getattr(request, 'query_params', {}),
json=getattr(request, 'data', {}), json=getattr(request, 'data', {}),
timeout=(31, 31), timeout=(31, 31),
) )
except requests.RequestException: except requests.RequestException:
# subscriptions credentials are not valid for basic auth, so just return 401 # subscriptions credentials are not valid for basic auth, so just return 401
if using_subscriptions_credentials: if using_subscriptions_credentials:
response = Response(status=status.HTTP_401_UNAUTHORIZED) response = Response(status=status.HTTP_401_UNAUTHORIZED)
else: else:
logger.error("Automation Analytics API request failed, trying base auth method") logger.error("Automation Analytics API request failed, trying base auth method")
response = self._base_auth_request(request, method, url, rh_user, rh_password, headers) response = self._base_auth_request(request, method, url, rh_user, rh_password, headers)
# #
# Missing or wrong user/pass # Missing or wrong user/pass
# #
@@ -257,67 +265,90 @@ class AnalyticsGenericView(APIView):
class AnalyticsGenericListView(AnalyticsGenericView): class AnalyticsGenericListView(AnalyticsGenericView):
resource_purpose = 'analytics api proxy list view'
@extend_schema_if_available(extensions={"x-ai-description": "Get analytics data from Red Hat Insights"})
def get(self, request, format=None): def get(self, request, format=None):
return self._send_to_analytics(request, method="GET") return self._send_to_analytics(request, method="GET")
@extend_schema_if_available(extensions={"x-ai-description": "Post query to Red Hat Insights analytics"})
def post(self, request, format=None): def post(self, request, format=None):
return self._send_to_analytics(request, method="POST") return self._send_to_analytics(request, method="POST")
@extend_schema_if_available(extensions={"x-ai-description": "Get analytics endpoint options"})
def options(self, request, format=None): def options(self, request, format=None):
return self._send_to_analytics(request, method="OPTIONS") return self._send_to_analytics(request, method="OPTIONS")
class AnalyticsGenericDetailView(AnalyticsGenericView): class AnalyticsGenericDetailView(AnalyticsGenericView):
resource_purpose = 'analytics api proxy detail view'
@extend_schema_if_available(extensions={"x-ai-description": "Get specific analytics resource from Red Hat Insights"})
def get(self, request, slug, format=None): def get(self, request, slug, format=None):
return self._send_to_analytics(request, method="GET") return self._send_to_analytics(request, method="GET")
@extend_schema_if_available(extensions={"x-ai-description": "Post query for specific analytics resource to Red Hat Insights"})
def post(self, request, slug, format=None): def post(self, request, slug, format=None):
return self._send_to_analytics(request, method="POST") return self._send_to_analytics(request, method="POST")
@extend_schema_if_available(extensions={"x-ai-description": "Get options for specific analytics resource"})
def options(self, request, slug, format=None): def options(self, request, slug, format=None):
return self._send_to_analytics(request, method="OPTIONS") return self._send_to_analytics(request, method="OPTIONS")
@extend_schema_if_available(
extensions={'x-ai-description': 'Check if the user has access to Red Hat Insights'},
)
class AnalyticsAuthorizedView(AnalyticsGenericListView): class AnalyticsAuthorizedView(AnalyticsGenericListView):
name = _("Authorized") name = _("Authorized")
resource_purpose = 'red hat insights authorization status'
class AnalyticsReportsList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsReportsList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("Reports") name = _("Reports")
swagger_topic = "Automation Analytics" resource_purpose = 'automation analytics reports'
class AnalyticsReportDetail(AnalyticsGenericDetailView): class AnalyticsReportDetail(AnalyticsGenericDetailView):
name = _("Report") name = _("Report")
resource_purpose = 'automation analytics report detail'
class AnalyticsReportOptionsList(AnalyticsGenericListView): class AnalyticsReportOptionsList(AnalyticsGenericListView):
name = _("Report Options") name = _("Report Options")
resource_purpose = 'automation analytics report options'
class AnalyticsAdoptionRateList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsAdoptionRateList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("Adoption Rate") name = _("Adoption Rate")
resource_purpose = 'automation analytics adoption rate data'
class AnalyticsEventExplorerList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsEventExplorerList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("Event Explorer") name = _("Event Explorer")
resource_purpose = 'automation analytics event explorer data'
class AnalyticsHostExplorerList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsHostExplorerList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("Host Explorer") name = _("Host Explorer")
resource_purpose = 'automation analytics host explorer data'
class AnalyticsJobExplorerList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsJobExplorerList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("Job Explorer") name = _("Job Explorer")
resource_purpose = 'automation analytics job explorer data'
class AnalyticsProbeTemplatesList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsProbeTemplatesList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("Probe Templates") name = _("Probe Templates")
resource_purpose = 'automation analytics probe templates'
class AnalyticsProbeTemplateForHostsList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsProbeTemplateForHostsList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("Probe Template For Hosts") name = _("Probe Template For Hosts")
resource_purpose = 'automation analytics probe templates for hosts'
class AnalyticsRoiTemplatesList(GetNotAllowedMixin, AnalyticsGenericListView): class AnalyticsRoiTemplatesList(GetNotAllowedMixin, AnalyticsGenericListView):
name = _("ROI Templates") name = _("ROI Templates")
resource_purpose = 'automation analytics roi templates'

View File

@@ -1,5 +1,7 @@
from collections import OrderedDict from collections import OrderedDict
from ansible_base.lib.utils.schema import extend_schema_if_available
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
from rest_framework.permissions import IsAuthenticated from rest_framework.permissions import IsAuthenticated
@@ -30,6 +32,7 @@ class BulkView(APIView):
] ]
allowed_methods = ['GET', 'OPTIONS'] allowed_methods = ['GET', 'OPTIONS']
@extend_schema_if_available(extensions={"x-ai-description": "Retrieves a list of available bulk actions"})
def get(self, request, format=None): def get(self, request, format=None):
'''List top level resources''' '''List top level resources'''
data = OrderedDict() data = OrderedDict()
@@ -45,11 +48,13 @@ class BulkJobLaunchView(GenericAPIView):
serializer_class = serializers.BulkJobLaunchSerializer serializer_class = serializers.BulkJobLaunchSerializer
allowed_methods = ['GET', 'POST', 'OPTIONS'] allowed_methods = ['GET', 'POST', 'OPTIONS']
@extend_schema_if_available(extensions={"x-ai-description": "Get information about bulk job launch endpoint"})
def get(self, request): def get(self, request):
data = OrderedDict() data = OrderedDict()
data['detail'] = "Specify a list of unified job templates to launch alongside their launchtime parameters" data['detail'] = "Specify a list of unified job templates to launch alongside their launchtime parameters"
return Response(data, status=status.HTTP_200_OK) return Response(data, status=status.HTTP_200_OK)
@extend_schema_if_available(extensions={"x-ai-description": "Bulk launch job templates"})
def post(self, request): def post(self, request):
bulkjob_serializer = serializers.BulkJobLaunchSerializer(data=request.data, context={'request': request}) bulkjob_serializer = serializers.BulkJobLaunchSerializer(data=request.data, context={'request': request})
if bulkjob_serializer.is_valid(): if bulkjob_serializer.is_valid():
@@ -64,9 +69,11 @@ class BulkHostCreateView(GenericAPIView):
serializer_class = serializers.BulkHostCreateSerializer serializer_class = serializers.BulkHostCreateSerializer
allowed_methods = ['GET', 'POST', 'OPTIONS'] allowed_methods = ['GET', 'POST', 'OPTIONS']
@extend_schema_if_available(extensions={"x-ai-description": "Get information about bulk host create endpoint"})
def get(self, request): def get(self, request):
return Response({"detail": "Bulk create hosts with this endpoint"}, status=status.HTTP_200_OK) return Response({"detail": "Bulk create hosts with this endpoint"}, status=status.HTTP_200_OK)
@extend_schema_if_available(extensions={"x-ai-description": "Bulk create hosts"})
def post(self, request): def post(self, request):
serializer = serializers.BulkHostCreateSerializer(data=request.data, context={'request': request}) serializer = serializers.BulkHostCreateSerializer(data=request.data, context={'request': request})
if serializer.is_valid(): if serializer.is_valid():
@@ -81,9 +88,11 @@ class BulkHostDeleteView(GenericAPIView):
serializer_class = serializers.BulkHostDeleteSerializer serializer_class = serializers.BulkHostDeleteSerializer
allowed_methods = ['GET', 'POST', 'OPTIONS'] allowed_methods = ['GET', 'POST', 'OPTIONS']
@extend_schema_if_available(extensions={"x-ai-description": "Get information about bulk host delete endpoint"})
def get(self, request): def get(self, request):
return Response({"detail": "Bulk delete hosts with this endpoint"}, status=status.HTTP_200_OK) return Response({"detail": "Bulk delete hosts with this endpoint"}, status=status.HTTP_200_OK)
@extend_schema_if_available(extensions={"x-ai-description": "Bulk delete hosts"})
def post(self, request): def post(self, request):
serializer = serializers.BulkHostDeleteSerializer(data=request.data, context={'request': request}) serializer = serializers.BulkHostDeleteSerializer(data=request.data, context={'request': request})
if serializer.is_valid(): if serializer.is_valid():

View File

@@ -5,6 +5,7 @@ from django.conf import settings
from rest_framework.permissions import AllowAny from rest_framework.permissions import AllowAny
from rest_framework.response import Response from rest_framework.response import Response
from awx.api.generics import APIView from awx.api.generics import APIView
from ansible_base.lib.utils.schema import extend_schema_if_available
from awx.main.scheduler import TaskManager, DependencyManager, WorkflowManager from awx.main.scheduler import TaskManager, DependencyManager, WorkflowManager
@@ -14,7 +15,9 @@ class TaskManagerDebugView(APIView):
exclude_from_schema = True exclude_from_schema = True
permission_classes = [AllowAny] permission_classes = [AllowAny]
prefix = 'Task' prefix = 'Task'
resource_purpose = 'debug task manager'
@extend_schema_if_available(extensions={"x-ai-description": "Trigger task manager scheduling"})
def get(self, request): def get(self, request):
TaskManager().schedule() TaskManager().schedule()
if not settings.AWX_DISABLE_TASK_MANAGERS: if not settings.AWX_DISABLE_TASK_MANAGERS:
@@ -29,7 +32,9 @@ class DependencyManagerDebugView(APIView):
exclude_from_schema = True exclude_from_schema = True
permission_classes = [AllowAny] permission_classes = [AllowAny]
prefix = 'Dependency' prefix = 'Dependency'
resource_purpose = 'debug dependency manager'
@extend_schema_if_available(extensions={"x-ai-description": "Trigger dependency manager scheduling"})
def get(self, request): def get(self, request):
DependencyManager().schedule() DependencyManager().schedule()
if not settings.AWX_DISABLE_TASK_MANAGERS: if not settings.AWX_DISABLE_TASK_MANAGERS:
@@ -44,7 +49,9 @@ class WorkflowManagerDebugView(APIView):
exclude_from_schema = True exclude_from_schema = True
permission_classes = [AllowAny] permission_classes = [AllowAny]
prefix = 'Workflow' prefix = 'Workflow'
resource_purpose = 'debug workflow manager'
@extend_schema_if_available(extensions={"x-ai-description": "Trigger workflow manager scheduling"})
def get(self, request): def get(self, request):
WorkflowManager().schedule() WorkflowManager().schedule()
if not settings.AWX_DISABLE_TASK_MANAGERS: if not settings.AWX_DISABLE_TASK_MANAGERS:
@@ -58,7 +65,9 @@ class DebugRootView(APIView):
_ignore_model_permissions = True _ignore_model_permissions = True
exclude_from_schema = True exclude_from_schema = True
permission_classes = [AllowAny] permission_classes = [AllowAny]
resource_purpose = 'debug endpoints root'
@extend_schema_if_available(extensions={"x-ai-description": "List available debug endpoints"})
def get(self, request, format=None): def get(self, request, format=None):
'''List of available debug urls''' '''List of available debug urls'''
data = OrderedDict() data = OrderedDict()

View File

@@ -10,6 +10,7 @@ import time
import re import re
import asn1 import asn1
from ansible_base.lib.utils.schema import extend_schema_if_available
from awx.api import serializers from awx.api import serializers
from awx.api.generics import GenericAPIView, Response from awx.api.generics import GenericAPIView, Response
from awx.api.permissions import IsSystemAdmin from awx.api.permissions import IsSystemAdmin
@@ -49,7 +50,9 @@ class InstanceInstallBundle(GenericAPIView):
model = models.Instance model = models.Instance
serializer_class = serializers.InstanceSerializer serializer_class = serializers.InstanceSerializer
permission_classes = (IsSystemAdmin,) permission_classes = (IsSystemAdmin,)
resource_purpose = 'install bundle'
@extend_schema_if_available(extensions={"x-ai-description": "Generate and download install bundle for an instance"})
def get(self, request, *args, **kwargs): def get(self, request, *args, **kwargs):
instance_obj = self.get_object() instance_obj = self.get_object()
@@ -195,8 +198,8 @@ def generate_receptor_tls(instance_obj):
.issuer_name(ca_cert.issuer) .issuer_name(ca_cert.issuer)
.public_key(csr.public_key()) .public_key(csr.public_key())
.serial_number(x509.random_serial_number()) .serial_number(x509.random_serial_number())
.not_valid_before(datetime.datetime.utcnow()) .not_valid_before(datetime.datetime.now(datetime.UTC))
.not_valid_after(datetime.datetime.utcnow() + datetime.timedelta(days=3650)) .not_valid_after(datetime.datetime.now(datetime.UTC) + datetime.timedelta(days=3650))
.add_extension( .add_extension(
csr.extensions.get_extension_for_class(x509.SubjectAlternativeName).value, csr.extensions.get_extension_for_class(x509.SubjectAlternativeName).value,
critical=csr.extensions.get_extension_for_class(x509.SubjectAlternativeName).critical, critical=csr.extensions.get_extension_for_class(x509.SubjectAlternativeName).critical,

View File

@@ -19,6 +19,8 @@ from rest_framework import serializers
# AWX # AWX
from awx.main.models import ActivityStream, Inventory, JobTemplate, Role, User, InstanceGroup, InventoryUpdateEvent, InventoryUpdate from awx.main.models import ActivityStream, Inventory, JobTemplate, Role, User, InstanceGroup, InventoryUpdateEvent, InventoryUpdate
from ansible_base.lib.utils.schema import extend_schema_if_available
from awx.api.generics import ( from awx.api.generics import (
ListCreateAPIView, ListCreateAPIView,
RetrieveUpdateDestroyAPIView, RetrieveUpdateDestroyAPIView,
@@ -43,7 +45,6 @@ from awx.api.views.mixin import RelatedJobsPreventDeleteMixin
from awx.api.pagination import UnifiedJobEventPagination from awx.api.pagination import UnifiedJobEventPagination
logger = logging.getLogger('awx.api.views.organization') logger = logging.getLogger('awx.api.views.organization')
@@ -55,6 +56,7 @@ class InventoryUpdateEventsList(SubListAPIView):
name = _('Inventory Update Events List') name = _('Inventory Update Events List')
search_fields = ('stdout',) search_fields = ('stdout',)
pagination_class = UnifiedJobEventPagination pagination_class = UnifiedJobEventPagination
resource_purpose = 'events of an inventory update'
def get_queryset(self): def get_queryset(self):
iu = self.get_parent_object() iu = self.get_parent_object()
@@ -69,11 +71,17 @@ class InventoryUpdateEventsList(SubListAPIView):
class InventoryList(ListCreateAPIView): class InventoryList(ListCreateAPIView):
model = Inventory model = Inventory
serializer_class = InventorySerializer serializer_class = InventorySerializer
resource_purpose = 'inventories'
@extend_schema_if_available(extensions={"x-ai-description": "A list of inventories."})
def get(self, request, *args, **kwargs):
return super().get(request, *args, **kwargs)
class InventoryDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView): class InventoryDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView):
model = Inventory model = Inventory
serializer_class = InventorySerializer serializer_class = InventorySerializer
resource_purpose = 'inventory detail'
def update(self, request, *args, **kwargs): def update(self, request, *args, **kwargs):
obj = self.get_object() obj = self.get_object()
@@ -100,33 +108,39 @@ class InventoryDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIVie
class ConstructedInventoryDetail(InventoryDetail): class ConstructedInventoryDetail(InventoryDetail):
serializer_class = ConstructedInventorySerializer serializer_class = ConstructedInventorySerializer
resource_purpose = 'constructed inventory detail'
class ConstructedInventoryList(InventoryList): class ConstructedInventoryList(InventoryList):
serializer_class = ConstructedInventorySerializer serializer_class = ConstructedInventorySerializer
resource_purpose = 'constructed inventories'
def get_queryset(self): def get_queryset(self):
r = super().get_queryset() r = super().get_queryset()
return r.filter(kind='constructed') return r.filter(kind='constructed')
@extend_schema_if_available(extensions={"x-ai-description": "Get or create input inventory inventory"})
class InventoryInputInventoriesList(SubListAttachDetachAPIView): class InventoryInputInventoriesList(SubListAttachDetachAPIView):
model = Inventory model = Inventory
serializer_class = InventorySerializer serializer_class = InventorySerializer
parent_model = Inventory parent_model = Inventory
relationship = 'input_inventories' relationship = 'input_inventories'
resource_purpose = 'input inventories of a constructed inventory'
def is_valid_relation(self, parent, sub, created=False): def is_valid_relation(self, parent, sub, created=False):
if sub.kind == 'constructed': if sub.kind == 'constructed':
raise serializers.ValidationError({'error': 'You cannot add a constructed inventory to another constructed inventory.'}) raise serializers.ValidationError({'error': 'You cannot add a constructed inventory to another constructed inventory.'})
@extend_schema_if_available(extensions={"x-ai-description": "Get activity stream for an inventory"})
class InventoryActivityStreamList(SubListAPIView): class InventoryActivityStreamList(SubListAPIView):
model = ActivityStream model = ActivityStream
serializer_class = ActivityStreamSerializer serializer_class = ActivityStreamSerializer
parent_model = Inventory parent_model = Inventory
relationship = 'activitystream_set' relationship = 'activitystream_set'
search_fields = ('changes',) search_fields = ('changes',)
resource_purpose = 'activity stream for an inventory'
def get_queryset(self): def get_queryset(self):
parent = self.get_parent_object() parent = self.get_parent_object()
@@ -140,11 +154,13 @@ class InventoryInstanceGroupsList(SubListAttachDetachAPIView):
serializer_class = InstanceGroupSerializer serializer_class = InstanceGroupSerializer
parent_model = Inventory parent_model = Inventory
relationship = 'instance_groups' relationship = 'instance_groups'
resource_purpose = 'instance groups of an inventory'
class InventoryAccessList(ResourceAccessList): class InventoryAccessList(ResourceAccessList):
model = User # needs to be User for AccessLists's model = User # needs to be User for AccessLists's
parent_model = Inventory parent_model = Inventory
resource_purpose = 'users who can access the inventory'
class InventoryObjectRolesList(SubListAPIView): class InventoryObjectRolesList(SubListAPIView):
@@ -153,6 +169,7 @@ class InventoryObjectRolesList(SubListAPIView):
parent_model = Inventory parent_model = Inventory
search_fields = ('role_field', 'content_type__model') search_fields = ('role_field', 'content_type__model')
deprecated = True deprecated = True
resource_purpose = 'roles of an inventory'
def get_queryset(self): def get_queryset(self):
po = self.get_parent_object() po = self.get_parent_object()
@@ -165,6 +182,7 @@ class InventoryJobTemplateList(SubListAPIView):
serializer_class = JobTemplateSerializer serializer_class = JobTemplateSerializer
parent_model = Inventory parent_model = Inventory
relationship = 'jobtemplates' relationship = 'jobtemplates'
resource_purpose = 'job templates using an inventory'
def get_queryset(self): def get_queryset(self):
parent = self.get_parent_object() parent = self.get_parent_object()
@@ -175,8 +193,10 @@ class InventoryJobTemplateList(SubListAPIView):
class InventoryLabelList(LabelSubListCreateAttachDetachView): class InventoryLabelList(LabelSubListCreateAttachDetachView):
parent_model = Inventory parent_model = Inventory
resource_purpose = 'labels of an inventory'
class InventoryCopy(CopyAPIView): class InventoryCopy(CopyAPIView):
model = Inventory model = Inventory
copy_return_serializer_class = InventorySerializer copy_return_serializer_class = InventorySerializer
resource_purpose = 'copy of an inventory'

View File

@@ -2,6 +2,7 @@
from awx.api.generics import SubListCreateAttachDetachAPIView, RetrieveUpdateAPIView, ListCreateAPIView from awx.api.generics import SubListCreateAttachDetachAPIView, RetrieveUpdateAPIView, ListCreateAPIView
from awx.main.models import Label from awx.main.models import Label
from awx.api.serializers import LabelSerializer from awx.api.serializers import LabelSerializer
from ansible_base.lib.utils.schema import extend_schema_if_available
# Django # Django
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
@@ -24,9 +25,10 @@ class LabelSubListCreateAttachDetachView(SubListCreateAttachDetachAPIView):
model = Label model = Label
serializer_class = LabelSerializer serializer_class = LabelSerializer
relationship = 'labels' relationship = 'labels'
resource_purpose = 'labels of a resource'
def unattach(self, request, *args, **kwargs): def unattach(self, request, *args, **kwargs):
(sub_id, res) = super().unattach_validate(request) sub_id, res = super().unattach_validate(request)
if res: if res:
return res return res
@@ -39,6 +41,7 @@ class LabelSubListCreateAttachDetachView(SubListCreateAttachDetachAPIView):
return res return res
@extend_schema_if_available(extensions={"x-ai-description": "Create or attach a label to a resource"})
def post(self, request, *args, **kwargs): def post(self, request, *args, **kwargs):
# If a label already exists in the database, attach it instead of erroring out # If a label already exists in the database, attach it instead of erroring out
# that it already exists # that it already exists
@@ -61,9 +64,11 @@ class LabelSubListCreateAttachDetachView(SubListCreateAttachDetachAPIView):
class LabelDetail(RetrieveUpdateAPIView): class LabelDetail(RetrieveUpdateAPIView):
model = Label model = Label
serializer_class = LabelSerializer serializer_class = LabelSerializer
resource_purpose = 'label detail'
class LabelList(ListCreateAPIView): class LabelList(ListCreateAPIView):
name = _("Labels") name = _("Labels")
model = Label model = Label
serializer_class = LabelSerializer serializer_class = LabelSerializer
resource_purpose = 'labels'

View File

@@ -2,6 +2,7 @@
# All Rights Reserved. # All Rights Reserved.
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
from ansible_base.lib.utils.schema import extend_schema_if_available
from awx.api.generics import APIView, Response from awx.api.generics import APIView, Response
from awx.api.permissions import IsSystemAdminOrAuditor from awx.api.permissions import IsSystemAdminOrAuditor
@@ -13,7 +14,9 @@ class MeshVisualizer(APIView):
name = _("Mesh Visualizer") name = _("Mesh Visualizer")
permission_classes = (IsSystemAdminOrAuditor,) permission_classes = (IsSystemAdminOrAuditor,)
swagger_topic = "System Configuration" swagger_topic = "System Configuration"
resource_purpose = 'mesh network topology visualization data'
@extend_schema_if_available(extensions={"x-ai-description": "Get mesh network topology visualization data"})
def get(self, request, format=None): def get(self, request, format=None):
data = { data = {
'nodes': InstanceNodeSerializer(Instance.objects.all(), many=True).data, 'nodes': InstanceNodeSerializer(Instance.objects.all(), many=True).data,

View File

@@ -7,13 +7,13 @@ import logging
# Django # Django
from django.conf import settings from django.conf import settings
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
from ansible_base.lib.utils.schema import extend_schema_if_available
# Django REST Framework # Django REST Framework
from rest_framework.permissions import AllowAny from rest_framework.permissions import AllowAny
from rest_framework.response import Response from rest_framework.response import Response
from rest_framework.exceptions import PermissionDenied from rest_framework.exceptions import PermissionDenied
# AWX # AWX
# from awx.main.analytics import collectors # from awx.main.analytics import collectors
import awx.main.analytics.subsystem_metrics as s_metrics import awx.main.analytics.subsystem_metrics as s_metrics
@@ -22,13 +22,13 @@ from awx.api import renderers
from awx.api.generics import APIView from awx.api.generics import APIView
logger = logging.getLogger('awx.analytics') logger = logging.getLogger('awx.analytics')
class MetricsView(APIView): class MetricsView(APIView):
name = _('Metrics') name = _('Metrics')
swagger_topic = 'Metrics' swagger_topic = 'Metrics'
resource_purpose = 'prometheus metrics data'
renderer_classes = [renderers.PlainTextRenderer, renderers.PrometheusJSONRenderer, renderers.BrowsableAPIRenderer] renderer_classes = [renderers.PlainTextRenderer, renderers.PrometheusJSONRenderer, renderers.BrowsableAPIRenderer]
@@ -37,6 +37,7 @@ class MetricsView(APIView):
self.permission_classes = (AllowAny,) self.permission_classes = (AllowAny,)
return super(APIView, self).initialize_request(request, *args, **kwargs) return super(APIView, self).initialize_request(request, *args, **kwargs)
@extend_schema_if_available(extensions={"x-ai-description": "Get Prometheus metrics data"})
def get(self, request): def get(self, request):
'''Show Metrics Details''' '''Show Metrics Details'''
if settings.ALLOW_METRICS_FOR_ANONYMOUS_USERS or request.user.is_superuser or request.user.is_system_auditor: if settings.ALLOW_METRICS_FOR_ANONYMOUS_USERS or request.user.is_superuser or request.user.is_system_auditor:

View File

@@ -60,11 +60,13 @@ logger = logging.getLogger('awx.api.views.organization')
class OrganizationList(OrganizationCountsMixin, ListCreateAPIView): class OrganizationList(OrganizationCountsMixin, ListCreateAPIView):
model = Organization model = Organization
serializer_class = OrganizationSerializer serializer_class = OrganizationSerializer
resource_purpose = 'organizations'
class OrganizationDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView): class OrganizationDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView):
model = Organization model = Organization
serializer_class = OrganizationSerializer serializer_class = OrganizationSerializer
resource_purpose = 'organization detail'
def get_serializer_context(self, *args, **kwargs): def get_serializer_context(self, *args, **kwargs):
full_context = super(OrganizationDetail, self).get_serializer_context(*args, **kwargs) full_context = super(OrganizationDetail, self).get_serializer_context(*args, **kwargs)
@@ -102,6 +104,7 @@ class OrganizationInventoriesList(SubListAPIView):
serializer_class = InventorySerializer serializer_class = InventorySerializer
parent_model = Organization parent_model = Organization
relationship = 'inventories' relationship = 'inventories'
resource_purpose = 'inventories of an organization'
class OrganizationUsersList(BaseUsersList): class OrganizationUsersList(BaseUsersList):
@@ -110,6 +113,7 @@ class OrganizationUsersList(BaseUsersList):
parent_model = Organization parent_model = Organization
relationship = 'member_role.members' relationship = 'member_role.members'
ordering = ('username',) ordering = ('username',)
resource_purpose = 'users of an organization'
class OrganizationAdminsList(BaseUsersList): class OrganizationAdminsList(BaseUsersList):
@@ -118,6 +122,7 @@ class OrganizationAdminsList(BaseUsersList):
parent_model = Organization parent_model = Organization
relationship = 'admin_role.members' relationship = 'admin_role.members'
ordering = ('username',) ordering = ('username',)
resource_purpose = 'administrators of an organization'
class OrganizationProjectsList(SubListCreateAPIView): class OrganizationProjectsList(SubListCreateAPIView):
@@ -125,6 +130,7 @@ class OrganizationProjectsList(SubListCreateAPIView):
serializer_class = ProjectSerializer serializer_class = ProjectSerializer
parent_model = Organization parent_model = Organization
parent_key = 'organization' parent_key = 'organization'
resource_purpose = 'projects of an organization'
class OrganizationExecutionEnvironmentsList(SubListCreateAttachDetachAPIView): class OrganizationExecutionEnvironmentsList(SubListCreateAttachDetachAPIView):
@@ -134,6 +140,7 @@ class OrganizationExecutionEnvironmentsList(SubListCreateAttachDetachAPIView):
relationship = 'executionenvironments' relationship = 'executionenvironments'
parent_key = 'organization' parent_key = 'organization'
swagger_topic = "Execution Environments" swagger_topic = "Execution Environments"
resource_purpose = 'execution environments of an organization'
class OrganizationJobTemplatesList(SubListCreateAPIView): class OrganizationJobTemplatesList(SubListCreateAPIView):
@@ -141,6 +148,7 @@ class OrganizationJobTemplatesList(SubListCreateAPIView):
serializer_class = JobTemplateSerializer serializer_class = JobTemplateSerializer
parent_model = Organization parent_model = Organization
parent_key = 'organization' parent_key = 'organization'
resource_purpose = 'job templates of an organization'
class OrganizationWorkflowJobTemplatesList(SubListCreateAPIView): class OrganizationWorkflowJobTemplatesList(SubListCreateAPIView):
@@ -148,6 +156,7 @@ class OrganizationWorkflowJobTemplatesList(SubListCreateAPIView):
serializer_class = WorkflowJobTemplateSerializer serializer_class = WorkflowJobTemplateSerializer
parent_model = Organization parent_model = Organization
parent_key = 'organization' parent_key = 'organization'
resource_purpose = 'workflow job templates of an organization'
class OrganizationTeamsList(SubListCreateAttachDetachAPIView): class OrganizationTeamsList(SubListCreateAttachDetachAPIView):
@@ -156,6 +165,7 @@ class OrganizationTeamsList(SubListCreateAttachDetachAPIView):
parent_model = Organization parent_model = Organization
relationship = 'teams' relationship = 'teams'
parent_key = 'organization' parent_key = 'organization'
resource_purpose = 'teams of an organization'
class OrganizationActivityStreamList(SubListAPIView): class OrganizationActivityStreamList(SubListAPIView):
@@ -164,6 +174,7 @@ class OrganizationActivityStreamList(SubListAPIView):
parent_model = Organization parent_model = Organization
relationship = 'activitystream_set' relationship = 'activitystream_set'
search_fields = ('changes',) search_fields = ('changes',)
resource_purpose = 'activity stream for an organization'
class OrganizationNotificationTemplatesList(SubListCreateAttachDetachAPIView): class OrganizationNotificationTemplatesList(SubListCreateAttachDetachAPIView):
@@ -172,28 +183,34 @@ class OrganizationNotificationTemplatesList(SubListCreateAttachDetachAPIView):
parent_model = Organization parent_model = Organization
relationship = 'notification_templates' relationship = 'notification_templates'
parent_key = 'organization' parent_key = 'organization'
resource_purpose = 'notification templates of an organization'
class OrganizationNotificationTemplatesAnyList(SubListCreateAttachDetachAPIView): class OrganizationNotificationTemplatesAnyList(SubListCreateAttachDetachAPIView):
model = NotificationTemplate model = NotificationTemplate
serializer_class = NotificationTemplateSerializer serializer_class = NotificationTemplateSerializer
parent_model = Organization parent_model = Organization
resource_purpose = 'base view for notification templates of an organization'
class OrganizationNotificationTemplatesStartedList(OrganizationNotificationTemplatesAnyList): class OrganizationNotificationTemplatesStartedList(OrganizationNotificationTemplatesAnyList):
relationship = 'notification_templates_started' relationship = 'notification_templates_started'
resource_purpose = 'notification templates for job started events of an organization'
class OrganizationNotificationTemplatesErrorList(OrganizationNotificationTemplatesAnyList): class OrganizationNotificationTemplatesErrorList(OrganizationNotificationTemplatesAnyList):
relationship = 'notification_templates_error' relationship = 'notification_templates_error'
resource_purpose = 'notification templates for job error events of an organization'
class OrganizationNotificationTemplatesSuccessList(OrganizationNotificationTemplatesAnyList): class OrganizationNotificationTemplatesSuccessList(OrganizationNotificationTemplatesAnyList):
relationship = 'notification_templates_success' relationship = 'notification_templates_success'
resource_purpose = 'notification templates for job success events of an organization'
class OrganizationNotificationTemplatesApprovalList(OrganizationNotificationTemplatesAnyList): class OrganizationNotificationTemplatesApprovalList(OrganizationNotificationTemplatesAnyList):
relationship = 'notification_templates_approvals' relationship = 'notification_templates_approvals'
resource_purpose = 'notification templates for workflow approval events of an organization'
class OrganizationInstanceGroupsList(OrganizationInstanceGroupMembershipMixin, SubListAttachDetachAPIView): class OrganizationInstanceGroupsList(OrganizationInstanceGroupMembershipMixin, SubListAttachDetachAPIView):
@@ -202,6 +219,7 @@ class OrganizationInstanceGroupsList(OrganizationInstanceGroupMembershipMixin, S
parent_model = Organization parent_model = Organization
relationship = 'instance_groups' relationship = 'instance_groups'
filter_read_permission = False filter_read_permission = False
resource_purpose = 'instance groups of an organization'
class OrganizationGalaxyCredentialsList(SubListAttachDetachAPIView): class OrganizationGalaxyCredentialsList(SubListAttachDetachAPIView):
@@ -210,6 +228,7 @@ class OrganizationGalaxyCredentialsList(SubListAttachDetachAPIView):
parent_model = Organization parent_model = Organization
relationship = 'galaxy_credentials' relationship = 'galaxy_credentials'
filter_read_permission = False filter_read_permission = False
resource_purpose = 'galaxy credentials of an organization'
def is_valid_relation(self, parent, sub, created=False): def is_valid_relation(self, parent, sub, created=False):
if sub.kind != 'galaxy_api_token': if sub.kind != 'galaxy_api_token':
@@ -219,6 +238,7 @@ class OrganizationGalaxyCredentialsList(SubListAttachDetachAPIView):
class OrganizationAccessList(ResourceAccessList): class OrganizationAccessList(ResourceAccessList):
model = User # needs to be User for AccessLists's model = User # needs to be User for AccessLists's
parent_model = Organization parent_model = Organization
resource_purpose = 'users who can access the organization'
class OrganizationObjectRolesList(SubListAPIView): class OrganizationObjectRolesList(SubListAPIView):
@@ -227,6 +247,7 @@ class OrganizationObjectRolesList(SubListAPIView):
parent_model = Organization parent_model = Organization
search_fields = ('role_field', 'content_type__model') search_fields = ('role_field', 'content_type__model')
deprecated = True deprecated = True
resource_purpose = 'roles of an organization'
def get_queryset(self): def get_queryset(self):
po = self.get_parent_object() po = self.get_parent_object()

View File

@@ -23,7 +23,8 @@ from rest_framework import status
import requests import requests
from awx import MODE from ansible_base.lib.utils.schema import extend_schema_if_available
from awx.api.generics import APIView from awx.api.generics import APIView
from awx.conf.registry import settings_registry from awx.conf.registry import settings_registry
from awx.main.analytics import all_collectors from awx.main.analytics import all_collectors
@@ -31,7 +32,7 @@ from awx.main.ha import is_ha_environment
from awx.main.tasks.system import clear_setting_cache from awx.main.tasks.system import clear_setting_cache
from awx.main.utils import get_awx_version, get_custom_venv_choices from awx.main.utils import get_awx_version, get_custom_venv_choices
from awx.main.utils.licensing import validate_entitlement_manifest from awx.main.utils.licensing import validate_entitlement_manifest
from awx.api.versioning import URLPathVersioning, reverse, drf_reverse from awx.api.versioning import URLPathVersioning, reverse
from awx.main.constants import PRIVILEGE_ESCALATION_METHODS from awx.main.constants import PRIVILEGE_ESCALATION_METHODS
from awx.main.models import Project, Organization, Instance, InstanceGroup, JobTemplate from awx.main.models import Project, Organization, Instance, InstanceGroup, JobTemplate
from awx.main.utils import set_environ from awx.main.utils import set_environ
@@ -46,8 +47,10 @@ class ApiRootView(APIView):
name = _('REST API') name = _('REST API')
versioning_class = URLPathVersioning versioning_class = URLPathVersioning
swagger_topic = 'Versioning' swagger_topic = 'Versioning'
resource_purpose = 'api root and version information'
@method_decorator(ensure_csrf_cookie) @method_decorator(ensure_csrf_cookie)
@extend_schema_if_available(extensions={"x-ai-description": "List supported API versions"})
def get(self, request, format=None): def get(self, request, format=None):
'''List supported API versions''' '''List supported API versions'''
v2 = reverse('api:api_v2_root_view', request=request, kwargs={'version': 'v2'}) v2 = reverse('api:api_v2_root_view', request=request, kwargs={'version': 'v2'})
@@ -58,15 +61,15 @@ class ApiRootView(APIView):
data['custom_logo'] = settings.CUSTOM_LOGO data['custom_logo'] = settings.CUSTOM_LOGO
data['custom_login_info'] = settings.CUSTOM_LOGIN_INFO data['custom_login_info'] = settings.CUSTOM_LOGIN_INFO
data['login_redirect_override'] = settings.LOGIN_REDIRECT_OVERRIDE data['login_redirect_override'] = settings.LOGIN_REDIRECT_OVERRIDE
if MODE == 'development':
data['swagger'] = drf_reverse('api:schema-swagger-ui')
return Response(data) return Response(data)
class ApiVersionRootView(APIView): class ApiVersionRootView(APIView):
permission_classes = (AllowAny,) permission_classes = (AllowAny,)
swagger_topic = 'Versioning' swagger_topic = 'Versioning'
resource_purpose = 'api top-level resources'
@extend_schema_if_available(extensions={"x-ai-description": "List top-level API resources"})
def get(self, request, format=None): def get(self, request, format=None):
'''List top level resources''' '''List top level resources'''
data = OrderedDict() data = OrderedDict()
@@ -126,6 +129,7 @@ class ApiVersionRootView(APIView):
class ApiV2RootView(ApiVersionRootView): class ApiV2RootView(ApiVersionRootView):
name = _('Version 2') name = _('Version 2')
resource_purpose = 'api v2 root'
class ApiV2PingView(APIView): class ApiV2PingView(APIView):
@@ -137,7 +141,11 @@ class ApiV2PingView(APIView):
authentication_classes = () authentication_classes = ()
name = _('Ping') name = _('Ping')
swagger_topic = 'System Configuration' swagger_topic = 'System Configuration'
resource_purpose = 'basic instance information'
@extend_schema_if_available(
extensions={'x-ai-description': 'Return basic information about this instance'},
)
def get(self, request, format=None): def get(self, request, format=None):
"""Return some basic information about this instance """Return some basic information about this instance
@@ -172,12 +180,16 @@ class ApiV2SubscriptionView(APIView):
permission_classes = (IsAuthenticated,) permission_classes = (IsAuthenticated,)
name = _('Subscriptions') name = _('Subscriptions')
swagger_topic = 'System Configuration' swagger_topic = 'System Configuration'
resource_purpose = 'aap subscription validation'
def check_permissions(self, request): def check_permissions(self, request):
super(ApiV2SubscriptionView, self).check_permissions(request) super(ApiV2SubscriptionView, self).check_permissions(request)
if not request.user.is_superuser and request.method.lower() not in {'options', 'head'}: if not request.user.is_superuser and request.method.lower() not in {'options', 'head'}:
self.permission_denied(request) # Raises PermissionDenied exception. self.permission_denied(request) # Raises PermissionDenied exception.
@extend_schema_if_available(
extensions={'x-ai-description': 'List valid AAP subscriptions'},
)
def post(self, request): def post(self, request):
data = request.data.copy() data = request.data.copy()
@@ -244,12 +256,16 @@ class ApiV2AttachView(APIView):
permission_classes = (IsAuthenticated,) permission_classes = (IsAuthenticated,)
name = _('Attach Subscription') name = _('Attach Subscription')
swagger_topic = 'System Configuration' swagger_topic = 'System Configuration'
resource_purpose = 'subscription attachment'
def check_permissions(self, request): def check_permissions(self, request):
super(ApiV2AttachView, self).check_permissions(request) super(ApiV2AttachView, self).check_permissions(request)
if not request.user.is_superuser and request.method.lower() not in {'options', 'head'}: if not request.user.is_superuser and request.method.lower() not in {'options', 'head'}:
self.permission_denied(request) # Raises PermissionDenied exception. self.permission_denied(request) # Raises PermissionDenied exception.
@extend_schema_if_available(
extensions={'x-ai-description': 'Attach a subscription'},
)
def post(self, request): def post(self, request):
data = request.data.copy() data = request.data.copy()
subscription_id = data.get('subscription_id', None) subscription_id = data.get('subscription_id', None)
@@ -299,12 +315,16 @@ class ApiV2ConfigView(APIView):
permission_classes = (IsAuthenticated,) permission_classes = (IsAuthenticated,)
name = _('Configuration') name = _('Configuration')
swagger_topic = 'System Configuration' swagger_topic = 'System Configuration'
resource_purpose = 'system configuration and license management'
def check_permissions(self, request): def check_permissions(self, request):
super(ApiV2ConfigView, self).check_permissions(request) super(ApiV2ConfigView, self).check_permissions(request)
if not request.user.is_superuser and request.method.lower() not in {'options', 'head', 'get'}: if not request.user.is_superuser and request.method.lower() not in {'options', 'head', 'get'}:
self.permission_denied(request) # Raises PermissionDenied exception. self.permission_denied(request) # Raises PermissionDenied exception.
@extend_schema_if_available(
extensions={'x-ai-description': 'Return various configuration settings'},
)
def get(self, request, format=None): def get(self, request, format=None):
'''Return various sitewide configuration settings''' '''Return various sitewide configuration settings'''
@@ -324,13 +344,22 @@ class ApiV2ConfigView(APIView):
become_methods=PRIVILEGE_ESCALATION_METHODS, become_methods=PRIVILEGE_ESCALATION_METHODS,
) )
if ( # Check superuser/auditor first
request.user.is_superuser if request.user.is_superuser or request.user.is_system_auditor:
or request.user.is_system_auditor has_org_access = True
or Organization.accessible_objects(request.user, 'admin_role').exists() else:
or Organization.accessible_objects(request.user, 'auditor_role').exists() # Single query checking all three organization role types at once
or Organization.accessible_objects(request.user, 'project_admin_role').exists() has_org_access = (
): (
Organization.access_qs(request.user, 'change')
| Organization.access_qs(request.user, 'audit')
| Organization.access_qs(request.user, 'add_project')
)
.distinct()
.exists()
)
if has_org_access:
data.update( data.update(
dict( dict(
project_base_dir=settings.PROJECTS_ROOT, project_base_dir=settings.PROJECTS_ROOT,
@@ -338,11 +367,14 @@ class ApiV2ConfigView(APIView):
custom_virtualenvs=get_custom_venv_choices(), custom_virtualenvs=get_custom_venv_choices(),
) )
) )
elif JobTemplate.accessible_objects(request.user, 'admin_role').exists(): else:
data['custom_virtualenvs'] = get_custom_venv_choices() # Only check JobTemplate access if org check failed
if JobTemplate.accessible_objects(request.user, 'admin_role').exists():
data['custom_virtualenvs'] = get_custom_venv_choices()
return Response(data) return Response(data)
@extend_schema_if_available(extensions={"x-ai-description": "Add or update a subscription manifest license"})
def post(self, request): def post(self, request):
if not isinstance(request.data, dict): if not isinstance(request.data, dict):
return Response({"error": _("Invalid subscription data")}, status=status.HTTP_400_BAD_REQUEST) return Response({"error": _("Invalid subscription data")}, status=status.HTTP_400_BAD_REQUEST)
@@ -388,6 +420,9 @@ class ApiV2ConfigView(APIView):
logger.warning(smart_str(u"Invalid subscription submitted."), extra=dict(actor=request.user.username)) logger.warning(smart_str(u"Invalid subscription submitted."), extra=dict(actor=request.user.username))
return Response({"error": _("Invalid subscription")}, status=status.HTTP_400_BAD_REQUEST) return Response({"error": _("Invalid subscription")}, status=status.HTTP_400_BAD_REQUEST)
@extend_schema_if_available(
extensions={'x-ai-description': 'Remove the current subscription'},
)
def delete(self, request): def delete(self, request):
try: try:
settings.LICENSE = {} settings.LICENSE = {}

View File

@@ -11,6 +11,7 @@ from rest_framework import status
from rest_framework.exceptions import PermissionDenied from rest_framework.exceptions import PermissionDenied
from rest_framework.permissions import AllowAny from rest_framework.permissions import AllowAny
from rest_framework.response import Response from rest_framework.response import Response
from ansible_base.lib.utils.schema import extend_schema_if_available
from awx.api import serializers from awx.api import serializers
from awx.api.generics import APIView, GenericAPIView from awx.api.generics import APIView, GenericAPIView
@@ -24,6 +25,7 @@ logger = logging.getLogger('awx.api.views.webhooks')
class WebhookKeyView(GenericAPIView): class WebhookKeyView(GenericAPIView):
serializer_class = serializers.EmptySerializer serializer_class = serializers.EmptySerializer
permission_classes = (WebhookKeyPermission,) permission_classes = (WebhookKeyPermission,)
resource_purpose = 'webhook key management'
def get_queryset(self): def get_queryset(self):
qs_models = {'job_templates': JobTemplate, 'workflow_job_templates': WorkflowJobTemplate} qs_models = {'job_templates': JobTemplate, 'workflow_job_templates': WorkflowJobTemplate}
@@ -31,11 +33,13 @@ class WebhookKeyView(GenericAPIView):
return super().get_queryset() return super().get_queryset()
@extend_schema_if_available(extensions={"x-ai-description": "Get the webhook key for a template"})
def get(self, request, *args, **kwargs): def get(self, request, *args, **kwargs):
obj = self.get_object() obj = self.get_object()
return Response({'webhook_key': obj.webhook_key}) return Response({'webhook_key': obj.webhook_key})
@extend_schema_if_available(extensions={"x-ai-description": "Rotate the webhook key for a template"})
def post(self, request, *args, **kwargs): def post(self, request, *args, **kwargs):
obj = self.get_object() obj = self.get_object()
obj.rotate_webhook_key() obj.rotate_webhook_key()
@@ -52,6 +56,7 @@ class WebhookReceiverBase(APIView):
authentication_classes = () authentication_classes = ()
ref_keys = {} ref_keys = {}
resource_purpose = 'webhook receiver for triggering jobs'
def get_queryset(self): def get_queryset(self):
qs_models = {'job_templates': JobTemplate, 'workflow_job_templates': WorkflowJobTemplate} qs_models = {'job_templates': JobTemplate, 'workflow_job_templates': WorkflowJobTemplate}
@@ -127,7 +132,8 @@ class WebhookReceiverBase(APIView):
raise PermissionDenied raise PermissionDenied
@csrf_exempt @csrf_exempt
def post(self, request, *args, **kwargs): @extend_schema_if_available(extensions={"x-ai-description": "Receive a webhook event and trigger a job"})
def post(self, request, *args, **kwargs_in):
# Ensure that the full contents of the request are captured for multiple uses. # Ensure that the full contents of the request are captured for multiple uses.
request.body request.body
@@ -175,6 +181,7 @@ class WebhookReceiverBase(APIView):
class GithubWebhookReceiver(WebhookReceiverBase): class GithubWebhookReceiver(WebhookReceiverBase):
service = 'github' service = 'github'
resource_purpose = 'github webhook receiver'
ref_keys = { ref_keys = {
'pull_request': 'pull_request.head.sha', 'pull_request': 'pull_request.head.sha',
@@ -212,6 +219,7 @@ class GithubWebhookReceiver(WebhookReceiverBase):
class GitlabWebhookReceiver(WebhookReceiverBase): class GitlabWebhookReceiver(WebhookReceiverBase):
service = 'gitlab' service = 'gitlab'
resource_purpose = 'gitlab webhook receiver'
ref_keys = {'Push Hook': 'checkout_sha', 'Tag Push Hook': 'checkout_sha', 'Merge Request Hook': 'object_attributes.last_commit.id'} ref_keys = {'Push Hook': 'checkout_sha', 'Tag Push Hook': 'checkout_sha', 'Merge Request Hook': 'object_attributes.last_commit.id'}
@@ -250,6 +258,7 @@ class GitlabWebhookReceiver(WebhookReceiverBase):
class BitbucketDcWebhookReceiver(WebhookReceiverBase): class BitbucketDcWebhookReceiver(WebhookReceiverBase):
service = 'bitbucket_dc' service = 'bitbucket_dc'
resource_purpose = 'bitbucket data center webhook receiver'
ref_keys = { ref_keys = {
'repo:refs_changed': 'changes.0.toHash', 'repo:refs_changed': 'changes.0.toHash',

View File

@@ -6,7 +6,7 @@ import urllib.parse as urlparse
from collections import OrderedDict from collections import OrderedDict
# Django # Django
from django.core.validators import URLValidator, _lazy_re_compile from django.core.validators import URLValidator, DomainNameValidator, _lazy_re_compile
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
# Django REST Framework # Django REST Framework
@@ -160,10 +160,11 @@ class StringListIsolatedPathField(StringListField):
class URLField(CharField): class URLField(CharField):
# these lines set up a custom regex that allow numbers in the # these lines set up a custom regex that allow numbers in the
# top-level domain # top-level domain
tld_re = ( tld_re = (
r'\.' # dot r'\.' # dot
r'(?!-)' # can't start with a dash r'(?!-)' # can't start with a dash
r'(?:[a-z' + URLValidator.ul + r'0-9' + '-]{2,63}' # domain label, this line was changed from the original URLValidator r'(?:[a-z' + DomainNameValidator.ul + r'0-9' + '-]{2,63}' # domain label, this line was changed from the original URLValidator
r'|xn--[a-z0-9]{1,59})' # or punycode label r'|xn--[a-z0-9]{1,59})' # or punycode label
r'(?<!-)' # can't end with a dash r'(?<!-)' # can't end with a dash
r'\.?' # may have a trailing dot r'\.?' # may have a trailing dot

View File

@@ -5,7 +5,6 @@ from django.urls import re_path
from awx.conf.views import SettingCategoryList, SettingSingletonDetail, SettingLoggingTest from awx.conf.views import SettingCategoryList, SettingSingletonDetail, SettingLoggingTest
urlpatterns = [ urlpatterns = [
re_path(r'^$', SettingCategoryList.as_view(), name='setting_category_list'), re_path(r'^$', SettingCategoryList.as_view(), name='setting_category_list'),
re_path(r'^(?P<category_slug>[a-z0-9-]+)/$', SettingSingletonDetail.as_view(), name='setting_singleton_detail'), re_path(r'^(?P<category_slug>[a-z0-9-]+)/$', SettingSingletonDetail.as_view(), name='setting_singleton_detail'),

View File

@@ -31,7 +31,7 @@ from awx.conf.models import Setting
from awx.conf.serializers import SettingCategorySerializer, SettingSingletonSerializer from awx.conf.serializers import SettingCategorySerializer, SettingSingletonSerializer
from awx.conf import settings_registry from awx.conf import settings_registry
from awx.main.utils.external_logging import reconfigure_rsyslog from awx.main.utils.external_logging import reconfigure_rsyslog
from ansible_base.lib.utils.schema import extend_schema_if_available
SettingCategory = collections.namedtuple('SettingCategory', ('url', 'slug', 'name')) SettingCategory = collections.namedtuple('SettingCategory', ('url', 'slug', 'name'))
@@ -42,6 +42,10 @@ class SettingCategoryList(ListAPIView):
filter_backends = [] filter_backends = []
name = _('Setting Categories') name = _('Setting Categories')
@extend_schema_if_available(extensions={"x-ai-description": "A list of additional API endpoints related to settings."})
def get(self, request, *args, **kwargs):
return super().get(request, *args, **kwargs)
def get_queryset(self): def get_queryset(self):
setting_categories = [] setting_categories = []
categories = settings_registry.get_registered_categories() categories = settings_registry.get_registered_categories()
@@ -63,6 +67,10 @@ class SettingSingletonDetail(RetrieveUpdateDestroyAPIView):
filter_backends = [] filter_backends = []
name = _('Setting Detail') name = _('Setting Detail')
@extend_schema_if_available(extensions={"x-ai-description": "Update system settings."})
def patch(self, request, *args, **kwargs):
return super().patch(request, *args, **kwargs)
def get_queryset(self): def get_queryset(self):
self.category_slug = self.kwargs.get('category_slug', 'all') self.category_slug = self.kwargs.get('category_slug', 'all')
all_category_slugs = list(settings_registry.get_registered_categories().keys()) all_category_slugs = list(settings_registry.get_registered_categories().keys())

View File

@@ -897,8 +897,6 @@ class HostAccess(BaseAccess):
'created_by', 'created_by',
'modified_by', 'modified_by',
'inventory', 'inventory',
'last_job__job_template',
'last_job_host_summary__job',
) )
prefetch_related = ('groups', 'inventory_sources') prefetch_related = ('groups', 'inventory_sources')

View File

@@ -1,15 +1,17 @@
# Python # Python
import logging import logging
# Dispatcherd
from dispatcherd.publish import task
# AWX # AWX
from awx.main.analytics.subsystem_metrics import DispatcherMetrics, CallbackReceiverMetrics from awx.main.analytics.subsystem_metrics import DispatcherMetrics, CallbackReceiverMetrics
from awx.main.dispatch.publish import task as task_awx
from awx.main.dispatch import get_task_queuename from awx.main.dispatch import get_task_queuename
logger = logging.getLogger('awx.main.scheduler') logger = logging.getLogger('awx.main.scheduler')
@task_awx(queue=get_task_queuename) @task(queue=get_task_queuename, timeout=300, on_duplicate='discard')
def send_subsystem_metrics(): def send_subsystem_metrics():
DispatcherMetrics().send_metrics() DispatcherMetrics().send_metrics()
CallbackReceiverMetrics().send_metrics() CallbackReceiverMetrics().send_metrics()

View File

@@ -1,8 +1,6 @@
import datetime import datetime
import asyncio import asyncio
import logging import logging
import redis
import redis.asyncio
import re import re
from prometheus_client import ( from prometheus_client import (
@@ -15,7 +13,7 @@ from prometheus_client import (
) )
from django.conf import settings from django.conf import settings
from awx.main.utils.redis import get_redis_client, get_redis_client_async
BROADCAST_WEBSOCKET_REDIS_KEY_NAME = 'broadcast_websocket_stats' BROADCAST_WEBSOCKET_REDIS_KEY_NAME = 'broadcast_websocket_stats'
@@ -66,6 +64,8 @@ class FixedSlidingWindow:
class RelayWebsocketStatsManager: class RelayWebsocketStatsManager:
_redis_client = None # Cached Redis client for get_stats_sync()
def __init__(self, local_hostname): def __init__(self, local_hostname):
self._local_hostname = local_hostname self._local_hostname = local_hostname
self._stats = dict() self._stats = dict()
@@ -80,7 +80,7 @@ class RelayWebsocketStatsManager:
async def run_loop(self): async def run_loop(self):
try: try:
redis_conn = await redis.asyncio.Redis.from_url(settings.BROKER_URL) redis_conn = get_redis_client_async()
while True: while True:
stats_data_str = ''.join(stat.serialize() for stat in self._stats.values()) stats_data_str = ''.join(stat.serialize() for stat in self._stats.values())
await redis_conn.set(self._redis_key, stats_data_str) await redis_conn.set(self._redis_key, stats_data_str)
@@ -103,8 +103,10 @@ class RelayWebsocketStatsManager:
""" """
Stringified verion of all the stats Stringified verion of all the stats
""" """
redis_conn = redis.Redis.from_url(settings.BROKER_URL) # Reuse cached Redis client to avoid creating new connection pools on every call
stats_str = redis_conn.get(BROADCAST_WEBSOCKET_REDIS_KEY_NAME) or b'' if cls._redis_client is None:
cls._redis_client = get_redis_client()
stats_str = cls._redis_client.get(BROADCAST_WEBSOCKET_REDIS_KEY_NAME) or b''
return parser.text_string_to_metric_families(stats_str.decode('UTF-8')) return parser.text_string_to_metric_families(stats_str.decode('UTF-8'))

View File

@@ -487,9 +487,7 @@ def unified_jobs_table(since, full_path, until, **kwargs):
OR (main_unifiedjob.finished > '{0}' AND main_unifiedjob.finished <= '{1}')) OR (main_unifiedjob.finished > '{0}' AND main_unifiedjob.finished <= '{1}'))
AND main_unifiedjob.launch_type != 'sync' AND main_unifiedjob.launch_type != 'sync'
ORDER BY main_unifiedjob.id ASC) TO STDOUT WITH CSV HEADER ORDER BY main_unifiedjob.id ASC) TO STDOUT WITH CSV HEADER
'''.format( '''.format(since.isoformat(), until.isoformat())
since.isoformat(), until.isoformat()
)
return _copy_table(table='unified_jobs', query=unified_job_query, path=full_path) return _copy_table(table='unified_jobs', query=unified_job_query, path=full_path)
@@ -550,9 +548,7 @@ def workflow_job_node_table(since, full_path, until, **kwargs):
) always_nodes ON main_workflowjobnode.id = always_nodes.from_workflowjobnode_id ) always_nodes ON main_workflowjobnode.id = always_nodes.from_workflowjobnode_id
WHERE (main_workflowjobnode.modified > '{}' AND main_workflowjobnode.modified <= '{}') WHERE (main_workflowjobnode.modified > '{}' AND main_workflowjobnode.modified <= '{}')
ORDER BY main_workflowjobnode.id ASC) TO STDOUT WITH CSV HEADER ORDER BY main_workflowjobnode.id ASC) TO STDOUT WITH CSV HEADER
'''.format( '''.format(since.isoformat(), until.isoformat())
since.isoformat(), until.isoformat()
)
return _copy_table(table='workflow_job_node', query=workflow_job_node_query, path=full_path) return _copy_table(table='workflow_job_node', query=workflow_job_node_query, path=full_path)

View File

@@ -8,6 +8,7 @@ import pathlib
import shutil import shutil
import tarfile import tarfile
import tempfile import tempfile
from urllib.parse import urlparse, urlunparse
from django.conf import settings from django.conf import settings
from django.core.serializers.json import DjangoJSONEncoder from django.core.serializers.json import DjangoJSONEncoder
@@ -23,6 +24,8 @@ from awx.main.models import Job
from awx.main.access import access_registry from awx.main.access import access_registry
from awx.main.utils import get_awx_http_client_headers, set_environ, datetime_hook from awx.main.utils import get_awx_http_client_headers, set_environ, datetime_hook
from awx.main.utils.analytics_proxy import OIDCClient from awx.main.utils.analytics_proxy import OIDCClient
from awx.main.utils.candlepin import get_or_generate_candlepin_certificate
from awx.main.utils.candlepin.client import _temp_cert_files
__all__ = ['register', 'gather', 'ship'] __all__ = ['register', 'gather', 'ship']
@@ -41,6 +44,76 @@ def _valid_license():
return True return True
def _get_cert_upload_url(url):
"""
Convert analytics URL to use 'cert.' subdomain for mTLS uploads.
Some analytics services use different hostnames for different auth methods:
- cert.example.com - for mTLS (certificate-based) uploads
- example.com - for OIDC (token-based) uploads
Args:
url: Original analytics URL
Returns:
URL with 'cert.' prepended to hostname if not already present
"""
try:
parsed = urlparse(url)
hostname = parsed.hostname
# Only modify if hostname doesn't already start with 'cert.'
if hostname and not hostname.startswith('cert.'):
new_hostname = f'cert.{hostname}'
# Reconstruct URL with new hostname
netloc = new_hostname
if parsed.port:
netloc = f'{new_hostname}:{parsed.port}'
new_parsed = parsed._replace(netloc=netloc)
return urlunparse(new_parsed)
return url
except Exception as e:
logger.warning(f'Could not modify URL for cert upload: {e}, using original URL')
return url
def _get_analytics_credentials():
"""
Get Red Hat Insights credentials from settings.
Attempts to retrieve credentials in the following priority order:
1. REDHAT_USERNAME / REDHAT_PASSWORD
2. SUBSCRIPTIONS_USERNAME / SUBSCRIPTIONS_PASSWORD
3. SUBSCRIPTIONS_CLIENT_ID / SUBSCRIPTIONS_CLIENT_SECRET
Returns:
tuple: (username, password) if credentials are found, (None, None) otherwise
"""
rh_id = getattr(settings, 'REDHAT_USERNAME', None)
rh_secret = getattr(settings, 'REDHAT_PASSWORD', None)
if rh_id and rh_secret:
return rh_id, rh_secret
# Try SUBSCRIPTIONS_USERNAME / SUBSCRIPTIONS_PASSWORD
rh_id = getattr(settings, 'SUBSCRIPTIONS_USERNAME', None)
rh_secret = getattr(settings, 'SUBSCRIPTIONS_PASSWORD', None)
if rh_id and rh_secret:
return rh_id, rh_secret
# Try SUBSCRIPTIONS_CLIENT_ID / SUBSCRIPTIONS_CLIENT_SECRET
rh_id = getattr(settings, 'SUBSCRIPTIONS_CLIENT_ID', None)
rh_secret = getattr(settings, 'SUBSCRIPTIONS_CLIENT_SECRET', None)
if rh_id and rh_secret:
return rh_id, rh_secret
return None, None
def all_collectors(): def all_collectors():
from awx.main.analytics import collectors from awx.main.analytics import collectors
@@ -184,10 +257,8 @@ def gather(dest=None, module=None, subset=None, since=None, until=None, collecti
logger.log(log_level, "Automation Analytics not enabled. Use --dry-run to gather locally without sending.") logger.log(log_level, "Automation Analytics not enabled. Use --dry-run to gather locally without sending.")
return None return None
if not ( rh_id, rh_secret = _get_analytics_credentials()
settings.AUTOMATION_ANALYTICS_URL if not (settings.AUTOMATION_ANALYTICS_URL and rh_id and rh_secret):
and ((settings.REDHAT_USERNAME and settings.REDHAT_PASSWORD) or (settings.SUBSCRIPTIONS_CLIENT_ID and settings.SUBSCRIPTIONS_CLIENT_SECRET))
):
logger.log(log_level, "Not gathering analytics, configuration is invalid. Use --dry-run to gather locally without sending.") logger.log(log_level, "Not gathering analytics, configuration is invalid. Use --dry-run to gather locally without sending.")
return None return None
@@ -368,19 +439,14 @@ def ship(path):
logger.error('AUTOMATION_ANALYTICS_URL is not set') logger.error('AUTOMATION_ANALYTICS_URL is not set')
return False return False
rh_id = getattr(settings, 'REDHAT_USERNAME', None) rh_id, rh_secret = _get_analytics_credentials()
rh_secret = getattr(settings, 'REDHAT_PASSWORD', None)
if not (rh_id and rh_secret):
rh_id = getattr(settings, 'SUBSCRIPTIONS_CLIENT_ID', None)
rh_secret = getattr(settings, 'SUBSCRIPTIONS_CLIENT_SECRET', None)
if not rh_id: if not rh_id:
logger.error('Neither REDHAT_USERNAME nor SUBSCRIPTIONS_CLIENT_ID are set') logger.error('No valid username found. Tried: REDHAT_USERNAME, SUBSCRIPTIONS_USERNAME, SUBSCRIPTIONS_CLIENT_ID')
return False return False
if not rh_secret: if not rh_secret:
logger.error('Neither REDHAT_PASSWORD nor SUBSCRIPTIONS_CLIENT_SECRET are set') logger.error('No valid password found. Tried: REDHAT_PASSWORD, SUBSCRIPTIONS_PASSWORD, SUBSCRIPTIONS_CLIENT_SECRET')
return False return False
with open(path, 'rb') as f: with open(path, 'rb') as f:
@@ -388,17 +454,40 @@ def ship(path):
s = requests.Session() s = requests.Session()
s.headers = get_awx_http_client_headers() s.headers = get_awx_http_client_headers()
s.headers.pop('Content-Type') s.headers.pop('Content-Type')
with set_environ(**settings.AWX_TASK_ENV): with set_environ(**settings.AWX_TASK_ENV):
# Try Certificate-based mTLS authentication (zero-touch)
cert_pem, key_pem = get_or_generate_candlepin_certificate()
if cert_pem and key_pem:
# Use cert. subdomain for mTLS uploads
cert_url = _get_cert_upload_url(url)
logger.debug("Attempting certificate-based authentication for analytics upload")
try:
with _temp_cert_files(cert_pem, key_pem) as (cert_path, key_path):
response = s.post(
cert_url, files=files, cert=(cert_path, key_path), verify=settings.INSIGHTS_CERT_PATH, headers=s.headers, timeout=(31, 31)
)
if response.status_code < 300:
return True
else:
logger.warning(
f'Certificate-based authentication failed with status {response.status_code}, {response.text}. Falling back to OIDC auth'
)
except Exception as e:
logger.warning(f"Certificate-based authentication failed: {e}, falling back to OIDC auth")
# Try OIDC authentication
logger.debug("Attempting OIDC authentication for analytics upload")
f.seek(0) # requests POST may read from the handler, so seek to beginning of file for the next POST attempt
try: try:
client = OIDCClient(rh_id, rh_secret) client = OIDCClient(rh_id, rh_secret)
response = client.make_request("POST", url, headers=s.headers, files=files, verify=settings.INSIGHTS_CERT_PATH, timeout=(31, 31)) response = client.make_request("POST", url, headers=s.headers, files=files, verify=settings.INSIGHTS_CERT_PATH, timeout=(31, 31))
except requests.RequestException:
logger.error("Automation Analytics API request failed, trying base auth method")
response = s.post(url, files=files, verify=settings.INSIGHTS_CERT_PATH, auth=(rh_id, rh_secret), headers=s.headers, timeout=(31, 31))
# Accept 2XX status_codes if response.status_code < 300:
if response.status_code >= 300: return True
logger.error('Upload failed with status {}, {}'.format(response.status_code, response.text)) else:
return False logger.error(f'OIDC authentication failed with status {response.status_code}, {response.text}')
return False
return True except requests.RequestException as e:
logger.error(f"OIDC authentication failed: {e}")
return False

View File

@@ -0,0 +1,41 @@
import http.client
import socket
import urllib.error
import urllib.request
import logging
from django.conf import settings
logger = logging.getLogger(__name__)
def get_dispatcherd_metrics(request):
metrics_cfg = settings.METRICS_SUBSYSTEM_CONFIG.get('server', {}).get(settings.METRICS_SERVICE_DISPATCHER, {})
host = metrics_cfg.get('host', 'localhost')
port = metrics_cfg.get('port', 8015)
metrics_filter = []
if request is not None and hasattr(request, "query_params"):
try:
nodes_filter = request.query_params.getlist("node")
except Exception:
nodes_filter = []
if nodes_filter and settings.CLUSTER_HOST_ID not in nodes_filter:
return ''
try:
metrics_filter = request.query_params.getlist("metric")
except Exception:
metrics_filter = []
if metrics_filter:
# Right now we have no way of filtering the dispatcherd metrics
# so just avoid getting in the way if another metric is filtered for
return ''
url = f"http://{host}:{port}/metrics"
try:
with urllib.request.urlopen(url, timeout=1.0) as response:
payload = response.read()
if not payload:
return ''
return payload.decode('utf-8')
except (urllib.error.URLError, UnicodeError, socket.timeout, TimeoutError, http.client.HTTPException) as exc:
logger.debug(f"Failed to collect dispatcherd metrics from {url}: {exc}")
return ''

View File

@@ -14,6 +14,8 @@ from rest_framework.request import Request
from awx.main.consumers import emit_channel_notification from awx.main.consumers import emit_channel_notification
from awx.main.utils import is_testing from awx.main.utils import is_testing
from awx.main.utils.redis import get_redis_client
from .dispatcherd_metrics import get_dispatcherd_metrics
root_key = settings.SUBSYSTEM_METRICS_REDIS_KEY_PREFIX root_key = settings.SUBSYSTEM_METRICS_REDIS_KEY_PREFIX
logger = logging.getLogger('awx.main.analytics') logger = logging.getLogger('awx.main.analytics')
@@ -198,8 +200,8 @@ class Metrics(MetricsNamespace):
def __init__(self, namespace, auto_pipe_execute=False, instance_name=None, metrics_have_changed=True, **kwargs): def __init__(self, namespace, auto_pipe_execute=False, instance_name=None, metrics_have_changed=True, **kwargs):
MetricsNamespace.__init__(self, namespace) MetricsNamespace.__init__(self, namespace)
self.pipe = redis.Redis.from_url(settings.BROKER_URL).pipeline() self.conn = get_redis_client()
self.conn = redis.Redis.from_url(settings.BROKER_URL) self.pipe = self.conn.pipeline()
self.last_pipe_execute = time.time() self.last_pipe_execute = time.time()
# track if metrics have been modified since last saved to redis # track if metrics have been modified since last saved to redis
# start with True so that we get an initial save to redis # start with True so that we get an initial save to redis
@@ -397,11 +399,6 @@ class DispatcherMetrics(Metrics):
SetFloatM('workflow_manager_recorded_timestamp', 'Unix timestamp when metrics were last recorded'), SetFloatM('workflow_manager_recorded_timestamp', 'Unix timestamp when metrics were last recorded'),
SetFloatM('workflow_manager_spawn_workflow_graph_jobs_seconds', 'Time spent spawning workflow tasks'), SetFloatM('workflow_manager_spawn_workflow_graph_jobs_seconds', 'Time spent spawning workflow tasks'),
SetFloatM('workflow_manager_get_tasks_seconds', 'Time spent loading workflow tasks from db'), SetFloatM('workflow_manager_get_tasks_seconds', 'Time spent loading workflow tasks from db'),
# dispatcher subsystem metrics
SetIntM('dispatcher_pool_scale_up_events', 'Number of times local dispatcher scaled up a worker since startup'),
SetIntM('dispatcher_pool_active_task_count', 'Number of active tasks in the worker pool when last task was submitted'),
SetIntM('dispatcher_pool_max_worker_count', 'Highest number of workers in worker pool in last collection interval, about 20s'),
SetFloatM('dispatcher_availability', 'Fraction of time (in last collection interval) dispatcher was able to receive messages'),
] ]
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
@@ -429,8 +426,12 @@ class CallbackReceiverMetrics(Metrics):
def metrics(request): def metrics(request):
output_text = '' output_text = ''
for m in [DispatcherMetrics(), CallbackReceiverMetrics()]: output_text += DispatcherMetrics().generate_metrics(request)
output_text += m.generate_metrics(request) output_text += CallbackReceiverMetrics().generate_metrics(request)
dispatcherd_metrics = get_dispatcherd_metrics(request)
if dispatcherd_metrics:
output_text += dispatcherd_metrics
return output_text return output_text
@@ -480,13 +481,6 @@ class CallbackReceiverMetricsServer(MetricsServer):
super().__init__(settings.METRICS_SERVICE_CALLBACK_RECEIVER, registry) super().__init__(settings.METRICS_SERVICE_CALLBACK_RECEIVER, registry)
class DispatcherMetricsServer(MetricsServer):
def __init__(self):
registry = CollectorRegistry(auto_describe=True)
registry.register(CustomToPrometheusMetricsCollector(DispatcherMetrics(metrics_have_changed=False)))
super().__init__(settings.METRICS_SERVICE_DISPATCHER, registry)
class WebsocketsMetricsServer(MetricsServer): class WebsocketsMetricsServer(MetricsServer):
def __init__(self): def __init__(self):
registry = CollectorRegistry(auto_describe=True) registry = CollectorRegistry(auto_describe=True)

View File

@@ -82,7 +82,7 @@ class MainConfig(AppConfig):
def configure_dispatcherd(self): def configure_dispatcherd(self):
"""This implements the default configuration for dispatcherd """This implements the default configuration for dispatcherd
If running the tasking service like awx-manage run_dispatcher, If running the tasking service like awx-manage dispatcherd,
some additional config will be applied on top of this. some additional config will be applied on top of this.
This configuration provides the minimum such that code can submit This configuration provides the minimum such that code can submit
tasks to pg_notify to run those tasks. tasks to pg_notify to run those tasks.

View File

@@ -213,6 +213,40 @@ register(
category_slug='system', category_slug='system',
) )
register(
'AWX_ANALYTICS_CANDLEPIN_CA',
field_class=fields.CharField,
default='/etc/rhsm/ca/redhat-uep.pem',
allow_blank=True,
label=_('Candlepin CA Certificate Path'),
help_text=_('Path to the CA certificate file for verifying TLS connections to Candlepin. Leave blank to use system certificates.'),
category=_('System'),
category_slug='system',
)
register(
'AWX_ANALYTICS_CANDLEPIN_RENEWAL_THRESHOLD_DAYS',
field_class=fields.IntegerField,
default=90,
min_value=1,
label=_('Candlepin Certificate Renewal Threshold'),
help_text=_('Number of days before certificate expiry to trigger automatic renewal of Candlepin identity certificates.'),
category=_('System'),
category_slug='system',
unit=_('days'),
)
register(
'AWX_ANALYTICS_CANDLEPIN_PROXY_URL',
field_class=fields.CharField,
default='',
allow_blank=True,
label=_('Candlepin Proxy URL'),
help_text=_('HTTP/HTTPS proxy URL for Candlepin API requests (e.g., http://proxy.example.com:8080). Leave blank for no proxy.'),
category=_('System'),
category_slug='system',
)
register( register(
'INSTALL_UUID', 'INSTALL_UUID',
field_class=fields.CharField, field_class=fields.CharField,
@@ -824,6 +858,58 @@ register(
unit=_('seconds'), unit=_('seconds'),
) )
register(
'CANDLEPIN_CONSUMER_UUID',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=False,
label=_('Candlepin Consumer UUID'),
help_text=_('UUID of the registered Candlepin consumer for this AAP instance.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'CANDLEPIN_CERT_PEM',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=True,
label=_('Candlepin Identity Certificate'),
help_text=_('PEM-encoded Candlepin identity certificate for mTLS authentication.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'CANDLEPIN_KEY_PEM',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=True,
label=_('Candlepin Identity Key'),
help_text=_('PEM-encoded private key for Candlepin identity certificate.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'CANDLEPIN_SERIAL_NUMBER',
field_class=fields.CharField,
default='',
allow_blank=True,
encrypted=False,
label=_('Candlepin Certificate Serial Number'),
help_text=_('Serial number of the Candlepin identity certificate for tracking.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register( register(
'IS_K8S', 'IS_K8S',
field_class=fields.BooleanField, field_class=fields.BooleanField,

View File

@@ -11,6 +11,7 @@ __all__ = [
'CAN_CANCEL', 'CAN_CANCEL',
'ACTIVE_STATES', 'ACTIVE_STATES',
'STANDARD_INVENTORY_UPDATE_ENV', 'STANDARD_INVENTORY_UPDATE_ENV',
'OIDC_CREDENTIAL_TYPE_NAMESPACES',
] ]
PRIVILEGE_ESCALATION_METHODS = [ PRIVILEGE_ESCALATION_METHODS = [
@@ -140,3 +141,6 @@ org_role_to_permission = {
'execution_environment_admin_role': 'add_executionenvironment', 'execution_environment_admin_role': 'add_executionenvironment',
'auditor_role': 'view_project', # TODO: also doesnt really work 'auditor_role': 'view_project', # TODO: also doesnt really work
} }
# OIDC credential type namespaces for feature flag filtering
OIDC_CREDENTIAL_TYPE_NAMESPACES = ['hashivault-kv-oidc', 'hashivault-ssh-oidc']

View File

@@ -3,7 +3,6 @@ import logging
import time import time
import hmac import hmac
import asyncio import asyncio
import redis
from django.core.serializers.json import DjangoJSONEncoder from django.core.serializers.json import DjangoJSONEncoder
from django.conf import settings from django.conf import settings
@@ -14,6 +13,8 @@ from channels.generic.websocket import AsyncJsonWebsocketConsumer
from channels.layers import get_channel_layer from channels.layers import get_channel_layer
from channels.db import database_sync_to_async from channels.db import database_sync_to_async
from awx.main.utils.redis import get_redis_client_async
logger = logging.getLogger('awx.main.consumers') logger = logging.getLogger('awx.main.consumers')
XRF_KEY = '_auth_user_xrf' XRF_KEY = '_auth_user_xrf'
@@ -40,10 +41,10 @@ class WebsocketSecretAuthHelper:
@classmethod @classmethod
def verify_secret(cls, s, nonce_tolerance=300): def verify_secret(cls, s, nonce_tolerance=300):
try: try:
(prefix, payload) = s.split(' ') prefix, payload = s.split(' ')
if prefix != 'HMAC-SHA256': if prefix != 'HMAC-SHA256':
raise ValueError('Unsupported encryption algorithm') raise ValueError('Unsupported encryption algorithm')
(nonce_parsed, secret_parsed) = payload.split(':') nonce_parsed, secret_parsed = payload.split(':')
except Exception: except Exception:
raise ValueError("Failed to parse secret") raise ValueError("Failed to parse secret")
@@ -94,6 +95,9 @@ class RelayConsumer(AsyncJsonWebsocketConsumer):
await self.channel_layer.group_add(settings.BROADCAST_WEBSOCKET_GROUP_NAME, self.channel_name) await self.channel_layer.group_add(settings.BROADCAST_WEBSOCKET_GROUP_NAME, self.channel_name)
logger.info(f"client '{self.channel_name}' joined the broadcast group.") logger.info(f"client '{self.channel_name}' joined the broadcast group.")
# Initialize Redis client once for reuse across all message handling
self._redis_conn = get_redis_client_async()
async def disconnect(self, code): async def disconnect(self, code):
logger.info(f"client '{self.channel_name}' disconnected from the broadcast group.") logger.info(f"client '{self.channel_name}' disconnected from the broadcast group.")
await self.channel_layer.group_discard(settings.BROADCAST_WEBSOCKET_GROUP_NAME, self.channel_name) await self.channel_layer.group_discard(settings.BROADCAST_WEBSOCKET_GROUP_NAME, self.channel_name)
@@ -102,11 +106,12 @@ class RelayConsumer(AsyncJsonWebsocketConsumer):
await self.send(event['text']) await self.send(event['text'])
async def receive_json(self, data): async def receive_json(self, data):
(group, message) = unwrap_broadcast_msg(data) group, message = unwrap_broadcast_msg(data)
if group == "metrics": if group == "metrics":
message = json.loads(message['text']) message = json.loads(message['text'])
conn = redis.Redis.from_url(settings.BROKER_URL) await self._redis_conn.set(
conn.set(settings.SUBSYSTEM_METRICS_REDIS_KEY_PREFIX + "-" + message['metrics_namespace'] + "_instance_" + message['instance'], message['metrics']) settings.SUBSYSTEM_METRICS_REDIS_KEY_PREFIX + "-" + message['metrics_namespace'] + "_instance_" + message['instance'], message['metrics']
)
else: else:
await self.channel_layer.group_send(group, message) await self.channel_layer.group_send(group, message)

View File

@@ -77,14 +77,13 @@ class PubSub(object):
n = psycopg.connection.Notify(pgn.relname.decode(enc), pgn.extra.decode(enc), pgn.be_pid) n = psycopg.connection.Notify(pgn.relname.decode(enc), pgn.extra.decode(enc), pgn.be_pid)
yield n yield n
def events(self, yield_timeouts=False): def events(self):
if not self.conn.autocommit: if not self.conn.autocommit:
raise RuntimeError('Listening for events can only be done in autocommit mode') raise RuntimeError('Listening for events can only be done in autocommit mode')
while True: while True:
if select.select([self.conn], [], [], self.select_timeout) == NOT_READY: if select.select([self.conn], [], [], self.select_timeout) == NOT_READY:
if yield_timeouts: yield None
yield None
else: else:
notification_generator = self.current_notifies(self.conn) notification_generator = self.current_notifies(self.conn)
for notification in notification_generator: for notification in notification_generator:

View File

@@ -2,7 +2,7 @@ from django.conf import settings
from ansible_base.lib.utils.db import get_pg_notify_params from ansible_base.lib.utils.db import get_pg_notify_params
from awx.main.dispatch import get_task_queuename from awx.main.dispatch import get_task_queuename
from awx.main.dispatch.pool import get_auto_max_workers from awx.main.utils.common import get_auto_max_workers
def get_dispatcherd_config(for_service: bool = False, mock_publish: bool = False) -> dict: def get_dispatcherd_config(for_service: bool = False, mock_publish: bool = False) -> dict:
@@ -11,28 +11,40 @@ def get_dispatcherd_config(for_service: bool = False, mock_publish: bool = False
Parameters: Parameters:
for_service: if True, include dynamic options needed for running the dispatcher service for_service: if True, include dynamic options needed for running the dispatcher service
this will require database access, you should delay evaluation until after app setup this will require database access, you should delay evaluation until after app setup
mock_publish: if True, use mock values that don't require database access
this is used during tests to avoid database queries during app initialization
""" """
# When mock_publish=True (e.g., during tests), use a default value to avoid
# database access in get_auto_max_workers() which queries settings.IS_K8S
if mock_publish:
max_workers = 20 # Reasonable default for tests
else:
max_workers = get_auto_max_workers()
config = { config = {
"version": 2, "version": 2,
"service": { "service": {
"pool_kwargs": { "pool_kwargs": {
"min_workers": settings.JOB_EVENT_WORKERS, "min_workers": settings.JOB_EVENT_WORKERS,
"max_workers": get_auto_max_workers(), "max_workers": max_workers,
# This must be less than max_workers to make sense, which is usually 4
# With reserve of 1, after a burst of tasks, load needs to down to 4-1=3
# before we return to min_workers
"scaledown_reserve": 1,
"worker_max_lifetime_seconds": settings.WORKER_MAX_LIFETIME_SECONDS,
}, },
"main_kwargs": {"node_id": settings.CLUSTER_HOST_ID}, "main_kwargs": {"node_id": settings.CLUSTER_HOST_ID},
"process_manager_cls": "ForkServerManager", "process_manager_cls": "ForkServerManager",
"process_manager_kwargs": {"preload_modules": ['awx.main.dispatch.hazmat']}, "process_manager_kwargs": {"preload_modules": ['awx.main.dispatch.prefork']},
}, },
"brokers": { "brokers": {},
"socket": {"socket_path": settings.DISPATCHERD_DEBUGGING_SOCKFILE}, "publish": {},
},
"publish": {"default_control_broker": "socket"},
"worker": {"worker_cls": "awx.main.dispatch.worker.dispatcherd.AWXTaskWorker"}, "worker": {"worker_cls": "awx.main.dispatch.worker.dispatcherd.AWXTaskWorker"},
} }
if mock_publish: if mock_publish:
config["brokers"]["noop"] = {} config["brokers"]["dispatcherd.testing.brokers.noop"] = {}
config["publish"]["default_broker"] = "noop" config["publish"]["default_broker"] = "dispatcherd.testing.brokers.noop"
else: else:
config["brokers"]["pg_notify"] = { config["brokers"]["pg_notify"] = {
"config": get_pg_notify_params(), "config": get_pg_notify_params(),
@@ -49,5 +61,11 @@ def get_dispatcherd_config(for_service: bool = False, mock_publish: bool = False
} }
config["brokers"]["pg_notify"]["channels"] = ['tower_broadcast_all', 'tower_settings_change', get_task_queuename()] config["brokers"]["pg_notify"]["channels"] = ['tower_broadcast_all', 'tower_settings_change', get_task_queuename()]
metrics_cfg = settings.METRICS_SUBSYSTEM_CONFIG.get('server', {}).get(settings.METRICS_SERVICE_DISPATCHER)
if metrics_cfg:
config["service"]["metrics_kwargs"] = {
"host": metrics_cfg.get("host", "localhost"),
"port": metrics_cfg.get("port", 8015),
}
return config return config

View File

@@ -1,78 +0,0 @@
import logging
import uuid
import json
from django.conf import settings
from django.db import connection
import redis
from awx.main.dispatch import get_task_queuename
from . import pg_bus_conn
logger = logging.getLogger('awx.main.dispatch')
class Control(object):
services = ('dispatcher', 'callback_receiver')
result = None
def __init__(self, service, host=None):
if service not in self.services:
raise RuntimeError('{} must be in {}'.format(service, self.services))
self.service = service
self.queuename = host or get_task_queuename()
def status(self, *args, **kwargs):
r = redis.Redis.from_url(settings.BROKER_URL)
if self.service == 'dispatcher':
stats = r.get(f'awx_{self.service}_statistics') or b''
return stats.decode('utf-8')
else:
workers = []
for key in r.keys('awx_callback_receiver_statistics_*'):
workers.append(r.get(key).decode('utf-8'))
return '\n'.join(workers)
def running(self, *args, **kwargs):
return self.control_with_reply('running', *args, **kwargs)
def cancel(self, task_ids, with_reply=True):
if with_reply:
return self.control_with_reply('cancel', extra_data={'task_ids': task_ids})
else:
self.control({'control': 'cancel', 'task_ids': task_ids, 'reply_to': None}, extra_data={'task_ids': task_ids})
def schedule(self, *args, **kwargs):
return self.control_with_reply('schedule', *args, **kwargs)
@classmethod
def generate_reply_queue_name(cls):
return f"reply_to_{str(uuid.uuid4()).replace('-','_')}"
def control_with_reply(self, command, timeout=5, extra_data=None):
logger.warning('checking {} {} for {}'.format(self.service, command, self.queuename))
reply_queue = Control.generate_reply_queue_name()
self.result = None
if not connection.get_autocommit():
raise RuntimeError('Control-with-reply messages can only be done in autocommit mode')
with pg_bus_conn(select_timeout=timeout) as conn:
conn.listen(reply_queue)
send_data = {'control': command, 'reply_to': reply_queue}
if extra_data:
send_data.update(extra_data)
conn.notify(self.queuename, json.dumps(send_data))
for reply in conn.events(yield_timeouts=True):
if reply is None:
logger.error(f'{self.service} did not reply within {timeout}s')
raise RuntimeError(f"{self.service} did not reply within {timeout}s")
break
return json.loads(reply.payload)
def control(self, msg, **kwargs):
with pg_bus_conn() as conn:
conn.notify(self.queuename, json.dumps(msg))

Some files were not shown because too many files have changed in this diff Show More