* Move PG version check to check_db command
Move to utils, check in pre_migrate signal
* Add back in environment var skip
* Add tests for compliance
tests Assisted-By: claude
* AAP-12516 [option 2] Handle nested workflow artifacts via root node `ancestor_artifacts` (#16381)
* Add new test for artfact precedence upstream node vs outer workflow
* Fix bugs, upstream artifacts come first for precedence
* Track nested artifacts path through ancestor_artifacts on root nodes
* Fix case where first root node did not get the vars
* touchup comment
* Prevent conflict with sliced jobs hack
* Reorder URLs so that Django debug toolbar can work (#16352)
* Reorder URLs so that Django debug toolbar can work
* Move comment with URL move
* feat: support for oidc credential /test endpoint (#16370)
Adds support for testing external credentials that use OIDC workload identity tokens.
When FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED is enabled, the /test endpoints return
JWT payload details alongside test results.
- Add OIDC credential test endpoints with job template selection
- Return JWT payload and secret value in test response
- Maintain backward compatibility (detail field for errors)
- Add comprehensive unit and functional tests
- Refactor shared error handling logic
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
* Bind the install bundle to the ansible.receptor collection 2.0.8 version (#16396)
* [Devel] Config Endpoint Optimization (#16389)
* Improved performance of the config endpoint by reducing database queries in GET /api/controller/v2/config/
* Fix OIDC workload identity for inventory sync (#16390)
The cloud credential used by inventory updates was not going through
the OIDC workload identity token flow because it lives outside the
normal _credentials list. This overrides populate_workload_identity_tokens
in RunInventoryUpdate to include the cloud credential as an
additional_credentials argument to the base implementation, and
patches get_cloud_credential on the instance so the injector picks up
the credential with OIDC context intact.
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: integrate awx-tui to the awx_devel image (#16399)
* Aap 45980 (#16395)
* support bitbucket_dc webhooks
* add test
* update docs
* fix import for refactored method (#16394)
retrieve_workload_identity_jwt_with_claims is now
in a separate utility file, not in jobs.py
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* AAP-70257 controller collection should retry transient HTTP errors with exponential backoff. (#16415)
controller collection should retry transient HTTP errors with exponential backoff
* AAP-71844 Fix rrule fast-forward across DST boundaries (#16407)
Fix rrule fast-forward producing wrong occurrences across DST boundaries
The UTC round-trip in _fast_forward_rrule shifts the dtstart's local
hour when the original and fast-forwarded times are in different DST
periods. Since dateutil generates HOURLY occurrences by stepping in
local time, the shifted hour changes the set of reachable hours. With
BYHOUR constraints this causes a ValueError crash; without BYHOUR,
occurrences are silently shifted by 1 hour.
Fix by performing all arithmetic in the dtstart's original timezone.
Python aware-datetime subtraction already computes absolute elapsed
time regardless of timezone, so the UTC conversion was unnecessary
for correctness and actively harmful during fall-back ambiguity.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Correctly restrict push actions to ownership repos (#16398)
* Correctly restrict push actions to ownership repos
* Use standard action to see if push actions should run
* Run spec job for 2.6 and higher
* Be even more restrictve, do not push if on a fork
* [Devel] Performance Optimization for Select Hosts Query (#16413)
* Fixed black reformating
* Make test simulate 500k hosts in real world scenario
* feat: improve unauthorized response on aap deployments (#16422)
* fix: do not include secret values in the credentials test endpoint an… (#16425)
fix: do not include secret values in the credentials test endpoint and add a guard to make sure credentials are testable
* [devel backport] AAP-41742: Fix workflow node update failing when JT has unprompted labels (#16426)
* AAP-41742: Fix workflow node update failing when JT has unprompted labels
PATCH extra_data on a workflow node fails with
{"labels":["Field is not configured to prompt on launch."]}
when the node has labels associated but the JT has
ask_labels_on_launch=False.
The serializer was passing all persisted M2M state from prompts_dict()
to _accept_or_ignore_job_kwargs() on every PATCH, re-validating
unchanged fields. Fix scopes validation to only the fields in the
request; full re-validation still occurs when unified_job_template
is being changed.
* Capture attrs keys before _build_mock_obj mutates them
_build_mock_obj() pops pseudo-fields (limit, scm_branch, job_tags,
etc.) from attrs. Computing requested_prompt_fields after the pop
would miss those fields and skip their ask_on_launch validation.
* Include survey_passwords when validating extra_vars prompts
prompts_dict() emits survey_passwords alongside extra_vars.
_accept_or_ignore_job_kwargs uses it to decrypt encrypted survey
values before validation. Without it, encrypted password blobs
are validated as-is against the survey spec.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add test to ensure credential secret values are not returned (#16434)
* AAP-68024 perf: derive last_job_host_summary from query instead of denormalized FK (#16332)
* perf: stop eagerly updating Host.last_job_host_summary on every job completion
The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.
Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.
Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK
The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix .pk AttributeError, add job_template annotations, annotate host sublists
- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs
Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove stale failures_url from deprecated DashboardView
The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to changed files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: replace 10 subquery annotations with bulk prefetch
Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').
This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.
- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary
Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.
- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: scope annotation to views, restore license_error/canceled_on
- Remove with_latest_summary_id() from HostManager.get_queryset() to
avoid applying the correlated subquery to every Host query globally
(count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
host-serving views only
- Restore license_error and canceled_on to last_job summary fields
to avoid breaking API change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard _fetch_all() to skip bulk-attach on non-annotated querysets
Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove name from last_job_host_summary and canceled_on from last_job summary
Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary
Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to test_host_queryset.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Fix flake8 F841: remove unused job1/job2 variables in tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Add comment explaining why Prefetch was not used for host latest summary
Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix null handling to keep old behavior
---------
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
* [AAP-72722] Use url instead of jwt_aud for workload identity audience (#16432)
* [AAP-72722] Use url instead of jwt_aud for workload identity audience
The OIDC credential plugin's jwt_aud field is being removed. Use the
plugin's url field as the audience when requesting workload identity
tokens, since the target service URL is the appropriate audience value.
Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [Devel] Optimize host_list_rbac query (#16408)
* Defer ansible_facts in HostManager to avoid fetching large JSON column in host list queries (AAP-68023)
The host list endpoint (GET /api/v2/hosts/) fetches the ansible_facts
JSON column unnecessarily, contributing to the 7.8s median query time
at scale. This column can be very large and is not used by the list
serializer.
Changes:
- HostManager.get_queryset() now defers ansible_facts
- finish_fact_cache call site uses .only(*HOST_FACTS_FIELDS) to eagerly
load ansible_facts when actually needed, avoiding N+1 queries
- Unit test mocks updated to support .only() queryset chaining
- Points DAB dependency at the RBAC query optimization branch for
combined testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
* fix: constructed inventories no longer increase the host count (#16433)
* Fix version worktree (#16431)
* git worktree friendly precomit install
* worktrees don't have a .git directory. Before, docker-compose would
trigger pre-commit install and fail.
* make docker-compose work in git worktree
* AWX tries to discover the version via info stored in .git/ dir.
setuptools-scm is capable of finding the .git/ dir, starting from a
worktree, but is unable because only the worktree is mapped into the
container, not the .git/ dir itself. Thus, we have to detect and pass
the version into the container from outside. That is why this change
landed in the Makefile.
* fix: as_user() gateway session cookie fallback (#16437)
Add a fallback that checks for `gateway_sessionid` when no cookie
matches `session_cookie_name`, mirroring the existing fallback in
`Connection.login()`. The finally block now cleans up whichever
cookie name was actually used.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Pass setting to dispatcherd so it can be configured (#16438)
* fix: allow blank password field to fix OpenAPI schema validation (#16440)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* first pass porting over metrics
* move settings to defaults
* add more unit tests
* update unit tests
* lint fixes
* more lint fixes
* refactor and address feedback
* remove the api views
* remove model and move helper functions out of licensing
* add settings to API, fix tests, refactoring
* fix circular import
* update tests
* remove duplicate code, handle edge cases, use clearer naming, add test coverage
* update test for changes in ship()
* remove unneeded setting
* _discover_org should account for verify-tls=False
* directly assign settings, detect url, update tests
* log errors close to occurance
* rename function for clarity, focus on critical tests
* rename for clarity, lint fixes
* fix test params, priority for org discovery
* fix test failures and linting
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
Co-authored-by: Tong He <68936428+unnecessary-username@users.noreply.github.com>
Co-authored-by: Stevenson Michel <iamstevensonmichel@outlook.com>
Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: Sean Sullivan <ssulliva@redhat.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>
Co-authored-by: Lila Yasin <lyasin@redhat.com>
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* Add dispatcherctl command
* Add tests for dispatcherctl command
* Exit early if sqlite3
* Switch to dispatcherd mgmt cmd
* Move unwanted command options to run_dispatcher
* Add test for new stuff
* Update the SOS report status command
* make docs always reference new command
* Consistently error if given config file
* Additional dispatcher removal simplifications and waiting repear updates
* Fix double call and logging message
* Implement bugbot comment, should reap running on lost instances
* Add test case for new pending behavior
* WIP First pass
* started removing feature flags and adjusting logic
* Add decorator
* moved to dispatcher decorator
* updated as many as I could find
* Keep callback receiver working
* remove any code that is not used by the call back receiver
* add back auto_max_workers
* added back get_auto_max_workers into common utils
* Remove control and hazmat (squash this not done)
* moved status out and deleted control as no longer needed
* removed unused imports
* adjusted test import to pull correct method
* fixed imports and addressed clusternode heartbeat test
* Update function comments
* Add back hazmat for config and remove baseworker
* added back hazmat per @alancoding feedback around config
* removed baseworker completely and refactored it into the callback
worker
* Fix dispatcher run call and remove dispatch setting
* remove dispatcher mock publish setting
* Adjust heartbeat arg and more formatting
* fixed the call to cluster_node_heartbeat missing binder
* Fix attribute error in server logs
Refactored code to use Python's built-in datetime.timezone and zoneinfo instead of pytz for timezone handling. This modernizes the codebase and removes the dependency on pytz, aligning with current best practices for timezone-aware datetime objects.
Deleted the awx/main/management/commands/graph_jobs.py file and removed the asciichartpy package from requirements. This cleans up unused code and dependencies related to terminal job status graphing.
* update to Python 3.12
* remove use of utcnow
* switch to timezone.utc
datetime.UTC is an alias of datetime.timezone.utc. if we're doing the double import for datetime it's more straightforward to just import timezone as well and get it directly
* debug python env version issue
* change python version
* pin to SHA and remove debug portion
* Remove the dynamic filter on dispatcher startup
Configure the dynamic logging level only on startup
* Special case for log level on settings change
* Add unit test for new behavior
* Add test for initial config
* Mark test django DB
* Do necessary requirement bump
* Delete cache in live test fixture
Upgrade to Django 5.2 LTS with compatibility fixes across fields, migrations, dispatch config, tests, and dev deps.
Dependencies:
- Upgrade django to 5.2.8 and relax requirements.in to >=5.2,<5.3.
- Bump django-debug-toolbar to >=6.0 for compatibility.
Backend:
- awx/conf/fields.py: switch URL TLD regex to use DomainNameValidator.ul in custom URLField.
- awx/main/management/commands/gather_analytics.py: use datetime.timezone.utc for naïve datetime handling.
- awx/main/dispatch/config.py: add mock_publish option; avoid DB access for test runs, set default max_workers, and support a noop broker.
Migrations (SQLite/Postgres compatibility):
- Add awx/main/migrations/_sqlite_helper.py with db-aware AlterIndexTogether/RenameIndex wrappers; consume in 0144_event_partitions.py and 0184_django_indexes.py.
- Update 0187_hop_nodes.py to use CheckConstraint(condition=...).
- Add 0205_alter_instance_peers_alter_job_hosts_and_more.py adjusting through_fields/relations on instance.peers, job.hosts, and role.ancestors.
- _dab_rbac.py: iterate roles with chunk_size=1000 for migration performance.
Tests:
Include hcp_terraform in default credential types in test_credential.py.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
Bump migrations and delete some files
Resolve remaining conflicts
Fix requirements
Flake8 fixes
Prefer devel changes for schema
Use correct versions
Remove sso connected stuff
Update to modern actions and collection fixes
Remove unwated alias
Version problems in actions
Fix more versioning problems
Update warning string
Messed it up again
Shorten exception
More removals
Remove pbr license
Remove tests deleted in devel
Remove unexpected files
Remove some content missed in the rebase
Use sleep_task from devel
Restore devel live conftest file
Add in settings that got missed
Prefer devel version of collection test
Finish repairing .github path
Remove unintended test file duplication
Undo more unintended file additions
* Added better error handling and messaging when the service token authentication is broken. Allowed for GATEWAY_BASE_URL to override the service token's base url if it is set in the environment variables.
Co-Authored-By: Cursor (claude-4-sonnet)
* Removed GATEWAY_BASE_URL override for service token auth.
* migrate settings using the existing authenticator framework
* add method to get settings value to gateway client
* add transformer functions for settings
* Switched back to PUT for settings updates
* Started wiring in testing changes
* Added settings_* aggregation results. Added skip-github option. Added tests.
Assisted-by: Cursor
* Added --skip-all-authenticators command line argument. Added GoogleOAuth testing. Added tests for skipping all authenticators.
Assisted-by: Cursor
* wip: migrate other missing settings
* update login_redirect_override in google_oauth2
* impement login redirect for azuread
* implement login redirect for github
* implement login redirect for saml
* set LOGIN_REDIRECT_OVERRIDE even if no authenticator matched
* extract logic for login redirect override to base class
* use urlparse to compare valid redirect urls
* Preserve the original query parameters
* Fix flake8 issues
* Preserve the query parameter in sso_login_url
Gateway sets the sso_login_url to
/api/gateway/social/login/aap-saml-keycloak/?idp=IdP
The idp needs to be preserved when creating the redirect
* Update awx/main/utils/gateway_client.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* Update awx/main/management/commands/import_auth_config_to_gateway.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* list of settings updated
* Update awx/main/utils/gateway_client.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* Update awx/sso/utils/base_migrator.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* fix tests
---------
Co-authored-by: Andrew Potozniak <potozniak@redhat.com>
Co-authored-by: Madhu Kanoor <mkanoor@redhat.com>
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* feat: AAP-48498 Radius authenticator migrator
Issue: AAP-48498
* fix: Namingm Style and tests
* enabled by default
* test: SECRET is now ignored unless --force is set
* add force flag to enforce updates even when authenticator already exists
* remove cleartext field
* update list of encrypted fields
* show updated and unchanged authenticators in report
* collect controller ldap configuration
* translate role mapping and submit ldap authenticator
* implement require and deny group mapping
* remove all references of awx in the naming
* fix linter issues
* address PR feedback
* update ldap authenticator naming
* update github authenticator naming
* assume that server_uri is always a string
* update order of evaluation for require and deny groups
* cleanup and move ldap related functions into the ldap migrator
* add skip option for saml
* update saml authenticator to new slug format
* update azuread authenticator to new slug format
This PR migrates the SAML configuration from the Controller
to the Gateway, it intentionally skips setting the CALLBACK_URL
so that the Gateway can fill in the appropriate URL.
* compare authenticators and mappers before recreating them
* add unit tests
* fix linter errors
* refactor and improve: better implementation for get_authenticator_by_slug and removal of redundant code
* add submit_authenticator method to handle create vs. update in a generic way
* remove unused import
* wip: management command for authenticator export to GateWay
* wip: implement ldap auth config migration
* refactor: split concerns into gathering config and converting / recreating config
* refactor: dry run by default
* use the authenticator slug for idempotency
* move to correct utils path
* use env vars instead of flags, fix linter errors
* remove unused import
* Delete existing all-group vars on inventory sync (with overwrite-vars=True) instead of merging them.
* Implementation of inv var handling with file as db.
* Improve serialization to file of inv vars for src update
* Include inventory-level variable editing into inventory source update handling
* Add group vars to inventory source update handling
* Add support for overwrite_vars to new inventory source handling
* Persist inventory var history in the database instead of a file.
* Remove logging which was needed during development.
* Remove further debugging code and improve comments
* Move special handling for user edits of variables into serializers
* Relate the inventory variable history model to its inventory
* Allow for inventory variables to have the value 'None'
* Fix KeyError in new inventory variable handling
* Add unique-together constraint for new model InventoryGroupVariablesWithHistory
* Use only one special invsrc_id for initial update and manual updates
* Fix internal server error when creating a new inventory
* Print the empty string for a variable with value 'None'
* Fix comment which incorrectly states old behaviour
* Fix inventory_group_variables_update tests which did not take the new handling of None into account
* Allow any type for Ansible-core variable values
* Refactor misleading method names
* Fix internal server error when savig vars from group form
* Remove superfluous json conversion in front of JSONField
* Call variable update from create/update instead from validate
* Use group_id instead of group_name in model InventoryGroupVariablesWithHistory
* Disable new variable update handling for all regular (non-'all') groups
* Add live test to verify AAP-17690 (inv var deleted from source)
* Add functional tests to verify inventory variables update logic
* Fix migration which was corrupted by a rebase
* Add a more complex live test and resolve linter complaints
* Force overwrite_vars=False for updates from source on all-group
* Change behavior with respect to overwrite_vars
Use dynamic AWX max_workers value
Make basic --status and --running commands work
Make feature flag enabled true by default for development
* [dispatcherd] Dispatcher socket-based `--status` demo working (#15908)
* Fix Task Decorator to Work With and Without Feature Flag (AAP-41775) (#15911)
* refactor(system): extract common heartbeat helpers and split cluster_node_heartbeat
Extract common heartbeat logic into helper functions: _heartbeat_instance_management: consolidates instance management, health checks, and lost-instance detection. _heartbeat_check_versions: compares instance versions and initiates shutdown when necessary. _heartbeat_handle_lost_instances: reaps jobs and marks lost instances offline.
Refactor the original cluster_node_heartbeat to use these helpers and retain legacy behavior (using bind_kwargs).
Introduce adispatch_cluster_node_heartbeat for dispatcherd: uses the control API to retrieve running tasks and reaps them.
Link the two implementations by attaching adispatch_cluster_node_heartbeat as the _new_method on cluster_node_heartbeat.
* feat(publish): delegate heartbeat task submission to new dispatcherd implementation
Update apply_async to check at runtime if FEATURE_NEW_DISPATCHER is enabled.
When the task is cluster_node_heartbeat and a _new_method is attached, delegate the task submission to the new dispatcherd implementation.
Preserve the original behavior for all other tasks and fallback on error.
* refactor(system): extract task ID retrieval from dispatcherd into helper function
Improves readability of adispatch_cluster_node_heartbeat by extracting
the complex UUID parsing logic into a dedicated helper function.
Adds clearer error handling and follows established code patterns.
* fix(dispatcher): Enable task decorator to work with and without feature flag
Implemented a new approach for handling task execution with feature flags
by attaching alternative implementations to apply_async._new_method. This
allows cluster_node_heartbeat to work correctly with both the legacy and
new dispatcher systems without modifying core decorator logic.
AAP-41775
* fix(dispatcher): Improve error handling and logging in feature flag implementation
- Add error handling when attaching alternative dispatcher implementation
- Fix method self-reference in apply_async to properly use cls.apply_async
- Document limitations of this targeted approach for specific tasks
- Add logging for better debugging of dispatcher selection
- Ensure decorator timing by keeping method attachment after function definitions
This completes the robust implementation for switching between dispatcher
implementations based on feature flags.
AAP-41775
* fix(dispatcher): Implement registry pattern for dispatcher feature flag compatibility
Replaces direct method attribute assignment with a global registry for
alternative implementations. The original approach tried to attach new
methods directly to apply_async bound methods, which fails because bound
methods don't support attribute assignment in Python.
The registry pattern:
- Creates a global ALTERNATIVE_TASK_IMPLEMENTATIONS dict in publish.py
- Registers alternative implementations by task name
- Modifies apply_async to check the registry when feature flag is enabled
- Adds extensive logging throughout the process for debugging
This enables cluster_node_heartbeat to work correctly with both the legacy
and new dispatcher implementations based on the FEATURE_NEW_DISPATCHER flag.
AAP-41775
* refactor(dispatcher): Remove excessive logging from dispatcher implementation
Reduces verbose debugging logs while maintaining essential logging for critical
operations. Preserves:
- Task implementation selection based on feature flag
- Registration success/failure messages
- Critical error reporting
Removed:
- Registry content debugging messages
- Repetitive task diagnostics
- Non-essential information logging
AAP-41775
* fix(dispatcher): Fix shallow copy in dispatcher schedule conversion
This resolves "AttributeError: 'float' object has no attribute 'total_seconds'"
errors when the dispatcher is restarted.
Refs: AAP-41775
* Use IPC mechanism to get running tasks (#15926)
* Allow tasks from tasks
* Fix failure to limit to waiting jobs
* Get job record with lock
* Fix failures in dispatcherd feature branch (#15930)
* Fully handle DispatcherCancel
* Complete rest of preload import work
* Complete dispatcherd integration & job cancellation (AAP-43033) (#15941)
* feat(dispatcher): Implement job cancellation for new dispatcher
Adds feature-flag-aware job cancellation that routes cancel requests to either
the legacy dispatcher or the new dispatcherd library based on the
FEATURE_NEW_DISPATCHER flag.
- Updates cancel_dispatcher_process() to use dispatcherd's control API when enabled
- Handles both direct cancellation and task manager workflow cancellation cases
- Works with DispatcherCancel exception handling to properly handle SIGUSR1 signals
AAP-43033
* fix(dispatcher): Update run_dispatcher.py to properly handle task cancellation
Modifies the cancel command in run_dispatcher.py to properly cancel tasks
when the FEATURE_NEW_DISPATCHER flag is enabled, rather than just listing
running tasks.
The implementation translates each task UUID to the appropriate
filter format expected by the dispatcherd control API, maintaining the same
behavior as the original implementation.
Part of: AAP-43033
* refactor(system): Refactor dispatch_startup() to extract common startup logic and branch based on feature flag
This commit refactors the dispatch_startup() function to improve clarity and consistency across the legacy
and new dispatcherd flows.
No dispatcher-specific functionality is needed beyond the changes made, so this refactoring improves robustness without
altering core behavior.
* refactor(system): Refactor inform_cluster_of_shutdown() for clarity
* refactor(tasks): Replace @task with @task_awx across 22 tasks for dispatcher compatibility
- Migrated all task decorators to use @task_awx, ensuring dispatcher-aware behavior.
- Tested each task with the new dispatcherd, verifying that tasks using the registry pattern execute correctly without needing binder‐based alternative implementations.
- Removed redundant logging and outdated comments.
- Legacy tasks that do not require special parameter extraction continue to use their original logic.
- This commit reflects our complete journey of testing and verifying dispatcherd compatibility across all 22 tasks.
* refactor(publish): fix linter
* Fix bug from the branch rebase
* AAP-43763 Add tests for connection management in dispatcherd workers (#15949)
* Add test for job cancel in live tests
* Fix bug from the branch rebase
* Add test for connection recovery after connection broke
* Add test for breaking connection
* Fix dispatcherd bugs: schedule aliases, job kwargs handling, cancel handling (#15960)
* Put in job kwargs handling, not done before
* AAP-44382 [dispatcherd] Fixes for running with feature flag off (#15973)
* Use correct decorator for test of tasks
* Finalize dispatcherd feature branch (#15975)
* Work dispatcherd into dependency management system
* Use util methods from DAB
* Rename the dispatcherd feature flag, and flip default to not-enabled
* Move to new submit_task method
* Update the location of the sock file
* AAP-44381 Make dispatcherd config loading more lazy (#15979)
* Make dispatcherd config loading more lazy
* Make submission error more obvious
* Fix signal handling gap, hijack SIGUSR1 from dispatcherd (#15983)
* Fix signal handling gap, hijack SIGUSR1 from dispatcherd
* Minor adjustments to dispatcherd status command
* [dispatcherd] Get rid of alternative task registry (#15984)
Get rid of alternative task registry
* Fix deadlock error and other cleanup errors (#15987)
* Move to proper error handling location
---------
Co-authored-by: artem_tiupin <70763601+art-tapin@users.noreply.github.com>
* Delete existing all-group vars on inventory sync (with overwrite-vars=True) instead of merging them.
* Implementation of inv var handling with file as db.
* Improve serialization to file of inv vars for src update
* Include inventory-level variable editing into inventory source update handling
* Add group vars to inventory source update handling
* Add support for overwrite_vars to new inventory source handling
* Persist inventory var history in the database instead of a file.
* Remove logging which was needed during development.
* Remove further debugging code and improve comments
* Move special handling for user edits of variables into serializers
* Relate the inventory variable history model to its inventory
* Allow for inventory variables to have the value 'None'
* Fix KeyError in new inventory variable handling
* Add unique-together constraint for new model InventoryGroupVariablesWithHistory
* Use only one special invsrc_id for initial update and manual updates
* Fix internal server error when creating a new inventory
* Print the empty string for a variable with value 'None'
* Fix comment which incorrectly states old behaviour
* Fix inventory_group_variables_update tests which did not take the new handling of None into account
* Allow any type for Ansible-core variable values
* Refactor misleading method names
* Fix internal server error when savig vars from group form
* Remove superfluous json conversion in front of JSONField
* Call variable update from create/update instead from validate
* Use group_id instead of group_name in model InventoryGroupVariablesWithHistory
* Disable new variable update handling for all regular (non-'all') groups
* Add live test to verify AAP-17690 (inv var deleted from source)
* Add functional tests to verify inventory variables update logic
* Fix migration which was corrupted by a rebase
* Add a more complex live test and resolve linter complaints
* Force overwrite_vars=False for updates from source on all-group
* Change behavior with respect to overwrite_vars
* Remove oauth provider
This removes the oauth provider functionality from awx. The
oauth2_provider app and all references to it have been removed.
Migrations to delete the two tables that locally overwrote
oauth2_provider tables are included. This change does not include
migrations to delete the tables provided by the oauth2_provider app.
Also not included here are changes to awxkit, awx_collection or the ui.
* Fix linters
* Update migrations after rebase
* Update collection tests for auth changes
The changes in https://github.com/ansible/awx/pull/15554 will cause a
few collection tests to fail, depending on what the test configuration
is. This changes the tests to look for a specific warning rather than
counting the number of warnings emitted.
* Update migration
* Removed unused oauth_scopes references
---------
Co-authored-by: Mike Graves <mgraves@redhat.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
* Register all discovered CredentialType(s) after Django finishes
loading
* Protect parallel registrations using shared postgres advisory lock
* The down-side of this is that this will run when it does not need to,
adding overhead to the init process.
* Only register discovered credential types in the database IF
migrations have ran and are up-to-date.
We have not identify the root cause of wsrelay failure but attempt to make wsrelay restart itself resulted in postgres and redis connection leak. We were not able to fully identify where the redis connection leak comes from so reverting back to failing and removing startsecs 30 will prevent wsrelay to FATAL
- when re-establishing connection to db close old connection
- re-initialize WebSocketRelayManager when restarting asyncio.run
- log and ignore error in cleanup_offline_host (this might come back to bite us)
- cleanup connection when WebSocketRelayManager crash
* Add dump_auth_config management cmd
- Dump SAML config from AWX to DAB authenticator config in json format
* Add dumping of LDAP settings
* add test for command
* Fix is_enabled
* fix command name typo
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
* add fields to config, add name to data
* break out IDP values
* change test fields and value comparison
* edit help text, reformat settings
---------
Co-authored-by: jessicamack <jmack@redhat.com>
