* INCLUDE_AWX_VAR_PREFIX to USE_TOWER_VAR_PREFIX boolean toggle replace the include legacy prefix boolean with a tower-or-awx toggle USE_TOWER_VAR_PREFIX=True (default) emits only tower_ prefixed variables, false emits only awx_ (deprecated)
* Clean up dead constant and cache get_job_variable_prefixes() calls
* Revise tests to reflect new behavior
* Fix fragile fallback test to actually exercise getattr default
* Fix mock target for settings fallback test
* perf: stop eagerly updating Host.last_job_host_summary on every job completion
The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.
Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.
Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK
The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix .pk AttributeError, add job_template annotations, annotate host sublists
- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs
Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove stale failures_url from deprecated DashboardView
The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to changed files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: replace 10 subquery annotations with bulk prefetch
Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').
This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.
- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary
Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.
- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: scope annotation to views, restore license_error/canceled_on
- Remove with_latest_summary_id() from HostManager.get_queryset() to
avoid applying the correlated subquery to every Host query globally
(count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
host-serving views only
- Restore license_error and canceled_on to last_job summary fields
to avoid breaking API change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard _fetch_all() to skip bulk-attach on non-annotated querysets
Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove name from last_job_host_summary and canceled_on from last_job summary
Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary
Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to test_host_queryset.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Fix flake8 F841: remove unused job1/job2 variables in tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Add comment explaining why Prefetch was not used for host latest summary
Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix null handling to keep old behavior
---------
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
* Add new tests for bug saving concurrent facts
* Fix first bug and improve tests
* Fix new bug where concurrent job clears facts from other job in unwanted way
* minor test fixes
* Add in missing playbook
* Fix host reference for constructed inventory
* Increase speed for concurrent fact tests
* Make test a bit faster
* Fix linters
* Add some functional tests
* Remove the sanity test
* Agent markers added
* Address SonarCloud
* Do backdating method, resolving stricter assertions
* Address coderabbit comments
* Address review comment with qs only method
* Delete missed sleep statement
* Add more coverage
* WIP First pass
* started removing feature flags and adjusting logic
* Add decorator
* moved to dispatcher decorator
* updated as many as I could find
* Keep callback receiver working
* remove any code that is not used by the call back receiver
* add back auto_max_workers
* added back get_auto_max_workers into common utils
* Remove control and hazmat (squash this not done)
* moved status out and deleted control as no longer needed
* removed unused imports
* adjusted test import to pull correct method
* fixed imports and addressed clusternode heartbeat test
* Update function comments
* Add back hazmat for config and remove baseworker
* added back hazmat per @alancoding feedback around config
* removed baseworker completely and refactored it into the callback
worker
* Fix dispatcher run call and remove dispatch setting
* remove dispatcher mock publish setting
* Adjust heartbeat arg and more formatting
* fixed the call to cluster_node_heartbeat missing binder
* Fix attribute error in server logs
Refactored code to use Python's built-in datetime.timezone and zoneinfo instead of pytz for timezone handling. This modernizes the codebase and removes the dependency on pytz, aligning with current best practices for timezone-aware datetime objects.
* Dump running tasks when running out of capacity
* Use same logic for max_workers and capacity
* Address case where CPU capacity is the constraint
* Add a test for correspondence
* Fake redis to make tests work
By stable, we mean future occurrences of the rrule
should be the same before and after the fast forward
operation.
The problem before was that we were fast forwarding to
7 days ago. For some rrules, this does not retain the old
occurrences. Thus, jobs would launch at unexpected times.
This change makes sure we fast forward in increments of
the rrule INTERVAL, thus the new dtstart should be in the
occurrence list of the old rrule.
Additionally, code is updated to fast forward
EXRULE (exclusion rules) in addition to RRULE
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Develop ability to list permissions for existing roles
Create a model registry for RBAC-tracked models
Write the data migration logic for creating
the preloaded role definitions
Write migration to migrate old Role into ObjectRole model
This loops over the old Role model, knowing it is unique
on object and role_field
Most of the logic is concerned with identifying the
needed permissions, and then corresponding role definition
As needed, object roles are created and users then teams
are assigned
Write re-computation of cache logic for teams
and then for object role permissions
Migrate new RBAC internals to ansible_base
Migrate tests to ansible_base
Implement solution for visible_roles
Expose URLs for DAB RBAC
* Bulk launch serializer RBAC and code structure review
Use WJ node as base in bulk job launch child
remove fields we get for free this way
Minor translation marking
Consolidate bulk API permission methods
split out permission check for each UJT type
Code consolidation for org check method
add a save before starting the workflow job
Remove corresponding views for job instance_groups
Validate job_slice_count in API
Remove defaults from some job launch view prompts
the null default is preferable
Additionally, move the inventory-specific hacks of yesteryear
into the prompts_dict method of the WorkflowJob model
try to make it clear exactly what this is hacking and why
Correctly summarize label prompts, and add missing EE
Expand unit tests to apply more fields
adding missing fields to preserve during copy to workflow.py
Fix bug where empty workflow job vars blanked node vars (#12904)
* Fix bug where empty workflow job vars blanked node vars
* Fix bug where workflow job has no extra_vars, add test
* Add empty workflow job extra vars to assure fix
This removes a loop that ran on import
the loop was giving the wrong behavior
and it initialized too many fields as char_prompts fields
With this, we will now enumerate the char_prompts type fields manually
* Making almost all fields promptable on job templates and config models
* Adding EE, IG and label access checks
* Changing jobs preferred instance group function to handle the new IG cache field
* Adding new ask fields to job template modules
* Address unit/functional tests
* Adding migration file
* Track combined artifacts on workflow jobs
* Avoid schema change for passing nested workflow artifacts
* Basic support for nested workflow artifacts, add test
* Forgot that only does not work with polymorphic
* Remove incorrect field
* Consolidate logic and prevent recursion with UJ artifacts method
* Stop trying to do precedence by status, filter for obvious ones
* Review comments about sets
* Fix up bug with convergence node paths and artifacts
* Added schedule_rruleset lookup plugin for awx.awx
* Added DB migration for rrule size
* Updated schedule docs
* The schedule API endpoint will now return an array of errors on rule validation to try and inform the user of all errors instead of just the first
* Primary development of integrating runner cleanup command
* Fixup image cleanup signals and their tests
* Use alphabetical sort to solve the cluster coordination problem
* Update test to new pattern
* Clarity edits to interface with ansible-runner cleanup method
* Another change corresponding to ansible-runner CLI updates
* Fix incomplete implementation of receptor remote cleanup
* Share receptor utils code between worker_info and cleanup
* Complete task logging from calling runner cleanup command
* Wrap up unit tests and some contract changes that fall out of those
* Fix bug in CLI construction
* Fix queryset filter bug
--- Removed reference to tower in InventorySource and InventoryUpdate model
--- Added a migration for above change
--- Added new CONTROLLER* variables in awx/main/models/credentials/__init__.py
--- Migrated awxkit to new CONTROLLER* variables
--- Updated the tests to use new CONTROLLER* variables
--- Fix some issues with upgrade path, rename more cases
