* Remove dead Host.last_job and Host.last_job_host_summary FK fields
PR #16332 replaced these denormalized ForeignKey columns with dynamic
derivation via Host.latest_summary, but left the columns in place.
Nothing writes to or reads from them anymore — drop the dead weight.
- Remove last_job and last_job_host_summary ForeignKey fields from Host
- Convert HostSerializer to use SerializerMethodField instead of FK-backed fields
- Remove redundant JobHostSummary.latest_for_host/latest_job_for_host classmethods
- Add migration 0206 to drop the columns
- Update tests to use Host.latest_summary instead of removed fields/methods
* Remove dead last_job_host_summary_id UPDATE from cleanup_jobs
The Host.last_job_host_summary FK was removed in the prior commit,
so the raw SQL UPDATE that NULLed it before deleting JHS rows now
references a non-existent column and crashes.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace ResourceMixin/accessible_objects/accessible_pk_qs with DAB RBAC access_qs/access_ids_qs
Remove the ResourceMixin class and migrate all callers to the new
DAB RBAC queryset API: access_qs replaces accessible_objects and
access_ids_qs replaces accessible_pk_qs, using permission strings
(view, change, use, etc.) instead of role field names.
Also updates prefetch_page_capabilities to use permission-based
capability definitions instead of role field references.
Restore UNION pattern in TeamAccess.filtered_queryset
The access_qs migration accidentally collapsed the UNION into an OR,
which can be 48x slower at scale (AAP-83319).
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Unify activity stream format for role assignments across old and new APIs
The old API (/api/v2/users/<pk>/roles/) and new API
(/api/v2/role_user_assignments/) were producing different activity stream
entries for the same logical action. The old API created entries via the
Role.members m2m_changed signal in the legacy format, while the new API
used the new-format handler added in AAP-82996.
This change makes the old API produce the same new-format entry by:
- Skipping the legacy activity_stream_associate for Role_members changes
- Letting the mirrored RoleUserAssignment creation fire the new-format
record_role_assignment_activity_stream handler instead
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The EE sidecar container creates /etc/receptor/receptor.conf on pod
startup. If the task container's dispatcherd reaches the heartbeat
before the config exists, the instance rejoins the cluster at full
capacity with no receptor — causing fatal job failures with no
recovery path.
Three layers of defense:
1. dispatcherd management command: poll for the receptor config file
before entering the main loop (10s timeout, exits with error to
let k8s restart the container).
2. Heartbeat: if get_receptor_ctl() raises FileNotFoundError (config
or socket missing), mark the instance offline so no jobs are
dispatched to it. Previously this was silently ignored and the
instance kept full capacity.
3. should_update_config(): catch FileNotFoundError and return True so
write_receptor_config() can create the file on first boot instead
of crashing.
Prior art: #12698 (closed without merge) took a broader approach with
custom exception types. See PR comment for overlap analysis.
Closes: AAP-89010
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-87084 — Revert UnifiedJob queryset to DAB APIs, scope actor role filter
The previous patch (7242fe89) inlined all DAB RBAC internals into
UnifiedJobAccess.filtered_queryset to pre-compute role IDs and avoid
repeated subqueries. While it fixed the UNION regression from c84575c,
it created 70 lines of duplicated logic inconsistent with the
UnifiedJobTemplate path and fetched ALL user roles regardless of
relevance.
This commit:
- Reverts filtered_queryset to the clean post-f8fa690 form that
delegates to DAB's access_ids_qs / accessible_pk_qs APIs
- Adds _scoped_actor_role_filter() as a temporary shim that pre-filters
RoleUserAssignment by content_type_id, auto-including Org and Team CTs
for inherited permissions and NULL for global roles
- Wires the scoped filter into UnifiedJobTemplate.accessible_pk_qs
- Adds tests for custom global roles, mixed permissions, and unrelated
role exclusion
The shim duplicates logic planned for DAB's _actor_role_filter
content_type_ids parameter; once that merges, replace and delete.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove unused ContentType and RoleUserAssignment imports to fix flake8
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-87084 — Remove _scoped_actor_role_filter shim, use DAB _actor_role_filter directly
The scoped content_type_ids parameter now lives in DAB's
_actor_role_filter. The AWX shim and its hardcoded Org/Team CT
inclusions are no longer needed. The unified job accessible_pk_qs
uses the unscoped _actor_role_filter for correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [AAP-82668] Skip old RBAC sync on cascade-deleted assignments
When a non-RBAC parent (e.g. Organization) is deleted, Django cascades
to RoleUserAssignment/RoleTeamAssignment and fires post_delete for each.
The sync_assignments_to_old_rbac_delete handler would then do 3-4
FK/GFK queries per assignment to sync removals to the old Role model —
entirely redundant since the old Role M2M tables cascade-delete from the
same content object.
Use Django's post_delete `origin` kwarg (4.1+) to detect this: when
origin is a Model instance whose app_label is not dab_rbac, the delete
is a cascade from a non-RBAC parent and the sync is skipped.
Measured on 150 teams / 321 users / 48K assignments:
Baseline: 78.8s
With fix: 20.4s (via service-index endpoint)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add regression tests for old RBAC sync after bulk claims
Tests that save_user_claims (which uses bulk_create, skipping signals)
correctly syncs old Role.members when run through AwxJWTAuthentication.
Covers add, remove, and multi-org/team scenarios.
* test: call process_permissions instead of internal _sync_old_rbac
Tests now exercise the public AwxJWTAuthentication.process_permissions()
API with the JWT layer mocked, rather than calling the private
_sync_old_rbac method directly.
The test_bulk_claims_skips_old_rbac_signals test asserts that
save_user_claims does NOT populate old Role.members via signals.
On current DAB, save_user_claims uses the serial give_permission
path which fires signals normally, so old Role.members IS populated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Move PG version check to check_db command
Move to utils, check in pre_migrate signal
* Add back in environment var skip
* Add tests for compliance
tests Assisted-By: claude
* Fix bash operator precedence in repo ownership check
The condition had || operators outside proper test block grouping,
which could cause the check to fail with a shell error. Wrap the
OR conditions in parentheses with explicit [[ ]] tests.
Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>
* Replace reusable workflow with direct if conditions for repo ownership check
The reusable workflow with job dependencies had a timing/evaluation issue
where jobs would still execute even when should_run=false. Using direct
if conditions with github context variables (repository, ref_name) is more
reliable and ensures jobs are properly skipped on fork pushes.
Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>
* Remove unused repo-owns-branch reusable workflow
No longer needed after replacing with direct if conditions.
Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>
* Address even more pytest warnings, co-authored with Opus 4.6
* Upgrade pyparsing
* Attempt to update smart inventory logic
* Move smart inventory tests here
* Fix some failing dev env tests
Assisted-by: claude
* Use shared fixture for teardown
* Fix test goof
assisted-by: claude Opus 4.6
* [AAP-57274] Fix creator permissions for models without old-style roles
NotificationTemplate has no old-style ImplicitRoleField (like admin_role)
because notification permissions were historically org-level only.
When a non-admin user creates a notification template,
give_creator_permissions tries to sync the DAB RBAC assignment back
to the old role system and hits an AttributeError.
Catch the AttributeError so the DAB RBAC assignment still succeeds.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Consolidate implementation of same-org validation rule
* Update tests for the simplified validation
* Still do validation with deferance to the new callback
* Correctly falsy handling in view logic
* Change fact processing loop to use file listing
* Fix some test
* Address coderabbit comments
* Handle saving facts in batches to keep memory low
* Improve log about mismatch in response to review comment
* Correctly restrict push actions to ownership repos
* Use standard action to see if push actions should run
* Run spec job for 2.6 and higher
* Be even more restrictve, do not push if on a fork
* Add new test for artfact precedence upstream node vs outer workflow
* Fix bugs, upstream artifacts come first for precedence
* Track nested artifacts path through ancestor_artifacts on root nodes
* Fix case where first root node did not get the vars
* touchup comment
* Prevent conflict with sliced jobs hack
* Fix job cancel chain bugs
* Early relief valve for canceled jobs, ATF related changes
* Add test and fix for approval nodes as well
* Revert unwanted change
* Refactor workflow approval nodes to make it more clean
* Revert data structure changes
* Delete local utility file
* Review comment addressing
* Use canceled status in websocket
* Delete slop
* Add agent marker
* Bugbot comment about status websocket mismatch
* Add new tests for bug saving concurrent facts
* Fix first bug and improve tests
* Fix new bug where concurrent job clears facts from other job in unwanted way
* minor test fixes
* Add in missing playbook
* Fix host reference for constructed inventory
* Increase speed for concurrent fact tests
* Make test a bit faster
* Fix linters
* Add some functional tests
* Remove the sanity test
* Agent markers added
* Address SonarCloud
* Do backdating method, resolving stricter assertions
* Address coderabbit comments
* Address review comment with qs only method
* Delete missed sleep statement
* Add more coverage
* Enable new fancy asyncio metrics for dispatcherd
Remove old dispatcher metrics and patch in new data from local whatever
Update test fixture to new dispatcherd version
* Update dispatcherd again
* Handle node filter in URL, and catch more errors
* Add test for metric filter
* Split module for dispatcherd metrics
* Add dispatcherctl command
* Add tests for dispatcherctl command
* Exit early if sqlite3
* Switch to dispatcherd mgmt cmd
* Move unwanted command options to run_dispatcher
* Add test for new stuff
* Update the SOS report status command
* make docs always reference new command
* Consistently error if given config file
* Additional dispatcher removal simplifications and waiting repear updates
* Fix double call and logging message
* Implement bugbot comment, should reap running on lost instances
* Add test case for new pending behavior
* Remove the dynamic filter on dispatcher startup
Configure the dynamic logging level only on startup
* Special case for log level on settings change
* Add unit test for new behavior
* Add test for initial config
* Mark test django DB
* Do necessary requirement bump
* Delete cache in live test fixture
* Disconnect logic to fill in role parents
Get tests passing hopefully
Whatever SonarCloud
* remove role parents/children endpoints and related views
* remove duplicate get_queryset method from RoleTeamsList
---------
Co-authored-by: Peter Braun <pbraun@redhat.com>
Bump migrations and delete some files
Resolve remaining conflicts
Fix requirements
Flake8 fixes
Prefer devel changes for schema
Use correct versions
Remove sso connected stuff
Update to modern actions and collection fixes
Remove unwated alias
Version problems in actions
Fix more versioning problems
Update warning string
Messed it up again
Shorten exception
More removals
Remove pbr license
Remove tests deleted in devel
Remove unexpected files
Remove some content missed in the rebase
Use sleep_task from devel
Restore devel live conftest file
Add in settings that got missed
Prefer devel version of collection test
Finish repairing .github path
Remove unintended test file duplication
Undo more unintended file additions
* Working branch for testing DAB RBAC changes
* AAP-48392 Handle DAB RBAC either before or after new type model (for merge) (#16045)
* Handle DAB RBAC either before or after new type model
* Translate CT to DAB CT
* Fix for rearrangement of post_migration methods
* Directly include RBAC service URLs
* Add a run before remote permission additions
* Sync old rbac to remote rbac (#7025)
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Set DAB requirement back to devel
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
* Handle DAB RBAC either before or after new type model
* Translate CT to DAB CT
* Fixes for content type switch
* Use more compatible coding pattern
* Deeper purge of content_type_id
* revert, turns out that did not work
* More content type replacements
* Revert changes to serializer
* Revert another content_type change
* Fix for rearrangement of post_migration methods
* Remove thing I am not going to do
* Revert branch pin that was temporary
* Mark the collection role module as deprecated
* Mark deprecated in DOCUMENTATION
* Add deprecation info
* Resolve validate-modules deprecation errors
---------
Co-authored-by: Luis <lvilla@redhat.com>
* 🧪 Unpersist Git creds @ cov combine job
This is one of the things Zizmor [[1]] warns about.
[1]: https://docs.zizmor.sh
* 🧪 Download all coverage artifacts in one go
* 🧪 Delegate artifact garbage collection to GH
This is implemented by setting the retention days input to 1 on the
initial upload.
Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <webknjaz@redhat.com>
* Make the JT name uniqueness enforced at the database level
* Forgot demo project fixture
* New approach, done by adding a new field
* Update for linters and failures
* Fix logical error in migration test
* Revert some test changes based on review comment
* Do not rename first template, add test
* Avoid name-too-long rename errors
* Insert migration into place
* Move existing files with git
* Bump migrations of existing
* Update migration test
* Awkward bump
* Fix migration file link
* update test reference again
* Add no cov on fail flag to fix CI
* Bump django to 4.2.21
* Coverage args
* Try cov equals awx
* Run migration test in parallel
* Ignore error and clean up commands
* Try to make schema check not hang
---------
Co-authored-by: Satoe Imaishi <simaishi@redhat.com>
* Clean up logging when receptor not running
* Make logging more concise for unregistered case
* Silence another unwanted traceback
* Silence a few more tracebacks
* Fix bug where collectstatic could error due to dispatcherd config
* Revert test because it will not work in test suite
* New publish mocking system
* Remove import of unused
* Fix default publish broker
Use dynamic AWX max_workers value
Make basic --status and --running commands work
Make feature flag enabled true by default for development
* [dispatcherd] Dispatcher socket-based `--status` demo working (#15908)
* Fix Task Decorator to Work With and Without Feature Flag (AAP-41775) (#15911)
* refactor(system): extract common heartbeat helpers and split cluster_node_heartbeat
Extract common heartbeat logic into helper functions: _heartbeat_instance_management: consolidates instance management, health checks, and lost-instance detection. _heartbeat_check_versions: compares instance versions and initiates shutdown when necessary. _heartbeat_handle_lost_instances: reaps jobs and marks lost instances offline.
Refactor the original cluster_node_heartbeat to use these helpers and retain legacy behavior (using bind_kwargs).
Introduce adispatch_cluster_node_heartbeat for dispatcherd: uses the control API to retrieve running tasks and reaps them.
Link the two implementations by attaching adispatch_cluster_node_heartbeat as the _new_method on cluster_node_heartbeat.
* feat(publish): delegate heartbeat task submission to new dispatcherd implementation
Update apply_async to check at runtime if FEATURE_NEW_DISPATCHER is enabled.
When the task is cluster_node_heartbeat and a _new_method is attached, delegate the task submission to the new dispatcherd implementation.
Preserve the original behavior for all other tasks and fallback on error.
* refactor(system): extract task ID retrieval from dispatcherd into helper function
Improves readability of adispatch_cluster_node_heartbeat by extracting
the complex UUID parsing logic into a dedicated helper function.
Adds clearer error handling and follows established code patterns.
* fix(dispatcher): Enable task decorator to work with and without feature flag
Implemented a new approach for handling task execution with feature flags
by attaching alternative implementations to apply_async._new_method. This
allows cluster_node_heartbeat to work correctly with both the legacy and
new dispatcher systems without modifying core decorator logic.
AAP-41775
* fix(dispatcher): Improve error handling and logging in feature flag implementation
- Add error handling when attaching alternative dispatcher implementation
- Fix method self-reference in apply_async to properly use cls.apply_async
- Document limitations of this targeted approach for specific tasks
- Add logging for better debugging of dispatcher selection
- Ensure decorator timing by keeping method attachment after function definitions
This completes the robust implementation for switching between dispatcher
implementations based on feature flags.
AAP-41775
* fix(dispatcher): Implement registry pattern for dispatcher feature flag compatibility
Replaces direct method attribute assignment with a global registry for
alternative implementations. The original approach tried to attach new
methods directly to apply_async bound methods, which fails because bound
methods don't support attribute assignment in Python.
The registry pattern:
- Creates a global ALTERNATIVE_TASK_IMPLEMENTATIONS dict in publish.py
- Registers alternative implementations by task name
- Modifies apply_async to check the registry when feature flag is enabled
- Adds extensive logging throughout the process for debugging
This enables cluster_node_heartbeat to work correctly with both the legacy
and new dispatcher implementations based on the FEATURE_NEW_DISPATCHER flag.
AAP-41775
* refactor(dispatcher): Remove excessive logging from dispatcher implementation
Reduces verbose debugging logs while maintaining essential logging for critical
operations. Preserves:
- Task implementation selection based on feature flag
- Registration success/failure messages
- Critical error reporting
Removed:
- Registry content debugging messages
- Repetitive task diagnostics
- Non-essential information logging
AAP-41775
* fix(dispatcher): Fix shallow copy in dispatcher schedule conversion
This resolves "AttributeError: 'float' object has no attribute 'total_seconds'"
errors when the dispatcher is restarted.
Refs: AAP-41775
* Use IPC mechanism to get running tasks (#15926)
* Allow tasks from tasks
* Fix failure to limit to waiting jobs
* Get job record with lock
* Fix failures in dispatcherd feature branch (#15930)
* Fully handle DispatcherCancel
* Complete rest of preload import work
* Complete dispatcherd integration & job cancellation (AAP-43033) (#15941)
* feat(dispatcher): Implement job cancellation for new dispatcher
Adds feature-flag-aware job cancellation that routes cancel requests to either
the legacy dispatcher or the new dispatcherd library based on the
FEATURE_NEW_DISPATCHER flag.
- Updates cancel_dispatcher_process() to use dispatcherd's control API when enabled
- Handles both direct cancellation and task manager workflow cancellation cases
- Works with DispatcherCancel exception handling to properly handle SIGUSR1 signals
AAP-43033
* fix(dispatcher): Update run_dispatcher.py to properly handle task cancellation
Modifies the cancel command in run_dispatcher.py to properly cancel tasks
when the FEATURE_NEW_DISPATCHER flag is enabled, rather than just listing
running tasks.
The implementation translates each task UUID to the appropriate
filter format expected by the dispatcherd control API, maintaining the same
behavior as the original implementation.
Part of: AAP-43033
* refactor(system): Refactor dispatch_startup() to extract common startup logic and branch based on feature flag
This commit refactors the dispatch_startup() function to improve clarity and consistency across the legacy
and new dispatcherd flows.
No dispatcher-specific functionality is needed beyond the changes made, so this refactoring improves robustness without
altering core behavior.
* refactor(system): Refactor inform_cluster_of_shutdown() for clarity
* refactor(tasks): Replace @task with @task_awx across 22 tasks for dispatcher compatibility
- Migrated all task decorators to use @task_awx, ensuring dispatcher-aware behavior.
- Tested each task with the new dispatcherd, verifying that tasks using the registry pattern execute correctly without needing binder‐based alternative implementations.
- Removed redundant logging and outdated comments.
- Legacy tasks that do not require special parameter extraction continue to use their original logic.
- This commit reflects our complete journey of testing and verifying dispatcherd compatibility across all 22 tasks.
* refactor(publish): fix linter
* Fix bug from the branch rebase
* AAP-43763 Add tests for connection management in dispatcherd workers (#15949)
* Add test for job cancel in live tests
* Fix bug from the branch rebase
* Add test for connection recovery after connection broke
* Add test for breaking connection
* Fix dispatcherd bugs: schedule aliases, job kwargs handling, cancel handling (#15960)
* Put in job kwargs handling, not done before
* AAP-44382 [dispatcherd] Fixes for running with feature flag off (#15973)
* Use correct decorator for test of tasks
* Finalize dispatcherd feature branch (#15975)
* Work dispatcherd into dependency management system
* Use util methods from DAB
* Rename the dispatcherd feature flag, and flip default to not-enabled
* Move to new submit_task method
* Update the location of the sock file
* AAP-44381 Make dispatcherd config loading more lazy (#15979)
* Make dispatcherd config loading more lazy
* Make submission error more obvious
* Fix signal handling gap, hijack SIGUSR1 from dispatcherd (#15983)
* Fix signal handling gap, hijack SIGUSR1 from dispatcherd
* Minor adjustments to dispatcherd status command
* [dispatcherd] Get rid of alternative task registry (#15984)
Get rid of alternative task registry
* Fix deadlock error and other cleanup errors (#15987)
* Move to proper error handling location
---------
Co-authored-by: artem_tiupin <70763601+art-tapin@users.noreply.github.com>
* Make the JT name uniqueness enforced at the database level
* Forgot demo project fixture
* New approach, done by adding a new field
* Update for linters and failures
* Fix logical error in migration test
* Revert some test changes based on review comment
* Do not rename first template, add test
* Avoid name-too-long rename errors
* Insert migration into place
* Move existing files with git
* Bump migrations of existing
* Update migration test
* Awkward bump
* Fix migration file link
* update test reference again
* Dump running tasks when running out of capacity
* Use same logic for max_workers and capacity
* Address case where CPU capacity is the constraint
* Add a test for correspondence
* Fake redis to make tests work
Alan RomingerandClaude Opus 4.6
