* AAP-96178: Fix unified_jobs pagination count to respect RBAC filtering
Non-admin users saw the unfiltered table-wide count instead of the count
matching their RBAC-visible results. Use the same primary-key subquery
approach already proven on the /api/v2/jobs/ endpoint.
* AAP-96178: Add docstrings and mixed-role count test for UnifiedJobPaginator
Clarify the PK subquery approach with expanded docstring and inline
comments per review feedback. Add a functional test verifying that a user
with a mix of JT and inventory roles sees a count matching exactly
their permitted results.
* AAP-96178: Add deduplication test for multiple RBAC access routes
Verify that a job reachable through three overlapping permission paths
(direct JT role, org audit role, and team JT role) appears exactly once
in both results and count.
Assisted-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add custom last_job_host_summary__failed filter to HostList
The denormalized last_job_host_summary FK was removed (AAP-68024), so
filtering by ?last_job_host_summary__failed=True on /v2/hosts/ would
400 or return stale results. Add custom handling that annotates the
queryset with a Subquery on JobHostSummary (same pattern used by the
dashboard counts) and reserves the param names from FieldLookupBackend
so the generic filter skips them.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Extend last_job_host_summary failed filter to inventory and group host lists
Build on AlanCoding's HostList filter (ansible/awx#16698) by moving the
Subquery-based latest-summary logic into HostRelatedSearchMixin so
InventoryHostsList, GroupHostsList, and GroupAllHostsList honor the same
query params as the global hosts list.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address review: validate filter values and apply both summary filters
Require True for last_job_host_summary failed query params, apply failed
and not-failed filters independently (AND semantics), add GroupHostsList
coverage, and document mixin helpers.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Update schema to add pattern fields for credential types
* Update pattern field to be camel case to match pattern field on OPTIONS
* Fix lint issues
* Add unit tests for clean text pattern hook
* [AAP-93690] Add CleanText validation and OPTIONS patterns to subscriptions endpoint
Add SubscriptionCredentialsSerializer (PlainSerializerCleanTextMixin) for
/api/v2/config/subscriptions/ so POST requests validate credential string
fields through the CleanText enforcement pipeline.
- subscriptions_client_id, subscriptions_username: Tier 2 (free-text)
- subscriptions_client_secret, subscriptions_password: excluded (secrets)
- name_fields = frozenset() ensures Tier 2, not Tier 1
- Fake Meta.model satisfies CleanTextMixin audit logging and OPTIONS metadata
- ApiV2SubscriptionView gains serializer_class and get_serializer() so the
existing metadata machinery advertises validation patterns on OPTIONS
- post() runs serializer.is_valid() before credential logic, preserving all
existing error messages, status codes, and $encrypted$ handling
- Functional tests: unsafe-input rejection, secret exclusion, safe passthrough,
OPTIONS pattern presence/absence, all existing happy/error paths still pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Chai Bot <chai-bot@redhat.com>
* [AAP-93690] Preserve credential whitespace and improve docstring coverage
DRF CharField strips leading/trailing whitespace by default, which
silently altered subscription credentials before passing them to
validate_rh. Set trim_whitespace=False on all four credential fields
so serializer.validated_data preserves the exact submitted values.
Add four regression tests proving whitespace is passed unchanged to
validate_rh for username, password, client_id, and client_secret.
Add docstrings to concrete_model(), check_permissions(), post(), and
the ApiV2SubscriptionView class to meet the 80% docstring coverage
threshold.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Chai Bot <chai-bot@redhat.com>
* Accept explicit JSON null on SubscriptionCredentialsSerializer fields
DRF CharField rejects explicit JSON null by default, causing a 400
"This field may not be null" error when a frontend sends null for the
unused credential pair (e.g. client_id=null, client_secret=null with a
valid username/password). Add allow_null=True on all four optional
fields so the serializer passes null through to the view, where the
existing falsy-check credential-selection logic already handles it
correctly.
Includes a focused regression test proving a valid username/password
pair with null client fields is accepted and calls validate_rh with
the basic-auth pair.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Chai Bot <chai-bot@redhat.com>
* fix(subscriptions): supply request/view context to SubscriptionCredentialsSerializer
ApiV2SubscriptionView.get_serializer() now injects the standard DRF
serializer context (request, format, view) so that CleanTextMixin's
_log_validation_failure() can attribute rejected input to the
authenticated user and client IP.
post() is updated to call self.get_serializer(data=request.data)
instead of constructing the serializer class directly, ensuring
consistent context propagation.
A focused regression test asserts that the CleanText warning emitted
for unsafe subscription input includes the authenticated username
and client IP address.
Signed-off-by: redhat-chai-bot <redhat-chai-bot@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: redhat-chai-bot <redhat-chai-bot@users.noreply.github.com>
* remove jira ref, decouple test from DAB working
---------
Signed-off-by: Chai Bot <chai-bot@redhat.com>
Signed-off-by: redhat-chai-bot <redhat-chai-bot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix 500 error on /api/v2/schedules/ from orphaned token cleanup schedule
delete_clear_tokens_sjt() relied on cascade to remove the associated
schedule when deleting the cleanup_tokens SystemJobTemplate. In migration
context with polymorphic models, the cascade did not reliably fire,
leaving an orphaned schedule whose unified_job_template referenced a
missing row.
Explicitly delete associated schedules and clear stale next_schedule
references before deleting the SJT. Add defensive handling in
ScheduleSerializer.get_related() and get_summary_fields() so orphaned
schedules degrade gracefully instead of raising RelatedObjectDoesNotExist.
Fix the existing test assertion that used a JOIN-based query incapable of
detecting orphans, and add coverage for next_schedule reference cleanup.
Fixes: AAP-92564
* Add migration to clean up orphaned token cleanup schedules
Databases that already ran migration 0204 may have orphaned "Cleanup
Expired OAuth 2 Tokens" schedules left behind by the broken cascade.
Remove them and clear any stale UnifiedJobTemplate.next_schedule
references. Add a test that simulates the orphan scenario by deleting
the UJT via raw SQL and verifying the migration repairs it.
Ref: AAP-92564
* Re-parent migration 0212 for gateway backport to stable-2.7
Apply the gateway backport migration pattern (AAP-87599): re-parent
0212_cleanup_orphaned_token_schedules to depend on 0207_merge (the
common ancestor with tower/stable-2.7) instead of 0211. Add
0213_merge_0211_0212 to converge the parallel paths on devel.
This allows the backport to tower/stable-2.7 to use the identical
0212 migration file with no changes.
Ref: AAP-92564
* Test next_schedule cleanup on a surviving template
Use the cleanup_sessions SystemJobTemplate (which survives the deletion)
as the next_schedule reference holder, so the assertion actually proves
the UPDATE ... SET next_schedule = NULL logic works rather than being
masked by the referenced UJT itself being deleted.
Ref: AAP-92564
* Fix test: create surviving SJT instead of relying on migration data
The CI test DB does not have the cleanup_sessions SystemJobTemplate
from migration 0078. Use get_or_create to ensure the surviving UJT
exists within the test itself.
Ref: AAP-92564
* Renumber cleanup migration to 0208 and bump 0208-0211 to 0209-0212
Replace the gateway backport pattern (re-parented migration + merge
migration) with a simpler approach: slot the cleanup migration as 0208
directly after the merge point, and renumber the subsequent migrations.
This keeps the migration graph linear on all branches and preserves
the ability to backport 0209-0212 individually in the future.
Ref: AAP-92564
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add cleantextmixin to settings serializer
* Make excluded_fields a cached property
* Add test coverage for category_slug all
* Fix OPTIONS 500 for settings CleanTextMixin pattern injection
* Apply validation for json and list fields
---------
Co-authored-by: Peter Braun <pbraun@redhat.com>
* feat(api): advertise validation patterns on JSON sub-key fields
Credential type inputs.fields/metadata and notification OPTIONS now include
pattern and pattern_description for non-secret string sub-keys when
ENHANCED_INPUT_VALIDATION_ENABLED is on.
* Fix lint issue
* Fix sonarqube issue
* feat(api): advertise validation patterns on top-level CharFields
Wires DAB's inject_clean_text_patterns into awx.api.metadata.Metadata so
top-level CharField OPTIONS metadata gets pattern/patternDescription/flags,
matching the JSON sub-key pattern exposure already in place. No-op until
ENHANCED_INPUT_VALIDATION_ENABLED is on and a serializer mixes in CleanTextMixin.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* Increase test coverage
* Fix lint issues
* Address review feedback on validation pattern injection
Import TIER2_PATTERN_DESCRIPTION from DAB instead of keeping a manually
synced duplicate, and short-circuit inject_patterns_into_field_list
before iterating fields when enhanced input validation is disabled.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* Avoid mutating shared field dicts in inject_patterns_into_field_list
Shallow-copy each field before injecting pattern metadata so callers
that pass module-level schema dicts (e.g. ManagedCredentialType.registry
entries) don't get them mutated in place, matching the existing
copy-before-mutate approach in inject_patterns_into_init_parameters.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* Add validation pattern for survey fields
---------
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
* AAP-78694 - Add CleanTextMixin to AWX resource serializers
Applies django-ansible-base's CleanTextMixin to the AWX serializers
listed in AAP-78694: Organization, User, Team, Credential(+Create),
CredentialType, Project, JobTemplate, WorkflowJobTemplate, Inventory,
Host, Group, InventorySource, InstanceGroup, Instance,
ExecutionEnvironment, NotificationTemplate, Schedule, AdHocCommand,
WorkflowJobTemplateNode, WorkflowJobTemplateNodeCreateApproval,
JobLaunch, and WorkflowJobLaunch.
Adds a local PromptFieldCleanTextMixin for serializers whose model
derives from LaunchTimeConfigBase, where scm_branch/limit/job_tags/
skip_tags are NullablePromptPseudoField descriptors invisible to the
mixin's normal model-field auto-discovery.
Per-serializer excluded_fields/name_fields overrides are documented
inline where a field's existing behavior would otherwise conflict
with Tier 1/2 validation (Jinja2-templated fields, secret fields,
and existing colon-containing name/hostname syntax).
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Cover CleanTextMixin gaps in Copy, Bulk, and Job Detail paths
Three write paths were found to bypass the Tier 1/2 validation applied
elsewhere for the same resources:
- CopySerializer (used by the Copy button/endpoint for EE, Project,
Credential, JobTemplate, WorkflowJobTemplate, NotificationTemplate)
accepted a user-supplied name with zero validation. Adds a new
PlainSerializerCleanTextMixin for plain serializers.Serializer
subclasses with no backing model.
- BulkJobNodeSerializer's validate() intentionally skips
LaunchConfigurationBaseSerializer.validate() (it assumes
unified_job_template/inventory/execution_environment are model
instances, but bulk nodes pass raw PKs to avoid a per-node DB
lookup), which meant it also skipped CleanTextMixin. Adds
PromptFieldCleanTextMixin._run_clean_text_validation() so the
Tier 1/2 checks still run despite that skip. BulkJobLaunchSerializer
and WorkflowJobNodeSerializer are also covered.
- JobSerializer (backing the narrow JobDetail PUT/PATCH window while
a job's status is "new") had no coverage at all.
Confirmed live against a running controller that a Copy with a
malicious name previously succeeded (201) where the equivalent PATCH
on the same resource was already rejected (400); both are rejected
consistently after this change.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Fix NotificationTemplateSerializer validate() bypass
validate() returned early without calling super().validate() whenever
notification_configuration was absent from the request (e.g. a PATCH
that only changes name/description), which skipped CleanTextMixin
entirely on that path. Same bug class as the BulkJobNodeSerializer
MRO-skip already fixed, caught by code review.
Confirmed live: a PATCH with only a malicious name (no
notification_configuration) previously succeeded; now rejected with
400, while a legitimate name-only PATCH still succeeds.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Add CleanTextMixin to the previously out-of-scope API-only serializers
LabelSerializer, CredentialInputSourceSerializer, and
WorkflowApprovalTemplateSerializer were originally scoped out ("no UI
exposure") but are now in scope per updated ticket guidance.
WorkflowApprovalTemplateSerializer backs the update path for the same
model WorkflowJobTemplateNodeCreateApprovalSerializer already protects
on create -- without this, an approval name rejected at creation could
be rewritten to anything via PATCH.
Confirmed live: Label create and WorkflowApprovalTemplate update both
reject a malicious name (400) and accept a valid one.
CredentialInputSourceSerializer verified at the code level only (plain
CharField input_field_name, no custom validators; metadata is a
JSONSchemaField holding vault paths/keys, not secret values) --
setting up a live external-credential-type chain in this environment
wasn't done.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Correct HostSerializer name_fields demotion comment
The comment claimed the Tier 1 -> Tier 2 demotion was needed for
"hostname:port" syntax, but that colon is stripped from attrs['name']
in validate() before it reaches CleanTextMixin -- that case never
needed the demotion. What actually needs it: names with any colon
count other than exactly 1, which _get_host_port_from_name passes
through untouched -- most notably IPv6 addresses, which its own
comment says it explicitly does not parse. Caught by code review.
Confirmed live: an IPv6-style host name (2 colons) is accepted
unmodified (201).
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Validate non-secret notification_configuration keys
Replaced the blanket notification_configuration exclusion with a
precise excluded_json_keys covering only the actual secret fields.
Unlike Credential.inputs (admin-definable custom types with arbitrary
secret fields), notification_type is a fixed models.CharField(choices)
-- every backend class ships in awx/main/notifications/ and is known
at code-authoring time, so the secret key set can be a static list
rather than resolved per-instance.
Enumerated every init_parameters entry with type "password" across
all 8 backends (email/irc/webhook password, slack/pagerduty token,
twilio account_token, grafana grafana_key, awssns
aws_secret_access_key/aws_session_token) and confirmed none of those
names are used for a non-secret value in a different backend.
Previously-unvalidated, UI-exposed fields like webhook url and slack
channels are now covered. Confirmed live: a javascript: webhook url
and a <script>-tagged slack channel are now rejected (400), while a
weird-looking token/password value is still accepted unchanged.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Exclude extra_data on launch-config serializers
extra_data (WorkflowJobTemplateNode/WorkflowJobNode/Schedule) plays
the same role as extra_vars elsewhere and is submitted in the same
two formats -- a dict, or a raw YAML/JSON string (parse_yaml_or_json
handles both). Its model field is JSONBlob, whose get_internal_type()
reports "TextField" for legacy DB-migration reasons, so
CleanTextMixin silently skipped dict payloads (not a str) while
string payloads hit Tier 2's template-injection blocklist and
incorrectly rejected legitimate Jinja variable references -- a real
regression once enforcement is on, caught by code review.
First attempt put the exclusion on the shared
LaunchConfigurationBaseSerializer base and silently did nothing:
CleanTextMixin's own excluded_fields = frozenset() default sits
earlier in the MRO than that base class for every subclass (mixin is
first in the bases tuple), so the base class's value was always
shadowed. Verified with a standalone MRO test before settling on
setting excluded_fields directly on each leaf serializer
(WorkflowJobTemplateNodeSerializer, WorkflowJobNodeSerializer,
ScheduleSerializer), matching every other exclusion in this file.
Confirmed live on both WorkflowJobTemplateNodeSerializer and
ScheduleSerializer: a YAML-string extra_data with a Jinja reference
is accepted (previously 400), while other Tier 1/2 fields on the same
serializers are still rejected as expected.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* AAP-78694 - Validate non-secret Credential.inputs keys dynamically
Unlike NotificationTemplate.notification_configuration (fixed set of
backend classes, static excluded_json_keys works), CredentialType is
admin-definable via a real "Create a new custom credential type"
endpoint, so the set of secret field names in Credential.inputs can't
be known at class-definition time -- it varies per credential's own
type, via that type's own inputs schema ("secret": true flags,
exposed as CredentialType.secret_fields).
validate() resolves the credential_type from attrs (create) or the
existing instance (update) and sets excluded_json_keys as an instance
attribute before chaining into CleanTextMixin.validate(). Setting an
instance attribute rather than a class attribute sidesteps the MRO
trap found while fixing extra_data on launch-config serializers:
CleanTextMixin's own excluded_fields/excluded_json_keys class-level
defaults sit earlier in the MRO than any shared base class further
down the chain, silently shadowing it -- an instance attribute always
wins regardless of MRO position, and correctly threads through
CredentialSerializerCreate and its User/Team/Organization subclasses
for free, since each of their validate() overrides chains through
super() into this one.
Falls back to excluding the whole inputs blob (prior, safe behavior)
if credential_type can't be resolved.
Confirmed live: malicious username (non-secret, including via PATCH
with no credential_type in the payload) is rejected; a dangerous-
looking password value and a real unencrypted SSH private key
(secret fields) are both still accepted unchanged.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* Increase test coverage
* Cover remaining serializer branches for Sonar gate
Line coverage is already over 90%; Sonar's quality gate uses combined
line+branch coverage. These unit tests exercise previously untested
methods so the PR can clear the 90% new-code threshold.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
* style(api): format uncovered serializer tests for ruff
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Assisted-by: Cursor Grok 4.6 (SpaceXAI / Cursor)
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(api): set Mock.name as an attribute, not a constructor kwarg
unittest.mock.Mock(name=...) only sets the mock's repr name, so
getattr(obj, 'name') returned another Mock and four coverage tests
failed in api-test.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Assisted-by: Cursor Grok 4.6 (SpaceXAI / Cursor)
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(api): keep a single call inside pytest.raises blocks
Sonar python:S5918 flags exception tests that construct mocks or
serializers inside the raises context.
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Assisted-by: Cursor Grok 4.6 (SpaceXAI / Cursor)
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add more test coverage
---------
Signed-off-by: Pratyush Bhandari <prbhanda@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Remove dead Host.last_job and Host.last_job_host_summary FK fields
PR #16332 replaced these denormalized ForeignKey columns with dynamic
derivation via Host.latest_summary, but left the columns in place.
Nothing writes to or reads from them anymore — drop the dead weight.
- Remove last_job and last_job_host_summary ForeignKey fields from Host
- Convert HostSerializer to use SerializerMethodField instead of FK-backed fields
- Remove redundant JobHostSummary.latest_for_host/latest_job_for_host classmethods
- Add migration 0206 to drop the columns
- Update tests to use Host.latest_summary instead of removed fields/methods
* Remove dead last_job_host_summary_id UPDATE from cleanup_jobs
The Host.last_job_host_summary FK was removed in the prior commit,
so the raw SQL UPDATE that NULLed it before deleting JHS rows now
references a non-existent column and crashes.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace ResourceMixin/accessible_objects/accessible_pk_qs with DAB RBAC access_qs/access_ids_qs
Remove the ResourceMixin class and migrate all callers to the new
DAB RBAC queryset API: access_qs replaces accessible_objects and
access_ids_qs replaces accessible_pk_qs, using permission strings
(view, change, use, etc.) instead of role field names.
Also updates prefetch_page_capabilities to use permission-based
capability definitions instead of role field references.
Restore UNION pattern in TeamAccess.filtered_queryset
The access_qs migration accidentally collapsed the UNION into an OR,
which can be 48x slower at scale (AAP-83319).
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-81173 — Replace 4-way OR with UNION in unified job list RBAC query
The UnifiedJobAccess.filtered_queryset() method used a 4-way OR to
determine which unified jobs a user can see. Under load, the OR forces
PostgreSQL to evaluate all four branches in a single plan, preventing
branch-specific index optimization and causing 35 hours of DB time per
30-minute Scale Lab window.
Split each RBAC branch (template read_role, inventory update, ad-hoc
command, org auditor) into separate querysets combined with UNION, giving
the planner an independent optimal plan per branch. The UNION result is
wrapped in pk__in= for compatibility with BaseAccess.get_queryset()
prefetch_related and the workflowapproval filter.
This follows the same pattern proven in AAP-83319 (team list UNION fix).
* AAP-81173 — Add UnifiedJobPagination to prevent COUNT regression
The pk__in UNION pattern used for RBAC filtering forces the large
outer table as the driving table for COUNT(*), requiring PostgreSQL
to materialize all subquery result sets. On large deployments this
produces catastrophic query times (see AAP-83773 for the identical
issue on activity_stream).
Override the paginator count to use an unfiltered
UnifiedJob.objects.count() — the over-count is acceptable for
pagination UI. Also fix test_unified_job_list_rando_sees_nothing
to assert on results length instead of count, since count is now
unfiltered.
* AAP-81173 — Add unit tests for UnifiedJobPagination coverage
Cover the UnifiedJobPaginator.count cached property and the
count_disabled branch in UnifiedJobPagination.paginate_queryset
to satisfy SonarCloud's 90% new-code coverage gate.
* AAP-81173 — Address review feedback: save/restore paginator class, format consistency
- Fix Pagination.paginate_queryset() to save/restore django_paginator_class
instead of hardcoding DjangoPaginator in the finally block. This lets
subclasses set the class attribute without needing to override the method.
- Remove UnifiedJobPagination.paginate_queryset() override — now only needs
to set django_paginator_class = UnifiedJobPaginator as a class attribute.
- Format by_org_auditor consistently with the other three UNION branches.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* AAP-83773 — Use unfiltered count for activity stream pagination
Forward-port of tower#7606 (stable-2.6).
The RBAC-filtered COUNT(*) on activity_stream takes ~36 min per call
on large tables (713K rows) due to the pk__in subquery shape introduced
by the AAP-81860 LEFT JOIN fix. Replace with an unfiltered table count
for pagination -- an approximate over-count is harmless for UI page
navigation while the actual page results remain RBAC-filtered.
* AAP-83773 — Add unit tests for ActivityStreamPagination
Cover ActivityStreamPaginator and ActivityStreamPagination to satisfy
SonarCloud coverage gate on new lines in awx/api/pagination.py.
* AAP-83773 — Save/restore django_paginator_class in base Pagination
Address review feedback from lallen92: use a save/restore pattern in
Pagination.paginate_queryset() so subclasses only need to set the class
attribute. Remove the now-redundant paginate_queryset override from
ActivityStreamPagination.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Liam Allen <lallen@redhat.com>
* AAP-82745 — Replace O(N) correlated subqueries in org list with bulk aggregation
The org list endpoint computed per-org user and admin counts using two
correlated subqueries against roleuserassignment — one per org per role
type. At scale this was the #1 worst query in the customer DB (677ms
mean, 336K calls, 3,801 min total DB time).
Replace with a single flat conditional-aggregation query that scans
roleuserassignment once regardless of org count. Also simplify the
detail view to use direct .count() calls and add a query count
regression test.
* Remove dead code: org_counts is always populated after refactor
* Address review feedback on detail view and test
- Detail view: combine two .count() calls into single .aggregate()
with conditional counts, matching the list-view pattern
- Use .update() in else branch for consistency with if branch
- Remove redundant setup_managed_roles fixture (transitive via
organization_resource_creator)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
feat: add exclude query parameter to unified_jobs and jobs list endpoints
Allow clients to exclude heavy fields (artifacts, extra_vars) from list
responses via ?exclude=artifacts,extra_vars. These fields are now included
by default instead of being stripped.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AAP-81517 — Optimize HostList API: conditional DISTINCT + composite index
Make .distinct() conditional on host_filter being set — without it the
RBAC IN subquery on a direct FK cannot produce duplicates, so DISTINCT
is pure overhead. Add composite index (host_id, id DESC) on
main_jobhostsummary so the with_latest_summary_id() correlated subquery
can use an index-only top-1 scan instead of scanning and sorting.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix cartesian product in organization user/admin count queries
The organizations list and detail endpoints annotated each org with user and admin counts using two Count() calls that traverse the Role.members M2M. Django generated two LEFT JOINs on the same through table, crossing every member row with every admin row before COUNT(DISTINCT) reduced the product.
At scale (2,617 members × 46,233 admins) this produced 120M intermediate rows and 96-second query times, causing 504 timeouts.
Replace with independent Subquery expressions that each query main_rbac_roles_members separately - no cross product.
Fixes: AAP-72817
Fixes: AAP-72480
* Fix variable names which do not meet coding standards
* Fix formatting inconsistency in organization detail subquery annotation
Break the long .annotate() line across multiple lines to match the style used in mixin.py.
* Rewrite org count subqueries to use DAB RBAC models
Replace old RBAC Role.members.through subqueries with
RoleUserAssignment-based correlated subqueries, querying
managed RoleDefinitions ('Organization Member' / 'Organization Admin')
directly. This aligns with the DAB RBAC migration direction and
eliminates dependency on the deprecated ImplicitRoleField M2M tables
for these counts.
Update test fixtures to use RoleDefinition.give_permission() and
add setup_managed_roles where needed.
* Fix collection tests: set up managed role definitions
The DAB RBAC migration to use RoleUserAssignment subqueries in
organization views requires managed role definitions (Organization
Member, Organization Admin) to exist in the test database.
Add an autouse fixture to the collection test conftest that calls
setup_managed_role_definitions() before each test.
* Add setup_managed_roles fixture to functional tests hitting org views
Tests that hit organization list/detail views now require the
setup_managed_roles fixture to pre-create the Organization Member
and Organization Admin RoleDefinition objects used by the DAB RBAC
subqueries.
* Revert setup_managed_roles from ext_auditor tests
The setup_managed_roles fixture conflicts with the ext_auditor_rd
fixture by deleting the Alien Auditor role definition. These tests
don't need it — the defensive view code handles missing role
definitions gracefully.
* Handle missing Organization Member/Admin role definitions gracefully
Use filter().first() instead of get() for RoleDefinition lookups in
organization list and detail views. Returns 0 for user/admin counts
when role definitions are not yet created, preventing 500 errors in
environments where post_migrate signals haven't run.
* Cast OuterRef('pk') to TextField for RoleUserAssignment.object_id comparison
RoleUserAssignment.object_id is a TextField, but OuterRef('pk') on
Organization produces an integer. PostgreSQL strictly rejects text = integer
comparisons. Use Cast() to explicitly convert the PK to text.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
feat: inject x-ai-description from overlay file during schema generation
Many endpoints have human-readable AI descriptions that were added
downstream in aap-mcp-server (PRs #73 and #119) but never backported
as @extend_schema_if_available decorators. This causes 470 out of 631
x-ai-description entries to be lost every time the spec is regenerated.
Add a JSON overlay file (openapi_ai_descriptions.json) containing the
missing descriptions keyed by operationId, and a drf-spectacular
postprocessing hook that merges them into the generated schema for any
operation that doesn't already have x-ai-description from a decorator.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move plugin loading to lazy-on-first-access, DB sync to dispatcher
Remove credential type and inventory plugin loading from Django's
app.ready() path. In-memory registries (ManagedCredentialType.registry
and InventorySourceOptions.injectors) are now populated lazily on first
access via LazyLoadDict, a dict subclass that calls a loader function
on the first read operation. This ensures web workers, dispatcher
workers, and management commands all get the registries populated
exactly when needed, without eager loading at startup.
The DB sync (CredentialType.setup_tower_managed_defaults) is moved to
the dispatcher's startup task, where it only needs to run once per
deployment rather than in every Django process.
Co-Authored-By: Alan Rominger <arominge@redhat.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* INCLUDE_AWX_VAR_PREFIX to USE_TOWER_VAR_PREFIX boolean toggle replace the include legacy prefix boolean with a tower-or-awx toggle USE_TOWER_VAR_PREFIX=True (default) emits only tower_ prefixed variables, false emits only awx_ (deprecated)
* Clean up dead constant and cache get_job_variable_prefixes() calls
* Revise tests to reflect new behavior
* Fix fragile fallback test to actually exercise getattr default
* Fix mock target for settings fallback test
* AAP-12516 [option 2] Handle nested workflow artifacts via root node `ancestor_artifacts` (#16381)
* Add new test for artfact precedence upstream node vs outer workflow
* Fix bugs, upstream artifacts come first for precedence
* Track nested artifacts path through ancestor_artifacts on root nodes
* Fix case where first root node did not get the vars
* touchup comment
* Prevent conflict with sliced jobs hack
* Reorder URLs so that Django debug toolbar can work (#16352)
* Reorder URLs so that Django debug toolbar can work
* Move comment with URL move
* feat: support for oidc credential /test endpoint (#16370)
Adds support for testing external credentials that use OIDC workload identity tokens.
When FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED is enabled, the /test endpoints return
JWT payload details alongside test results.
- Add OIDC credential test endpoints with job template selection
- Return JWT payload and secret value in test response
- Maintain backward compatibility (detail field for errors)
- Add comprehensive unit and functional tests
- Refactor shared error handling logic
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
* Bind the install bundle to the ansible.receptor collection 2.0.8 version (#16396)
* [Devel] Config Endpoint Optimization (#16389)
* Improved performance of the config endpoint by reducing database queries in GET /api/controller/v2/config/
* Fix OIDC workload identity for inventory sync (#16390)
The cloud credential used by inventory updates was not going through
the OIDC workload identity token flow because it lives outside the
normal _credentials list. This overrides populate_workload_identity_tokens
in RunInventoryUpdate to include the cloud credential as an
additional_credentials argument to the base implementation, and
patches get_cloud_credential on the instance so the injector picks up
the credential with OIDC context intact.
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: integrate awx-tui to the awx_devel image (#16399)
* Aap 45980 (#16395)
* support bitbucket_dc webhooks
* add test
* update docs
* fix import for refactored method (#16394)
retrieve_workload_identity_jwt_with_claims is now
in a separate utility file, not in jobs.py
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* AAP-70257 controller collection should retry transient HTTP errors with exponential backoff. (#16415)
controller collection should retry transient HTTP errors with exponential backoff
* AAP-71844 Fix rrule fast-forward across DST boundaries (#16407)
Fix rrule fast-forward producing wrong occurrences across DST boundaries
The UTC round-trip in _fast_forward_rrule shifts the dtstart's local
hour when the original and fast-forwarded times are in different DST
periods. Since dateutil generates HOURLY occurrences by stepping in
local time, the shifted hour changes the set of reachable hours. With
BYHOUR constraints this causes a ValueError crash; without BYHOUR,
occurrences are silently shifted by 1 hour.
Fix by performing all arithmetic in the dtstart's original timezone.
Python aware-datetime subtraction already computes absolute elapsed
time regardless of timezone, so the UTC conversion was unnecessary
for correctness and actively harmful during fall-back ambiguity.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Correctly restrict push actions to ownership repos (#16398)
* Correctly restrict push actions to ownership repos
* Use standard action to see if push actions should run
* Run spec job for 2.6 and higher
* Be even more restrictve, do not push if on a fork
* [Devel] Performance Optimization for Select Hosts Query (#16413)
* Fixed black reformating
* Make test simulate 500k hosts in real world scenario
* feat: improve unauthorized response on aap deployments (#16422)
* fix: do not include secret values in the credentials test endpoint an… (#16425)
fix: do not include secret values in the credentials test endpoint and add a guard to make sure credentials are testable
* [devel backport] AAP-41742: Fix workflow node update failing when JT has unprompted labels (#16426)
* AAP-41742: Fix workflow node update failing when JT has unprompted labels
PATCH extra_data on a workflow node fails with
{"labels":["Field is not configured to prompt on launch."]}
when the node has labels associated but the JT has
ask_labels_on_launch=False.
The serializer was passing all persisted M2M state from prompts_dict()
to _accept_or_ignore_job_kwargs() on every PATCH, re-validating
unchanged fields. Fix scopes validation to only the fields in the
request; full re-validation still occurs when unified_job_template
is being changed.
* Capture attrs keys before _build_mock_obj mutates them
_build_mock_obj() pops pseudo-fields (limit, scm_branch, job_tags,
etc.) from attrs. Computing requested_prompt_fields after the pop
would miss those fields and skip their ask_on_launch validation.
* Include survey_passwords when validating extra_vars prompts
prompts_dict() emits survey_passwords alongside extra_vars.
_accept_or_ignore_job_kwargs uses it to decrypt encrypted survey
values before validation. Without it, encrypted password blobs
are validated as-is against the survey spec.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add test to ensure credential secret values are not returned (#16434)
* AAP-68024 perf: derive last_job_host_summary from query instead of denormalized FK (#16332)
* perf: stop eagerly updating Host.last_job_host_summary on every job completion
The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.
Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.
Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK
The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix .pk AttributeError, add job_template annotations, annotate host sublists
- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs
Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove stale failures_url from deprecated DashboardView
The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to changed files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: replace 10 subquery annotations with bulk prefetch
Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').
This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.
- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary
Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.
- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: scope annotation to views, restore license_error/canceled_on
- Remove with_latest_summary_id() from HostManager.get_queryset() to
avoid applying the correlated subquery to every Host query globally
(count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
host-serving views only
- Restore license_error and canceled_on to last_job summary fields
to avoid breaking API change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard _fetch_all() to skip bulk-attach on non-annotated querysets
Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove name from last_job_host_summary and canceled_on from last_job summary
Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary
Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to test_host_queryset.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Fix flake8 F841: remove unused job1/job2 variables in tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Add comment explaining why Prefetch was not used for host latest summary
Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix null handling to keep old behavior
---------
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
* [AAP-72722] Use url instead of jwt_aud for workload identity audience (#16432)
* [AAP-72722] Use url instead of jwt_aud for workload identity audience
The OIDC credential plugin's jwt_aud field is being removed. Use the
plugin's url field as the audience when requesting workload identity
tokens, since the target service URL is the appropriate audience value.
Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [Devel] Optimize host_list_rbac query (#16408)
* Defer ansible_facts in HostManager to avoid fetching large JSON column in host list queries (AAP-68023)
The host list endpoint (GET /api/v2/hosts/) fetches the ansible_facts
JSON column unnecessarily, contributing to the 7.8s median query time
at scale. This column can be very large and is not used by the list
serializer.
Changes:
- HostManager.get_queryset() now defers ansible_facts
- finish_fact_cache call site uses .only(*HOST_FACTS_FIELDS) to eagerly
load ansible_facts when actually needed, avoiding N+1 queries
- Unit test mocks updated to support .only() queryset chaining
- Points DAB dependency at the RBAC query optimization branch for
combined testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
* fix: constructed inventories no longer increase the host count (#16433)
* Fix version worktree (#16431)
* git worktree friendly precomit install
* worktrees don't have a .git directory. Before, docker-compose would
trigger pre-commit install and fail.
* make docker-compose work in git worktree
* AWX tries to discover the version via info stored in .git/ dir.
setuptools-scm is capable of finding the .git/ dir, starting from a
worktree, but is unable because only the worktree is mapped into the
container, not the .git/ dir itself. Thus, we have to detect and pass
the version into the container from outside. That is why this change
landed in the Makefile.
* fix: as_user() gateway session cookie fallback (#16437)
Add a fallback that checks for `gateway_sessionid` when no cookie
matches `session_cookie_name`, mirroring the existing fallback in
`Connection.login()`. The finally block now cleans up whichever
cookie name was actually used.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Pass setting to dispatcherd so it can be configured (#16438)
* fix: allow blank password field to fix OpenAPI schema validation (#16440)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* first pass porting over metrics
* move settings to defaults
* add more unit tests
* update unit tests
* lint fixes
* more lint fixes
* refactor and address feedback
* remove the api views
* remove model and move helper functions out of licensing
* add settings to API, fix tests, refactoring
* fix circular import
* update tests
* remove duplicate code, handle edge cases, use clearer naming, add test coverage
* update test for changes in ship()
* remove unneeded setting
* _discover_org should account for verify-tls=False
* directly assign settings, detect url, update tests
* log errors close to occurance
* rename function for clarity, focus on critical tests
* rename for clarity, lint fixes
* fix test params, priority for org discovery
* fix test failures and linting
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
Co-authored-by: Tong He <68936428+unnecessary-username@users.noreply.github.com>
Co-authored-by: Stevenson Michel <iamstevensonmichel@outlook.com>
Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
Co-authored-by: Dave Mulford <dmulford@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: Sean Sullivan <ssulliva@redhat.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Dan Leehr <dleehr@users.noreply.github.com>
Co-authored-by: Lila Yasin <lyasin@redhat.com>
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* Consolidate implementation of same-org validation rule
* Update tests for the simplified validation
* Still do validation with deferance to the new callback
* Correctly falsy handling in view logic
* [AAP-72722] Use url instead of jwt_aud for workload identity audience
The OIDC credential plugin's jwt_aud field is being removed. Use the
plugin's url field as the audience when requesting workload identity
tokens, since the target service URL is the appropriate audience value.
Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: stop eagerly updating Host.last_job_host_summary on every job completion
The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id
on every host touched by a job. In the Q4CY25 scale lab this query had
a median execution time of 75 seconds due to index churn on main_host.
Replace all reads of the denormalized FK with a new classmethod
JobHostSummary.latest_for_host(host_id) that queries for the most
recent summary on demand. This eliminates the write-side bulk_update
of last_job_host_summary_id entirely.
Changes:
- Add JobHostSummary.latest_for_host() classmethod
- Serializer: use latest_for_host() instead of obj.last_job_host_summary
- Dashboard view: use subquery instead of FK traversal for failed hosts
- Inventory.update_computed_fields: use subquery for failed host count
- events.py: remove last_job_host_summary_id from bulk_update
- signals.py: simplify _update_host_last_jhs to only update last_job
- access.py/managers.py: remove select_related/defer through the FK
The FK field on Host is left in place for now (removal requires a
migration) but is no longer written to.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix .pk AttributeError, add job_template annotations, annotate host sublists
- Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related)
- Add job_template_id and job_template_name to subquery annotations so list
views include these fields in summary_fields.last_job (matching detail views)
- Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob
subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue)
- Annotate all host sublist views (InventoryHostsList, GroupHostsList,
GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update test_events to use JobHostSummary.latest_for_host instead of stale FKs
Tests were asserting host.last_job_id and host.last_job_host_summary_id
which are no longer updated. Use JobHostSummary.latest_for_host() to
derive the same data, matching the new read-time derivation approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove stale failures_url from deprecated DashboardView
The failures_url linked to ?last_job_host_summary__failed=True which
filters on the now-stale FK. The dashboard count itself was already
fixed to use a subquery annotation. Since DashboardView is deprecated
and has_active_failures is a SerializerMethodField (not filterable),
remove the failures_url entirely rather than creating a custom filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to changed files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: replace 10 subquery annotations with bulk prefetch
Instead of annotating every host queryset with 10 correlated subqueries
(summary + job + job_template fields), annotate only _latest_summary_id
and bulk-fetch the full JobHostSummary objects after pagination via
select_related('job', 'job__job_template').
This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN
query, addressing review feedback about annotation overhead on host list
views.
- _annotate_host_latest_summary: only annotates _latest_summary_id
- _prefetch_latest_summaries: bulk-fetches and attaches to host objects
- HostSummaryPrefetchMixin: hooks into list() after pagination
- Serializer uses real JobHostSummary objects (no more AnnotatedSummary)
- to_representation always overwrites stale FK values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary
Per review feedback, replace the view-level HostSummaryPrefetchMixin
with a custom QuerySet that bulk-attaches summaries at evaluation time
(like prefetch_related), and a Host.latest_summary property as the
single access point.
- HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch
JobHostSummary objects with select_related after queryset evaluation
- HostManager now inherits from the custom queryset via from_queryset()
- Host.latest_summary property: uses cache if available, falls back to
individual query
- Remove _annotate_host_latest_summary, _prefetch_latest_summaries,
HostSummaryPrefetchMixin from views — no more list() override needed
- Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS
- Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: scope annotation to views, restore license_error/canceled_on
- Remove with_latest_summary_id() from HostManager.get_queryset() to
avoid applying the correlated subquery to every Host query globally
(count, exists, internal relations)
- Apply with_latest_summary_id() in get_queryset() of the 6
host-serving views only
- Restore license_error and canceled_on to last_job summary fields
to avoid breaking API change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Guard _fetch_all() to skip bulk-attach on non-annotated querysets
Without this guard, _fetch_all() would set _latest_summary_cache=None
on every host in non-annotated querysets (e.g. Host.objects.filter()),
masking the per-object fallback query in Host.latest_summary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove name from last_job_host_summary and canceled_on from last_job summary
Per reviewer feedback: these fields were not in the original API contract
via SUMMARIZABLE_FK_FIELDS and their addition would be an API change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary
Tests cover:
- with_latest_summary_id() annotation and most-recent selection
- _fetch_all() bulk-attach behavior on annotated querysets
- _fetch_all() skips non-annotated querysets (preserves fallback)
- .count() and .exists() do NOT trigger _fetch_all
- Host.latest_summary cache hits (zero queries) and fallback
- Host.latest_job property
- select_related on bulk-attached summaries (no N+1)
- Chaining preserves annotation
- Multiple jobs / partial host coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply black formatting to test_host_queryset.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Fix flake8 F841: remove unused job1/job2 variables in tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
* Add comment explaining why Prefetch was not used for host latest summary
Django Prefetch cannot handle latest per group -- [:1] slicing fetches
1 record globally, not per host (Django ticket #26780). The custom
_fetch_all override uses the same 2-query pattern as prefetch_related
internally, customized for this use case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix null handling to keep old behavior
---------
Signed-off-by: Ben Thomasson <bthomass@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: AlanCoding <arominge@redhat.com>
* AAP-41742: Fix workflow node update failing when JT has unprompted labels
PATCH extra_data on a workflow node fails with
{"labels":["Field is not configured to prompt on launch."]}
when the node has labels associated but the JT has
ask_labels_on_launch=False.
The serializer was passing all persisted M2M state from prompts_dict()
to _accept_or_ignore_job_kwargs() on every PATCH, re-validating
unchanged fields. Fix scopes validation to only the fields in the
request; full re-validation still occurs when unified_job_template
is being changed.
* Capture attrs keys before _build_mock_obj mutates them
_build_mock_obj() pops pseudo-fields (limit, scm_branch, job_tags,
etc.) from attrs. Computing requested_prompt_fields after the pop
would miss those fields and skip their ask_on_launch validation.
* Include survey_passwords when validating extra_vars prompts
prompts_dict() emits survey_passwords alongside extra_vars.
_accept_or_ignore_job_kwargs uses it to decrypt encrypted survey
values before validation. Without it, encrypted password blobs
are validated as-is against the survey spec.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Adds support for testing external credentials that use OIDC workload identity tokens.
When FEATURE_OIDC_WORKLOAD_IDENTITY_ENABLED is enabled, the /test endpoints return
JWT payload details alongside test results.
- Add OIDC credential test endpoints with job template selection
- Return JWT payload and secret value in test response
- Maintain backward compatibility (detail field for errors)
- Add comprehensive unit and functional tests
- Refactor shared error handling logic
Co-authored-by: Daniel Finca <dfinca@redhat.com>
Co-authored-by: melissalkelly <melissalkelly1@gmail.com>
Django 5.2 restricts LogoutView to POST only (deprecated in 4.1,
removed in 5.0+). Without this fix, GET requests to /api/logout/
return 405 Method Not Allowed.
Add http_method_names override and a get() method that delegates
to post() where auth_logout() actually runs
The schema-swagger-ui URL was removed from awx/api/urls/urls.py in
d7eb714859 when docs endpoints moved to DAB's api_documentation app,
but the reverse call in ApiRootView was not removed, causing a
NoReverseMatch error in development mode.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Jake Jackson
lgalisandCursor
Elijah DeLeeandJake Jackson
Pratyush Bhandari
Maeve HofferandClaude Opus 4.6
Liam Allen
Dirk JulichandClaude Opus 4.6
Alan RomingerandClaude Opus 4.6
Dirk Julich
Chris SantiagoandClaude Opus 4.6
Rodrigo Toshiaki HorieandClaude Opus 4.6
Lila Yasin
Stevenson Michel
Dan Leehr
Chris Meyers
Chris Meyers
