mirror of
https://github.com/ansible/awx.git
synced 2026-06-23 15:47:49 -02:30
d1b3ae53ae
* perf: stop eagerly updating Host.last_job_host_summary on every job completion The playbook_on_stats wrapup path bulk-updates last_job_host_summary_id on every host touched by a job. In the Q4CY25 scale lab this query had a median execution time of 75 seconds due to index churn on main_host. Replace all reads of the denormalized FK with a new classmethod JobHostSummary.latest_for_host(host_id) that queries for the most recent summary on demand. This eliminates the write-side bulk_update of last_job_host_summary_id entirely. Changes: - Add JobHostSummary.latest_for_host() classmethod - Serializer: use latest_for_host() instead of obj.last_job_host_summary - Dashboard view: use subquery instead of FK traversal for failed hosts - Inventory.update_computed_fields: use subquery for failed host count - events.py: remove last_job_host_summary_id from bulk_update - signals.py: simplify _update_host_last_jhs to only update last_job - access.py/managers.py: remove select_related/defer through the FK The FK field on Host is left in place for now (removal requires a migration) but is no longer written to. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix .pk AttributeError, add job_template annotations, annotate host sublists - Add 'pk' to AnnotatedSummary dynamic type (fixes AttributeError in get_related) - Add job_template_id and job_template_name to subquery annotations so list views include these fields in summary_fields.last_job (matching detail views) - Traverse job__ FK from JobHostSummary instead of using separate UnifiedJob subquery with OuterRef on another annotation (cleaner SQL, avoids alias issue) - Annotate all host sublist views (InventoryHostsList, GroupHostsList, GroupAllHostsList, InventorySourceHostsList) to prevent N+1 queries Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update test_events to use JobHostSummary.latest_for_host instead of stale FKs Tests were asserting host.last_job_id and host.last_job_host_summary_id which are no longer updated. Use JobHostSummary.latest_for_host() to derive the same data, matching the new read-time derivation approach. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove stale failures_url from deprecated DashboardView The failures_url linked to ?last_job_host_summary__failed=True which filters on the now-stale FK. The dashboard count itself was already fixed to use a subquery annotation. Since DashboardView is deprecated and has_active_failures is a SerializerMethodField (not filterable), remove the failures_url entirely rather than creating a custom filter. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply black formatting to changed files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor: replace 10 subquery annotations with bulk prefetch Instead of annotating every host queryset with 10 correlated subqueries (summary + job + job_template fields), annotate only _latest_summary_id and bulk-fetch the full JobHostSummary objects after pagination via select_related('job', 'job__job_template'). This reduces the SQL from 10 correlated subqueries to 1 subquery + 1 IN query, addressing review feedback about annotation overhead on host list views. - _annotate_host_latest_summary: only annotates _latest_summary_id - _prefetch_latest_summaries: bulk-fetches and attaches to host objects - HostSummaryPrefetchMixin: hooks into list() after pagination - Serializer uses real JobHostSummary objects (no more AnnotatedSummary) - to_representation always overwrites stale FK values Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor: move latest summary to QuerySet._fetch_all + Host.latest_summary Per review feedback, replace the view-level HostSummaryPrefetchMixin with a custom QuerySet that bulk-attaches summaries at evaluation time (like prefetch_related), and a Host.latest_summary property as the single access point. - HostLatestSummaryQuerySet: overrides _fetch_all() to bulk-fetch JobHostSummary objects with select_related after queryset evaluation - HostManager now inherits from the custom queryset via from_queryset() - Host.latest_summary property: uses cache if available, falls back to individual query - Remove _annotate_host_latest_summary, _prefetch_latest_summaries, HostSummaryPrefetchMixin from views — no more list() override needed - Remove last_job/last_job_host_summary from SUMMARIZABLE_FK_FIELDS - Serializer uses obj.latest_summary and DEFAULT_SUMMARY_FIELDS loop Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix: scope annotation to views, restore license_error/canceled_on - Remove with_latest_summary_id() from HostManager.get_queryset() to avoid applying the correlated subquery to every Host query globally (count, exists, internal relations) - Apply with_latest_summary_id() in get_queryset() of the 6 host-serving views only - Restore license_error and canceled_on to last_job summary fields to avoid breaking API change Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Guard _fetch_all() to skip bulk-attach on non-annotated querysets Without this guard, _fetch_all() would set _latest_summary_cache=None on every host in non-annotated querysets (e.g. Host.objects.filter()), masking the per-object fallback query in Host.latest_summary. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove name from last_job_host_summary and canceled_on from last_job summary Per reviewer feedback: these fields were not in the original API contract via SUMMARIZABLE_FK_FIELDS and their addition would be an API change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add functional tests for HostLatestSummaryQuerySet and Host.latest_summary Tests cover: - with_latest_summary_id() annotation and most-recent selection - _fetch_all() bulk-attach behavior on annotated querysets - _fetch_all() skips non-annotated querysets (preserves fallback) - .count() and .exists() do NOT trigger _fetch_all - Host.latest_summary cache hits (zero queries) and fallback - Host.latest_job property - select_related on bulk-attached summaries (no N+1) - Chaining preserves annotation - Multiple jobs / partial host coverage Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply black formatting to test_host_queryset.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ben Thomasson <bthomass@redhat.com> * Fix flake8 F841: remove unused job1/job2 variables in tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ben Thomasson <bthomass@redhat.com> * Add comment explaining why Prefetch was not used for host latest summary Django Prefetch cannot handle latest per group -- [:1] slicing fetches 1 record globally, not per host (Django ticket #26780). The custom _fetch_all override uses the same 2-query pattern as prefetch_related internally, customized for this use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix null handling to keep old behavior --------- Signed-off-by: Ben Thomasson <bthomass@redhat.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: AlanCoding <arominge@redhat.com>