mirror of
https://github.com/ansible/awx.git
synced 2026-10-29 22:03:20 -02:30
8565291e43e17cd1ee5333e82e57aeb230e6574e
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Dirk JulichandClaude Opus 4.6
|
a17f5c167b |
Add SSRF egress filtering and allowlist to notification backends (#16679)
* AAP-94827 Add SSRF egress filtering and allowlist to notification backends
Add url_validation.py to block notification URLs that resolve to
private, loopback, link-local, or reserved addresses. Admins can
allowlist specific IPs, CIDR ranges, or hostnames via the new
NOTIFICATION_IP_ALLOW_LIST system setting.
The allowlist ships together with the filter to avoid breaking
webhook integrations targeting internal (RFC1918) endpoints.
* AAP-94827 Mock validate_url in functional redirect test
The functional test uses a fake hostname ('some.fake.url') that fails
DNS resolution when validate_url tries to resolve it in CI.
* AAP-94163 Add non-overridable deny list for link-local addresses
Link-local addresses (169.254.0.0/16, fe80::/10) are now always blocked
regardless of allowlist settings, protecting cloud metadata endpoints
even if DNS is subverted for an allowlisted hostname.
Requested by Product Security as a condition for approving hostname
support in the NOTIFICATION_IP_ALLOW_LIST setting.
* AAP-94163 Block IPv4-mapped link-local addresses in deny list
Unwrap IPv4-mapped IPv6 addresses (e.g. ::ffff:169.254.169.254) before
checking the link-local deny list, preventing a bypass where the mapped
address doesn't match either the IPv4 or IPv6 deny network directly.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Dirk JulichandClaude Opus 4.6
|
5613b5ac6c |
AAP-96085: Decouple collection cache from project cache_id (#16710)
* AAP-96085: Decouple collection cache from project cache_id Introduce a requirements-hash-keyed cache (reqcache_<hash>/) so that ansible-galaxy collection/role installs are reused across project commits when requirements files haven't changed. Previously, any commit to the project repo invalidated the entire cache, triggering a full reinstall of all collections — a single collection like community.general 11.0.0 (~3,500 files) adds 40-90s of I/O on SAN-backed storage. * Address review feedback: atomic cache publish, bounded eviction, path-aware hash - Atomic publish: _sync_requirements_cache() copies into a temp directory and renames into place, so a failed copytree cannot leave a partial reqcache directory that looks valid to subsequent cache-hit checks - Bounded eviction: clear_project_cache() evicts the oldest reqcache_* directories when the count exceeds max_reqcache (default 10), preventing unbounded storage growth from repeated requirements changes - Path-aware hash: requirements_hash now includes file paths alongside checksums via zip(), so swapping content between requirements files correctly invalidates the cache * Protect current reqcache from eviction during cleanup Read the requirements hash from stage/.requirements_hash and exclude the matching reqcache_* entry from age-based eviction. Without this, a cache hit on the oldest entry could be evicted before the job uses it. * AAP-96176: Cache tarball for make_local_copy() to reduce small-file I/O After populating reqcache_<hash>/, create a tarball of each subfolder (requirements_collections.tar, requirements_roles.tar). make_local_copy() prefers the tarball when available — copies and extracts one file instead of shutil.copytree() across potentially thousands of individual collection files. Falls back to copytree when no tarball exists (upgrade compatibility). On SAN-backed storage, this significantly reduces per-job copy I/O by eliminating per-file source reads. * Fix direct-cache tarball cross-reference and partial extraction fallback Remove reqcache tarball lookup on direct cache hits — the direct cache could diverge from the reqcache (e.g. galaxy server changes not in the requirements hash), serving stale content. The tarball optimization still works for reqcache fallback hits where cache_subpath + '.tar' resolves directly. Clean up partially extracted destination before copytree fallback so a failed tarball extraction doesn't leave behind a directory that causes FileExistsError. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Dirk JulichandClaude Opus 4.6
|
918ba2ef36 |
AAP-95887 Use SKIP LOCKED to prevent lock convoy on parent JT update (#16719)
* AAP-95887 Use SKIP LOCKED to prevent lock convoy on parent JT update When allow_simultaneous is enabled and many jobs complete concurrently, _update_parent_instance() serializes all sessions on the same JT row lock. In an example customer incident, single-row UPDATEs averaged 11-25 minutes with 635+ sessions queued. Use select_for_update(skip_locked=True) for the on_commit callback path so concurrent updaters that cannot acquire the row lock return immediately instead of queuing. This is safe because all fields on the JT (current_job, status, last_job, last_job_failed) are "latest wins" — at least one session always acquires the lock and writes the current state. Non-allow_simultaneous jobs are unchanged (synchronous, no skip). * Skip lock contention test on SQLite select_for_update(skip_locked=True) is silently a no-op on SQLite. Mark the lock contention test to skip when running against SQLite (the default test database). The test runs when tests are executed against PostgreSQL. * Use mock for lock contention test instead of threading The test suite uses SQLite where select_for_update is a no-op. Follow codebase convention (e.g. sqlite_copy fixture) of mocking PostgreSQL-specific behavior: mock select_for_update to return an empty queryset, simulating a locked row. Removes the separate test class and threading/psycopg dependencies. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Dirk JulichandClaude Opus 4.6
|
dad4da74b9 |
Fix 500 on /api/v2/schedules/ from orphaned token cleanup schedule (#16660)
* Fix 500 error on /api/v2/schedules/ from orphaned token cleanup schedule delete_clear_tokens_sjt() relied on cascade to remove the associated schedule when deleting the cleanup_tokens SystemJobTemplate. In migration context with polymorphic models, the cascade did not reliably fire, leaving an orphaned schedule whose unified_job_template referenced a missing row. Explicitly delete associated schedules and clear stale next_schedule references before deleting the SJT. Add defensive handling in ScheduleSerializer.get_related() and get_summary_fields() so orphaned schedules degrade gracefully instead of raising RelatedObjectDoesNotExist. Fix the existing test assertion that used a JOIN-based query incapable of detecting orphans, and add coverage for next_schedule reference cleanup. Fixes: AAP-92564 * Add migration to clean up orphaned token cleanup schedules Databases that already ran migration 0204 may have orphaned "Cleanup Expired OAuth 2 Tokens" schedules left behind by the broken cascade. Remove them and clear any stale UnifiedJobTemplate.next_schedule references. Add a test that simulates the orphan scenario by deleting the UJT via raw SQL and verifying the migration repairs it. Ref: AAP-92564 * Re-parent migration 0212 for gateway backport to stable-2.7 Apply the gateway backport migration pattern (AAP-87599): re-parent 0212_cleanup_orphaned_token_schedules to depend on 0207_merge (the common ancestor with tower/stable-2.7) instead of 0211. Add 0213_merge_0211_0212 to converge the parallel paths on devel. This allows the backport to tower/stable-2.7 to use the identical 0212 migration file with no changes. Ref: AAP-92564 * Test next_schedule cleanup on a surviving template Use the cleanup_sessions SystemJobTemplate (which survives the deletion) as the next_schedule reference holder, so the assertion actually proves the UPDATE ... SET next_schedule = NULL logic works rather than being masked by the referenced UJT itself being deleted. Ref: AAP-92564 * Fix test: create surviving SJT instead of relying on migration data The CI test DB does not have the cleanup_sessions SystemJobTemplate from migration 0078. Use get_or_create to ensure the surviving UJT exists within the test itself. Ref: AAP-92564 * Renumber cleanup migration to 0208 and bump 0208-0211 to 0209-0212 Replace the gateway backport pattern (re-parented migration + merge migration) with a simpler approach: slot the cleanup migration as 0208 directly after the merge point, and renumber the subsequent migrations. This keeps the migration graph linear on all branches and preserves the ability to backport 0209-0212 individually in the future. Ref: AAP-92564 --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Dirk JulichandClaude Opus 4.6
|
2a6dd41186 |
AAP-89067: Fix stale image purge when EE image reference form changes (#16664)
* AAP-89067: Fix stale image purge when EE image reference form changes The _handle_image_cleanup signal compared images by string only, so two references to the same underlying image (e.g. tag vs digest) were treated as different. This caused podman rmi to strip all names from a still-needed image, breaking environments especially in disconnected installs. Add layered guards: skip cleanup for managed images (settings and managed=True EE objects), detect tag-vs-digest reference form changes for the same repository, and check other EEs for cross-reference form conflicts on delete. Genuinely different versions (v1→v2) still get cleaned up. * Reduce cognitive complexity of _handle_image_cleanup Extract the delete-path loop into _other_ee_shares_repo_with_different_ref to flatten nesting and bring cognitive complexity under the SonarQube threshold of 15. * Extend managed-image guard to match by repo when ref forms differ _is_managed_image previously only did exact string comparison against managed settings, so repo@sha256:... would not match repo:tag in CONTROL_PLANE_EXECUTION_ENVIRONMENT or GLOBAL_JOB_EXECUTION_ENVIRONMENTS. Now also compares by repository + reference form (tag vs digest). * Add docstring to _handle_image_cleanup --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |