Dirk JulichandClaude Opus 4.6
5613b5ac6c
* AAP-96085: Decouple collection cache from project cache_id Introduce a requirements-hash-keyed cache (reqcache_<hash>/) so that ansible-galaxy collection/role installs are reused across project commits when requirements files haven't changed. Previously, any commit to the project repo invalidated the entire cache, triggering a full reinstall of all collections — a single collection like community.general 11.0.0 (~3,500 files) adds 40-90s of I/O on SAN-backed storage. * Address review feedback: atomic cache publish, bounded eviction, path-aware hash - Atomic publish: _sync_requirements_cache() copies into a temp directory and renames into place, so a failed copytree cannot leave a partial reqcache directory that looks valid to subsequent cache-hit checks - Bounded eviction: clear_project_cache() evicts the oldest reqcache_* directories when the count exceeds max_reqcache (default 10), preventing unbounded storage growth from repeated requirements changes - Path-aware hash: requirements_hash now includes file paths alongside checksums via zip(), so swapping content between requirements files correctly invalidates the cache * Protect current reqcache from eviction during cleanup Read the requirements hash from stage/.requirements_hash and exclude the matching reqcache_* entry from age-based eviction. Without this, a cache hit on the oldest entry could be evicted before the job uses it. * AAP-96176: Cache tarball for make_local_copy() to reduce small-file I/O After populating reqcache_<hash>/, create a tarball of each subfolder (requirements_collections.tar, requirements_roles.tar). make_local_copy() prefers the tarball when available — copies and extracts one file instead of shutil.copytree() across potentially thousands of individual collection files. Falls back to copytree when no tarball exists (upgrade compatibility). On SAN-backed storage, this significantly reduces per-job copy I/O by eliminating per-file source reads. * Fix direct-cache tarball cross-reference and partial extraction fallback Remove reqcache tarball lookup on direct cache hits — the direct cache could diverge from the reqcache (e.g. galaxy server changes not in the requirements hash), serving stale content. The tarball optimization still works for reqcache fallback hits where cache_subpath + '.tar' resolves directly. Clean up partially extracted destination before copytree fallback so a failed tarball extraction doesn't leave behind a directory that causes FileExistsError. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Caution
The last release of this repository was released on Jul 2, 2024. Releases of this project are now paused during a large scale refactoring. For more information, follow the Forum and - more specifically - see the various communications on the matter:
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
To install AWX, please view the Install guide.
To learn more about using AWX, view the AWX docs site.
The AWX Project Frequently Asked Questions can be found here.
The AWX logos and branding assets are covered by our trademark guidelines.
Contributing
- Refer to the Contributing guide to get started developing, testing, and building AWX.
- All code submissions are made through pull requests against the
develbranch. - All contributors must use
git commit --signofffor any commit to be merged and agree that usage of--signoffconstitutes agreement with the terms of DCO 1.1 - Take care to make sure no merge commits are in the submission, and use
git rebasevs.git mergefor this reason. - If submitting a large code change, it's a good idea to join discuss via the Ansible Forum. This helps everyone know what's going on, and it also helps save time and effort if the community decides some changes are needed.
Reporting Issues
If you're experiencing a problem that you feel is a bug in AWX or have ideas for improving AWX, we encourage you to open an issue and share your feedback. But before opening a new issue, we ask that you please take a look at our Issues guide.
Code of Conduct
We require all of our community members and contributors to adhere to the Ansible code of conduct. If you have questions or need assistance, please reach out to our community team at codeofconduct@ansible.com
Get Involved
We welcome your feedback and ideas via the Ansible Forum.
For a full list of all the ways to talk with the Ansible Community, see the AWX Communication guide.