Deleted the awx/main/management/commands/graph_jobs.py file and removed the asciichartpy package from requirements. This cleans up unused code and dependencies related to terminal job status graphing.
Upgrade to Django 5.2 LTS with compatibility fixes across fields, migrations, dispatch config, tests, and dev deps.
Dependencies:
- Upgrade django to 5.2.8 and relax requirements.in to >=5.2,<5.3.
- Bump django-debug-toolbar to >=6.0 for compatibility.
Backend:
- awx/conf/fields.py: switch URL TLD regex to use DomainNameValidator.ul in custom URLField.
- awx/main/management/commands/gather_analytics.py: use datetime.timezone.utc for naïve datetime handling.
- awx/main/dispatch/config.py: add mock_publish option; avoid DB access for test runs, set default max_workers, and support a noop broker.
Migrations (SQLite/Postgres compatibility):
- Add awx/main/migrations/_sqlite_helper.py with db-aware AlterIndexTogether/RenameIndex wrappers; consume in 0144_event_partitions.py and 0184_django_indexes.py.
- Update 0187_hop_nodes.py to use CheckConstraint(condition=...).
- Add 0205_alter_instance_peers_alter_job_hosts_and_more.py adjusting through_fields/relations on instance.peers, job.hosts, and role.ancestors.
- _dab_rbac.py: iterate roles with chunk_size=1000 for migration performance.
Tests:
Include hcp_terraform in default credential types in test_credential.py.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
Moved the AddField operation before the RunPython operations for 'rename_jts' and 'rename_projects' in migration 0200_template_name_constraint.py. This ensures the new 'org_unique' field exists before related data migrations are executed.
Fix
```
django.db.utils.ProgrammingError: column main_unifiedjobtemplate.org_unique does not exist
```
while applying migration 0200_template_name_constraint.py
when there's a job template or poject with duplicate name in the same org
Updated setuptools version from 78.1.1 to 80.9.0 in Makefile, requirements.in, and requirements.txt to ensure compatibility and address any potential issues with older versions.
* Update Python dependencies
Relaxed or updated version constraints for several dependencies in requirements files and Makefile, including Cython, asciichartpy, msgpack, python-daemon, and pyyaml. These changes address build issues, remove unnecessary pins, and update to newer compatible versions.
* remove docutils license
* we no longer have this as a dep so we don't need to carry its license
* Update dependencies to address security vulnerabilities
Bumped versions of cryptography, protobuf, and idna in requirements to address CVE-2024-26130, CVE-2025-4565, and CVE-2024-3651. These updates improve security by resolving known vulnerabilities in the affected packages.
---------
Co-authored-by: thedoubl3j <jljacks93@gmail.com>
Remove ALLOW_LOCAL_RESOURCE_MANAGEMENT setting and enable local resource management
This commit removes the ALLOW_LOCAL_RESOURCE_MANAGEMENT setting and all associated
functionality, making the behavior as if the setting is always enabled.
Changes:
- Remove ALLOW_LOCAL_RESOURCE_MANAGEMENT setting from defaults.py
- Remove @immutablesharedfields decorator and all related logic
- Remove decorator applications from Organization, Team, and User API views
- Remove role assignment restrictions in UserRolesList and RoleUsersList
- Remove test file for immutablesharedfields functionality
- Clean up unused imports
Result: Organizations, Teams, and Users can now always be created, modified,
and deleted via the API without platform ingress restrictions.
Remove ALLOW_LOCAL_RESOURCE_MANAGEMENT setting and enable local resource management
This commit removes the ALLOW_LOCAL_RESOURCE_MANAGEMENT setting and all associated
functionality, making the behavior as if the setting is always enabled.
Changes:
- Remove ALLOW_LOCAL_RESOURCE_MANAGEMENT setting from defaults.py
- Remove @immutablesharedfields decorator and all related logic
- Remove decorator applications from Organization, Team, and User API views
- Remove role assignment restrictions in UserRolesList and RoleUsersList
- Remove test file for immutablesharedfields functionality
- Clean up unused imports
Result: Organizations, Teams, and Users can now always be created, modified,
and deleted via the API without platform ingress restrictions.
```
/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/python.py:163:
PytestReturnNotNoneWarning: Expected None, but
awx/main/tests/unit/test_tasks.py::TestJobCredentials::test_custom_environment_injectors_with_boolean_extra_vars
returned ['successful', 0], which will be an error in a future version
of pytest. Did you mean to use `assert` instead of `return`?
```
* Dug into the git blame for this one
060585434a is the commit for any
historians. It was wrongfully carried over from a mock pexpect
implementation. Our new tests are nice. They don't go as far as trying
to run the task so they do not need to mock pexpect. That is why it is
safe to remove this code without finding it a new home.
Co-authored-by: Chris Meyers <chris.meyers.fsu@gmail.com>
* Prevent job pod from mounting serviceaccount token
* Add serializer validation for cg pod_spec_override
Prevent automountServiceAccountToken to be set to true and provide an error message when automountServiceAccountToken is being set to true
* Add `opa_query_path field` for Inventory, Organization and JobTemplate models (#6850)
Add `opa_query_path` model field to Inventory, Organizatio and JobTemplate. Add migration file and expose opa_query_path field in the related API serializers.
* Gather and evaluate `opa_query_path` fields and raise violation exceptions (#6864)
gather and evaluate all opa query related to a job execution during policy evaluation phase
* Add OPA_AUTH_CUSTOM_HEADERS support (#6863)
* Extend policy input data serializers (#6890)
* Extend policy input data serializers
* Update help text for PaC related fields (#6891)
* Remove encrypted from OPA_AUTH_CUSTOMER_HEADER
Unable to encrypt a dict field
---------
Co-authored-by: Jiří Jeřábek (Jiri Jerabek) <Jerabekjirka@email.cz>
Co-authored-by: Alexander Saprykin <cutwatercore@gmail.com>
Co-authored-by: Tina Tien <98424339+tiyiprh@users.noreply.github.com>
* Publish image base on git repo name instead of hard coded to AWX (#15828)
* Fix git credential for devel_image build (#15834)
* Continue if pre-warm cache fail in container build (#15835)
* Use correct devel image for docker-compose (#15836)
* [AAP-39138] - Add DAB Feature Flag common API (#15786)
* Update django-ansible-base reference to ansible-automation-platform/django-ansible-base@stable-2.5
---------
Co-authored-by: Zack Kayyali <zkayyali@redhat.com>
UI_NEXT is no longer being served in 4.6
Removing static file dir for ui-next fixes
```
WARNINGS:
?: (staticfiles.W004) The directory '/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/ui_next/build' in the STATICFILES_DIRS setting does not exist.
CommandError: Inventory with id = 1038181458411569545 cannot be found
```
* Fix CI for newer debian image (#15583)
* Fix CI for newer debian image
Signed-off-by: Rick Elrod <rick@elrod.me>
* Missed one
Signed-off-by: Rick Elrod <rick@elrod.me>
---------
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update ci.yml
---------
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
# Add a postfix to the UI URL patterns for UI URL generated by the API
# example if set to '' UI URL generated by the API for jobs would be $TOWER_URL/jobs
# example if set to 'execution' UI URL generated by the API for jobs would be $TOWER_URL/execution/jobs
* Prevent job pod from mounting serviceaccount token
* Add serializer validation for cg pod_spec_override
Prevent automountServiceAccountToken to be set to true and provide an error message when automountServiceAccountToken is being set to true
* Fallback to use subscription cred for analytic
Fall back to use SUBSCRIPTION_USERNAME/PASSWORD to upload analytic to if REDHAT_USERNAME/PASSWORD are not set
* Improve error message
* Guard against request with no query or data
* Add test for _send_to_analytics
Focus on credentials
* Supress sonarcloud warning about password
* Add test for analytic ship
- use asyncio.get_running_loop() instead of passing around event_loops
- add name to all of the asyncio tasks for easier debugging
we are trying to figure out which task is
```
Task was destroyed but it is pending!
task: <Task pending name='Task-<id>' coro=<RedisConnection._read_data() done, defined at /var/lib/awx/venv/awx/lib64/python3.9/site-packages/aioredis/connection.py:180> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fba77bf1700>()]> cb=[RedisConnection.__init__.<locals>.<lambda>() at /var/lib/awx/venv/awx/lib64/python3.9/site-packages/aioredis/connection.py:168]>
```
is referring to
If RECEPTOR_KEEP_WORK_ON_ERROR is set to true receptor work unit will not be automatically released
Co-Authored-By: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
Links are use to indicate network connectivity and optionally provide alias
it is not needed for communication since all the container are on the awx network
in the prometheus container case since awx_ container now have valid hostname it's no longer required (also i think the link is missing a `-` anyway...)
links also implicitly imply dependency between services in this i see awx container depends on redis and postgres so i switch to depends_on to retain that
Making this change to be podman compatible
because i get
```
Error response from daemon: bad parameter: link is not supported
```
Fix command to set db session timeout
Add quote around the value of the setting
Example failures
```
2024-07-10 13:33:29,237 ERROR [a7e55a64e6744a0e920bb1fd78615e5f] awx.main.dispatch Worker failed to run task awx.main.tasks.system.awx_periodic_scheduler(*[], **{}
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/backends/utils.py", line 87, in _execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/psycopg/cursor.py", line 732, in execute
raise ex.with_traceback(None)
psycopg.errors.SyntaxError: trailing junk after numeric literal at or near "1d"
LINE 1: SET idle_in_transaction_session_timeout = 1d
^
```
Timestamp for activity stream is not indexed result in slow query. switching to ID (which effectively will is order by created time) to improve performance
* Add better 403 error message for Job template create
To create Job template u need access to projects and inventory
---------
Co-authored-by: Chris Meyers <chris.meyers.fsu@gmail.com>
* Add TASK_MANAGER_LOCK_TIMEOUT
`TASK_MANAGER_LOCK_TIMEOUT` controls the `idle_in_transaction_session_timeout` and `idle_session_timeout` configuration for task manager connections and lock in database
hope to prevent the situation that the task instance that holds the lock becomes unresponsive and preventing other instance to be able to run task manager
* Add session timeout to periodic scheduler and all sub task manager locks
Workaround
```
ERROR awx/main/tests/functional/test_licenses.py - pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib'
```
* Periodically sync from share resource provider
- add periodic task `periodic_resource_sync` run once every 15 min
- if `RESOURCE_SERVER` is not configured sync will not run
- only 1 node
example RESOURCE_SERVER configuration
```
RESOURCE_SERVER = {
"URL": "<resource server url>",
"SECRET_KEY": "<resource server auth token>",
"VALIDATE_HTTPS": <True/False>,
}
RESOURCE_SERVICE_PATH = <resource_service_path>
```
- switch to galaxy search API for determining if the version we want to publish already exist
- switch from github action variable to env var for easier copy and paste testing
We have not identify the root cause of wsrelay failure but attempt to make wsrelay restart itself resulted in postgres and redis connection leak. We were not able to fully identify where the redis connection leak comes from so reverting back to failing and removing startsecs 30 will prevent wsrelay to FATAL
```
ERRO[0000] path "/var/lib/awx/.config" exists and it is not owned by the current user
```
start to happen with podman 5
it seems that the config files are no longer needed removing it fixes the problem
- when re-establishing connection to db close old connection
- re-initialize WebSocketRelayManager when restarting asyncio.run
- log and ignore error in cleanup_offline_host (this might come back to bite us)
- cleanup connection when WebSocketRelayManager crash
Hao Liu
