Adding AWX_CONTAINER_GROUP_POD_LAUNCH_TIMEOUT in defaults.py
--- Adding a new global variable AWX_CONTAINER_GROUP_POD_LAUNCH_TIMEOUT in defaults.py
--- Passing this parameter in receptor_params in AWXReceptorJob class
SUMMARY
Related issue : ansible/receptor#315
Corresponding PR on receptor side : ansible/receptor#316
I have added a global variable AWX_CONTAINER_GROUP_POD_LAUNCH_TIMEOUT ( and set default value as 5 minutes ).
This will be passed onto Receptor in case a job is container group task.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
ADDITIONAL INFORMATION
File Name : awx/main/tasks.py
Class Name : AWXReceptorJob
Function name: receptor_params(self)
Before the change the value of receptor_params before returning was :
{'secret_kube_config': '---\n'
'apiVersion: v1\n'
'clusters:\n'
'- cluster:\n'
' insecure-skip-tls-verify: true\n'
' server: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'contexts:\n'
'- context:\n'
' cluster: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' namespace: default\n'
' user: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'current-context: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'kind: Config\n'
'preferences: {}\n'
'users:\n'
'- name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' user:\n'
' token: '
'sha256~otM0Z2vC8qfzm3eV0tfz-0V4DXKPqLC_Pp1gxXKuC-c\n',
'secret_kube_pod': '---\n'
'apiVersion: v1\n'
'kind: Pod\n'
'metadata:\n'
' labels:\n'
' ansible-awx: 04366fdb-65ec-4efc-bb01-685e9ca7e05b\n'
" ansible-awx-job-id: '21'\n"
' name: automation-job-21\n'
' namespace: default\n'
'spec:\n'
' containers:\n'
' - args:\n'
' - ansible-runner\n'
' - worker\n'
' - --private-data-dir=/runner\n'
' image: quay.io/ansible/awx-ee:0.2.0\n'
' name: worker\n'}
>>>
After the change the value of receptor_params before returning is :
{'pod_pending_timeout': '5m',
'secret_kube_config': '---\n'
'apiVersion: v1\n'
'clusters:\n'
'- cluster:\n'
' insecure-skip-tls-verify: true\n'
' server: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'contexts:\n'
'- context:\n'
' cluster: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' namespace: default\n'
' user: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'current-context: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'kind: Config\n'
'preferences: {}\n'
'users:\n'
'- name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' user:\n'
' token: '
'sha256~otM0Z2vC8qfzm3eV0tfz-0V4DXKPqLC_Pp1gxXKuC-c\n',
'secret_kube_pod': '---\n'
'apiVersion: v1\n'
'kind: Pod\n'
'metadata:\n'
' labels:\n'
' ansible-awx: 04366fdb-65ec-4efc-bb01-685e9ca7e05b\n'
" ansible-awx-job-id: '21'\n"
' name: automation-job-21\n'
' namespace: default\n'
'spec:\n'
' containers:\n'
' - args:\n'
' - ansible-runner\n'
' - worker\n'
' - --private-data-dir=/runner\n'
' image: quay.io/ansible/awx-ee:0.2.0\n'
' name: worker\n'}
>>>
We can see that an extra parameter has been added : 'pod_pending_timeout': '5m',
####Additional changes :
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRIES from defaults.py as it was not being used anywhere
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRY_DELAY from defaults.py as it was not being used anywhere
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
Reviewed-by: Seth Foster <None>
--- Passing this parameter in receptor_params in AWXReceptorJob class
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRIES from defaults.py as it was not being used anywhere
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRY_DELAY from defaults.py as it was not being used anywhere
Adding host_metrics for customers who don't have automation analytics
SUMMARY
I am trying to accomplish the following:
Create a new model that stores details about hostname
create corresponding migrations for it
Update the table (main_hostmetrics) after playbook execution
retrieve the data with custom awx-manage command : awx-manage host_metrics --since <date> --until <date>
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
ADDITIONAL INFORMATION
Reviewed-by: Ryan Petrello <ryan@ryanpetrello.com>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Use inventory and env private_data_dir subfolders
SUMMARY
This is another part of trying to get AWX to follow ansible-runner best practices.
See docs:
https://ansible-runner.readthedocs.io/en/stable/intro.html#runner-input-directory-hierarchy
So what runner & people expect is:
.
├── env
│ ├── envvars
│ ├── extravars
│ ├── passwords
│ ├── cmdline
│ ├── settings
│ └── ssh_key
├── inventory
│ └── hosts
└── project
├── test.yml
└── roles
└── testrole
├── defaults
├── handlers
├── meta
├── README.md
├── tasks
├── tests
└── vars
What we were producing was something more like
├── cp
├── env
│ ├── cmdline
│ ├── envvars
│ ├── extravars
│ ├── passwords
│ ├── settings
│ └── ssh_key
├── project
│ ├── ansible.cfg
│ ├── ansible_env.yml
│ ├── async_tasks.yml
<snip>
│ └── vault.yml
└── tmp3evnorsu
There was no inventory folder, and inventory was stored in tmp3evnorsu as a top-level file. Any credential files were also saved in that same structure.
With this change, it's more like:
├── cp
├── env
│ ├── cmdline
│ ├── envvars
│ ├── extravars
│ ├── passwords
│ └── settings
├── inventory
│ └── tmp94xjm0d7
└── project
├── ansible.cfg
├── ansible_env.yml
├── async_tasks.yml
<snip>
└── vault.yml
and any credential files (starting with tmp) are dumped into the env folder.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
14.1.0
ADDITIONAL INFORMATION
The motivation for this is that ansible-runner may require specific mounts for every file of directory in the private_data_dir. As such, we really don't want randomized filenames in the top level directory. I don't know for sure this is the direction we are going to go, but this seems like a defensible change on its own, to better adhere to the practices someone familiar with ansible-runner would expect.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Chris Meyers <None>
Updates LingUI Again
SUMMARY
This updates ling ui to bring in their latest release so that we don't have to run npm extract-strings every time we add/update a <Plural> or a string with a variable inside it.
ISSUE TYPE
-dependency upgrade
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
close db and cache connection in new threads
It's not safe to share or re-use django db connections nor django cache connections across new threads nor processes.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Commits settings.DEFAULT_EXECUTION_ENVIRONMENTS updates into the database
SUMMARY
Fixes: ansible/awx-operator#256
cc: @shanemcd
Cannot update execution environment version in place by updating the AWX spec via awx-operator. For example, consider the scenario below:
spec:
tower_ee_images:
- image: registry.tatu.home/ansible/awx-ee:0.1.0
name: My Custom Execution Environment
The expectation would be once you modify the AWX spec to the version below, the EE should reflect this change once the awx-operator finishes its reconciliation.
After:
spec:
tower_ee_images:
- image: registry.tatu.home/ansible/awx-ee:0.2.0
name: My Custom Execution Environment
-- deployment got updated
- args:
- receptor
- --config
- /etc/receptor.conf
image: registry.tatu.home/ansible/awx-ee:0.2.0
imagePullPolicy: IfNotPresent
name: awx-devel-ee-ee
Besides the awx-operator updated the YAML deployment as expected, the change did not hit the database.
awx=# select * from main_executionenvironment;
id | created | modified | description | image | managed_by_tower | created_by_id | credential_id | modified_by_id | organization_id | name | pull
----+---------------+---------------+-------------+----------------+------------------+---------------+---------------+----------------+-----------------+----------------+------
1 | 2021-05-11 15.| 2021-05-11 15.| | registry.tatu..| t | 1 | | 1 | | My Custom Exec.|
|.:24:02.674302.|.:24:02.674327.| |.home/ansible/a.| | | | | |.ution Environm.|
|.+00 |.+00 | |.wx-ee:0.1.0 | | | | | |.ent |
(1 row)
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
devel
ADDITIONAL INFORMATION
Quick proof of concept
>>> settings.DEFAULT_EXECUTION_ENVIRONMENTS
[{'name': 'My Custom Execution Environment', 'image': 'registry.tatu.home/ansible/awx-ee:0.2.0'}]
>>> [a.name for a in ExecutionEnvironment.objects.all()]
['My Custom Execution Environment']
>>> [a.image for a in ExecutionEnvironment.objects.all()]
['registry.tatu.home/ansible/awx-ee:0.2.0']
*** updating to new but keeping name
>>> _, created = ExecutionEnvironment.objects.update_or_create(name='My Custom Execution Environment', defaults={'image': 'registry.tatu.home/ansible/awx-ee:0.3.0', 'managed_by_tower': True})
>>> [a.name for a in ExecutionEnvironment.objects.all()]
['My Custom Execution Environment']
>>> [a.image for a in ExecutionEnvironment.objects.all()]
['registry.tatu.home/ansible/awx-ee:0.3.0']
*** adding a new name
>>> _, created = ExecutionEnvironment.objects.update_or_create(name='My Custom Execution Environment2', defaults={'image': 'registry.tatu.home/ansible/awx-ee:0.3.0', 'managed_by_tower': True})
>>> [a.name for a in ExecutionEnvironment.objects.all()]
['My Custom Execution Environment2', 'My Custom Execution Environment']
>>> [a.image for a in ExecutionEnvironment.objects.all()]
['registry.tatu.home/ansible/awx-ee:0.3.0', 'registry.tatu.home/ansible/awx-ee:0.3.0']
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Add a check for the existence of pre-commit.sh
SUMMARY
so that you don't get stuck when working on branches that don't have this yet.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Have the project update playbook warn if role/collection syncing is disabled
In recent AWX, a galaxy credential (even a blank one for galaxy.ansible.com) is required to sync role/collection content. This is done so that server precedence can be properly set, and so that it can be configured to pull only from a private content host.
This does lead to bug reports where the credentials are not set, and users don't understand why their content is not syncing. This makes that more clear.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Adds support for pendo initialization across the app
SUMMARY
We were already bootstrapping pendo as part of the subscription code I just moved that code to a more general place. When the app container mounts (after login or on refresh) we check to see if the pendo flag is turned on. If it is, we initialize pendo. If it's not then we do nothing. If a user goes into settings and manually changes the pendo tracking setting then we trigger a hard reload of the browser tab (to take the new setting into account and either initialize or not). This functionality existed in the old UI as well.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Michael Abashian <None>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add default error boundary around screens
SUMMARY
Catch any unhandled non-async errors with an error boundary around screens. This will show a generic error message instead
of crashing the page.
The react docs decently summarize the general concept of error boundaries. The example code is informative, but there appears to be a few more edge cases that need to be dealt with. Rather than re-implement solutions for those, I decided to pull in a simple and well-tested library.
ADDITIONAL INFORMATION
before
after
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Update with force option if branch_override
Addresses a reported issue where force-pushing to feature branches did not work.
Connect #9371 (comment)
Reviewed-by: Bill Nottingham <None>
created a TransmitterThread class to start transmit jobs within. Surr…
SUMMARY
Surfaced this bug while fixing issue #9952.
Receptor-ctl would hang trying to read from a socket pair that is never closed, when the transmit job was executed yet threw an exception. This anomaly occurred when using a k8s cluster (Openshift in our case) as the execution environment.
FYI - for now, this only applies to container groups.
We moved the transmit function to execute in its own thread, and rejoin the calling function (_run_internal) when it finishes transmitting, and bubbling up exceptions if any occurred in the thread.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
ADDITIONAL INFORMATION
Steps to reproduce bug:
Create an ansible project with some files (or roles) and commit to a git repository
Purposely commit a broken symlink on any file in the project files to git
Create a credential set that authenticates to your k8s cluster in AWX
Create a container instance group that leverages the credentials from Step 3 in AWX
Create a project in AWX that leverages the ansible project/git repo from Step 1
Create a job template that leverages the AWX project from Step 4 and set your EE to the container instance group from Step 3
Execute the job from Step 5
Reviewed-by: Shane McDonald <me@shanemcd.com>
converts inventory groups related groups and related hosts to tables
SUMMARY
This addresses #10132
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Remove source_script from tower_inventory_source integration test
SUMMARY
source_script parameter is on longer available in the current version AWX API(#9822), so we need to remove it from the integration test of tower_inventory_source module
Fixes#10104
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
awx_collections
AWX VERSION
devel
ADDITIONAL INFORMATION
None
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Hideki Saito <saito@fgrep.org>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Use development container when updating dependencies
GOAL
When adding, updating, or removing a package, the resultant changeset to the package.json and package-lock.json files that we check into devel should be consistent and reproducible, regardless of who is doing the update or what machine they're doing the update on.
SUMMARY OF CHANGES
Even though we usually run the ui development tooling outside of the awx development container, we should standardize on using this environment to update packages to ensure some consistency.
Use npm ci instead of npm install as the default package installation command, since we generally want to modify package files only when updating deps
You can run into memory limits from docker or node when building in the container, so increase the node memory limit and add a note to the docs that you may need to increase the container memory limit in your docker-for-mac settings (or similar tool, if you use it)
Reviewed-by: Kersom <None>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add an nginx location directive for the favicon
SUMMARY
Add an nginx location directive for the favicon so that the rewrite rule that adds slashes to the ends of requested
urls doesn't get to it.
related #8618
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
UI
AWX VERSION
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Update login message to avoid duplicate brand name
Returned brandName already contain the name brand Ansible. Update to
avoid name brand repetition.
Also update brandName on document title.
See: #9126
Reviewed-by: Kersom <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fixes bug where workflow approvals page would crash if deleted username was referenced
SUMMARY
link #9163
The tooltip on the status labels for completed workflow approvals attempts to show the username of the user that acted on it. If that user has been deleted then the key is not present in the object. This would cause the page to crash.
To fix this, I conditionally show a different string when the username is not available. If a user wants to see which deleted user acted on the approval they'll have to go to the activity stream.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Add Variables field validation in launch prompt
SUMMARY
Adds validation for the Variables field during launch prompts.
Prevents unexpected expanding of YAML expressions in the default value (similar to #7506) — this was caused by the default value being parsed into a JavaScript object and then re-translated back to YAML when mounting the "Other Prompts" step
addresses #8907
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add instance_group to inventory update serializer
SUMMARY
Inventory updates run remotely, in the execution plane. This adds the instance_group field to the inventory update serializer to help clarify this reality.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Remove all AWX_ISOLATED* fields
I believe this should address #10123. The issue mentions a worker-json.js 404 and, while we should fix that at some point, I don't think it's actually related to the save not working.
SUMMARY
I noticed the job settings form wasn't saving on devel.
After some debugging, I found that removing the AWX_ISOLATED_CHECK_INTERVAL and AWX_ISOLATED_LAUNCH_TIMEOUT made the form saveable again.
It doesn't seem like we use any of the AWX_ISOLATED* fields anymore (not to be confused with AWX_ISOLATION* fields, which we do use) so this PR removes them all.
cc @wenottingham @shanemcd @marshmalien @gamuniz @tiagodread @jbradberry
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
