Add managed by tower as part of the EE details page
Add managed by tower as part of the EE details page.
See: #8171
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Enable ?page_size=1 to Fetch Correct Objects on JT Endpoint
SUMMARY
Problem:
When multiple schedules are made from a single JT and a URL sort is done via:
api/v2/job_templates/[id]/schedules/?page_size=1
... the first half of the results come back with the same ID. 🤔
Solution:
Implementing a backup sort via ID on the schedules model fixes this problem, and now all schedules on the job template endpoint are being fetched properly.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
ADDITIONAL INFORMATION
This might fix#9632
Reviewed-by: Seth Foster <None>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Update the versioning on the docker-compose template
SUMMARY
Some versions of docker-compose will break with the new addition of
name parameters without this.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Installer
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Hide commands that are being run for `make black`
Before:
$ make black
command -v black >/dev/null 2>&1 || { echo "could not find black on your PATH, you may need to \`pip install black\`, or set AWX_IGNORE_BLACK=1" && exit 1; }
could not find black on your PATH, you may need to `pip install black`, or set AWX_IGNORE_BLACK=1
make: *** [Makefile:275: black] Error 1
After:
$ make black
could not find black on your PATH, you may need to `pip install black`, or set AWX_IGNORE_BLACK=1
make: *** [Makefile:275: black] Error 1
Reviewed-by: Ryan Petrello <None>
Remove/Modify Usage of tower-cli (Deprecated) in Collections Test
tower-cli is no longer being maintained; this PR removes three test tasks that are no longer necessary from awx_collection/tests/integration/targets/tower_project_manual/tasks/create_project_dir.yml, and it modifies the last task in that test file to use awx-cli instead.
I also noticed this warning while running Collections tests:
awx/main/tasks.py:2109
/awx_devel/awx/main/tasks.py:2109: DeprecationWarning: invalid escape sequence \s
d['Password:\s*?$'] = 'scm_password' # noqa
This PR modifies that part of /awx_devel/awx/main/tasks.py into a raw string to fix this issue.
FYI, some deprecated Collections modules depend on the usage of tower-cli, but those tests are currently getting skipped:
awx_collection/test/awx/test_send_receive.py::test_receive_send_jt SKIPPED (The tower-cli library is needed to run th...) [ 56%]
awx_collection/test/awx/test_workflow_template.py::test_create_workflow_job_template SKIPPED (The tower-cli library i...) [ 73%]
awx_collection/test/awx/test_workflow_template.py::test_with_nested_workflow SKIPPED (The tower-cli library is needed...) [ 74%]
awx_collection/test/awx/test_workflow_template.py::test_schema_with_branches SKIPPED (The tower-cli library is needed...) [ 74%]
awx_collection/test/awx/test_workflow_template.py::test_with_missing_ujt SKIPPED (The tower-cli library is needed to ...) [ 75%]
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Ryan Petrello <None>
fix up a bug in rsyslogd error handling
@kdelee I'm unclear on why this is actually happening, so I'm just gonna cheat and generate a datestring
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Add EE to the settings page
Allow a system admin to set the global default execution environment.
See: #9088
This PR is also addressing the issue: #9669
Edit:
Details
Reviewed-by: Kersom <None>
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
remove ansible_version from the API config and metrics endpoints
AWX no longer includes Ansible on the control plane and there is no
"global" version of Ansible aside from what's configured at the
Execution Environment level
see: #9472
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Yanis Guenane <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Set a custom name for Docker volumes
SUMMARY
This pull request is related to #9695 and will make sure that the command make docker-compose does not create additional Docker volumes with the wrong prefix.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Installer
AWX VERSION
awx: 18.0.0
ADDITIONAL INFORMATION
The current version of docker-compose.yml.j2 does not care about the Docker volumes created during the execution of migrate.yml. This means new Docker volumes will be created with the wrong prefix, which only contain the preloaded data.
This does not make sense, because I want to keep my data and according to the docs, the following command is necessary to load the demo data.
$ docker exec tools_awx_1 awx-manage create_preload_data
Reviewed-by: Shane McDonald <me@shanemcd.com>
let jupyter install ipython
running akit with ipython on py38:
In [1]: jt = v2.job_templates.create()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/usr/lib64/python3.8/codeop.py in __call__(self, source, filename, symbol)
141
142 def __call__(self, source, filename, symbol):
--> 143 codeob = compile(source, filename, symbol, self.flags, 1)
144 for feature in _features:
145 if codeob.co_flags & feature.compiler_flag:
TypeError: required field "type_ignores" missing from Module
Looks like we need a newer version of ipython:
ipython/ipython#12558 (comment)
.. because we pinned ipython in 2017
a39b1e8
If I try to install the newer ipython alongside jupyter, pip's dep resolver mentions a conflict
.. but if we just install jupyter there are no conflicts and a newer ipython (that avoids the original issue) gets installed.
pip freeze | grep "ipython\|jupyter"
ipython==7.21.0
ipython-genutils==0.2.0
jupyter==1.0.0
jupyter-client==6.1.12
jupyter-console==6.4.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
In [1]: jt = v2.job_templates.create()
In [2]: jt
Out[2]:
{
"id": 12,
"type": "job_template",
"url": "/api/v2/job_templates/12/",
Reviewed-by: Shane McDonald <me@shanemcd.com>
Add subsystem metrics that propagate through Redis
SUMMARY
#9019 -- list of metrics and their purpose / description
#9012#9056#8629
Use Redis to store metrics pertaining to the performance and health of subsystems such as the callback receiver and task manager. It is thread / multiprocess safe and should be fast enough to handle a high volume of data.
This data shows up at the /api/v2/metrics endpoint
You can filter down nodes using /api/v2/metrics/?subsystemonly=1&node=awx-1
You can also filter down to a specific metric,
/api/v2/metrics/?subsystemonly=1&metrics=callback_receiver_events_insert_db_seconds&node=awx-1
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Seth Foster <None>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Loosen Collections vs Tower Version Check
SUMMARY
Connecting issue #9532
This change will make it so that if the major version numbers of Collections + Tower matches, a warning will not get activated.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Collections
AWX VERSION
awx: 17.0.1
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: John Westcott IV <None>
Fix api/v2/metrics data displaying incorrect value
SUMMARY
How to reproduce bug
Disable all instances
Queue up 5 jobs, (if using same JT, enable concurrent jobs)
Enable instance so jobs will start running
Refresh /api/v2/metrics endpoint. You should see a metric that says awx_status_total{status="pending"} 5.0. Once you see this, don't refresh again.
Wait for all jobs to finish.
Now start refreshing the api/v2/metrics endpoint. Every once in a while you will see this pending jobs metric show (awx_status_total{status="pending"} 5.0) even though there are no jobs in pending state
Fix
Use a locally defined prometheus registry instead of a global registry. Each time the endpoint is refreshed, a new, local registry is set up with prometheus objects (Gauge, Info). Since we aren't actually incrementing these metrics across api calls, we should be safe. That is to say, we just lookup the values from the database and set the prometheus values explicitly.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
Reviewed-by: Ryan Petrello <None>
Fixes a bug with ResourceAccess Add
SUMMARY
This addresses #9456 and #9466
The user can now properly add roles to a resource.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Support job cancellation through details panel
SUMMARY
for #7657
edit: also addresses #8838 cc @nixocio
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Adds delete warnings on lists
SUMMARY
This addresses #5546.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Keith Grant <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Kersom <None>
Python: Bump to python 3.8
SUMMARY
Bumping default python used in awx and ansible venv to python 3.8
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
devel
ADDITIONAL INFORMATION
N/A
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Yanis Guenane <None>
Properly set launched_by for jobs launched by scheduled workflows : Issue 5057
SUMMARY
with respect to the issue 5057, there should be a way by which the UI should know , if a job was launched from a workflow , and that workflow was launched from a schedule.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
ADDITIONAL INFORMATION
This PR tries to solve the issue 5057.
There is an edge case , where the API response is not very clear. When a JT is invoked from a workflow , and that workflow is invoked from a schedule, the API response does not convey how the job was launched.
So, I have added the schedule id of the schedule that invoked the workflow and which in turn invokes the JT. The new key in API response is 'launched_by', and it has the schedule id. In all the 5 previous cases mentioned in the initial issue , the 'launched_by' field is blank ( that is empty string ( "" )).
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
Reviewed-by: Nana <natr@hey.com>
fix the large file parsing in project sync
SUMMARY
Fixes the issue of scm update stuck on reading large files because of islice trying to read the whole file.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
Reviewed-by: Ryan Petrello <None>
Reviewed-by: None <None>
Filter credential type by Container Registry for EE
Filter credential type by Container Registry for EE form.
See: #9628
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fix service reference error for cred plugin dev env
SUMMARY
Intended to fix the error I see when running COMPOSE_TAG=devel make docker-compose-credential-plugins:
Reviewed-by: Ryan Petrello <None>
move code linting to a stricter pep8-esque auto-formatting tool, black
black (https://pypi.org/project/black/#description) is a strict superset of PEP8
It's also a tool that auto-formats your code on the fly for you based on its ruleset.
With this PR, you can run make check, and any style issues will automatically be applied.
Additionally, with this PR, if you spin up the development environment using our make targets, you'll automatically get a pre-commit hook installed that automatically runs linting prior to commit.
If you don't like this behavior, or don't want it locally, you can:
~ export AWX_IGNORE_BLACK=1
...but it's important to note that we won't merge your PR if it doesn't adhere to our style guidelines (which will run automatically as part of pre-merge CI).
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Seth Foster <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Update some links and notes in the changelog
SUMMARY
Fix typo, remove duplicate change note, fix a wrong link, add link to the ui virtualenv removal
Reviewed-by: Ryan Petrello <None>
Allow one to select non-global execution environments for organizations
Allow one to select non-global EE when editing an Organization.
See: #9592
All those EE should be present as a choice when editing the Default organization.
Editing Default organization.
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
fix port conflict in dev cluster
SUMMARY
problem: loop adds 100 to ports 7899 and 7999, which would yield 7999 to 8099 on the next iteration, so the 7999 is conflicting
fix: add 1000 instead
Also, haproxy was being defined twice, now it renders once.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
Reviewed-by: Seth Foster <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Instruct git to ignore the .vscode/ directory
SUMMARY
Instruct git to ignore the .vscode/ directory
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
add ouiaID to select and cancel buttons on modals
SUMMARY
Add ouiaId prop to select and cancel button within modals
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Fixes silent error on SCM subform
SUMMARY
This addresses #9373. It disallows the user to select both Update on launch and update on project update. It also adds a bit of info to the tool tip including a link to the project in question so the user can edit the project to allow them to update on launch and on project update
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Create a wrapper directory for the private data dir
Reviewed-by: None <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Fixes Several Bugs
SUMMARY
This address #9485 (Job template project field validate), #9319 (Job Details view only would show job type run, even if it was a job type check, #7516 (changes the Completed Jobs tab for a JT or WFJT to show Jobs since it show completed and pending/running jobs).
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: John Mitchell <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Prevent users from selecting credentials that prompt for passwords on workflow nodes and schedules
SUMMARY
link #8921
If a user selects a job template with default credentials that prompt for passwords (but does not prompt for credentials) then the user should not be allowed to create the node and a different JT must be selected:
If a user selects a credential that prompts for passwords when creating/editing a workflow node or schedule then we show this error:
If a user removes a credential that exists in the default collection of credentials on the JT then it must be replaced. This is the error we show:
If a user attempts to create a schedule for a job template with default credentials that prompt (but does not prompt for credentials) then the API responds with this error:
I believe this UX is consistent with the old UI but I am double checking that now.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fixes crashing wizard, and adds error handle on adding role
SUMMARY
This addresses #8769. It also adds error handling if there is some sort of request error during the submit request.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: John Mitchell <None>
Adds ouiaId's to various buttons
SUMMARY
@tiagodread @unlikelyzero @one-t @akus062381 this will likely break something because I changed some existing ouia-id's so that they are a consistent structure.
^^ Let's let one of them merge this
I also removed an unused component
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: John Hill <johill@redhat.com>
Adds support for html in custom login text
SUMMARY
link #7603
I couldn't come up with a way to do this without breaking up the component and discontinuing use of the LoginPage PF component. This is because LoginPage expects the textContent component (what we use to display the custom login text) to be a string. By using the underlying LoginPage components I reconstructed the login page and got more control over that prop.
The custom message in the old UI supported both strings and HTML:
So we need to support rendering HTML but we need to do it in a safe way. Our solution to that was https://docs.angularjs.org/api/ngSanitize. React doesn't seem to have anything like this built in so I went looking for outside help. html-entities is already included in our project but as best as I can tell that lib is mainly focused on swapping special characters out for html entities. I wanted something that was going to strip the HTML of bits that could be exploited by a malicious actor.
I settled on https://www.npmjs.com/package/sanitize-html because it was a) small and b) actively maintained. The API was simple and let me sanitize the HTML before setting it using dangerouslySetInnerHTML. If we need to tweak the configuration away from the default values then we can certainly do that.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Only attempt to fetch event options on non workflow jobs
SUMMARY
link #9640
This was fallout from output search filtering. We need this request for non workflow jobs so that we can build the search options.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add support for replace/revert on secret credential fields
SUMMARY
link #7256
Note that this only applies to editing an existing credential. You should not see this button on fields when adding a new credential.
When editing an existing credential the replace button should show up on fields where secret is true and the field has an existing value that is not an external credential. Examples:
Fields with external credentials should look the same:
Initially the button tooltip should say Replace. Clicking Replace will clear out the previously saved value and enable the form field:
The tooltip will change to Revert. Clicking Revert will take the field back to it's original state.
I also noticed a race condition which would result in the input fields (subform) not being populated due to the form rendering before the request(s) were completed. I fixed this.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add support for filtering and pagination on job output
SUMMARY
link #6612
link #5906
This PR adds the ability to filter job events and also includes logic to handle fetching filtered job events across different pages.
Note that the verbosity dropdown included in #5906 is not included in this work. I don't think that's possible without api changes.
As part of this work, I converted JobOutput.jsx from a class based component to a functional component. I've tried my best to make sure that all existing functionality has remained the same by comparing the experience of this branch to devel.
Like the old UI, the output filter is disabled while the job is running.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Enable Ansible version to be collected from EEs
SUMMARY
Connecting issue #9473
This PR, along with this Ansible-Runner PR, enables us to obtain the Ansible (core) version for each execution environment that is utilized. This info can be gathered from the new ansible_version column on the main_unifiedjobs table.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
ADDITIONAL INFORMATION
Screenshot/example of the DB output:
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Do not allow user to modify EE managed by tower
Do not allow user to attempt to modify EE managed by tower.
See: #9250
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
refactor payload construction for awxkit
This fixes container_group creation to allow passing
"is_container_group" and "credential" to the "create" method
on instance groups, and refactors other page objects
to use a common utility function to eliminate copy-pasted code
This will help us update to set is_container_group correctly as is now needed since de52ade
Reviewed-by: Ryan Petrello <None>
Remove custom virtual env
Remove custom virtual from the UI.
Also, surface missing-resource warnings on list items for UJTs that were using
custom virtualenvs. And related details page.
See: #9190
Also: #9207
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Add support for Centrify Vault as a credential plugin
replaces #8952
cc @surbhijain1502 @Asharma-bhavna @badrogh
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Chris Meyers <None>
Dont require is_container_group in payload when creating InstanceGroups
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Ryan Petrello <None>
Hashicorp Vault Credential Plugin : Support for namespace
SUMMARY
Added the support for Vault Namespace (Enterprise feature)
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
credential_plugins/hashivault.py
AWX VERSION
1.7.0
ADDITIONAL INFORMATION
Adding specific X-Vault-Namespace header when Namespace option is set.
Reviewed-by: Ryan Petrello <None>
