Compare commits

..

71 Commits

Author SHA1 Message Date
Luiz Costa
9e7486b024 WIP Makefile 2022-11-16 16:04:12 -03:00
Shane McDonald
d30c5ca9cd Merge pull request #13200 from shanemcd/disable-work-signing
Disable work signing by default in dev env
2022-11-16 11:23:53 -05:00
Lila Yasin
a3b21b261c Merge pull request #13178 from john-westcott-iv/update_django_patch
Updating the patch release of django per dependabot alerts
2022-11-16 10:58:38 -05:00
Sean Sullivan
d1d60c9ef1 update awx collection workflow module schema with new options (#13162) 2022-11-16 10:47:31 -03:00
Shane McDonald
925e055bb3 Merge pull request #13199 from shanemcd/default-no-external-nodes
Default to 0 execution nodes in dev env
2022-11-15 18:29:08 -05:00
Shane McDonald
9f40d7a05c Disable work signing by default in dev env
Certs are generated on the host and there is currently an issue due to openssl version mispatch between Fedora 36 and CentOS Stream 8 which causes:

tools_awx_1     | ERROR 2022/11/15 17:09:17 could not load signing key file: unknown block type PRIVATE KEY
tools_awx_1     | ERROR 2022/11/15 17:09:17 could not load signing key file: unknown block type PRIVATE KEY
2022-11-15 17:16:07 -05:00
Shane McDonald
968c316c0c Default to 0 execution nodes in dev env 2022-11-15 15:30:11 -05:00
Alan Rominger
2fdce43f9e Bulk save facts, and move to before status change (#12998)
* Facts scaling fixes for large inventory, timing issue

Move save of Ansible facts to before the job status changes
  this is considered an acceptable delay with the other
  performance fixes here

Remove completely unrelated unused facts method

Scale related changes to facts saving:
  Use .iterator() on queryset when looping
  Change save to bulk_update
  Apply bulk_update in batches of 100, to reduce memory
  Only save a single file modtime, avoiding large dict

Use decorator for long func time logging
  update decorator to fill in format statement
2022-11-15 15:18:06 -05:00
jessicamack
0933a96d60 Merge pull request #13176 from jessicamack/13019_add_node_type
Add node_type to instance info and capacity metrics
2022-11-15 12:43:53 -05:00
John Westcott IV
1106367962 Doing a hard pin on django 2022-11-11 13:37:15 -05:00
Jeff Bradberry
721e19e1c8 Merge pull request #13181 from jbradberry/remove-qsstats
Replace the querysets provided by django-qsstats-magic
2022-11-11 10:58:51 -05:00
TVo
87363af615 Merge pull request #13184 from ansible/tvo318-patch-1
Update conf.py
2022-11-10 15:18:57 -07:00
TVo
332c433b6e Update conf.py
Updated variables to reflect automation controller name.
2022-11-10 14:18:25 -07:00
Jeff Bradberry
e029cf7196 Remove the django-qsstats-magic dependency 2022-11-10 15:37:44 -05:00
Joe Garcia
a1d34462b0 Add proper declaration of secret version if present (#13071) 2022-11-10 13:23:29 -05:00
Joe Garcia
e4283841d6 Fixes #13119 #13120 Conjur Cloud support & update brand (#13121)
* Fixes #13119 #13120 Cloud support & update brand

* rm base64 import to pass lint

* Update references across the board

* Removed final reference to CyberArk Conjur Secret Lookup
2022-11-10 13:22:54 -05:00
Jeff Bradberry
477a63d1b4 Replace the querysets provided by django-qsstats-magic
with ones that are constructed to do the same thing, without any
extraneous functionality.
2022-11-10 11:00:39 -05:00
John Westcott IV
4a30cc244f Merge pull request #13177 from john-westcott-iv/updater_script_enhancement
Modify the update.sh script that manages dependencies
2022-11-10 09:15:01 -05:00
Alan Rominger
1f939aa25e Merge pull request #12884 from AlanCoding/is_testing
[tech debt] Move the IS_TESTING method out of settings
2022-11-09 15:29:35 -05:00
Jeff Bradberry
86b0a3d4f1 Merge pull request #13174 from jbradberry/fix-3.11-awxkit
Avoid adding the 'config' subparser twice
2022-11-09 13:37:03 -05:00
John Westcott IV
b269ed48ee Updating the patch release of django per dependabot alerts 2022-11-09 10:24:16 -05:00
John Westcott IV
fe1b37afaf Updating README and update.sh usage 2022-11-09 10:15:36 -05:00
Jessica Mack
87dd8c118d add node_type to instance info and capacity metrics 2022-11-09 07:32:58 -05:00
Alan Rominger
d6004fd2d3 Merge pull request #13153 from AlanCoding/cleanup_args
Refactor test_get_cleanup_task_kwargs_active_jobs and add new test
2022-11-08 16:06:13 -05:00
Jeff Bradberry
3d3e4ad150 Avoid adding the 'config' subparser twice
Once since it is defined as a CustomCommand subclass, and once because
it is an endpoint at the /api/v2/ level.  With Python 3.11 argparse
has become more strict and will raise an exception when you try to
inject duplicate subparsers.
2022-11-08 15:13:35 -05:00
Sarah Akus
81821fd378 Merge pull request #13148 from kialam/fix-host-event-modal-type-error
Fix HostEvent modal TypeError
2022-11-08 14:23:41 -05:00
Alan Rominger
8e5af2b5f2 Merge pull request #13092 from AlanCoding/tags_no_record
Fix error when applying taggit app tags to objects
2022-11-07 23:07:59 -05:00
Sarabraj Singh
918db89dc8 fixed GET and POST descriptors in swagger.json - /api/v2/job_templates/{id}/launch/ endpoint 2022-11-07 16:23:11 -05:00
Hao Liu
6e25a552d3 enable feature branch api-schema test (#12550)
- enable schema upload to s3 bucket for feature branch
- add workflow to delete schema from s3 bucket when feature branch is deleted

Signed-off-by: Hao Liu <haoli@redhat.com>
2022-11-04 16:14:55 +00:00
Michael Abashian
83c48bb5fa Revert "Merge pull request #12993 from AlexSCorey/fixWorkflow"
This reverts commit 75e6366c5e, reversing
changes made to af6fec5592.
2022-11-03 14:39:31 -04:00
Rick Elrod
1c65339a24 No InventoryUpdates when source Project is failed (#13063)
Previously, in some cases, an InventoryUpdate sourced by an SCM project
would still run and be successful even after the project it is sourced
from failed to update. This would happen because the InventoryUpdate
would revert the project back to its last working revision. This
behavior is confusing and inconsistent with how we handle jobs (which
just refuse to launch when the project is failed).

This change pulls out the logic that the job launch serializer and
RunJob#pre_run_hook had implemented (independently) to check if the
project is in a failed state, and puts it into a method on the Project
model. This is then checked in the project launch serializer as well as
the inventory update serializer, along with
SourceControlMixin#sync_and_copy as a fallback for things that don't run
the serializer validation (such as scheduled jobs and WFJT jobs).

Signed-off-by: Rick Elrod <rick@elrod.me>
2022-11-03 13:23:34 -05:00
Alex Corey
75e6366c5e Merge pull request #12993 from AlexSCorey/fixWorkflow
Gives enhanced permissions so that it can update dependabot pr bodies
2022-11-03 12:02:35 -04:00
John Westcott IV
af6fec5592 Merge pull request #12971 from mayaCostantini/precise-migrate-playbook-path
Precise migrate.yml path in development docs
2022-11-03 10:06:18 -04:00
Alan Rominger
893dba7076 Merge pull request #13114 from AlanCoding/use_seth
Remove ssh_key_data fix, handled in runner now
2022-11-02 15:41:00 -04:00
Alan Rominger
d571b9bbbc Refactor test_get_cleanup_task_kwargs_active_jobs and add new test
This takes some logic out of the queryset logic,
  using some established assumptions about the task manager
  if a job lands on a hybrid node (or is a project update) then
  it will have the same controller and execution node

With that established, the queryset can be simplified
2022-11-02 15:14:16 -04:00
Alex Corey
b28cc34ff3 Merge pull request #13141 from ansible/dependabot/npm_and_yarn/awx/ui/devel/patternfly/react-core-4.250.1
Bump @patternfly/react-core from 4.239.0 to 4.250.1 in /awx/ui
2022-11-02 10:49:48 -04:00
dependabot[bot]
776d39f057 Bump @patternfly/react-core from 4.239.0 to 4.250.1 in /awx/ui
Bumps [@patternfly/react-core](https://github.com/patternfly/patternfly-react) from 4.239.0 to 4.250.1.
- [Release notes](https://github.com/patternfly/patternfly-react/releases)
- [Commits](https://github.com/patternfly/patternfly-react/compare/@patternfly/react-core@4.239.0...@patternfly/react-core@4.250.1)

---
updated-dependencies:
- dependency-name: "@patternfly/react-core"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-02 14:29:33 +00:00
Sarabraj Singh
61b242d194 initial commit of new machinery to handle redirects for webhook notifications (#13083) 2022-11-02 11:28:18 -03:00
Alex Corey
22b81f5dd3 Merge pull request #13150 from ansible/dependabot/npm_and_yarn/awx/ui/devel/patternfly/react-icons-4.92.10
Bump @patternfly/react-icons from 4.90.0 to 4.92.10 in /awx/ui
2022-11-02 10:27:25 -04:00
dependabot[bot]
99e1920d42 Bump @patternfly/react-icons from 4.90.0 to 4.92.10 in /awx/ui
Bumps [@patternfly/react-icons](https://github.com/patternfly/patternfly-react) from 4.90.0 to 4.92.10.
- [Release notes](https://github.com/patternfly/patternfly-react/releases)
- [Commits](https://github.com/patternfly/patternfly-react/compare/@patternfly/react-icons@4.90.0...@patternfly/react-icons@4.92.10)

---
updated-dependencies:
- dependency-name: "@patternfly/react-icons"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-02 14:01:59 +00:00
Alex Corey
2218fd5c25 Merge pull request #13144 from ansible/dependabot/npm_and_yarn/awx/ui/devel/patternfly/patternfly-4.217.1
Bump @patternfly/patternfly from 4.210.2 to 4.217.1 in /awx/ui
2022-11-02 10:01:43 -04:00
Kia Lam
3c656842f0 Use optional chaining operator to prevent the modal from throwing an error. 2022-11-01 15:09:30 -07:00
Rebeccah Hunter
bd7635e74e Merge pull request #13146 from ansible/dedup-timeout
de-duplicate default settings
2022-11-01 16:47:00 -04:00
Elijah DeLee
0faa999ceb de-duplicate default settings 2022-11-01 14:50:32 -04:00
Rick Elrod
1bedf32baf Fix traceback on timeout with slicing + facts (#13139)
Slicing a QS with a step parameter forces the QS and returns a list.

Fixes #13131

Signed-off-by: Rick Elrod <rick@elrod.me>
2022-11-01 09:11:20 -05:00
dependabot[bot]
577f102e53 Bump @patternfly/patternfly from 4.210.2 to 4.217.1 in /awx/ui
Bumps [@patternfly/patternfly](https://github.com/patternfly/patternfly) from 4.210.2 to 4.217.1.
- [Release notes](https://github.com/patternfly/patternfly/releases)
- [Changelog](https://github.com/patternfly/patternfly/blob/main/RELEASE-NOTES.md)
- [Commits](https://github.com/patternfly/patternfly/compare/prerelease-v4.210.2...prerelease-v4.217.1)

---
updated-dependencies:
- dependency-name: "@patternfly/patternfly"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-01 08:18:34 +00:00
Jeff Bradberry
c5cf39abb7 Merge pull request #13132 from jbradberry/import-project-schedules
Expand the dependencies of the Schedule page type
2022-10-31 16:29:04 -04:00
Seth Foster
6b315f39de Merge pull request #12963 from fosterseth/minikube_cg
Make installing minikube optional in dev setup
2022-10-31 14:53:36 -04:00
Seth Foster
529a936d0a Make installing minikube optional in dev setup 2022-10-31 13:52:48 -04:00
Alan Rominger
6538d34b48 Remove ssh_key_data fix, handled in runner now 2022-10-31 11:01:28 -04:00
kialam
e40824bded Fix Schedules Form date validation same day different time scenario (#13062)
* Format datetime and convert to ms to compare which date is larger.

* Add supporting unit test.
2022-10-28 19:59:54 -03:00
Alan Rominger
ed318ea784 Merge pull request #13082 from AlanCoding/health_check_stdout
Include stdout from health check if it is not nothing
2022-10-28 13:59:21 -04:00
Alex Corey
d2b69e05f6 Merge pull request #13041 from ansible/12966-DisableManualHealthCheck
Allows health checks on only execution nodes
2022-10-28 13:26:11 -04:00
Jeff Bradberry
b57ae592ed Expand the dependencies of the Schedule page type
Really these could get any of the unified job template types, not just
system job templates, so importing e.g. a project with a schedule was
doing them in the wrong order.

Also, bump the timeout of the project update and make sure that we
stash it in the page cache even if it doesn't finish in 5 minutes.
2022-10-28 12:56:35 -04:00
John Westcott IV
e22f887765 Merge pull request #13129 from john-westcott-iv/update_pr_body_check
Fix pr_body_check
2022-10-27 16:05:03 -04:00
John Westcott IV
fc838ba44b Fix pr_body_check 2022-10-27 15:39:18 -04:00
Darshan
b19aa4a88d Fixes confusing Error when trying to sync project set to scm_type Manual (#13080)
Signed-off-by: darshanip <darshancoding@gmail.com>
Co-authored-by: Rick Elrod <rick@elrod.me>
2022-10-27 15:04:26 -04:00
Hao Liu
eba24db74c Merge pull request #13103 from saito-hideki/pr/add_arm64_arch_to_fact_list
Add arm64 architecture mapping to image_architecture for m1mac
2022-10-27 13:23:59 -04:00
Jeff Bradberry
153a197fad Merge pull request #13125 from jbradberry/improve-staticfile-management
UI static files are now directly copied to the proper static dir
2022-10-27 09:25:09 -04:00
Jeff Bradberry
8f4c329c2a UI static files are now directly copied to the proper static dir
when running `make ui-devel`.  Previously they were going to
/awx_devel/awx/public/static, but that directory is no longer being
served up by nginx, which forced us to have to run `make
collectstatic` (or equivalent) to get the files to the right place.
2022-10-26 17:51:29 -04:00
Seth Foster
368eb46f5b Merge pull request #13097 from fosterseth/cyan_log_lifecyle
make job lifecycle Cyan again
2022-10-26 16:57:03 -04:00
Alan Rominger
d6fea77082 Include stdout from health check if it is not nothing 2022-10-26 16:26:59 -04:00
Alex Corey
aaf6f5f17e Merge pull request #13104 from rooftopcellist/translations_updated_2022-10-25_09_13_58
Pushing updated strings for localization
2022-10-26 11:23:41 -04:00
Christian Adams
3303f7bfcf Pushing updated strings for localization 2022-10-26 11:05:00 -04:00
Alex Corey
41fd6ea37f Prevents health checks on all node types except for Execution nodes 2022-10-25 10:11:45 -04:00
Hideki Saito
4808a0053f Add arm64 architecture mapping to image_architecture for m1mac
* Addresses "make docker-compose-build" failure due to missing architecture mapping.

Signed-off-by: Hideki Saito <saito@fgrep.org>
2022-10-25 11:44:59 +09:00
Seth Foster
de41601f27 make job lifecycle Cyan again 2022-10-24 13:50:42 -04:00
Alan Rominger
487efb77ce Fix error when apply taggit app tags to objects 2022-10-21 14:53:31 -04:00
Alex Corey
aae57378f0 Gives enhanced permissions so that it can update dependabot pr bodies 2022-10-03 16:28:04 -04:00
Alan Rominger
cfce31419d Move the IS_TESTING method out of settings 2022-09-28 11:19:10 -04:00
Maya Costantini
8e83f9b134 Precise migrate.yml path in development docs 2022-09-28 10:57:40 +02:00
101 changed files with 204794 additions and 542 deletions

View File

@@ -0,0 +1,24 @@
---
name: Feature branch deletion cleanup
on:
delete:
branches:
- feature_**
jobs:
push:
runs-on: ubuntu-latest
permissions:
packages: write
contents: read
steps:
- name: Delete API Schema
env:
AWS_ACCESS_KEY: ${{ secrets.AWS_ACCESS_KEY }}
AWS_SECRET_KEY: ${{ secrets.AWS_SECRET_KEY }}
AWS_REGION: 'us-east-1'
run: |
ansible localhost -c local, -m command -a "{{ ansible_python_interpreter + ' -m pip install boto3'}}"
ansible localhost -c local -m aws_s3 \
-a "bucket=awx-public-ci-files object=${GITHUB_REF##*/}/schema.json mode=delete permission=public-read"

View File

@@ -13,21 +13,13 @@ jobs:
packages: write
contents: read
steps:
- name: Write PR body to a file
run: |
cat >> pr.body << __SOME_RANDOM_PR_EOF__
${{ github.event.pull_request.body }}
__SOME_RANDOM_PR_EOF__
- name: Display the received body for troubleshooting
run: cat pr.body
# We want to write these out individually just incase the options were joined on a single line
- name: Check for each of the lines
env:
PR_BODY: ${{ github.event.pull_request.body }}
run: |
grep "Bug, Docs Fix or other nominal change" pr.body > Z
grep "New or Enhanced Feature" pr.body > Y
grep "Breaking Change" pr.body > X
echo $PR_BODY | grep "Bug, Docs Fix or other nominal change" > Z
echo $PR_BODY | grep "New or Enhanced Feature" > Y
echo $PR_BODY | grep "Breaking Change" > X
exit 0
# We exit 0 and set the shell to prevent the returns from the greps from failing this step
# See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#exit-codes-and-error-action-preference

View File

@@ -5,6 +5,7 @@ on:
branches:
- devel
- release_**
- feature_**
jobs:
push:
runs-on: ubuntu-latest

176
Makefile
View File

@@ -54,46 +54,6 @@ I18N_FLAG_FILE = .i18n_built
VERSION PYTHON_VERSION docker-compose-sources \
.git/hooks/pre-commit
clean-tmp:
rm -rf tmp/
clean-venv:
rm -rf venv/
clean-dist:
rm -rf dist
clean-schema:
rm -rf swagger.json
rm -rf schema.json
rm -rf reference-schema.json
clean-languages:
rm -f $(I18N_FLAG_FILE)
find ./awx/locale/ -type f -regex ".*\.mo$" -delete
## Remove temporary build files, compiled Python files.
clean: clean-ui clean-api clean-awxkit clean-dist
rm -rf awx/public
rm -rf awx/lib/site-packages
rm -rf awx/job_status
rm -rf awx/job_output
rm -rf reports
rm -rf tmp
rm -rf $(I18N_FLAG_FILE)
mkdir tmp
clean-api:
rm -rf build $(NAME)-$(VERSION) *.egg-info
find . -type f -regex ".*\.py[co]$$" -delete
find . -type d -name "__pycache__" -delete
rm -f awx/awx_test.sqlite3*
rm -rf requirements/vendor
rm -rf awx/projects
clean-awxkit:
rm -rf awxkit/*.egg-info awxkit/.tox awxkit/build/*
## convenience target to assert environment variables are defined
guard-%:
@if [ "$${$*}" = "" ]; then \
@@ -117,7 +77,7 @@ virtualenv_awx:
fi; \
fi
## Install third-party requirements needed for AWX's environment.
## Install third-party requirements needed for AWX's environment.
# this does not use system site packages intentionally
requirements_awx: virtualenv_awx
if [[ "$(PIP_OPTIONS)" == *"--no-index"* ]]; then \
@@ -181,7 +141,7 @@ collectstatic:
@if [ "$(VENV_BASE)" ]; then \
. $(VENV_BASE)/awx/bin/activate; \
fi; \
mkdir -p awx/public/static && $(PYTHON) manage.py collectstatic --clear --noinput > /dev/null 2>&1
$(PYTHON) manage.py collectstatic --clear --noinput > /dev/null 2>&1
DEV_RELOAD_COMMAND ?= supervisorctl restart tower-processes:*
@@ -371,13 +331,6 @@ bulk_data:
UI_BUILD_FLAG_FILE = awx/ui/.ui-built
clean-ui:
rm -rf node_modules
rm -rf awx/ui/node_modules
rm -rf awx/ui/build
rm -rf awx/ui/src/locales/_build
rm -rf $(UI_BUILD_FLAG_FILE)
awx/ui/node_modules:
NODE_OPTIONS=--max-old-space-size=6144 $(NPM_BIN) --prefix awx/ui --loglevel warn --force ci
@@ -386,16 +339,14 @@ $(UI_BUILD_FLAG_FILE):
$(PYTHON) tools/scripts/compilemessages.py
$(NPM_BIN) --prefix awx/ui --loglevel warn run compile-strings
$(NPM_BIN) --prefix awx/ui --loglevel warn run build
mkdir -p awx/public/static/css
mkdir -p awx/public/static/js
mkdir -p awx/public/static/media
cp -r awx/ui/build/static/css/* awx/public/static/css
cp -r awx/ui/build/static/js/* awx/public/static/js
cp -r awx/ui/build/static/media/* awx/public/static/media
mkdir -p /var/lib/awx/public/static/css
mkdir -p /var/lib/awx/public/static/js
mkdir -p /var/lib/awx/public/static/media
cp -r awx/ui/build/static/css/* /var/lib/awx/public/static/css
cp -r awx/ui/build/static/js/* /var/lib/awx/public/static/js
cp -r awx/ui/build/static/media/* /var/lib/awx/public/static/media
touch $@
ui-release: $(UI_BUILD_FLAG_FILE)
ui-devel: awx/ui/node_modules
@@ -451,8 +402,9 @@ awx/projects:
COMPOSE_UP_OPTS ?=
COMPOSE_OPTS ?=
CONTROL_PLANE_NODE_COUNT ?= 1
EXECUTION_NODE_COUNT ?= 2
EXECUTION_NODE_COUNT ?= 0
MINIKUBE_CONTAINER_GROUP ?= false
MINIKUBE_SETUP ?= false # if false, run minikube separately
EXTRA_SOURCES_ANSIBLE_OPTS ?=
ifneq ($(ADMIN_PASSWORD),)
@@ -461,7 +413,7 @@ endif
docker-compose-sources: .git/hooks/pre-commit
@if [ $(MINIKUBE_CONTAINER_GROUP) = true ]; then\
ansible-playbook -i tools/docker-compose/inventory tools/docker-compose-minikube/deploy.yml; \
ansible-playbook -i tools/docker-compose/inventory -e minikube_setup=$(MINIKUBE_SETUP) tools/docker-compose-minikube/deploy.yml; \
fi;
ansible-playbook -i tools/docker-compose/inventory tools/docker-compose/ansible/sources.yml \
@@ -501,15 +453,6 @@ detect-schema-change: genschema
# Ignore differences in whitespace with -b
diff -u -b reference-schema.json schema.json
docker-compose-clean: awx/projects
docker-compose -f tools/docker-compose/_sources/docker-compose.yml rm -sf
docker-compose-container-group-clean:
@if [ -f "tools/docker-compose-minikube/_sources/minikube" ]; then \
tools/docker-compose-minikube/_sources/minikube delete; \
fi
rm -rf tools/docker-compose-minikube/_sources/
## Base development image build
docker-compose-build:
ansible-playbook tools/ansible/dockerfile.yml -e build_dev=True -e receptor_image=$(RECEPTOR_IMAGE)
@@ -517,15 +460,6 @@ docker-compose-build:
--build-arg BUILDKIT_INLINE_CACHE=1 \
--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) .
docker-clean:
$(foreach container_id,$(shell docker ps -f name=tools_awx -aq && docker ps -f name=tools_receptor -aq),docker stop $(container_id); docker rm -f $(container_id);)
if [ "$(shell docker images | grep awx_devel)" ]; then \
docker images | grep awx_devel | awk '{print $$3}' | xargs docker rmi --force; \
fi
docker-clean-volumes: docker-compose-clean docker-compose-container-group-clean
docker volume rm -f tools_awx_db tools_grafana_storage tools_prometheus_storage $(docker volume ls --filter name=tools_redis_socket_ -q)
docker-refresh: docker-clean docker-compose
## Docker Development Environment with Elastic Stack Connected
@@ -538,14 +472,6 @@ docker-compose-cluster-elk: awx/projects docker-compose-sources
docker-compose-container-group:
MINIKUBE_CONTAINER_GROUP=true make docker-compose
clean-elk:
docker stop tools_kibana_1
docker stop tools_logstash_1
docker stop tools_elasticsearch_1
docker rm tools_logstash_1
docker rm tools_elasticsearch_1
docker rm tools_kibana_1
psql-container:
docker run -it --net tools_default --rm postgres:12 sh -c 'exec psql -h "postgres" -p "5432" -U postgres'
@@ -602,6 +528,84 @@ messages:
print-%:
@echo $($*)
# Cleaning
# --------------------------------------
## Remove temporary build files, compiled Python files.
clean: clean-ui clean-api clean-awxkit clean-dist
rm -rf awx/public
rm -rf awx/lib/site-packages
rm -rf awx/job_status
rm -rf awx/job_output
rm -rf reports
rm -rf tmp
rm -rf $(I18N_FLAG_FILE)
mkdir tmp
clean-elk:
docker stop tools_kibana_1
docker stop tools_logstash_1
docker stop tools_elasticsearch_1
docker rm tools_logstash_1
docker rm tools_elasticsearch_1
docker rm tools_kibana_1
clean-ui:
rm -rf node_modules
rm -rf awx/ui/node_modules
rm -rf awx/ui/build
rm -rf awx/ui/src/locales/_build
rm -rf $(UI_BUILD_FLAG_FILE)
# the collectstatic command doesn't like it if this dir doesn't exist.
mkdir -p awx/ui/build/static
clean-tmp:
rm -rf tmp/
clean-venv:
rm -rf venv/
clean-dist:
rm -rf dist
clean-schema:
rm -rf swagger.json
rm -rf schema.json
rm -rf reference-schema.json
clean-languages:
rm -f $(I18N_FLAG_FILE)
find ./awx/locale/ -type f -regex ".*\.mo$" -delete
clean-api:
rm -rf build $(NAME)-$(VERSION) *.egg-info
rm -rf .tox
find . -type f -regex ".*\.py[co]$$" -delete
find . -type d -name "__pycache__" -delete
rm -f awx/awx_test.sqlite3*
rm -rf requirements/vendor
rm -rf awx/projects
clean-awxkit:
rm -rf awxkit/*.egg-info awxkit/.tox awxkit/build/*
docker-compose-clean: awx/projects
docker-compose -f tools/docker-compose/_sources/docker-compose.yml rm -sf
docker-compose-container-group-clean:
@if [ -f "tools/docker-compose-minikube/_sources/minikube" ]; then \
tools/docker-compose-minikube/_sources/minikube delete; \
fi
rm -rf tools/docker-compose-minikube/_sources/
docker-clean:
$(foreach container_id,$(shell docker ps -f name=tools_awx -aq && docker ps -f name=tools_receptor -aq),docker stop $(container_id); docker rm -f $(container_id);)
if [ "$(shell docker images | grep awx_devel)" ]; then \
docker images | grep awx_devel | awk '{print $$3}' | xargs docker rmi --force; \
fi
docker-clean-volumes: docker-compose-clean docker-compose-container-group-clean
docker volume rm -f tools_awx_db tools_grafana_storage tools_prometheus_storage $(docker volume ls --filter name=tools_redis_socket_ -q)
# HELP related targets
# --------------------------------------
@@ -635,4 +639,4 @@ help/generate:
} \
} \
{ lastLine = $$0 }' $(MAKEFILE_LIST) | sort -u
@printf "\n"
@printf "\n"

View File

@@ -2221,6 +2221,15 @@ class InventorySourceUpdateSerializer(InventorySourceSerializer):
class Meta:
fields = ('can_update',)
def validate(self, attrs):
project = self.instance.source_project
if project:
failed_reason = project.get_reason_if_failed()
if failed_reason:
raise serializers.ValidationError(failed_reason)
return super(InventorySourceUpdateSerializer, self).validate(attrs)
class InventoryUpdateSerializer(UnifiedJobSerializer, InventorySourceOptionsSerializer):
@@ -4272,17 +4281,10 @@ class JobLaunchSerializer(BaseSerializer):
# Basic validation - cannot run a playbook without a playbook
if not template.project:
errors['project'] = _("A project is required to run a job.")
elif template.project.status in ('error', 'failed'):
errors['playbook'] = _("Missing a revision to run due to failed project update.")
latest_update = template.project.project_updates.last()
if latest_update is not None and latest_update.failed:
failed_validation_tasks = latest_update.project_update_events.filter(
event='runner_on_failed',
play="Perform project signature/checksum verification",
)
if failed_validation_tasks:
errors['playbook'] = _("Last project update failed due to signature validation failure.")
else:
failure_reason = template.project.get_reason_if_failed()
if failure_reason:
errors['playbook'] = failure_reason
# cannot run a playbook without an inventory
if template.inventory and template.inventory.pending_deletion is True:
@@ -4952,7 +4954,7 @@ class InstanceSerializer(BaseSerializer):
res['install_bundle'] = self.reverse('api:instance_install_bundle', kwargs={'pk': obj.pk})
res['peers'] = self.reverse('api:instance_peers_list', kwargs={"pk": obj.pk})
if self.context['request'].user.is_superuser or self.context['request'].user.is_system_auditor:
if obj.node_type != 'hop':
if obj.node_type == 'execution':
res['health_check'] = self.reverse('api:instance_health_check', kwargs={'pk': obj.pk})
return res

View File

@@ -1,5 +1,5 @@
Launch a Job Template:
{% ifmeth GET %}
Make a GET request to this resource to determine if the job_template can be
launched and whether any passwords are required to launch the job_template.
The response will include the following fields:
@@ -29,8 +29,8 @@ The response will include the following fields:
* `inventory_needed_to_start`: Flag indicating the presence of an inventory
associated with the job template. If not then one should be supplied when
launching the job (boolean, read-only)
Make a POST request to this resource to launch the job_template. If any
{% endifmeth %}
{% ifmeth POST %}Make a POST request to this resource to launch the job_template. If any
passwords, inventory, or extra variables (extra_vars) are required, they must
be passed via POST data, with extra_vars given as a YAML or JSON string and
escaped parentheses. If the `inventory_needed_to_start` is `True` then the
@@ -41,3 +41,4 @@ are not provided, a 400 status code will be returned. If the job cannot be
launched, a 405 status code will be returned. If the provided credential or
inventory are not allowed to be used by the user, then a 403 status code will
be returned.
{% endifmeth %}

View File

@@ -7,7 +7,7 @@ receptor_work_commands:
command: ansible-runner
params: worker
allowruntimeparams: true
verifysignature: true
verifysignature: {{ sign_work }}
custom_worksign_public_keyfile: receptor/work-public-key.pem
custom_tls_certfile: receptor/tls/receptor.crt
custom_tls_keyfile: receptor/tls/receptor.key

View File

@@ -5,6 +5,7 @@
import dateutil
import functools
import html
import itertools
import logging
import re
import requests
@@ -20,9 +21,10 @@ from urllib3.exceptions import ConnectTimeoutError
# Django
from django.conf import settings
from django.core.exceptions import FieldError, ObjectDoesNotExist
from django.db.models import Q, Sum
from django.db.models import Q, Sum, Count
from django.db import IntegrityError, ProgrammingError, transaction, connection
from django.db.models.fields.related import ManyToManyField, ForeignKey
from django.db.models.functions import Trunc
from django.shortcuts import get_object_or_404
from django.utils.safestring import mark_safe
from django.utils.timezone import now
@@ -47,9 +49,6 @@ from rest_framework import status
from rest_framework_yaml.parsers import YAMLParser
from rest_framework_yaml.renderers import YAMLRenderer
# QSStats
import qsstats
# ANSIConv
import ansiconv
@@ -283,30 +282,50 @@ class DashboardJobsGraphView(APIView):
success_query = success_query.filter(instance_of=models.ProjectUpdate)
failed_query = failed_query.filter(instance_of=models.ProjectUpdate)
success_qss = qsstats.QuerySetStats(success_query, 'finished')
failed_qss = qsstats.QuerySetStats(failed_query, 'finished')
start_date = now()
end = now()
interval = 'day'
if period == 'month':
end_date = start_date - dateutil.relativedelta.relativedelta(months=1)
interval = 'days'
start = end - dateutil.relativedelta.relativedelta(months=1)
elif period == 'two_weeks':
end_date = start_date - dateutil.relativedelta.relativedelta(weeks=2)
interval = 'days'
start = end - dateutil.relativedelta.relativedelta(weeks=2)
elif period == 'week':
end_date = start_date - dateutil.relativedelta.relativedelta(weeks=1)
interval = 'days'
start = end - dateutil.relativedelta.relativedelta(weeks=1)
elif period == 'day':
end_date = start_date - dateutil.relativedelta.relativedelta(days=1)
interval = 'hours'
start = end - dateutil.relativedelta.relativedelta(days=1)
interval = 'hour'
else:
return Response({'error': _('Unknown period "%s"') % str(period)}, status=status.HTTP_400_BAD_REQUEST)
dashboard_data = {"jobs": {"successful": [], "failed": []}}
for element in success_qss.time_series(end_date, start_date, interval=interval):
dashboard_data['jobs']['successful'].append([time.mktime(element[0].timetuple()), element[1]])
for element in failed_qss.time_series(end_date, start_date, interval=interval):
dashboard_data['jobs']['failed'].append([time.mktime(element[0].timetuple()), element[1]])
succ_list = dashboard_data['jobs']['successful']
fail_list = dashboard_data['jobs']['failed']
qs_s = (
success_query.filter(finished__range=(start, end))
.annotate(d=Trunc('finished', interval, tzinfo=end.tzinfo))
.order_by()
.values('d')
.annotate(agg=Count('id', distinct=True))
)
data_s = {item['d']: item['agg'] for item in qs_s}
qs_f = (
failed_query.filter(finished__range=(start, end))
.annotate(d=Trunc('finished', interval, tzinfo=end.tzinfo))
.order_by()
.values('d')
.annotate(agg=Count('id', distinct=True))
)
data_f = {item['d']: item['agg'] for item in qs_f}
start_date = start.replace(hour=0, minute=0, second=0, microsecond=0)
for d in itertools.count():
date = start_date + dateutil.relativedelta.relativedelta(days=d)
if date > end:
break
succ_list.append([time.mktime(date.timetuple()), data_s.get(date, 0)])
fail_list.append([time.mktime(date.timetuple()), data_f.get(date, 0)])
return Response(dashboard_data)
@@ -392,8 +411,8 @@ class InstanceHealthCheck(GenericAPIView):
permission_classes = (IsSystemAdminOrAuditor,)
def get_queryset(self):
return super().get_queryset().filter(node_type='execution')
# FIXME: For now, we don't have a good way of checking the health of a hop node.
return super().get_queryset().exclude(node_type='hop')
def get(self, request, *args, **kwargs):
obj = self.get_object()
@@ -413,9 +432,10 @@ class InstanceHealthCheck(GenericAPIView):
execution_node_health_check.apply_async([obj.hostname])
else:
from awx.main.tasks.system import cluster_node_health_check
cluster_node_health_check.apply_async([obj.hostname], queue=obj.hostname)
return Response(
{"error": f"Cannot run a health check on instances of type {obj.node_type}. Health checks can only be run on execution nodes."},
status=status.HTTP_400_BAD_REQUEST,
)
return Response({'msg': f"Health check is running for {obj.hostname}."}, status=status.HTTP_200_OK)
@@ -2220,6 +2240,8 @@ class InventorySourceUpdateView(RetrieveAPIView):
def post(self, request, *args, **kwargs):
obj = self.get_object()
serializer = self.get_serializer(instance=obj, data=request.data)
serializer.is_valid(raise_exception=True)
if obj.can_update:
update = obj.update()
if not update:

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -238,7 +238,9 @@ def instance_info(since, include_hostnames=False, **kwargs):
info = {}
# Use same method that the TaskManager does to compute consumed capacity without querying all running jobs for each Instance
active_tasks = models.UnifiedJob.objects.filter(status__in=['running', 'waiting']).only('task_impact', 'controller_node', 'execution_node')
tm_instances = TaskManagerInstances(active_tasks, instance_fields=['uuid', 'version', 'capacity', 'cpu', 'memory', 'managed_by_policy', 'enabled'])
tm_instances = TaskManagerInstances(
active_tasks, instance_fields=['uuid', 'version', 'capacity', 'cpu', 'memory', 'managed_by_policy', 'enabled', 'node_type']
)
for tm_instance in tm_instances.instances_by_hostname.values():
instance = tm_instance.obj
instance_info = {
@@ -251,6 +253,7 @@ def instance_info(since, include_hostnames=False, **kwargs):
'enabled': instance.enabled,
'consumed_capacity': tm_instance.consumed_capacity,
'remaining_capacity': instance.capacity - tm_instance.consumed_capacity,
'node_type': instance.node_type,
}
if include_hostnames is True:
instance_info['hostname'] = instance.hostname

View File

@@ -57,6 +57,7 @@ def metrics():
[
'hostname',
'instance_uuid',
'node_type',
],
registry=REGISTRY,
)
@@ -84,6 +85,7 @@ def metrics():
[
'hostname',
'instance_uuid',
'node_type',
],
registry=REGISTRY,
)
@@ -111,6 +113,7 @@ def metrics():
[
'hostname',
'instance_uuid',
'node_type',
],
registry=REGISTRY,
)
@@ -120,6 +123,7 @@ def metrics():
[
'hostname',
'instance_uuid',
'node_type',
],
registry=REGISTRY,
)
@@ -180,12 +184,13 @@ def metrics():
instance_data = instance_info(None, include_hostnames=True)
for uuid, info in instance_data.items():
hostname = info['hostname']
INSTANCE_CAPACITY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['capacity'])
node_type = info['node_type']
INSTANCE_CAPACITY.labels(hostname=hostname, instance_uuid=uuid, node_type=node_type).set(instance_data[uuid]['capacity'])
INSTANCE_CPU.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['cpu'])
INSTANCE_MEMORY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['memory'])
INSTANCE_CONSUMED_CAPACITY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['consumed_capacity'])
INSTANCE_REMAINING_CAPACITY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['remaining_capacity'])
INSTANCE_INFO.labels(hostname=hostname, instance_uuid=uuid).info(
INSTANCE_CONSUMED_CAPACITY.labels(hostname=hostname, instance_uuid=uuid, node_type=node_type).set(instance_data[uuid]['consumed_capacity'])
INSTANCE_REMAINING_CAPACITY.labels(hostname=hostname, instance_uuid=uuid, node_type=node_type).set(instance_data[uuid]['remaining_capacity'])
INSTANCE_INFO.labels(hostname=hostname, instance_uuid=uuid, node_type=node_type).info(
{
'enabled': str(instance_data[uuid]['enabled']),
'managed_by_policy': str(instance_data[uuid]['managed_by_policy']),

View File

@@ -5,7 +5,9 @@ import logging
from django.conf import settings
from django.apps import apps
from awx.main.consumers import emit_channel_notification
from awx.main.utils import is_testing
root_key = 'awx_metrics'
logger = logging.getLogger('awx.main.analytics')
@@ -163,7 +165,7 @@ class Metrics:
Instance = apps.get_model('main', 'Instance')
if instance_name:
self.instance_name = instance_name
elif settings.IS_TESTING():
elif is_testing():
self.instance_name = "awx_testing"
else:
self.instance_name = Instance.objects.my_hostname()

View File

@@ -1,6 +1,5 @@
from .plugin import CredentialPlugin, CertFiles, raise_for_status
import base64
from urllib.parse import urljoin, quote
from django.utils.translation import gettext_lazy as _
@@ -61,7 +60,7 @@ def conjur_backend(**kwargs):
cacert = kwargs.get('cacert', None)
auth_kwargs = {
'headers': {'Content-Type': 'text/plain'},
'headers': {'Content-Type': 'text/plain', 'Accept-Encoding': 'base64'},
'data': api_key,
'allow_redirects': False,
}
@@ -69,9 +68,9 @@ def conjur_backend(**kwargs):
with CertFiles(cacert) as cert:
# https://www.conjur.org/api.html#authentication-authenticate-post
auth_kwargs['verify'] = cert
resp = requests.post(urljoin(url, '/'.join(['authn', account, username, 'authenticate'])), **auth_kwargs)
resp = requests.post(urljoin(url, '/'.join(['api', 'authn', account, username, 'authenticate'])), **auth_kwargs)
raise_for_status(resp)
token = base64.b64encode(resp.content).decode('utf-8')
token = resp.content.decode('utf-8')
lookup_kwargs = {
'headers': {'Authorization': 'Token token="{}"'.format(token)},
@@ -79,9 +78,10 @@ def conjur_backend(**kwargs):
}
# https://www.conjur.org/api.html#secrets-retrieve-a-secret-get
path = urljoin(url, '/'.join(['secrets', account, 'variable', secret_path]))
path = urljoin(url, '/'.join(['api', 'secrets', account, 'variable', secret_path]))
if version:
path = '?'.join([path, version])
ver = "version={}".format(version)
path = '?'.join([path, ver])
with CertFiles(cacert) as cert:
lookup_kwargs['verify'] = cert
@@ -90,4 +90,4 @@ def conjur_backend(**kwargs):
return resp.text
conjur_plugin = CredentialPlugin('CyberArk Conjur Secret Lookup', inputs=conjur_inputs, backend=conjur_backend)
conjur_plugin = CredentialPlugin('CyberArk Conjur Secrets Manager Lookup', inputs=conjur_inputs, backend=conjur_backend)

View File

@@ -466,7 +466,7 @@ class AutoscalePool(WorkerPool):
task_name = 'unknown'
if isinstance(body, dict):
task_name = body.get('task')
logger.warn(f'Workers maxed, queuing {task_name}, load: {sum(len(w.managed_tasks) for w in self.workers)} / {len(self.workers)}')
logger.warning(f'Workers maxed, queuing {task_name}, load: {sum(len(w.managed_tasks) for w in self.workers)} / {len(self.workers)}')
return super(AutoscalePool, self).write(preferred_queue, body)
except Exception:
for conn in connections.all():

View File

@@ -1,14 +1,13 @@
import inspect
import logging
import sys
import json
import time
from uuid import uuid4
from django.conf import settings
from django_guid import get_guid
from . import pg_bus_conn
from awx.main.utils import is_testing
logger = logging.getLogger('awx.main.dispatch')
@@ -93,7 +92,7 @@ class task:
obj.update(**kw)
if callable(queue):
queue = queue()
if not settings.IS_TESTING(sys.argv):
if not is_testing():
with pg_bus_conn() as conn:
conn.notify(queue, json.dumps(obj))
return (obj, queue)

View File

@@ -233,11 +233,12 @@ class Instance(HasPolicyEditsMixin, BaseModel):
if not isinstance(vargs.get('grace_period'), int):
vargs['grace_period'] = 60 # grace period of 60 minutes, need to set because CLI default will not take effect
if 'exclude_strings' not in vargs and vargs.get('file_pattern'):
active_pks = list(
UnifiedJob.objects.filter(
(models.Q(execution_node=self.hostname) | models.Q(controller_node=self.hostname)) & models.Q(status__in=('running', 'waiting'))
).values_list('pk', flat=True)
)
active_job_qs = UnifiedJob.objects.filter(status__in=('running', 'waiting'))
if self.node_type == 'execution':
active_job_qs = active_job_qs.filter(execution_node=self.hostname)
else:
active_job_qs = active_job_qs.filter(controller_node=self.hostname)
active_pks = list(active_job_qs.values_list('pk', flat=True))
if active_pks:
vargs['exclude_strings'] = [JOB_FOLDER_PREFIX % job_id for job_id in active_pks]
if 'remove_images' in vargs or 'image_prune' in vargs:

View File

@@ -247,6 +247,19 @@ class Inventory(CommonModelNameNotUnique, ResourceMixin, RelatedJobsMixin):
return (number, step)
def get_sliced_hosts(self, host_queryset, slice_number, slice_count):
"""
Returns a slice of Hosts given a slice number and total slice count, or
the original queryset if slicing is not requested.
NOTE: If slicing is performed, this will return a List[Host] with the
resulting slice. If slicing is not performed it will return the
original queryset (not evaluating it or forcing it to a list). This
puts the burden on the caller to check the resulting type. This is
non-ideal because it's easy to get wrong, but I think the only way
around it is to force the queryset which has memory implications for
large inventories.
"""
if slice_count > 1 and slice_number > 0:
offset = slice_number - 1
host_queryset = host_queryset[offset::slice_count]
@@ -554,17 +567,6 @@ class Host(CommonModelNameNotUnique, RelatedJobsMixin):
# Use .job_host_summaries.all() to get jobs affecting this host.
# Use .job_events.all() to get events affecting this host.
'''
We don't use timestamp, but we may in the future.
'''
def update_ansible_facts(self, module, facts, timestamp=None):
if module == "ansible":
self.ansible_facts.update(facts)
else:
self.ansible_facts[module] = facts
self.save()
def get_effective_host_name(self):
"""
Return the name of the host that will be used in actual ansible

View File

@@ -15,6 +15,7 @@ from urllib.parse import urljoin
from django.conf import settings
from django.core.exceptions import ValidationError
from django.db import models
from django.db.models.query import QuerySet
# from django.core.cache import cache
from django.utils.encoding import smart_str
@@ -43,7 +44,7 @@ from awx.main.models.notifications import (
NotificationTemplate,
JobNotificationMixin,
)
from awx.main.utils import parse_yaml_or_json, getattr_dne, NullablePromptPseudoField, polymorphic
from awx.main.utils import parse_yaml_or_json, getattr_dne, NullablePromptPseudoField, polymorphic, log_excess_runtime
from awx.main.fields import ImplicitRoleField, AskForField, JSONBlob, OrderedManyToManyField
from awx.main.models.mixins import (
ResourceMixin,
@@ -844,22 +845,35 @@ class Job(UnifiedJob, JobOptions, SurveyJobMixin, JobNotificationMixin, TaskMana
def get_notification_friendly_name(self):
return "Job"
def _get_inventory_hosts(self, only=['name', 'ansible_facts', 'ansible_facts_modified', 'modified', 'inventory_id']):
def _get_inventory_hosts(self, only=('name', 'ansible_facts', 'ansible_facts_modified', 'modified', 'inventory_id'), **filters):
"""Return value is an iterable for the relevant hosts for this job"""
if not self.inventory:
return []
host_queryset = self.inventory.hosts.only(*only)
return self.inventory.get_sliced_hosts(host_queryset, self.job_slice_number, self.job_slice_count)
if filters:
host_queryset = host_queryset.filter(**filters)
host_queryset = self.inventory.get_sliced_hosts(host_queryset, self.job_slice_number, self.job_slice_count)
if isinstance(host_queryset, QuerySet):
return host_queryset.iterator()
return host_queryset
def start_job_fact_cache(self, destination, modification_times, timeout=None):
@log_excess_runtime(logger, debug_cutoff=0.01, msg='Job {job_id} host facts prepared for {written_ct} hosts, took {delta:.3f} s', add_log_data=True)
def start_job_fact_cache(self, destination, log_data, timeout=None):
self.log_lifecycle("start_job_fact_cache")
log_data['job_id'] = self.id
log_data['written_ct'] = 0
os.makedirs(destination, mode=0o700)
hosts = self._get_inventory_hosts()
if timeout is None:
timeout = settings.ANSIBLE_FACT_CACHE_TIMEOUT
if timeout > 0:
# exclude hosts with fact data older than `settings.ANSIBLE_FACT_CACHE_TIMEOUT seconds`
timeout = now() - datetime.timedelta(seconds=timeout)
hosts = hosts.filter(ansible_facts_modified__gte=timeout)
hosts = self._get_inventory_hosts(ansible_facts_modified__gte=timeout)
else:
hosts = self._get_inventory_hosts()
last_filepath_written = None
for host in hosts:
filepath = os.sep.join(map(str, [destination, host.name]))
if not os.path.realpath(filepath).startswith(destination):
@@ -869,23 +883,38 @@ class Job(UnifiedJob, JobOptions, SurveyJobMixin, JobNotificationMixin, TaskMana
with codecs.open(filepath, 'w', encoding='utf-8') as f:
os.chmod(f.name, 0o600)
json.dump(host.ansible_facts, f)
log_data['written_ct'] += 1
last_filepath_written = filepath
except IOError:
system_tracking_logger.error('facts for host {} could not be cached'.format(smart_str(host.name)))
continue
# make note of the time we wrote the file so we can check if it changed later
modification_times[filepath] = os.path.getmtime(filepath)
# make note of the time we wrote the last file so we can check if any file changed later
if last_filepath_written:
return os.path.getmtime(last_filepath_written)
return None
def finish_job_fact_cache(self, destination, modification_times):
@log_excess_runtime(
logger,
debug_cutoff=0.01,
msg='Job {job_id} host facts: updated {updated_ct}, cleared {cleared_ct}, unchanged {unmodified_ct}, took {delta:.3f} s',
add_log_data=True,
)
def finish_job_fact_cache(self, destination, facts_write_time, log_data):
self.log_lifecycle("finish_job_fact_cache")
log_data['job_id'] = self.id
log_data['updated_ct'] = 0
log_data['unmodified_ct'] = 0
log_data['cleared_ct'] = 0
hosts_to_update = []
for host in self._get_inventory_hosts():
filepath = os.sep.join(map(str, [destination, host.name]))
if not os.path.realpath(filepath).startswith(destination):
system_tracking_logger.error('facts for host {} could not be cached'.format(smart_str(host.name)))
continue
if os.path.exists(filepath):
# If the file changed since we wrote it pre-playbook run...
# If the file changed since we wrote the last facts file, pre-playbook run...
modified = os.path.getmtime(filepath)
if modified > modification_times.get(filepath, 0):
if (not facts_write_time) or modified > facts_write_time:
with codecs.open(filepath, 'r', encoding='utf-8') as f:
try:
ansible_facts = json.load(f)
@@ -893,7 +922,7 @@ class Job(UnifiedJob, JobOptions, SurveyJobMixin, JobNotificationMixin, TaskMana
continue
host.ansible_facts = ansible_facts
host.ansible_facts_modified = now()
host.save(update_fields=['ansible_facts', 'ansible_facts_modified'])
hosts_to_update.append(host)
system_tracking_logger.info(
'New fact for inventory {} host {}'.format(smart_str(host.inventory.name), smart_str(host.name)),
extra=dict(
@@ -904,12 +933,21 @@ class Job(UnifiedJob, JobOptions, SurveyJobMixin, JobNotificationMixin, TaskMana
job_id=self.id,
),
)
log_data['updated_ct'] += 1
else:
log_data['unmodified_ct'] += 1
else:
# if the file goes missing, ansible removed it (likely via clear_facts)
host.ansible_facts = {}
host.ansible_facts_modified = now()
hosts_to_update.append(host)
system_tracking_logger.info('Facts cleared for inventory {} host {}'.format(smart_str(host.inventory.name), smart_str(host.name)))
host.save()
log_data['cleared_ct'] += 1
if len(hosts_to_update) > 100:
self.inventory.hosts.bulk_update(hosts_to_update, ['ansible_facts', 'ansible_facts_modified'])
hosts_to_update = []
if hosts_to_update:
self.inventory.hosts.bulk_update(hosts_to_update, ['ansible_facts', 'ansible_facts_modified'])
class LaunchTimeConfigBase(BaseModel):

View File

@@ -471,6 +471,29 @@ class Project(UnifiedJobTemplate, ProjectOptions, ResourceMixin, CustomVirtualEn
def get_absolute_url(self, request=None):
return reverse('api:project_detail', kwargs={'pk': self.pk}, request=request)
def get_reason_if_failed(self):
"""
If the project is in a failed or errored state, return a human-readable
error message explaining why. Otherwise return None.
This is used during validation in the serializer and also by
RunProjectUpdate/RunInventoryUpdate.
"""
if self.status not in ('error', 'failed'):
return None
latest_update = self.project_updates.last()
if latest_update is not None and latest_update.failed:
failed_validation_tasks = latest_update.project_update_events.filter(
event='runner_on_failed',
play="Perform project signature/checksum verification",
)
if failed_validation_tasks:
return _("Last project update failed due to signature validation failure.")
return _("Missing a revision to run due to failed project update.")
'''
RelatedJobsMixin
'''

View File

@@ -5,9 +5,6 @@ import json
import logging
import requests
from django.utils.encoding import smart_str
from django.utils.translation import gettext_lazy as _
from awx.main.notifications.base import AWXBaseEmailBackend
from awx.main.utils import get_awx_http_client_headers
from awx.main.notifications.custom_notification_base import CustomNotificationBase
@@ -17,6 +14,8 @@ logger = logging.getLogger('awx.main.notifications.webhook_backend')
class WebhookBackend(AWXBaseEmailBackend, CustomNotificationBase):
MAX_RETRIES = 5
init_parameters = {
"url": {"label": "Target URL", "type": "string"},
"http_method": {"label": "HTTP Method", "type": "string", "default": "POST"},
@@ -64,20 +63,67 @@ class WebhookBackend(AWXBaseEmailBackend, CustomNotificationBase):
if self.http_method.lower() not in ['put', 'post']:
raise ValueError("HTTP method must be either 'POST' or 'PUT'.")
chosen_method = getattr(requests, self.http_method.lower(), None)
for m in messages:
auth = None
if self.username or self.password:
auth = (self.username, self.password)
r = chosen_method(
"{}".format(m.recipients()[0]),
auth=auth,
data=json.dumps(m.body, ensure_ascii=False).encode('utf-8'),
headers=dict(list(get_awx_http_client_headers().items()) + list((self.headers or {}).items())),
verify=(not self.disable_ssl_verification),
)
if r.status_code >= 400:
logger.error(smart_str(_("Error sending notification webhook: {}").format(r.status_code)))
# the constructor for EmailMessage - https://docs.djangoproject.com/en/4.1/_modules/django/core/mail/message will turn an empty dictionary to an empty string
# sometimes an empty dict is intentional and we added this conditional to enforce that
if not m.body:
m.body = {}
url = str(m.recipients()[0])
data = json.dumps(m.body, ensure_ascii=False).encode('utf-8')
headers = {**(get_awx_http_client_headers()), **(self.headers or {})}
err = None
for retries in range(self.MAX_RETRIES):
# Sometimes we hit redirect URLs. We must account for this. We still extract the redirect URL from the response headers and try again. Max retires == 5
resp = chosen_method(
url=url,
auth=auth,
data=data,
headers=headers,
verify=(not self.disable_ssl_verification),
allow_redirects=False, # override default behaviour for redirects
)
# either success or error reached if this conditional fires
if resp.status_code not in [301, 307]:
break
# we've hit a redirect. extract the redirect URL out of the first response header and try again
logger.warning(
f"Received a {resp.status_code} from {url}, trying to reach redirect url {resp.headers.get('Location', None)}; attempt #{retries+1}"
)
# take the first redirect URL in the response header and try that
url = resp.headers.get("Location", None)
if url is None:
err = f"Webhook notification received redirect to a blank URL from {url}. Response headers={resp.headers}"
break
else:
# no break condition in the loop encountered; therefore we have hit the maximum number of retries
err = f"Webhook notification max number of retries [{self.MAX_RETRIES}] exceeded. Failed to send webhook notification to {url}"
if resp.status_code >= 400:
err = f"Error sending webhook notification: {resp.status_code}"
# log error message
if err:
logger.error(err)
if not self.fail_silently:
raise Exception(smart_str(_("Error sending notification webhook: {}").format(r.status_code)))
sent_messages += 1
raise Exception(err)
# no errors were encountered therefore we successfully sent off the notification webhook
if resp.status_code in range(200, 299):
logger.debug(f"Notification webhook successfully sent to {url}. Received {resp.status_code}")
sent_messages += 1
return sent_messages

View File

@@ -3,6 +3,8 @@
from django.db.models.signals import pre_save, post_save, pre_delete, m2m_changed
from taggit.managers import TaggableManager
class ActivityStreamRegistrar(object):
def __init__(self):
@@ -19,6 +21,8 @@ class ActivityStreamRegistrar(object):
pre_delete.connect(activity_stream_delete, sender=model, dispatch_uid=str(self.__class__) + str(model) + "_delete")
for m2mfield in model._meta.many_to_many:
if isinstance(m2mfield, TaggableManager):
continue # Special case for taggit app
try:
m2m_attr = getattr(model, m2mfield.name)
m2m_changed.connect(

View File

@@ -39,7 +39,7 @@ from awx.main.utils import (
ScheduleTaskManager,
ScheduleWorkflowManager,
)
from awx.main.utils.common import task_manager_bulk_reschedule
from awx.main.utils.common import task_manager_bulk_reschedule, is_testing
from awx.main.signals import disable_activity_stream
from awx.main.constants import ACTIVE_STATES
from awx.main.scheduler.dependency_graph import DependencyGraph
@@ -97,7 +97,7 @@ class TaskBase:
self.all_tasks = [t for t in qs]
def record_aggregate_metrics(self, *args):
if not settings.IS_TESTING():
if not is_testing():
# increment task_manager_schedule_calls regardless if the other
# metrics are recorded
s_metrics.Metrics(auto_pipe_execute=True).inc(f"{self.prefix}__schedule_calls", 1)

View File

@@ -2,8 +2,6 @@ import json
import time
import logging
from collections import deque
import os
import stat
# Django
from django.conf import settings
@@ -206,21 +204,6 @@ class RunnerCallback:
self.instance = self.update_model(self.instance.pk, job_args=json.dumps(runner_config.command), job_cwd=runner_config.cwd, job_env=job_env)
# We opened a connection just for that save, close it here now
connections.close_all()
elif status_data['status'] == 'failed':
# For encrypted ssh_key_data, ansible-runner worker will open and write the
# ssh_key_data to a named pipe. Then, once the podman container starts, ssh-agent will
# read from this named pipe so that the key can be used in ansible-playbook.
# Once the podman container exits, the named pipe is deleted.
# However, if the podman container fails to start in the first place, e.g. the image
# name is incorrect, then this pipe is not cleaned up. Eventually ansible-runner
# processor will attempt to write artifacts to the private data dir via unstream_dir, requiring
# that it open this named pipe. This leads to a hang. Thus, before any artifacts
# are written by the processor, it's important to remove this ssh_key_data pipe.
private_data_dir = self.instance.job_env.get('AWX_PRIVATE_DATA_DIR', None)
if private_data_dir:
key_data_file = os.path.join(private_data_dir, 'artifacts', str(self.instance.id), 'ssh_key_data')
if os.path.exists(key_data_file) and stat.S_ISFIFO(os.stat(key_data_file).st_mode):
os.remove(key_data_file)
elif status_data['status'] == 'error':
result_traceback = status_data.get('result_traceback', None)
if result_traceback:

View File

@@ -426,7 +426,7 @@ class BaseTask(object):
"""
instance.log_lifecycle("post_run")
def final_run_hook(self, instance, status, private_data_dir, fact_modification_times):
def final_run_hook(self, instance, status, private_data_dir):
"""
Hook for any steps to run after job/task is marked as complete.
"""
@@ -469,7 +469,6 @@ class BaseTask(object):
self.instance = self.update_model(pk, status='running', start_args='') # blank field to remove encrypted passwords
self.instance.websocket_emit_status("running")
status, rc = 'error', None
fact_modification_times = {}
self.runner_callback.event_ct = 0
'''
@@ -498,14 +497,6 @@ class BaseTask(object):
if not os.path.exists(settings.AWX_ISOLATION_BASE_PATH):
raise RuntimeError('AWX_ISOLATION_BASE_PATH=%s does not exist' % settings.AWX_ISOLATION_BASE_PATH)
# Fetch "cached" fact data from prior runs and put on the disk
# where ansible expects to find it
if getattr(self.instance, 'use_fact_cache', False):
self.instance.start_job_fact_cache(
os.path.join(private_data_dir, 'artifacts', str(self.instance.id), 'fact_cache'),
fact_modification_times,
)
# May have to serialize the value
private_data_files, ssh_key_data = self.build_private_data_files(self.instance, private_data_dir)
passwords = self.build_passwords(self.instance, kwargs)
@@ -646,7 +637,7 @@ class BaseTask(object):
self.instance.send_notification_templates('succeeded' if status == 'successful' else 'failed')
try:
self.final_run_hook(self.instance, status, private_data_dir, fact_modification_times)
self.final_run_hook(self.instance, status, private_data_dir)
except Exception:
logger.exception('{} Final run hook errored.'.format(self.instance.log_format))
@@ -767,6 +758,10 @@ class SourceControlMixin(BaseTask):
try:
original_branch = None
failed_reason = project.get_reason_if_failed()
if failed_reason:
self.update_model(self.instance.pk, status='failed', job_explanation=failed_reason)
raise RuntimeError(failed_reason)
project_path = project.get_project_path(check_if_exists=False)
if project.scm_type == 'git' and (scm_branch and scm_branch != project.scm_branch):
if os.path.exists(project_path):
@@ -1056,22 +1051,25 @@ class RunJob(SourceControlMixin, BaseTask):
error = _('Job could not start because no Execution Environment could be found.')
self.update_model(job.pk, status='error', job_explanation=error)
raise RuntimeError(error)
elif job.project.status in ('error', 'failed'):
msg = _('The project revision for this job template is unknown due to a failed update.')
job = self.update_model(job.pk, status='failed', job_explanation=msg)
raise RuntimeError(msg)
if job.inventory.kind == 'smart':
# cache smart inventory memberships so that the host_filter query is not
# ran inside of the event saving code
update_smart_memberships_for_inventory(job.inventory)
# Fetch "cached" fact data from prior runs and put on the disk
# where ansible expects to find it
if job.use_fact_cache:
self.facts_write_time = self.instance.start_job_fact_cache(os.path.join(private_data_dir, 'artifacts', str(job.id), 'fact_cache'))
def build_project_dir(self, job, private_data_dir):
self.sync_and_copy(job.project, private_data_dir, scm_branch=job.scm_branch)
def final_run_hook(self, job, status, private_data_dir, fact_modification_times):
super(RunJob, self).final_run_hook(job, status, private_data_dir, fact_modification_times)
if not private_data_dir:
def post_run_hook(self, job, status):
super(RunJob, self).post_run_hook(job, status)
job.refresh_from_db(fields=['job_env'])
private_data_dir = job.job_env.get('AWX_PRIVATE_DATA_DIR')
if (not private_data_dir) or (not hasattr(self, 'facts_write_time')):
# If there's no private data dir, that means we didn't get into the
# actual `run()` call; this _usually_ means something failed in
# the pre_run_hook method
@@ -1079,9 +1077,11 @@ class RunJob(SourceControlMixin, BaseTask):
if job.use_fact_cache:
job.finish_job_fact_cache(
os.path.join(private_data_dir, 'artifacts', str(job.id), 'fact_cache'),
fact_modification_times,
self.facts_write_time,
)
def final_run_hook(self, job, status, private_data_dir):
super(RunJob, self).final_run_hook(job, status, private_data_dir)
try:
inventory = job.inventory
except Inventory.DoesNotExist:

View File

@@ -208,7 +208,10 @@ def run_until_complete(node, timing_data=None, **kwargs):
if state_name.lower() == 'failed':
work_detail = status.get('Detail', '')
if work_detail:
raise RemoteJobError(f'Receptor error from {node}, detail:\n{work_detail}')
if stdout:
raise RemoteJobError(f'Receptor error from {node}, detail:\n{work_detail}\nstdout:\n{stdout}')
else:
raise RemoteJobError(f'Receptor error from {node}, detail:\n{work_detail}')
else:
raise RemoteJobError(f'Unknown ansible-runner error on node {node}, stdout:\n{stdout}')

View File

@@ -7,7 +7,7 @@ from awx.main.models.ha import Instance
from django.test.utils import override_settings
INSTANCE_KWARGS = dict(hostname='example-host', cpu=6, memory=36000000000, cpu_capacity=6, mem_capacity=42)
INSTANCE_KWARGS = dict(hostname='example-host', cpu=6, node_type='execution', memory=36000000000, cpu_capacity=6, mem_capacity=42)
@pytest.mark.django_db

View File

@@ -1,7 +1,7 @@
import pytest
from unittest import mock
from awx.main.models import AdHocCommand, InventoryUpdate, JobTemplate
from awx.main.models import AdHocCommand, InventoryUpdate, JobTemplate, Job
from awx.main.models.activity_stream import ActivityStream
from awx.main.models.ha import Instance, InstanceGroup
from awx.main.tasks.system import apply_cluster_membership_policies
@@ -15,6 +15,24 @@ def test_default_tower_instance_group(default_instance_group, job_factory):
assert default_instance_group in job_factory().preferred_instance_groups
@pytest.mark.django_db
@pytest.mark.parametrize('node_type', ('execution', 'control'))
@pytest.mark.parametrize('active', (True, False))
def test_get_cleanup_task_kwargs_active_jobs(node_type, active):
instance = Instance.objects.create(hostname='foobar', node_type=node_type)
job_kwargs = dict()
job_kwargs['controller_node' if node_type == 'control' else 'execution_node'] = instance.hostname
job_kwargs['status'] = 'running' if active else 'successful'
job = Job.objects.create(**job_kwargs)
kwargs = instance.get_cleanup_task_kwargs()
if active:
assert kwargs['exclude_strings'] == [f'awx_{job.pk}_']
else:
assert 'exclude_strings' not in kwargs
@pytest.mark.django_db
class TestPolicyTaskScheduling:
"""Tests make assertions about when the policy task gets scheduled"""

View File

@@ -75,6 +75,7 @@ def test_encrypted_subfields(get, post, user, organization):
url = reverse('api:notification_template_detail', kwargs={'pk': response.data['id']})
response = get(url, u)
assert response.data['notification_configuration']['account_token'] == "$encrypted$"
with mock.patch.object(notification_template_actual.notification_class, "send_messages", assert_send):
notification_template_actual.send("Test", {'body': "Test"})
@@ -175,3 +176,46 @@ def test_custom_environment_injection(post, user, organization):
fake_send.side_effect = _send_side_effect
template.send('subject', 'message')
def mock_post(*args, **kwargs):
class MockGoodResponse:
def __init__(self):
self.status_code = 200
class MockRedirectResponse:
def __init__(self):
self.status_code = 301
self.headers = {"Location": "http://goodendpoint"}
if kwargs['url'] == "http://goodendpoint":
return MockGoodResponse()
else:
return MockRedirectResponse()
@pytest.mark.django_db
@mock.patch('requests.post', side_effect=mock_post)
def test_webhook_notification_pointed_to_a_redirect_launch_endpoint(post, admin, organization):
n1 = NotificationTemplate.objects.create(
name="test-webhook",
description="test webhook",
organization=organization,
notification_type="webhook",
notification_configuration=dict(
url="http://some.fake.url",
disable_ssl_verification=True,
http_method="POST",
headers={
"Content-Type": "application/json",
},
username=admin.username,
password=admin.password,
),
messages={
"success": {"message": "", "body": "{}"},
},
)
assert n1.send("", n1.messages.get("success").get("body")) == 1

View File

@@ -36,15 +36,14 @@ def job(mocker, hosts, inventory):
def test_start_job_fact_cache(hosts, job, inventory, tmpdir):
fact_cache = os.path.join(tmpdir, 'facts')
modified_times = {}
job.start_job_fact_cache(fact_cache, modified_times, 0)
last_modified = job.start_job_fact_cache(fact_cache, timeout=0)
for host in hosts:
filepath = os.path.join(fact_cache, host.name)
assert os.path.exists(filepath)
with open(filepath, 'r') as f:
assert f.read() == json.dumps(host.ansible_facts)
assert filepath in modified_times
assert os.path.getmtime(filepath) <= last_modified
def test_fact_cache_with_invalid_path_traversal(job, inventory, tmpdir, mocker):
@@ -58,18 +57,16 @@ def test_fact_cache_with_invalid_path_traversal(job, inventory, tmpdir, mocker):
)
fact_cache = os.path.join(tmpdir, 'facts')
job.start_job_fact_cache(fact_cache, {}, 0)
job.start_job_fact_cache(fact_cache, timeout=0)
# a file called "foo" should _not_ be written outside the facts dir
assert os.listdir(os.path.join(fact_cache, '..')) == ['facts']
def test_finish_job_fact_cache_with_existing_data(job, hosts, inventory, mocker, tmpdir):
fact_cache = os.path.join(tmpdir, 'facts')
modified_times = {}
job.start_job_fact_cache(fact_cache, modified_times, 0)
last_modified = job.start_job_fact_cache(fact_cache, timeout=0)
for h in hosts:
h.save = mocker.Mock()
bulk_update = mocker.patch('django.db.models.query.QuerySet.bulk_update')
ansible_facts_new = {"foo": "bar"}
filepath = os.path.join(fact_cache, hosts[1].name)
@@ -83,23 +80,20 @@ def test_finish_job_fact_cache_with_existing_data(job, hosts, inventory, mocker,
new_modification_time = time.time() + 3600
os.utime(filepath, (new_modification_time, new_modification_time))
job.finish_job_fact_cache(fact_cache, modified_times)
job.finish_job_fact_cache(fact_cache, last_modified)
for host in (hosts[0], hosts[2], hosts[3]):
host.save.assert_not_called()
assert host.ansible_facts == {"a": 1, "b": 2}
assert host.ansible_facts_modified is None
assert hosts[1].ansible_facts == ansible_facts_new
hosts[1].save.assert_called_once_with(update_fields=['ansible_facts', 'ansible_facts_modified'])
bulk_update.assert_called_once_with([hosts[1]], ['ansible_facts', 'ansible_facts_modified'])
def test_finish_job_fact_cache_with_bad_data(job, hosts, inventory, mocker, tmpdir):
fact_cache = os.path.join(tmpdir, 'facts')
modified_times = {}
job.start_job_fact_cache(fact_cache, modified_times, 0)
last_modified = job.start_job_fact_cache(fact_cache, timeout=0)
for h in hosts:
h.save = mocker.Mock()
bulk_update = mocker.patch('django.db.models.query.QuerySet.bulk_update')
for h in hosts:
filepath = os.path.join(fact_cache, h.name)
@@ -109,26 +103,22 @@ def test_finish_job_fact_cache_with_bad_data(job, hosts, inventory, mocker, tmpd
new_modification_time = time.time() + 3600
os.utime(filepath, (new_modification_time, new_modification_time))
job.finish_job_fact_cache(fact_cache, modified_times)
job.finish_job_fact_cache(fact_cache, last_modified)
for h in hosts:
h.save.assert_not_called()
bulk_update.assert_not_called()
def test_finish_job_fact_cache_clear(job, hosts, inventory, mocker, tmpdir):
fact_cache = os.path.join(tmpdir, 'facts')
modified_times = {}
job.start_job_fact_cache(fact_cache, modified_times, 0)
last_modified = job.start_job_fact_cache(fact_cache, timeout=0)
for h in hosts:
h.save = mocker.Mock()
bulk_update = mocker.patch('django.db.models.query.QuerySet.bulk_update')
os.remove(os.path.join(fact_cache, hosts[1].name))
job.finish_job_fact_cache(fact_cache, modified_times)
job.finish_job_fact_cache(fact_cache, last_modified)
for host in (hosts[0], hosts[2], hosts[3]):
host.save.assert_not_called()
assert host.ansible_facts == {"a": 1, "b": 2}
assert host.ansible_facts_modified is None
assert hosts[1].ansible_facts == {}
hosts[1].save.assert_called_once_with()
bulk_update.assert_called_once_with([hosts[1]], ['ansible_facts', 'ansible_facts_modified'])

View File

@@ -27,11 +27,12 @@ def test_send_messages_as_POST():
]
)
requests_mock.post.assert_called_once_with(
'http://example.com',
url='http://example.com',
auth=None,
data=json.dumps({'text': 'test body'}, ensure_ascii=False).encode('utf-8'),
headers={'Content-Type': 'application/json', 'User-Agent': 'AWX 0.0.1.dev (open)'},
verify=True,
allow_redirects=False,
)
assert sent_messages == 1
@@ -57,11 +58,12 @@ def test_send_messages_as_PUT():
]
)
requests_mock.put.assert_called_once_with(
'http://example.com',
url='http://example.com',
auth=None,
data=json.dumps({'text': 'test body 2'}, ensure_ascii=False).encode('utf-8'),
headers={'Content-Type': 'application/json', 'User-Agent': 'AWX 0.0.1.dev (open)'},
verify=True,
allow_redirects=False,
)
assert sent_messages == 1
@@ -87,11 +89,12 @@ def test_send_messages_with_username():
]
)
requests_mock.post.assert_called_once_with(
'http://example.com',
url='http://example.com',
auth=('userstring', None),
data=json.dumps({'text': 'test body'}, ensure_ascii=False).encode('utf-8'),
headers={'Content-Type': 'application/json', 'User-Agent': 'AWX 0.0.1.dev (open)'},
verify=True,
allow_redirects=False,
)
assert sent_messages == 1
@@ -117,11 +120,12 @@ def test_send_messages_with_password():
]
)
requests_mock.post.assert_called_once_with(
'http://example.com',
url='http://example.com',
auth=(None, 'passwordstring'),
data=json.dumps({'text': 'test body'}, ensure_ascii=False).encode('utf-8'),
headers={'Content-Type': 'application/json', 'User-Agent': 'AWX 0.0.1.dev (open)'},
verify=True,
allow_redirects=False,
)
assert sent_messages == 1
@@ -147,11 +151,12 @@ def test_send_messages_with_username_and_password():
]
)
requests_mock.post.assert_called_once_with(
'http://example.com',
url='http://example.com',
auth=('userstring', 'passwordstring'),
data=json.dumps({'text': 'test body'}, ensure_ascii=False).encode('utf-8'),
headers={'Content-Type': 'application/json', 'User-Agent': 'AWX 0.0.1.dev (open)'},
verify=True,
allow_redirects=False,
)
assert sent_messages == 1
@@ -177,11 +182,12 @@ def test_send_messages_with_no_verify_ssl():
]
)
requests_mock.post.assert_called_once_with(
'http://example.com',
url='http://example.com',
auth=None,
data=json.dumps({'text': 'test body'}, ensure_ascii=False).encode('utf-8'),
headers={'Content-Type': 'application/json', 'User-Agent': 'AWX 0.0.1.dev (open)'},
verify=False,
allow_redirects=False,
)
assert sent_messages == 1
@@ -207,7 +213,7 @@ def test_send_messages_with_additional_headers():
]
)
requests_mock.post.assert_called_once_with(
'http://example.com',
url='http://example.com',
auth=None,
data=json.dumps({'text': 'test body'}, ensure_ascii=False).encode('utf-8'),
headers={
@@ -217,5 +223,6 @@ def test_send_messages_with_additional_headers():
'X-Test-Header2': 'test-content-2',
},
verify=True,
allow_redirects=False,
)
assert sent_messages == 1

View File

@@ -11,11 +11,12 @@ import os
import subprocess
import re
import stat
import sys
import urllib.parse
import threading
import contextlib
import tempfile
from functools import reduce, wraps
import functools
# Django
from django.core.exceptions import ObjectDoesNotExist, FieldDoesNotExist
@@ -73,6 +74,7 @@ __all__ = [
'NullablePromptPseudoField',
'model_instance_diff',
'parse_yaml_or_json',
'is_testing',
'RequireDebugTrueOrTest',
'has_model_field_prefetched',
'set_environ',
@@ -88,6 +90,7 @@ __all__ = [
'deepmerge',
'get_event_partition_epoch',
'cleanup_new_process',
'log_excess_runtime',
]
@@ -144,6 +147,19 @@ def underscore_to_camelcase(s):
return ''.join(x.capitalize() or '_' for x in s.split('_'))
@functools.cache
def is_testing(argv=None):
'''Return True if running django or py.test unit tests.'''
if 'PYTEST_CURRENT_TEST' in os.environ.keys():
return True
argv = sys.argv if argv is None else argv
if len(argv) >= 1 and ('py.test' in argv[0] or 'py/test.py' in argv[0]):
return True
elif len(argv) >= 2 and argv[1] == 'test':
return True
return False
class RequireDebugTrueOrTest(logging.Filter):
"""
Logging filter to output when in DEBUG mode or running tests.
@@ -152,7 +168,7 @@ class RequireDebugTrueOrTest(logging.Filter):
def filter(self, record):
from django.conf import settings
return settings.DEBUG or settings.IS_TESTING()
return settings.DEBUG or is_testing()
class IllegalArgumentError(ValueError):
@@ -174,7 +190,7 @@ def memoize(ttl=60, cache_key=None, track_function=False, cache=None):
cache = cache or get_memoize_cache()
def memoize_decorator(f):
@wraps(f)
@functools.wraps(f)
def _memoizer(*args, **kwargs):
if track_function:
cache_dict_key = slugify('%r %r' % (args, kwargs))
@@ -992,7 +1008,7 @@ def getattrd(obj, name, default=NoDefaultProvided):
"""
try:
return reduce(getattr, name.split("."), obj)
return functools.reduce(getattr, name.split("."), obj)
except AttributeError:
if default != NoDefaultProvided:
return default
@@ -1188,7 +1204,7 @@ def cleanup_new_process(func):
Cleanup django connection, cache connection, before executing new thread or processes entry point, func.
"""
@wraps(func)
@functools.wraps(func)
def wrapper_cleanup_new_process(*args, **kwargs):
from awx.conf.settings import SettingsWrapper # noqa
@@ -1200,15 +1216,30 @@ def cleanup_new_process(func):
return wrapper_cleanup_new_process
def log_excess_runtime(func_logger, cutoff=5.0):
def log_excess_runtime(func_logger, cutoff=5.0, debug_cutoff=5.0, msg=None, add_log_data=False):
def log_excess_runtime_decorator(func):
@wraps(func)
@functools.wraps(func)
def _new_func(*args, **kwargs):
start_time = time.time()
return_value = func(*args, **kwargs)
delta = time.time() - start_time
if delta > cutoff:
logger.info(f'Running {func.__name__!r} took {delta:.2f}s')
log_data = {'name': repr(func.__name__)}
if add_log_data:
return_value = func(*args, log_data=log_data, **kwargs)
else:
return_value = func(*args, **kwargs)
log_data['delta'] = time.time() - start_time
if isinstance(return_value, dict):
log_data.update(return_value)
if msg is None:
record_msg = 'Running {name} took {delta:.2f}s'
else:
record_msg = msg
if log_data['delta'] > cutoff:
func_logger.info(record_msg.format(**log_data))
elif log_data['delta'] > debug_cutoff:
func_logger.debug(record_msg.format(**log_data))
return return_value
return _new_func

View File

@@ -110,7 +110,7 @@ if settings.COLOR_LOGS is True:
# logs rendered with cyan text
previous_level_map = self.level_map.copy()
if record.name == "awx.analytics.job_lifecycle":
self.level_map[logging.DEBUG] = (None, 'cyan', True)
self.level_map[logging.INFO] = (None, 'cyan', True)
msg = super(ColorHandler, self).colorize(line, record)
self.level_map = previous_level_map
return msg

View File

@@ -10,28 +10,6 @@ import socket
from datetime import timedelta
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(__file__))
def is_testing(argv=None):
import sys
'''Return True if running django or py.test unit tests.'''
if 'PYTEST_CURRENT_TEST' in os.environ.keys():
return True
argv = sys.argv if argv is None else argv
if len(argv) >= 1 and ('py.test' in argv[0] or 'py/test.py' in argv[0]):
return True
elif len(argv) >= 2 and argv[1] == 'test':
return True
return False
def IS_TESTING(argv=None):
return is_testing(argv)
if "pytest" in sys.modules:
from unittest import mock
@@ -40,9 +18,13 @@ if "pytest" in sys.modules:
else:
import ldap
DEBUG = True
SQL_DEBUG = DEBUG
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(__file__))
# FIXME: it would be nice to cycle back around and allow this to be
# BigAutoField going forward, but we'd have to be explicit about our
# existing models.
@@ -101,7 +83,7 @@ USE_L10N = True
USE_TZ = True
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'ui', 'build', 'static'), os.path.join(BASE_DIR, 'static'))
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'ui', 'build', 'static'), os.path.join(BASE_DIR, 'static')]
# Absolute filesystem path to the directory where static file are collected via
# the collectstatic command.
@@ -254,6 +236,14 @@ START_TASK_LIMIT = 100
TASK_MANAGER_TIMEOUT = 300
TASK_MANAGER_TIMEOUT_GRACE_PERIOD = 60
# Number of seconds _in addition to_ the task manager timeout a job can stay
# in waiting without being reaped
JOB_WAITING_GRACE_PERIOD = 60
# Number of seconds after a container group job finished time to wait
# before the awx_k8s_reaper task will tear down the pods
K8S_POD_REAPER_GRACE_PERIOD = 60
# Disallow sending session cookies over insecure connections
SESSION_COOKIE_SECURE = True
@@ -1004,16 +994,5 @@ DEFAULT_CONTAINER_RUN_OPTIONS = ['--network', 'slirp4netns:enable_ipv6=true']
# Mount exposed paths as hostPath resource in k8s/ocp
AWX_MOUNT_ISOLATED_PATHS_ON_K8S = False
# Time out task managers if they take longer than this many seconds
TASK_MANAGER_TIMEOUT = 300
# Number of seconds _in addition to_ the task manager timeout a job can stay
# in waiting without being reaped
JOB_WAITING_GRACE_PERIOD = 60
# Number of seconds after a container group job finished time to wait
# before the awx_k8s_reaper task will tear down the pods
K8S_POD_REAPER_GRACE_PERIOD = 60
# This is overridden downstream via /etc/tower/conf.d/cluster_host_id.py
CLUSTER_HOST_ID = socket.gethostname()

View File

@@ -7,9 +7,9 @@
"name": "ui",
"dependencies": {
"@lingui/react": "3.14.0",
"@patternfly/patternfly": "4.210.2",
"@patternfly/react-core": "^4.239.0",
"@patternfly/react-icons": "4.90.0",
"@patternfly/patternfly": "4.217.1",
"@patternfly/react-core": "^4.250.1",
"@patternfly/react-icons": "4.92.10",
"@patternfly/react-table": "4.108.0",
"ace-builds": "^1.10.1",
"ansi-to-html": "0.7.2",
@@ -3747,26 +3747,26 @@
"dev": true
},
"node_modules/@patternfly/patternfly": {
"version": "4.210.2",
"resolved": "https://registry.npmjs.org/@patternfly/patternfly/-/patternfly-4.210.2.tgz",
"integrity": "sha512-aZiW24Bxi6uVmk5RyNTp+6q6ThtlJZotNRJfWVeGuwu1UlbBuV4DFa1bpjA6jfTZpfEpX2YL5+R+4ZVSCFAVdw=="
"version": "4.217.1",
"resolved": "https://registry.npmjs.org/@patternfly/patternfly/-/patternfly-4.217.1.tgz",
"integrity": "sha512-uN7JgfQsyR16YHkuGRCTIcBcnyKIqKjGkB2SGk9x1XXH3yYGenL83kpAavX9Xtozqp17KppOlybJuzcKvZMrgw=="
},
"node_modules/@patternfly/react-core": {
"version": "4.239.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-core/-/react-core-4.239.0.tgz",
"integrity": "sha512-6CmYABCJLUXTlzCk6C3WouMNZpS0BCT+aHU8CvYpFQ/NrpYp3MJaDsYbqgCRWV42rmIO5iXun/4WhXBJzJEoQg==",
"version": "4.250.1",
"resolved": "https://registry.npmjs.org/@patternfly/react-core/-/react-core-4.250.1.tgz",
"integrity": "sha512-vAOZPQdZzYXl/vkHnHMIt1eC3nrPDdsuuErPatkNPwmSvilXuXmWP5wxoJ36FbSNRRURkprFwx52zMmWS3iHJA==",
"dependencies": {
"@patternfly/react-icons": "^4.90.0",
"@patternfly/react-styles": "^4.89.0",
"@patternfly/react-tokens": "^4.91.0",
"@patternfly/react-icons": "^4.92.6",
"@patternfly/react-styles": "^4.91.6",
"@patternfly/react-tokens": "^4.93.6",
"focus-trap": "6.9.2",
"react-dropzone": "9.0.0",
"tippy.js": "5.1.2",
"tslib": "^2.0.0"
},
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0",
"react-dom": "^16.8.0 || ^17.0.0"
"react": "^16.8 || ^17 || ^18",
"react-dom": "^16.8 || ^17 || ^18"
}
},
"node_modules/@patternfly/react-core/node_modules/tslib": {
@@ -3775,18 +3775,18 @@
"integrity": "sha512-77EbyPPpMz+FRFRuAFlWMtmgUWGe9UOG2Z25NqCwiIjRhOf5iKGuzSe5P2w1laq+FkRy4p+PCuVkJSGkzTEKVw=="
},
"node_modules/@patternfly/react-icons": {
"version": "4.90.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-icons/-/react-icons-4.90.0.tgz",
"integrity": "sha512-qEnQKbxbUgyiosiKSkeKEBwmhgJwWEqniIAFyoxj+kpzAdeu7ueWe5iBbqo06mvDOedecFiM5mIE1N0MXwk8Yw==",
"version": "4.92.10",
"resolved": "https://registry.npmjs.org/@patternfly/react-icons/-/react-icons-4.92.10.tgz",
"integrity": "sha512-vwCy7b+OyyuvLDSLqLUG2DkJZgMDogjld8tJTdAaG8HiEhC1sJPZac+5wD7AuS3ym/sQolS4vYtNiVDnMEORxA==",
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0",
"react-dom": "^16.8.0 || ^17.0.0"
"react": "^16.8 || ^17 || ^18",
"react-dom": "^16.8 || ^17 || ^18"
}
},
"node_modules/@patternfly/react-styles": {
"version": "4.89.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-styles/-/react-styles-4.89.0.tgz",
"integrity": "sha512-SkT+qx3Xqu70T5s+i/AUT2hI2sKAPDX4ffeiJIUDu/oyWiFdk+/9DEivnLSyJMruroXXN33zKibvzb5rH7DKTQ=="
"version": "4.91.10",
"resolved": "https://registry.npmjs.org/@patternfly/react-styles/-/react-styles-4.91.10.tgz",
"integrity": "sha512-fAG4Vjp63ohiR92F4e/Gkw5q1DSSckHKqdnEF75KUpSSBORzYP0EKMpupSd6ItpQFJw3iWs3MJi3/KIAAfU1Jw=="
},
"node_modules/@patternfly/react-table": {
"version": "4.108.0",
@@ -3811,9 +3811,9 @@
"integrity": "sha512-d6xOpEDfsi2CZVlPQzGeux8XMwLT9hssAsaPYExaQMuYskwb+x1x7J371tWlbBdWHroy99KnVB6qIkUbs5X3UQ=="
},
"node_modules/@patternfly/react-tokens": {
"version": "4.91.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-tokens/-/react-tokens-4.91.0.tgz",
"integrity": "sha512-QeQCy8o8E/16fAr8mxqXIYRmpTsjCHJXi5p5jmgEDFmYMesN6Pqfv6N5D0FHb+CIaNOZWRps7GkWvlIMIE81sw=="
"version": "4.93.10",
"resolved": "https://registry.npmjs.org/@patternfly/react-tokens/-/react-tokens-4.93.10.tgz",
"integrity": "sha512-F+j1irDc9M6zvY6qNtDryhbpnHz3R8ymHRdGelNHQzPTIK88YSWEnT1c9iUI+uM/iuZol7sJmO5STtg2aPIDRQ=="
},
"node_modules/@pmmmwh/react-refresh-webpack-plugin": {
"version": "0.5.4",
@@ -25089,18 +25089,18 @@
"dev": true
},
"@patternfly/patternfly": {
"version": "4.210.2",
"resolved": "https://registry.npmjs.org/@patternfly/patternfly/-/patternfly-4.210.2.tgz",
"integrity": "sha512-aZiW24Bxi6uVmk5RyNTp+6q6ThtlJZotNRJfWVeGuwu1UlbBuV4DFa1bpjA6jfTZpfEpX2YL5+R+4ZVSCFAVdw=="
"version": "4.217.1",
"resolved": "https://registry.npmjs.org/@patternfly/patternfly/-/patternfly-4.217.1.tgz",
"integrity": "sha512-uN7JgfQsyR16YHkuGRCTIcBcnyKIqKjGkB2SGk9x1XXH3yYGenL83kpAavX9Xtozqp17KppOlybJuzcKvZMrgw=="
},
"@patternfly/react-core": {
"version": "4.239.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-core/-/react-core-4.239.0.tgz",
"integrity": "sha512-6CmYABCJLUXTlzCk6C3WouMNZpS0BCT+aHU8CvYpFQ/NrpYp3MJaDsYbqgCRWV42rmIO5iXun/4WhXBJzJEoQg==",
"version": "4.250.1",
"resolved": "https://registry.npmjs.org/@patternfly/react-core/-/react-core-4.250.1.tgz",
"integrity": "sha512-vAOZPQdZzYXl/vkHnHMIt1eC3nrPDdsuuErPatkNPwmSvilXuXmWP5wxoJ36FbSNRRURkprFwx52zMmWS3iHJA==",
"requires": {
"@patternfly/react-icons": "^4.90.0",
"@patternfly/react-styles": "^4.89.0",
"@patternfly/react-tokens": "^4.91.0",
"@patternfly/react-icons": "^4.92.6",
"@patternfly/react-styles": "^4.91.6",
"@patternfly/react-tokens": "^4.93.6",
"focus-trap": "6.9.2",
"react-dropzone": "9.0.0",
"tippy.js": "5.1.2",
@@ -25115,15 +25115,15 @@
}
},
"@patternfly/react-icons": {
"version": "4.90.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-icons/-/react-icons-4.90.0.tgz",
"integrity": "sha512-qEnQKbxbUgyiosiKSkeKEBwmhgJwWEqniIAFyoxj+kpzAdeu7ueWe5iBbqo06mvDOedecFiM5mIE1N0MXwk8Yw==",
"version": "4.92.10",
"resolved": "https://registry.npmjs.org/@patternfly/react-icons/-/react-icons-4.92.10.tgz",
"integrity": "sha512-vwCy7b+OyyuvLDSLqLUG2DkJZgMDogjld8tJTdAaG8HiEhC1sJPZac+5wD7AuS3ym/sQolS4vYtNiVDnMEORxA==",
"requires": {}
},
"@patternfly/react-styles": {
"version": "4.89.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-styles/-/react-styles-4.89.0.tgz",
"integrity": "sha512-SkT+qx3Xqu70T5s+i/AUT2hI2sKAPDX4ffeiJIUDu/oyWiFdk+/9DEivnLSyJMruroXXN33zKibvzb5rH7DKTQ=="
"version": "4.91.10",
"resolved": "https://registry.npmjs.org/@patternfly/react-styles/-/react-styles-4.91.10.tgz",
"integrity": "sha512-fAG4Vjp63ohiR92F4e/Gkw5q1DSSckHKqdnEF75KUpSSBORzYP0EKMpupSd6ItpQFJw3iWs3MJi3/KIAAfU1Jw=="
},
"@patternfly/react-table": {
"version": "4.108.0",
@@ -25146,9 +25146,9 @@
}
},
"@patternfly/react-tokens": {
"version": "4.91.0",
"resolved": "https://registry.npmjs.org/@patternfly/react-tokens/-/react-tokens-4.91.0.tgz",
"integrity": "sha512-QeQCy8o8E/16fAr8mxqXIYRmpTsjCHJXi5p5jmgEDFmYMesN6Pqfv6N5D0FHb+CIaNOZWRps7GkWvlIMIE81sw=="
"version": "4.93.10",
"resolved": "https://registry.npmjs.org/@patternfly/react-tokens/-/react-tokens-4.93.10.tgz",
"integrity": "sha512-F+j1irDc9M6zvY6qNtDryhbpnHz3R8ymHRdGelNHQzPTIK88YSWEnT1c9iUI+uM/iuZol7sJmO5STtg2aPIDRQ=="
},
"@pmmmwh/react-refresh-webpack-plugin": {
"version": "0.5.4",

View File

@@ -7,9 +7,9 @@
},
"dependencies": {
"@lingui/react": "3.14.0",
"@patternfly/patternfly": "4.210.2",
"@patternfly/react-core": "^4.239.0",
"@patternfly/react-icons": "4.90.0",
"@patternfly/patternfly": "4.217.1",
"@patternfly/react-core": "^4.250.1",
"@patternfly/react-icons": "4.92.10",
"@patternfly/react-table": "4.108.0",
"ace-builds": "^1.10.1",
"ansi-to-html": "0.7.2",

View File

@@ -1,49 +0,0 @@
import React from 'react';
import { arrayOf, bool, number, shape, string } from 'prop-types';
import { Label, LabelGroup } from '@patternfly/react-core';
import { Link } from 'react-router-dom';
function InstanceGroupLabels({ labels, isLinkable }) {
const buildLinkURL = (isContainerGroup) =>
isContainerGroup
? '/instance_groups/container_group/'
: '/instance_groups/';
return (
<LabelGroup numLabels={5}>
{labels.map(({ id, name, is_container_group }) =>
isLinkable ? (
<Label
color="blue"
key={id}
render={({ className, content, componentRef }) => (
<Link
className={className}
innerRef={componentRef}
to={`${buildLinkURL(is_container_group)}${id}/details`}
>
{content}
</Link>
)}
>
{name}
</Label>
) : (
<Label color="blue" key={id}>
{name}
</Label>
)
)}
</LabelGroup>
);
}
InstanceGroupLabels.propTypes = {
labels: arrayOf(shape({ id: number.isRequired, name: string.isRequired }))
.isRequired,
isLinkable: bool,
};
InstanceGroupLabels.defaultProps = { isLinkable: false };
export default InstanceGroupLabels;

View File

@@ -1 +0,0 @@
export { default } from './InstanceGroupLabels';

View File

@@ -6,7 +6,6 @@ import { Link } from 'react-router-dom';
import styled from 'styled-components';
import { Chip, Divider, Title } from '@patternfly/react-core';
import { toTitleCase } from 'util/strings';
import InstanceGroupLabels from 'components/InstanceGroupLabels';
import CredentialChip from '../CredentialChip';
import ChipGroup from '../ChipGroup';
import { DetailList, Detail, UserDateDetail } from '../DetailList';
@@ -228,7 +227,21 @@ function PromptDetail({
label={t`Instance Groups`}
rows={4}
value={
<InstanceGroupLabels labels={overrides.instance_groups} />
<ChipGroup
numChips={5}
totalChips={overrides.instance_groups.length}
ouiaId="prompt-instance-groups-chips"
>
{overrides.instance_groups.map((instance_group) => (
<Chip
key={instance_group.id}
ouiaId={`instance-group-${instance_group.id}-chip`}
isReadOnly
>
{instance_group.name}
</Chip>
))}
</ChipGroup>
}
/>
)}

View File

@@ -10,7 +10,6 @@ import useRequest, { useDismissableError } from 'hooks/useRequest';
import { JobTemplatesAPI, SchedulesAPI, WorkflowJobTemplatesAPI } from 'api';
import { parseVariableField, jsonToYaml } from 'util/yaml';
import { useConfig } from 'contexts/Config';
import InstanceGroupLabels from 'components/InstanceGroupLabels';
import parseRuleObj from '../shared/parseRuleObj';
import FrequencyDetails from './FrequencyDetails';
import AlertModal from '../../AlertModal';
@@ -28,6 +27,11 @@ import { VariablesDetail } from '../../CodeEditor';
import { VERBOSITY } from '../../VerbositySelectField';
import getHelpText from '../../../screens/Template/shared/JobTemplate.helptext';
const buildLinkURL = (instance) =>
instance.is_container_group
? '/instance_groups/container_group/'
: '/instance_groups/';
const PromptDivider = styled(Divider)`
margin-top: var(--pf-global--spacer--lg);
margin-bottom: var(--pf-global--spacer--lg);
@@ -494,7 +498,26 @@ function ScheduleDetail({ hasDaysToKeepField, schedule, surveyConfig }) {
fullWidth
label={t`Instance Groups`}
value={
<InstanceGroupLabels labels={instanceGroups} isLinkable />
<ChipGroup
numChips={5}
totalChips={instanceGroups.length}
ouiaId="instance-group-chips"
>
{instanceGroups.map((ig) => (
<Link
to={`${buildLinkURL(ig)}${ig.id}/details`}
key={ig.id}
>
<Chip
key={ig.id}
ouiaId={`instance-group-${ig.id}-chip`}
isReadOnly
>
{ig.name}
</Chip>
</Link>
))}
</ChipGroup>
}
isEmpty={instanceGroups.length === 0}
/>

View File

@@ -416,8 +416,14 @@ function ScheduleForm({
if (options.end === 'onDate') {
if (
DateTime.fromISO(values.startDate) >=
DateTime.fromISO(options.endDate)
DateTime.fromFormat(
`${values.startDate} ${values.startTime}`,
'yyyy-LL-dd h:mm a'
).toMillis() >=
DateTime.fromFormat(
`${options.endDate} ${options.endTime}`,
'yyyy-LL-dd h:mm a'
).toMillis()
) {
freqErrors.endDate = t`Please select an end date/time that comes after the start date/time.`;
}

View File

@@ -900,6 +900,36 @@ describe('<ScheduleForm />', () => {
);
});
test('should create schedule with the same start and end date provided that the end date is at a later time', async () => {
const today = DateTime.now().toFormat('yyyy-LL-dd');
const laterTime = DateTime.now().plus({ hours: 1 }).toFormat('h:mm a');
await act(async () => {
wrapper.find('DatePicker[aria-label="End date"]').prop('onChange')(
today,
new Date(today)
);
});
wrapper.update();
expect(
wrapper
.find('FormGroup[data-cy="schedule-End date/time"]')
.prop('helperTextInvalid')
).toBe(
'Please select an end date/time that comes after the start date/time.'
);
await act(async () => {
wrapper.find('TimePicker[aria-label="End time"]').prop('onChange')(
laterTime
);
});
wrapper.update();
expect(
wrapper
.find('FormGroup[data-cy="schedule-End date/time"]')
.prop('helperTextInvalid')
).toBe(undefined);
});
test('error shown when on day number is not between 1 and 31', async () => {
await act(async () => {
wrapper.find('FrequencySelect#schedule-frequency').invoke('onChange')([

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -282,7 +282,7 @@ const mockInputSources = {
summary_fields: {
source_credential: {
id: 20,
name: 'CyberArk Conjur Secret Lookup',
name: 'CyberArk Conjur Secrets Manager Lookup',
description: '',
kind: 'conjur',
cloud: false,
@@ -301,7 +301,7 @@ const mockInputSources = {
summary_fields: {
source_credential: {
id: 20,
name: 'CyberArk Conjur Secret Lookup',
name: 'CyberArk Conjur Secrets Manager Lookup',
description: '',
kind: 'conjur',
cloud: false,

View File

@@ -36,14 +36,14 @@ const mockCredentialTypeDetail = {
url: '/api/v2/credential_types/20/',
related: {
named_url:
'/api/v2/credential_types/CyberArk Conjur Secret Lookup+external/',
'/api/v2/credential_types/CyberArk Conjur Secrets Manager Lookup+external/',
credentials: '/api/v2/credential_types/20/credentials/',
activity_stream: '/api/v2/credential_types/20/activity_stream/',
},
summary_fields: { user_capabilities: { edit: false, delete: false } },
created: '2020-05-18T21:53:35.398260Z',
modified: '2020-05-18T21:54:05.451444Z',
name: 'CyberArk Conjur Secret Lookup',
name: 'CyberArk Conjur Secrets Manager Lookup',
description: '',
kind: 'external',
namespace: 'conjur',

View File

@@ -546,7 +546,7 @@
},
"created": "2020-05-18T21:53:35.398260Z",
"modified": "2020-05-18T21:54:05.451444Z",
"name": "CyberArk Conjur Secret Lookup",
"name": "CyberArk Conjur Secrets Manager Lookup",
"description": "",
"kind": "external",
"namespace": "conjur",

View File

@@ -3,7 +3,7 @@
"type": "credential",
"url": "/api/v2/credentials/1/",
"related": {
"named_url": "/api/v2/credentials/CyberArk Conjur Secret Lookup++CyberArk Conjur Secret Lookup+external++/",
"named_url": "/api/v2/credentials/CyberArk Conjur Secrets Manager Lookup+external++/",
"created_by": "/api/v2/users/1/",
"modified_by": "/api/v2/users/1/",
"activity_stream": "/api/v2/credentials/1/activity_stream/",
@@ -19,7 +19,7 @@
"summary_fields": {
"credential_type": {
"id": 20,
"name": "CyberArk Conjur Secret Lookup",
"name": "CyberArk Conjur Secrets Manager Lookup",
"description": ""
},
"created_by": {
@@ -69,7 +69,7 @@
},
"created": "2020-05-19T12:51:36.956029Z",
"modified": "2020-05-19T12:51:36.956086Z",
"name": "CyberArk Conjur Secret Lookup",
"name": "CyberArk Conjur Secrets Manager Lookup",
"description": "",
"organization": null,
"credential_type": 20,

View File

@@ -1,6 +1,6 @@
import React, { useCallback, useEffect, useState } from 'react';
import { useHistory, useParams } from 'react-router-dom';
import { Link, useHistory, useParams } from 'react-router-dom';
import { t, Plural } from '@lingui/macro';
import {
Button,
@@ -11,6 +11,7 @@ import {
CodeBlockCode,
Tooltip,
Slider,
Label,
} from '@patternfly/react-core';
import { DownloadIcon, OutlinedClockIcon } from '@patternfly/react-icons';
import styled from 'styled-components';
@@ -33,7 +34,6 @@ import useRequest, {
useDismissableError,
} from 'hooks/useRequest';
import HealthCheckAlert from 'components/HealthCheckAlert';
import InstanceGroupLabels from 'components/InstanceGroupLabels';
import RemoveInstanceButton from '../Shared/RemoveInstanceButton';
const Unavailable = styled.span`
@@ -156,6 +156,11 @@ function InstanceDetail({ setBreadcrumb, isK8s }) {
</>
);
const buildLinkURL = (inst) =>
inst.is_container_group
? '/instance_groups/container_group/'
: '/instance_groups/';
const { error, dismissError } = useDismissableError(
updateInstanceError || healthCheckError
);
@@ -220,9 +225,25 @@ function InstanceDetail({ setBreadcrumb, isK8s }) {
label={t`Instance Groups`}
dataCy="instance-groups"
helpText={t`The Instance Groups to which this instance belongs.`}
value={
<InstanceGroupLabels labels={instanceGroups} isLinkable />
}
value={instanceGroups.map((ig) => (
<React.Fragment key={ig.id}>
<Label
color="blue"
isTruncated
render={({ className, content, componentRef }) => (
<Link
to={`${buildLinkURL(ig)}${ig.id}/details`}
className={className}
innerRef={componentRef}
>
{content}
</Link>
)}
>
{ig.name}
</Label>{' '}
</React.Fragment>
))}
isEmpty={instanceGroups.length === 0}
/>
)}

View File

@@ -23,7 +23,6 @@ import { InventoriesAPI } from 'api';
import useRequest, { useDismissableError } from 'hooks/useRequest';
import { Inventory } from 'types';
import { relatedResourceDeleteRequests } from 'util/getRelatedResourceDeleteDetails';
import InstanceGroupLabels from 'components/InstanceGroupLabels';
import getHelpText from '../shared/Inventory.helptext';
function InventoryDetail({ inventory }) {
@@ -106,7 +105,23 @@ function InventoryDetail({ inventory }) {
<Detail
fullWidth
label={t`Instance Groups`}
value={<InstanceGroupLabels labels={instanceGroups} isLinkable />}
value={
<ChipGroup
numChips={5}
totalChips={instanceGroups?.length}
ouiaId="instance-group-chips"
>
{instanceGroups?.map((ig) => (
<Chip
key={ig.id}
isReadOnly
ouiaId={`instance-group-${ig.id}-chip`}
>
{ig.name}
</Chip>
))}
</ChipGroup>
}
isEmpty={instanceGroups.length === 0}
/>
)}

View File

@@ -131,8 +131,9 @@ describe('<InventoryDetail />', () => {
expect(InventoriesAPI.readInstanceGroups).toHaveBeenCalledWith(
mockInventory.id
);
const label = wrapper.find('Label').at(0);
expect(label.prop('children')).toEqual('Foo');
const chip = wrapper.find('Chip').at(0);
expect(chip.prop('isReadOnly')).toEqual(true);
expect(chip.prop('children')).toEqual('Foo');
});
test('should not load instance groups', async () => {

View File

@@ -2,7 +2,7 @@ import React, { useCallback, useEffect } from 'react';
import { Link, useHistory } from 'react-router-dom';
import { t } from '@lingui/macro';
import { Button, Label } from '@patternfly/react-core';
import { Button, Chip, Label } from '@patternfly/react-core';
import { Inventory } from 'types';
import { InventoriesAPI, UnifiedJobsAPI } from 'api';
@@ -10,6 +10,7 @@ import useRequest, { useDismissableError } from 'hooks/useRequest';
import AlertModal from 'components/AlertModal';
import { CardBody, CardActionsRow } from 'components/Card';
import ChipGroup from 'components/ChipGroup';
import { VariablesDetail } from 'components/CodeEditor';
import ContentError from 'components/ContentError';
import ContentLoading from 'components/ContentLoading';
@@ -17,7 +18,6 @@ import DeleteButton from 'components/DeleteButton';
import { DetailList, Detail, UserDateDetail } from 'components/DetailList';
import ErrorDetail from 'components/ErrorDetail';
import Sparkline from 'components/Sparkline';
import InstanceGroupLabels from 'components/InstanceGroupLabels';
function SmartInventoryDetail({ inventory }) {
const history = useHistory();
@@ -120,7 +120,23 @@ function SmartInventoryDetail({ inventory }) {
<Detail
fullWidth
label={t`Instance groups`}
value={<InstanceGroupLabels labels={instanceGroups} />}
value={
<ChipGroup
numChips={5}
totalChips={instanceGroups.length}
ouiaId="instance-group-chips"
>
{instanceGroups.map((ig) => (
<Chip
key={ig.id}
isReadOnly
ouiaId={`instance-group-${ig.id}-chip`}
>
{ig.name}
</Chip>
))}
</ChipGroup>
}
isEmpty={instanceGroups.length === 0}
/>
<VariablesDetail

View File

@@ -70,7 +70,6 @@ const getStdOutValue = (hostEvent) => {
function HostEventModal({ onClose, hostEvent = {}, isOpen = false }) {
const [hostStatus, setHostStatus] = useState(null);
const [activeTabKey, setActiveTabKey] = useState(0);
useEffect(() => {
setHostStatus(processEventStatus(hostEvent));
}, [setHostStatus, hostEvent]);
@@ -108,11 +107,11 @@ function HostEventModal({ onClose, hostEvent = {}, isOpen = false }) {
style={{ alignItems: 'center', marginTop: '20px' }}
gutter="sm"
>
<Detail label={t`Host`} value={hostEvent.host_name} />
{hostEvent.summary_fields.host?.description ? (
<Detail label={t`Host`} value={hostEvent.event_data?.host} />
{hostEvent.summary_fields?.host?.description ? (
<Detail
label={t`Description`}
value={hostEvent.summary_fields.host.description}
value={hostEvent.summary_fields?.host?.description}
/>
) : null}
{hostStatus ? (
@@ -125,12 +124,9 @@ function HostEventModal({ onClose, hostEvent = {}, isOpen = false }) {
<Detail label={t`Task`} value={hostEvent.task} />
<Detail
label={t`Module`}
value={hostEvent.event_data.task_action || t`No result found`}
/>
<Detail
label={t`Command`}
value={hostEvent?.event_data?.res?.cmd}
value={hostEvent.event_data?.task_action || t`No result found`}
/>
<Detail label={t`Command`} value={hostEvent.event_data?.res?.cmd} />
</DetailList>
</Tab>
<Tab

View File

@@ -52,6 +52,47 @@ const hostEvent = {
},
};
const partialHostEvent = {
changed: true,
event: 'runner_on_ok',
event_data: {
host: 'foo',
play: 'all',
playbook: 'run_command.yml',
res: {
ansible_loop_var: 'item',
changed: true,
item: '1',
msg: 'This is a debug message: 1',
stdout:
' total used free shared buff/cache available\nMem: 7973 3005 960 30 4007 4582\nSwap: 1023 0 1023',
stderr: 'problems',
cmd: ['free', '-m'],
stderr_lines: [],
stdout_lines: [
' total used free shared buff/cache available',
'Mem: 7973 3005 960 30 4007 4582',
'Swap: 1023 0 1023',
],
},
task: 'command',
task_action: 'command',
},
event_display: 'Host OK',
event_level: 3,
failed: false,
host: 1,
id: 123,
job: 4,
play: 'all',
playbook: 'run_command.yml',
stdout: `stdout: "changed: [localhost] => {"changed": true, "cmd": ["free", "-m"], "delta": "0:00:01.479609", "end": "2019-09-10 14:21:45.469533", "rc": 0, "start": "2019-09-10 14:21:43.989924", "stderr": "", "stderr_lines": [], "stdout": " total used free shared buff/cache available\nMem: 7973 3005 960 30 4007 4582\nSwap: 1023 0 1023", "stdout_lines": [" total used free shared buff/cache available", "Mem: 7973 3005 960 30 4007 4582", "Swap: 1023 0 1023"]}"
`,
task: 'command',
type: 'job_event',
url: '/api/v2/job_events/123/',
};
/*
Some libraries return a list of string in stdout
Example: https://github.com/ansible-collections/cisco.ios/blob/main/plugins/modules/ios_command.py#L124-L128
@@ -134,6 +175,13 @@ describe('HostEventModal', () => {
expect(wrapper).toHaveLength(1);
});
test('renders successfully with partial data', () => {
const wrapper = shallow(
<HostEventModal hostEvent={partialHostEvent} onClose={() => {}} />
);
expect(wrapper).toHaveLength(1);
});
test('should render all tabs', () => {
const wrapper = shallow(
<HostEventModal hostEvent={hostEvent} onClose={() => {}} isOpen />

View File

@@ -2,7 +2,7 @@ import React, { useEffect, useState, useCallback } from 'react';
import { Link, useHistory, useRouteMatch } from 'react-router-dom';
import { t } from '@lingui/macro';
import { Button } from '@patternfly/react-core';
import { Button, Chip } from '@patternfly/react-core';
import { OrganizationsAPI } from 'api';
import { DetailList, Detail, UserDateDetail } from 'components/DetailList';
import { CardBody, CardActionsRow } from 'components/Card';
@@ -16,7 +16,6 @@ import ErrorDetail from 'components/ErrorDetail';
import useRequest, { useDismissableError } from 'hooks/useRequest';
import { useConfig } from 'contexts/Config';
import ExecutionEnvironmentDetail from 'components/ExecutionEnvironmentDetail';
import InstanceGroupLabels from 'components/InstanceGroupLabels';
import { relatedResourceDeleteRequests } from 'util/getRelatedResourceDeleteDetails';
function OrganizationDetail({ organization }) {
@@ -80,6 +79,11 @@ function OrganizationDetail({ organization }) {
return <ContentError error={contentError} />;
}
const buildLinkURL = (instance) =>
instance.is_container_group
? '/instance_groups/container_group/'
: '/instance_groups/';
return (
<CardBody>
<DetailList>
@@ -122,7 +126,25 @@ function OrganizationDetail({ organization }) {
fullWidth
label={t`Instance Groups`}
helpText={t`The Instance Groups for this Organization to run on.`}
value={<InstanceGroupLabels labels={instanceGroups} isLinkable />}
value={
<ChipGroup
numChips={5}
totalChips={instanceGroups.length}
ouiaId="instance-group-chips"
>
{instanceGroups.map((ig) => (
<Link to={`${buildLinkURL(ig)}${ig.id}/details`} key={ig.id}>
<Chip
key={ig.id}
isReadOnly
ouiaId={`instance-group-${ig.id}-chip`}
>
{ig.name}
</Chip>
</Link>
))}
</ChipGroup>
}
isEmpty={instanceGroups.length === 0}
/>
)}

View File

@@ -90,7 +90,7 @@ describe('<OrganizationDetail />', () => {
await waitForElement(component, 'ContentLoading', (el) => el.length === 0);
expect(
component
.find('Label')
.find('Chip')
.findWhere((el) => el.text() === 'One')
.exists()
).toBe(true);

View File

@@ -34,7 +34,6 @@ import useRequest, { useDismissableError } from 'hooks/useRequest';
import useBrandName from 'hooks/useBrandName';
import ExecutionEnvironmentDetail from 'components/ExecutionEnvironmentDetail';
import { relatedResourceDeleteRequests } from 'util/getRelatedResourceDeleteDetails';
import InstanceGroupLabels from 'components/InstanceGroupLabels';
import getHelpText from '../shared/JobTemplate.helptext';
function JobTemplateDetail({ template }) {
@@ -168,6 +167,11 @@ function JobTemplateDetail({ template }) {
);
};
const buildLinkURL = (instance) =>
instance.is_container_group
? '/instance_groups/container_group/'
: '/instance_groups/';
if (instanceGroupsError) {
return <ContentError error={instanceGroupsError} />;
}
@@ -418,7 +422,25 @@ function JobTemplateDetail({ template }) {
label={t`Instance Groups`}
dataCy="jt-detail-instance-groups"
helpText={helpText.instanceGroups}
value={<InstanceGroupLabels labels={instanceGroups} isLinkable />}
value={
<ChipGroup
numChips={5}
totalChips={instanceGroups.length}
ouiaId="instance-group-chips"
>
{instanceGroups.map((ig) => (
<Link to={`${buildLinkURL(ig)}${ig.id}/details`} key={ig.id}>
<Chip
key={ig.id}
ouiaId={`instance-group-${ig.id}-chip`}
isReadOnly
>
{ig.name}
</Chip>
</Link>
))}
</ChipGroup>
}
isEmpty={instanceGroups.length === 0}
/>
{job_tags && (

View File

@@ -52,7 +52,7 @@ options:
- The credential type being created.
- Can be a built-in credential type such as "Machine", or a custom credential type such as "My Credential Type"
- Choices include Amazon Web Services, Ansible Galaxy/Automation Hub API Token, Centrify Vault Credential Provider Lookup,
Container Registry, CyberArk AIM Central Credential Provider Lookup, CyberArk Conjur Secret Lookup, Google Compute Engine,
Container Registry, CyberArk AIM Central Credential Provider Lookup, CyberArk Conjur Secrets Manager Lookup, Google Compute Engine,
GitHub Personal Access Token, GitLab Personal Access Token, GPG Public Key, HashiCorp Vault Secret Lookup, HashiCorp Vault Signed SSH,
Insights, Machine, Microsoft Azure Key Vault, Microsoft Azure Resource Manager, Network, OpenShift or Kubernetes API
Bearer Token, OpenStack, Red Hat Ansible Automation Platform, Red Hat Satellite 6, Red Hat Virtualization, Source Control,

View File

@@ -84,7 +84,7 @@ options:
type: str
execution_environment:
description:
- Execution Environment to use for the JT.
- Execution Environment to use for the job template.
type: str
custom_virtualenv:
description:

View File

@@ -114,7 +114,12 @@ def main():
# Update the project
result = module.post_endpoint(project['related']['update'])
if result['status_code'] != 202:
if result['status_code'] == 405:
module.fail_json(
msg="Unable to trigger a project update because the project scm_type ({0}) does not support it.".format(project['scm_type']),
response=result
)
elif result['status_code'] != 202:
module.fail_json(msg="Failed to update project, see response for details", response=result)
module.json_output['changed'] = True

View File

@@ -208,6 +208,29 @@ options:
description:
- Limit to act on, applied as a prompt, if job template prompts for limit
type: str
forks:
description:
- The number of parallel or simultaneous processes to use while executing the playbook, if job template prompts for forks
type: int
job_slice_count:
description:
- The number of jobs to slice into at runtime, if job template prompts for job slices. Will cause the Job Template to launch a workflow if value is greater than 1.
type: int
default: '1'
timeout:
description:
- Maximum time in seconds to wait for a job to finish (server-side), if job template prompts for timeout.
type: int
execution_environment:
description:
- Name of Execution Environment to be applied to job as launch-time prompts.
type: dict
suboptions:
name:
description:
- Name of Execution Environment to be applied to job as launch-time prompts.
- Uniqueness is not handled rigorously.
type: str
diff_mode:
description:
- Run diff mode, applied as a prompt, if job template prompts for diff mode
@@ -298,7 +321,6 @@ options:
related:
description:
- Related items to this workflow node.
- Must include credentials, failure_nodes, always_nodes, success_nodes, even if empty.
type: dict
suboptions:
always_nodes:
@@ -342,6 +364,46 @@ options:
description:
- Name Credentials to be applied to job as launch-time prompts.
elements: str
organization:
description:
- Name of key for use in model for organizational reference
type: dict
suboptions:
name:
description:
- The organization of the credentials exists in.
type: str
labels:
description:
- Labels to be applied to job as launch-time prompts.
- List of Label names.
- Uniqueness is not handled rigorously.
type: list
suboptions:
name:
description:
- Name Labels to be applied to job as launch-time prompts.
elements: str
organization:
description:
- Name of key for use in model for organizational reference
type: dict
suboptions:
name:
description:
- The organization of the label node exists in.
type: str
instance_groups:
description:
- Instance groups to be applied to job as launch-time prompts.
- List of Instance group names.
- Uniqueness is not handled rigorously.
type: list
suboptions:
name:
description:
- Name of Instance groups to be applied to job as launch-time prompts.
elements: str
destroy_current_nodes:
description:
- Set in order to destroy current workflow_nodes on the workflow.
@@ -474,11 +536,21 @@ EXAMPLES = '''
name: Default
name: job template 2
type: job_template
execution_environment:
name: My EE
related:
success_nodes: []
failure_nodes: []
always_nodes: []
credentials: []
credentials:
- name: cyberark
organization:
name: Default
instance_groups:
- name: SunCavanaugh Cloud
- name: default
labels:
- name: Custom Label
- name: Another Custom Label
organization:
name: Default
register: result
'''
@@ -547,6 +619,9 @@ def create_workflow_nodes(module, response, workflow_nodes, workflow_id):
'limit',
'diff_mode',
'verbosity',
'forks',
'job_slice_count',
'timeout',
'all_parents_must_converge',
'state',
):
@@ -555,6 +630,10 @@ def create_workflow_nodes(module, response, workflow_nodes, workflow_id):
workflow_node_fields[field_name] = field_val
if workflow_node['identifier']:
search_fields = {'identifier': workflow_node['identifier']}
if 'execution_environment' in workflow_node:
workflow_node_fields['execution_environment'] = module.get_one(
'execution_environments', name_or_id=workflow_node['execution_environment']['name']
)['id']
# Set Search fields
search_fields['workflow_job_template'] = workflow_node_fields['workflow_job_template'] = workflow_id
@@ -641,15 +720,26 @@ def create_workflow_nodes_association(module, response, workflow_nodes, workflow
# Get id's for association fields
association_fields = {}
for association in ('always_nodes', 'success_nodes', 'failure_nodes', 'credentials'):
for association in (
'always_nodes',
'success_nodes',
'failure_nodes',
'credentials',
'labels',
'instance_groups',
):
# Extract out information if it exists
# Test if it is defined, else move to next association.
prompt_lookup = ['credentials', 'labels', 'instance_groups']
if association in workflow_node['related']:
id_list = []
lookup_data = {}
for sub_name in workflow_node['related'][association]:
if association == 'credentials':
endpoint = 'credentials'
lookup_data = {'name': sub_name['name']}
if association in prompt_lookup:
endpoint = association
if 'organization' in sub_name:
lookup_data['organization'] = module.resolve_name_to_id('organizations', sub_name['organization']['name'])
lookup_data['name'] = sub_name['name']
else:
endpoint = 'workflow_job_template_nodes'
lookup_data = {'identifier': sub_name['identifier']}

View File

@@ -729,6 +729,24 @@
organization:
name: Default
type: workflow_job_template
forks: 12
job_slice_count: 2
timeout: 23
execution_environment:
name: "{{ ee1 }}"
related:
credentials:
- name: "{{ scm_cred_name }}"
organization:
name: Default
instance_groups:
- name: "{{ ig1 }}"
- name: "{{ ig2 }}"
labels:
- name: "{{ label1 }}"
- name: "{{ label2 }}"
organization:
name: "{{ org_name }}"
register: result
- name: Delete copied workflow job template

View File

@@ -275,7 +275,13 @@ class ApiV2(base.Base):
# When creating a project, we need to wait for its
# first project update to finish so that associated
# JTs have valid options for playbook names
_page.wait_until_completed()
try:
_page.wait_until_completed(timeout=300)
except AssertionError:
# If the project update times out, try to
# carry on in the hopes that it will
# finish before it is needed.
pass
else:
# If we are an existing project and our scm_tpye is not changing don't try and import the local_path setting
if asset['natural_key']['type'] == 'project' and 'local_path' in post_data and _page['scm_type'] == post_data['scm_type']:

View File

@@ -1,6 +1,7 @@
from contextlib import suppress
from awxkit.api.pages import SystemJobTemplate
from awxkit.api.pages import JobTemplate, SystemJobTemplate, Project, InventorySource
from awxkit.api.pages.workflow_job_templates import WorkflowJobTemplate
from awxkit.api.mixins import HasCreate
from awxkit.api.resources import resources
from awxkit.config import config
@@ -11,7 +12,7 @@ from . import base
class Schedule(HasCreate, base.Base):
dependencies = [SystemJobTemplate]
dependencies = [JobTemplate, SystemJobTemplate, Project, InventorySource, WorkflowJobTemplate]
NATURAL_KEY = ('unified_job_template', 'name')
def silent_delete(self):

View File

@@ -52,6 +52,7 @@ html_static_path = ['_static']
rst_epilog = '''
.. |prog| replace:: awx
.. |at| replace:: Ansible Tower
.. |RHAT| replace:: Red Hat Ansible Tower
.. |at| replace:: automation controller
.. |At| replace:: Automation controller
.. |RHAT| replace:: Red Hat Ansible Automation Platform controller
'''

View File

@@ -197,8 +197,10 @@ def parse_resource(client, skip_deprecated=False):
if hasattr(client, 'v2'):
for k in client.v2.json.keys():
if k in ('dashboard',):
# the Dashboard API is deprecated and not supported
if k in ('dashboard', 'config'):
# - the Dashboard API is deprecated and not supported
# - the Config command is already dealt with by the
# CustomCommand section above
continue
# argparse aliases are *only* supported in Python3 (not 2.7)

View File

@@ -1,24 +0,0 @@
Copyright (c) 2010, Matt Croydon, Mikhail Korobov
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the tastypie nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL MATT CROYDON BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -1,25 +1,22 @@
# Dependency Management
The `requirements.txt` file is generated from `requirements.in`, using `pip-tools` `pip-compile`.
The `requirements.txt` file is generated from `requirements.in` and `requirements_git.txt`, using `pip-tools` and `pip-compile`.
## How To Use
Commands should be run from inside the `./requirements` directory of the awx repository.
Commands should be run in the awx container from inside the `./requirements` directory of the awx repository.
### Upgrading or Adding Select Libraries
If you need to add or upgrade one targeted library, then modify `requirements.in`,
then run the script:
`./updater.sh`
NOTE: `./updater.sh` uses /usr/bin/python3.6, to match the current python version
(3.6) used to build releases.
`./updater.sh run`
#### Upgrading Unpinned Dependency
If you require a new version of a dependency that does not have a pinned version
for a fix or feature, pin a minimum version and run `./updater.sh`. For example,
for a fix or feature, pin a minimum version in `requirements.in` and run `./updater.sh run`. For example,
replace the line `asgi-amqp` with `asgi-amqp>=1.1.4`, and consider leaving a
note.

View File

@@ -10,7 +10,7 @@ cryptography>=36.0.2,<37.0.0 # Until paramiko fixes https://github.com/paramiko/
Cython<3 # Since the bump to PyYAML 5.4.1 this is now a mandatory dep
daphne
distro
django==3.2.13 # see UPGRADE BLOCKERs
django==3.2.16 # see UPGRADE BLOCKERs https://github.com/ansible/awx/security/dependabot/67
django-auth-ldap
django-cors-headers>=3.5.0
django-crum
@@ -19,7 +19,6 @@ django-guid==3.2.1
django-oauth-toolkit==1.4.1
django-polymorphic
django-pglocks
django-qsstats-magic
django-redis
django-solo
django-split-settings

View File

@@ -86,7 +86,7 @@ defusedxml==0.6.0
# social-auth-core
distro==1.5.0
# via -r /awx_devel/requirements/requirements.in
django==3.2.13
django==3.2.16
# via
# -r /awx_devel/requirements/requirements.in
# channels
@@ -115,9 +115,6 @@ django-pglocks==1.0.4
# via -r /awx_devel/requirements/requirements.in
django-polymorphic==3.1.0
# via -r /awx_devel/requirements/requirements.in
django-qsstats-magic==1.1.0
# via -r /awx_devel/requirements/requirements.in
# via -r /awx_devel/requirements/requirements_git.txt
django-redis==4.5.0
# via -r /awx_devel/requirements/requirements.in
django-solo==2.0.0

View File

@@ -33,11 +33,47 @@ generate_requirements() {
main() {
base_dir=$(pwd)
_tmp="$(mktemp -d --suffix .awx-requirements XXXX -p /tmp)"
_tmp=$(python -c "import tempfile; print(tempfile.mkdtemp(suffix='.awx-requirements', dir='/tmp'))")
trap _cleanup INT TERM EXIT
if [ "$1" = "upgrade" ]; then
case $1 in
"run")
NEEDS_HELP=0
;;
"upgrade")
NEEDS_HELP=0
pip_compile="${pip_compile} --upgrade"
;;
"help")
NEEDS_HELP=1
;;
*)
echo ""
echo "ERROR: Parameter $1 not valid"
echo ""
NEEDS_HELP=1
;;
esac
if [[ "$NEEDS_HELP" == "1" ]] ; then
echo "This script generates requirements.txt from requirements.in and requirements_git.in"
echo "It should be run from within the awx container"
echo ""
echo "Usage: $0 [run|upgrade]"
echo ""
echo "Commands:"
echo "help Print this message"
echo "run Run the process only upgrading pinned libraries from requirements.in"
echo "upgrade Upgrade all libraries to latest while respecting pinnings"
echo ""
exit
fi
if [[ ! -d /awx_devel ]] ; then
echo "This script should be run inside the awx container"
exit
fi
cp -vf requirements.txt "${_tmp}"

View File

@@ -9,4 +9,4 @@ template_dest: '_build'
receptor_image: quay.io/ansible/receptor:devel
# Helper vars to construct the proper download URL for the current architecture
image_architecture: '{{ { "x86_64": "amd64", "aarch64": "arm64", "armv7": "arm", "ppc64le": "ppc64le" }[ansible_facts.architecture] }}'
image_architecture: '{{ { "x86_64": "amd64", "aarch64": "arm64", "armv7": "arm", "arm64": "arm64", "ppc64le": "ppc64le" }[ansible_facts.architecture] }}'

View File

@@ -9,8 +9,8 @@ addons:
minikube_url_linux: 'https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64'
minikube_url_macos: 'https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64'
kubectl_url_linux: 'https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl'
kubectl_url_macos: 'https://dl.k8s.io/release/v1.21.0/bin/darwin/amd64/kubectl'
kubectl_url_linux: 'https://dl.k8s.io/release/v1.25.0/bin/linux/amd64/kubectl'
kubectl_url_macos: 'https://dl.k8s.io/release/v1.25.0/bin/darwin/amd64/kubectl'
# Service Account Name
minikube_service_account_name: 'awx-devel'

View File

@@ -8,6 +8,10 @@
state: 'directory'
mode: '0700'
- name: debug minikube_setup
debug:
var: minikube_setup
# Linux block
- block:
- name: Download Minikube
@@ -24,6 +28,7 @@
when:
- ansible_architecture == "x86_64"
- ansible_system == "Linux"
- minikube_setup | default(False) | bool
# MacOS block
- block:
@@ -41,25 +46,29 @@
when:
- ansible_architecture == "x86_64"
- ansible_system == "Darwin"
- minikube_setup | default(False) | bool
- name: Starting Minikube
shell: "{{ sources_dest }}/minikube start --driver={{ driver }} --install-addons=true --addons={{ addons | join(',') }}"
register: minikube_stdout
- block:
- name: Starting Minikube
shell: "{{ sources_dest }}/minikube start --driver={{ driver }} --install-addons=true --addons={{ addons | join(',') }}"
register: minikube_stdout
- name: Enable Ingress Controller on Minikube
shell: "{{ sources_dest }}/minikube addons enable ingress"
- name: Enable Ingress Controller on Minikube
shell: "{{ sources_dest }}/minikube addons enable ingress"
when:
- minikube_stdout.rc == 0
register: _minikube_ingress
ignore_errors: true
- name: Show Minikube Ingress known-issue 7332 warning
pause:
seconds: 5
prompt: "The Minikube Ingress addon has been disabled since it looks like you are hitting https://github.com/kubernetes/minikube/issues/7332"
when:
- '"minikube/issues/7332" in _minikube_ingress.stderr'
- ansible_system == "Darwin"
when:
- minikube_stdout.rc == 0
register: _minikube_ingress
ignore_errors: true
- name: Show Minikube Ingress known-issue 7332 warning
pause:
seconds: 5
prompt: "The Minikube Ingress addon has been disabled since it looks like you are hitting https://github.com/kubernetes/minikube/issues/7332"
when:
- '"minikube/issues/7332" in _minikube_ingress.stderr'
- ansible_system == "Darwin"
- minikube_setup | default(False) | bool
- name: Create ServiceAccount and clusterRoleBinding
k8s:

View File

@@ -28,7 +28,7 @@ Here are the main `make` targets:
Notable files:
- `tools/docker-compose/inventory` file - used to configure the AWX development environment.
- `migrate.yml` - playbook for migrating data from Local Docker to the Development Environment
- `tools/docker-compose/ansible/migrate.yml` - playbook for migrating data from Local Docker to the Development Environment
### Prerequisites
@@ -301,11 +301,19 @@ Note that you may see multiple messages of the form `2021-03-04 20:11:47,666 WAR
To bring up a 1 node AWX + minikube that is accessible from AWX run the following.
Start minikube
```bash
(host)$minikube start --cpus=4 --memory=8g --addons=ingress`
```
Start AWX
```bash
(host)$ make docker-compose-container-group
```
Alternatively, you can set the env var `MINIKUBE_CONTAINER_GROUP=true` to use the default dev env bring up. his way you can use other env flags like the cluster node count.
Alternatively, you can set the env var `MINIKUBE_CONTAINER_GROUP=true` to use the default dev env bring up. his way you can use other env flags like the cluster node count. Set `MINIKUBE_SETUP=true` to make the roles download, install and run minikube for you, but if you run into issues with this just start minikube yourself.
```bash
(host)$ MINIKUBE_CONTAINER_GROUP=true make docker-compose

View File

@@ -13,6 +13,7 @@ receptor_image: quay.io/ansible/receptor:devel
# Keys for signing work
receptor_rsa_bits: 4096
receptor_work_sign_reconfigure: false
sign_work: no # currently defaults to no because openssl version mismatch causes "unknown block type PRIVATE KEY"
work_sign_key_dir: '../_sources/receptor'
work_sign_private_keyfile: "{{ work_sign_key_dir }}/work_private_key.pem"
work_sign_public_keyfile: "{{ work_sign_key_dir }}/work_public_key.pem"

View File

@@ -86,11 +86,13 @@
command: openssl genrsa -out {{ work_sign_private_keyfile }} {{ receptor_rsa_bits }}
args:
creates: "{{ work_sign_private_keyfile }}"
when: sign_work | bool
- name: Generate public RSA key for signing work
command: openssl rsa -in {{ work_sign_private_keyfile }} -out {{ work_sign_public_keyfile }} -outform PEM -pubout
args:
creates: "{{ work_sign_public_keyfile }}"
when: sign_work | bool
- name: Include LDAP tasks if enabled
include_tasks: ldap.yml
@@ -128,6 +130,8 @@
src: "receptor-hop.conf.j2"
dest: "{{ sources_dest }}/receptor/receptor-hop.conf"
mode: '0600'
when:
- execution_node_count | int > 0
- name: Render Receptor Worker Config(s)
template:

View File

@@ -43,8 +43,10 @@ services:
- "../../docker-compose/_sources/SECRET_KEY:/etc/tower/SECRET_KEY"
- "../../docker-compose/_sources/receptor/receptor-awx-{{ loop.index }}.conf:/etc/receptor/receptor.conf"
- "../../docker-compose/_sources/receptor/receptor-awx-{{ loop.index }}.conf.lock:/etc/receptor/receptor.conf.lock"
{% if sign_work|bool %}
- "../../docker-compose/_sources/receptor/work_public_key.pem:/etc/receptor/work_public_key.pem"
- "../../docker-compose/_sources/receptor/work_private_key.pem:/etc/receptor/work_private_key.pem"
{% endif %}
# - "../../docker-compose/_sources/certs:/etc/receptor/certs" # TODO: optionally generate certs
- "/sys/fs/cgroup:/sys/fs/cgroup"
- "~/.kube/config:/var/lib/awx/.kube/config"

View File

@@ -11,12 +11,16 @@
- tcp-listener:
port: 2222
{% if sign_work|bool %}
- work-signing:
privatekey: /etc/receptor/work_private_key.pem
tokenexpiration: 1m
{% endif %}
{% if sign_work|bool %}
- work-verification:
publickey: /etc/receptor/work_public_key.pem
{% endif %}
{% for i in range(item | int + 1, control_plane_node_count | int + 1) %}
- tcp-peer:
@@ -40,7 +44,7 @@
command: ansible-runner
params: worker
allowruntimeparams: true
verifysignature: true
verifysignature: {{ sign_work }}
- work-kubernetes:
worktype: kubernetes-runtime-auth
@@ -48,7 +52,7 @@
allowruntimeauth: true
allowruntimepod: true
allowruntimeparams: true
verifysignature: true
verifysignature: {{ sign_work }}
- work-kubernetes:
worktype: kubernetes-incluster-auth
@@ -56,4 +60,4 @@
allowruntimeauth: true
allowruntimepod: true
allowruntimeparams: true
verifysignature: true
verifysignature: {{ sign_work }}

View File

@@ -16,7 +16,7 @@
command: ansible-runner
params: worker
allowruntimeparams: true
verifysignature: true
verifysignature: {{ sign_work }}
- control-service:
service: control

Some files were not shown because too many files have changed in this diff Show More