Merge pull request #6704 from ryanpetrello/11-0-0-release-version

bump version for 11.0.0

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul

.github/BOTMETA.yml

@@ -1,3 +1,4 @@
+---
 files:
     awx/ui/:
         labels: component:ui

.github/ISSUE_TEMPLATE.md

@@ -30,8 +30,9 @@ https://www.ansible.com/security
 
 ##### STEPS TO REPRODUCE
 
-<!-- For bugs, please show exactly how to reproduce the problem. For new
-features, show how the feature would be used.  -->
+<!-- For new features, show how the feature would be used. For bugs, please show
+exactly how to reproduce the problem. Ideally, provide all steps and data needed
+to recreate the bug from a new awx install. -->
 
 ##### EXPECTED RESULTS
 

.gitignore

@@ -31,6 +31,7 @@ awx/ui/templates/ui/installing.html
 awx/ui_next/node_modules/
 awx/ui_next/coverage/
 awx/ui_next/build/locales/_build
+rsyslog.pid
 /tower-license
 /tower-license/**
 tools/prometheus/data
@@ -135,9 +136,10 @@ use_dev_supervisor.txt
 
 
 # Ansible module tests
-awx_collection_test_venv/
-awx_collection/*.tar.gz
-awx_collection/galaxy.yml
+/awx_collection_test_venv/
+/awx_collection/*.tar.gz
+/awx_collection/galaxy.yml
+/sanity/
 
 .idea/*
 *.unison.tmp

.yamllint

@@ -0,0 +1,12 @@
+---
+ignore: |
+  .tox
+  awx/main/tests/data/inventory/plugins/**
+  # vault files
+  awx/main/tests/data/ansible_utils/playbooks/valid/vault.yml
+  awx/ui/test/e2e/tests/smoke-vars.yml
+
+extends: default
+
+rules:
+  line-length: disable

CHANGELOG.md

@@ -0,0 +1,198 @@
+# Changelog
+
+This is a list of high-level changes for each release of AWX. A full list of commits can be found at `https://github.com/ansible/awx/releases/tag/<version>`.
+
+## 11.0.0 (Apr 16, 2020)
+- As of AWX 11.0.0, Kubernetes-based deployments use a Deployment rather than a StatefulSet.
+- Reimplemented external logging support using rsyslogd to improve reliability and address a number of issues (https://github.com/ansible/awx/issues/5155)
+- Changed activity stream logs to include summary fields for related objects (https://github.com/ansible/awx/issues/1761)
+- Added code to more gracefully attempt to reconnect to redis if it restarts/becomes unavailable (https://github.com/ansible/awx/pull/6670)
+- Fixed a bug that caused REFRESH_TOKEN_EXPIRE_SECONDS to not properly be respected for OAuth2.0 refresh tokens generated by AWX (https://github.com/ansible/awx/issues/6630)
+- Fixed a bug that broke schedules containing RRULES with very old DTSTART dates (https://github.com/ansible/awx/pull/6550)
+- Fixed a bug that broke installs on older versions of Ansible packaged with certain Linux distributions (https://github.com/ansible/awx/issues/5501)
+- Fixed a bug that caused the activity stream to sometimes report the incorrect actor when associating user membership on SAML login (https://github.com/ansible/awx/pull/6525)
+- Fixed a bug in AWX's Grafana notification support when annotation tags are omitted (https://github.com/ansible/awx/issues/6580)
+- Fixed a bug that prevented some users from searching for Source Control credentials in the AWX user interface (https://github.com/ansible/awx/issues/6600)
+- Fixed a bug that prevented disassociating orphaned users from credentials (https://github.com/ansible/awx/pull/6554)
+- Updated Twisted to address CVE-2020-10108 and CVE-2020-10109.
+
+## 10.0.0 (Mar 30, 2020)
+- As of AWX 10.0.0, the official AWX CLI no longer supports Python 2 (it requires at least Python 3.6) (https://github.com/ansible/awx/pull/6327)
+- AWX no longer relies on RabbitMQ; Redis is added as a new dependency (https://github.com/ansible/awx/issues/5443)
+- Altered AWX's event tables to allow more than ~2 billion total events (https://github.com/ansible/awx/issues/6010)
+- Improved the performance (time to execute, and memory consumption) of the periodic job cleanup system job (https://github.com/ansible/awx/pull/6166)
+- Updated Job Templates so they now have an explicit Organization field (it is no longer inferred from the associated Project) (https://github.com/ansible/awx/issues/3903)
+- Updated social-auth-core to address an upcoming GitHub API deprecation (https://github.com/ansible/awx/issues/5970)
+- Updated to ansible-runner 1.4.6 to address various bugs.
+- Updated Django to address CVE-2020-9402
+- Updated pyyaml version to address CVE-2017-18342
+- Fixed a bug which prevented the new `scm_branch` field from being used in custom notification templates (https://github.com/ansible/awx/issues/6258)
+- Fixed a race condition that sometimes causes success/failure notifications to include an incomplete list of hosts (https://github.com/ansible/awx/pull/6290)
+- Fixed a bug that can cause certain setting pages to lose unsaved form edits when a playbook is launched (https://github.com/ansible/awx/issues/5265)
+- Fixed a bug that can prevent the "Use TLS/SSL" field from properly saving when editing email notification templates (https://github.com/ansible/awx/issues/6383)
+- Fixed a race condition that sometimes broke event/stdout processing for jobs launched in container groups (https://github.com/ansible/awx/issues/6280)
+
+## 9.3.0 (Mar 12, 2020)
+- Added the ability to specify an OAuth2 token description in the AWX CLI (https://github.com/ansible/awx/issues/6122)
+- Added support for K8S service account annotations to the installer (https://github.com/ansible/awx/pull/6007)
+- Added support for K8S imagePullSecrets to the installer (https://github.com/ansible/awx/pull/5989)
+- Launching jobs (and workflows) using the --monitor flag in the AWX CLI now returns a non-zero exit code on job failure (https://github.com/ansible/awx/issues/5920)
+- Improved UI performance for various job views when many simultaneous users are logged into AWX (https://github.com/ansible/awx/issues/5883)
+- Updated to the latest version of Django to address a few open CVEs (https://github.com/ansible/awx/pull/6080)
+- Fixed a critical bug which can cause AWX to hang and stop launching playbooks after a periodic of time (https://github.com/ansible/awx/issues/5617)
+- Fixed a bug which caused delays in project update stdout for certain large SCM clones (as of Ansible 2.9+) (https://github.com/ansible/awx/pull/6254)
+- Fixed a bug which caused certain smart inventory filters to mistakenly return duplicate hosts (https://github.com/ansible/awx/pull/5972)
+- Fixed an unclear server error when creating smart inventories with the AWX collection (https://github.com/ansible/awx/issues/6250)
+- Fixed a bug that broke Grafana notification support (https://github.com/ansible/awx/issues/6137)
+- Fixed a UI bug which prevent users with read access to an organization from editing credentials for that organization (https://github.com/ansible/awx/pull/6241)
+- Fixed a bug which prevent workflow approval records from recording a `started` and `elapsed` date (https://github.com/ansible/awx/issues/6202)
+- Fixed a bug which caused workflow nodes to have a confusing option for `verbosity` (https://github.com/ansible/awx/issues/6196)
+- Fixed an RBAC bug which prevented projects and inventory schedules from being created by certain users in certain contexts (https://github.com/ansible/awx/issues/5717)
+- Fixed a bug that caused `role_path` in a project's config to not be respected due to an error processing `/etc/ansible/ansible.cfg` (https://github.com/ansible/awx/pull/6038)
+- Fixed a bug that broke inventory updates for installs with custom home directories for the awx user (https://github.com/ansible/awx/pull/6152)
+- Fixed a bug that broke fact data collection when AWX encounters invalid/unexpected fact data (https://github.com/ansible/awx/issues/5935)
+
+
+## 9.2.0 (Feb 12, 2020)
+- Added the ability to configure the convergence behavior of workflow nodes https://github.com/ansible/awx/issues/3054
+- AWX now allows for a configurable global limit for fork count (per-job run).  The default maximum is 200. https://github.com/ansible/awx/pull/5604
+- Added the ability to specify AZURE_PUBLIC_CLOUD (for e.g., Azure Government KeyVault support) for the Azure credential plugin https://github.com/ansible/awx/issues/5138
+- Added support for several additional parameters for Satellite dynamic inventory https://github.com/ansible/awx/pull/5598
+- Added a new field to jobs for tracking the date/time a job is cancelled https://github.com/ansible/awx/pull/5610
+- Made a series of additional optimizations to the callback receiver to further improve stdout write speed for running playbooks https://github.com/ansible/awx/pull/5677 https://github.com/ansible/awx/pull/5739
+- Updated AWX to be compatible with Helm 3.x (https://github.com/ansible/awx/pull/5776)
+- Optimized AWX's job dependency/scheduling code to drastically improve processing time in scenarios where there are many pending jobs scheduled simultaneously https://github.com/ansible/awx/issues/5154
+- Fixed a bug which could cause SCM authentication details (basic auth passwords) to be reported to external loggers in certain failure scenarios (e.g., when a git clone fails and ansible itself prints an error message to stdout) https://github.com/ansible/awx/pull/5812
+- Fixed a k8s installer bug that caused installs to fail in certain situations https://github.com/ansible/awx/issues/5574
+- Fixed a number of issues that caused analytics gathering and reporting to run more often than necessary https://github.com/ansible/awx/pull/5721
+- Fixed a bug in the AWX CLI that prevented JSON-type settings from saving properly https://github.com/ansible/awx/issues/5528
+- Improved support for fetching custom virtualenv dependencies when AWX is installed behind a proxy https://github.com/ansible/awx/pull/5805
+- Updated the bundled version of openstacksdk to address a known issue https://github.com/ansible/awx/issues/5821
+- Updated the bundled vmware_inventory plugin to the latest version to address a bug https://github.com/ansible/awx/pull/5668
+- Fixed a bug that can cause inventory updates to fail to properly save their output when run within a workflow https://github.com/ansible/awx/pull/5666
+- Removed a number of pre-computed fields from the Host and Group models to improve AWX performance.  As part of this change, inventory group UIs throughout the interface no longer display status icons https://github.com/ansible/awx/pull/5448 
+
+## 9.1.1 (Jan 14, 2020)
+
+- Fixed a bug that caused database migrations on Kubernetes installs to hang https://github.com/ansible/awx/pull/5579
+- Upgraded Python-level app dependencies in AWX virtual environment https://github.com/ansible/awx/pull/5407
+- Running jobs no longer block associated inventory updates https://github.com/ansible/awx/pull/5519
+- Fixed invalid_response SAML error https://github.com/ansible/awx/pull/5577
+- Optimized the callback receiver to drastically improve the write speed of stdout for parallel jobs (https://github.com/ansible/awx/pull/5618)
+
+## 9.1.0 (Dec 17, 2019)
+- Added a command to generate a new SECRET_KEY and rekey the secrets in the database
+- Removed project update locking when jobs using it are running
+- Fixed slow queries for /api/v2/instances and /api/v2/instance_groups when smart inventories are used
+- Fixed a partial password disclosure when special characters existed in the RabbitMQ password (CVE-2019-19342)
+- Fixed hang in error handling for source control checkouts
+- Fixed an error on subsequent job runs that override the branch of a project on an instance that did not have a prior project checkout
+- Fixed an issue where jobs launched in isolated or container groups would incorrectly timeout
+- Fixed an incorrect link to instance groups documentation in the user interface
+- Fixed editing of inventory on Workflow templates
+- Fixed multiple issues with OAuth2 token cleanup system jobs
+- Fixed a bug that broke email notifications for workflow approval/deny https://github.com/ansible/awx/issues/5401
+- Updated SAML implementation to automatically login if authorization already exists
+- Updated AngularJS to 1.7.9 for CVE-2019-10768
+
+## 9.0.1 (Nov 4, 2019)
+
+- Fixed a bug in the installer that broke certain types of k8s installs https://github.com/ansible/awx/issues/5205
+
+## 9.0.0 (Oct 31, 2019)
+
+- Updated AWX images to use centos:8 as the parent image.
+- Updated to ansible-runner 1.4.4 to address various bugs.
+- Added oc and kubectl to the AWX images to support new container-based execution introduced in 8.0.0.
+- Added some optimizations to speed up the deletion of large Inventory Groups.
+- Fixed a bug that broke webhook launches for Job Templates that define a survey (https://github.com/ansible/awx/issues/5062).
+- Fixed a bug in the CLI which incorrectly parsed launch time arguments for `awx job_templates launch` and `awx workflow_job_templates launch` (https://github.com/ansible/awx/issues/5093).
+- Fixed a bug that caused inventory updates using "sourced from a project" to stop working (https://github.com/ansible/awx/issues/4750).
+- Fixed a bug that caused Slack notifications to sometimes show the wrong bot avatar (https://github.com/ansible/awx/pull/5125).
+- Fixed a bug that prevented the use of digits in AWX's URL settings (https://github.com/ansible/awx/issues/5081).
+
+## 8.0.0 (Oct 21, 2019)
+
+- The Ansible Tower Ansible modules have been migrated to a new official Ansible AWX collection: https://galaxy.ansible.com/awx/AWX
+  Please note that this functionality is only supported in Ansible 2.9+
+- AWX now supports the ability to launch jobs from external webhooks (GitHub and GitLab integration are supported).
+- AWX now supports Container Groups, a new feature that allows you to schedule and run playbooks on single-use kubernetes pods on-demand.
+- AWX now supports sending notifications when Workflow steps are approved, denied, or time out.
+- AWX now records the user who approved or denied Workflow steps.
+- AWX now supports fetching Ansible Collections from private galaxy servers.
+- AWX now checks the user's ansible.cfg for paths where role/collections may live when running project updates.
+- AWX now uses PostgreSQL 10 by default.
+- AWX now warns more loudly about underlying AMQP connectivity issues (https://github.com/ansible/awx/pull/4857).
+- Added a few optimizations to drastically improve dashboard performance for larger AWX installs (installs with several hundred thousand jobs or more).
+- Updated to the latest version of Ansible's VMWare inventory script (which adds support for vmware_guest_facts).
+- Deprecated /api/v2/inventory_scripts/ (this endpoint - and the Custom Inventory Script feature - will be removed in a future release of AWX).
+- Fixed a bug which prevented Organization Admins from removing users from their own Organization (https://github.com/ansible/awx/issues/2979)
+- Fixed a bug which sometimes caused cluster nodes to fail to re-join with a cryptic error, "No instance found with the current cluster host id" (https://github.com/ansible/awx/issues/4294)
+- Fixed a bug that prevented the use of launch-time passphrases when using credential plugins (https://github.com/ansible/awx/pull/4807)
+- Fixed a bug that caused notifications assigned at the Organization level not to take effect for Workflows in that Organization (https://github.com/ansible/awx/issues/4712)
+- Fixed a bug which caused a notable amount of CPU overhead on RabbitMQ health checks (https://github.com/ansible/awx/pull/5009)
+- Fixed a bug which sometimes caused the <return> key to stop functioning in <textarea> elements (https://github.com/ansible/awx/issues/4192)
+- Fixed a bug which caused request contention when the same OAuth2.0 token was used in multiple simultaneous requests (https://github.com/ansible/awx/issues/4694)
+- Fixed a bug related to parsing multiple choice survey options (https://github.com/ansible/awx/issues/4452).
+- Fixed a bug that caused single-sign-on icons on the login page to fail to render in certain Windows browsers (https://github.com/ansible/awx/issues/3924)
+- Fixed a number of bugs that caused certain OAuth2 settings to not be properly respected, such as REFRESH_TOKEN_EXPIRE_SECONDS.
+- Fixed a number of bugs in the AWX CLI, including a bug which sometimes caused long lines of stdout output to be unexpectedly truncated.
+- Fixed a number of bugs on the job details UI which sometimes caused auto-scrolling stdout to become stuck.
+- Fixed a bug which caused LDAP authentication to fail if the TLD of the server URL contained digits (https://github.com/ansible/awx/issues/3646)
+- Fixed a bug which broke HashiCorp Vault integration on older versions of HashiCorp Vault.
+
+## 7.0.0 (Sept 4, 2019)
+
+- AWX now detects and installs Ansible Collections defined in your project (note - this feature only works in Ansible 2.9+) (https://github.com/ansible/awx/issues/2534)
+- AWX now includes an official command line client.  Keep an eye out for a follow-up email on this mailing list for information on how to install it and try it out.
+- Added the ability to provide a specific SCM branch on jobs (https://github.com/ansible/awx/issues/282)
+- Added support for Workflow Approval Nodes, a new feature which allows you to add "pause and wait for approval" steps into your workflows (https://github.com/ansible/awx/issues/1206)
+- Added the ability to specify a specific HTTP method for webhook notifications (POST vs PUT) (https://github.com/ansible/awx/pull/4124)
+- Added the ability to specify a username and password for HTTP Basic Authorization for webhook notifications (https://github.com/ansible/awx/pull/4124)
+- Added support for customizing the text content of notifications (https://github.com/ansible/awx/issues/79)
+- Added the ability to enable and disable hosts in dynamic inventory (https://github.com/ansible/awx/pull/4420)
+- Added the description (if any) to the Job Template list (https://github.com/ansible/awx/issues/4359)
+- Added new metrics for instance hostnames and pending jobs to the /api/v2/metrics/ endpoint (https://github.com/ansible/awx/pull/4375)
+- Changed AWX's on/off toggle buttons to a non-text based style to simplify internationalization (https://github.com/ansible/awx/pull/4425)
+- Events emitted by ansible for adhoc commands are now sent to the external log aggregrator (https://github.com/ansible/awx/issues/4545)
+- Fixed a bug which allowed a user to make an organization credential in another organization without permissions to that organization (https://github.com/ansible/awx/pull/4483)
+- Fixed a bug that caused `extra_vars` on workflows to break when edited (https://github.com/ansible/awx/issues/4293)
+- Fixed a slow SQL query that caused performance issues when large numbers of groups exist (https://github.com/ansible/awx/issues/4461)
+- Fixed a few minor bugs in survey field validation (https://github.com/ansible/awx/pull/4509) (https://github.com/ansible/awx/pull/4479)
+- Fixed a bug that sometimes resulted in orphaned `ansible_runner_pi` directories in `/tmp` after playbook execution (https://github.com/ansible/awx/pull/4409)
+- Fixed a bug that caused the `is_system_auditor` flag in LDAP configuration to not work (https://github.com/ansible/awx/pull/4396)
+- Fixed a bug which caused schedules to disappear from the UI when toggled off (https://github.com/ansible/awx/pull/4378)
+- Fixed a bug that sometimes caused stdout content to contain extraneous blank lines in newer versions of Ansible (https://github.com/ansible/awx/pull/4391)
+- Updated to the latest Django security release, 2.2.4 (https://github.com/ansible/awx/pull/4410) (https://www.djangoproject.com/weblog/2019/aug/01/security-releases/)
+- Updated the default version of git to a version that includes support for x509 certificates (https://github.com/ansible/awx/issues/4362)
+- Removed the deprecated `credential` field from `/api/v2/workflow_job_templates/N/` (as part of the `/api/v1/` removal in prior AWX versions - https://github.com/ansible/awx/pull/4490).
+
+## 6.1.0 (Jul 18, 2019)
+
+- Updated AWX to use Django 2.2.2.
+- Updated the provided openstacksdk version to support new functionality (such as Nova scheduler_hints)
+- Added the ability to specify a custom cacert for the HashiCorp Vault credential plugin
+- Fixed a number of bugs related to path lookups for the HashiCorp Vault credential plugin
+- Fixed a bug which prevented signed SSH certificates from working, including the HashiCorp Vault Signed SSH backend
+- Fixed a bug which prevented custom logos from displaying on the login page (as a result of a new Content Security Policy in 6.0.0)
+- Fixed a bug which broke websocket connectivity in Apple Safari (as a result of a new Content Security Policy in 6.0.0)
+- Fixed a bug on the job output page that occasionally caused the "up" and "down" buttons to not load additional output
+- Fixed a bug on the job output page that caused quoted task names to display incorrectly
+
+## 6.0.0 (Jul 1, 2019)
+
+- Removed support for "Any" notification templates and their API endpoints e.g., /api/v2/job_templates/N/notification_templates/any/ (https://github.com/ansible/awx/issues/4022)
+- Fixed a bug which prevented credentials from properly being applied to inventory sources (https://github.com/ansible/awx/issues/4059)
+- Fixed a bug which can cause the task dispatcher to hang indefinitely when external logging support (e.g., Splunk, Logstash) is enabled (https://github.com/ansible/awx/issues/4181)
+- Fixed a bug which causes slow stdout display when running jobs against smart inventories. (https://github.com/ansible/awx/issues/3106)
+- Fixed a bug that caused SSL verification flags to fail to be respected for LDAP authentication in certain environments. (https://github.com/ansible/awx/pull/4190)
+- Added a simple Content Security Policy (https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP) to restrict access to third-party resources in the browser. (https://github.com/ansible/awx/pull/4167)
+- Updated ovirt4 library dependencies to work with newer versions of oVirt (https://github.com/ansible/awx/issues/4138)
+
+## 5.0.0 (Jun 21, 2019)
+
+- Bump Django Rest Framework from 3.7.7 to 3.9.4
+- Bump setuptools / pip dependencies
+- Fixed bug where Recent Notification list would not appear
+- Added notifications on job start
+- Default to Ansible 2.8

CONTRIBUTING.md

@@ -155,12 +155,11 @@ If you start a second terminal session, you can take a look at the running conta
 (host)$ docker ps
 
 $ docker ps
-CONTAINER ID        IMAGE                                              COMMAND                  CREATED             STATUS              PORTS                                                                                                                                      NAMES
-aa4a75d6d77b        gcr.io/ansible-tower-engineering/awx_devel:devel   "/tini -- /bin/sh ..."   23 seconds ago      Up 15 seconds       0.0.0.0:5555->5555/tcp, 0.0.0.0:7899-7999->7899-7999/tcp, 0.0.0.0:8013->8013/tcp, 0.0.0.0:8043->8043/tcp, 22/tcp, 0.0.0.0:8080->8080/tcp   tools_awx_1
-e4c0afeb548c        postgres:10                                       "docker-entrypoint..."   26 seconds ago      Up 23 seconds       5432/tcp                                                                                                                                   tools_postgres_1
-0089699d5afd        tools_logstash                                     "/docker-entrypoin..."   26 seconds ago      Up 25 seconds                                                                                                                                                  tools_logstash_1
-4d4ff0ced266        memcached:alpine                                   "docker-entrypoint..."   26 seconds ago      Up 25 seconds       0.0.0.0:11211->11211/tcp                                                                                                                   tools_memcached_1
-92842acd64cd        rabbitmq:3-management                              "docker-entrypoint..."   26 seconds ago      Up 24 seconds       4369/tcp, 5671-5672/tcp, 15671/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp                                                                    tools_rabbitmq_1
+CONTAINER ID        IMAGE                                              COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                              NAMES
+44251b476f98        gcr.io/ansible-tower-engineering/awx_devel:devel   "/entrypoint.sh /bin…"   27 seconds ago      Up 23 seconds       0.0.0.0:6899->6899/tcp, 0.0.0.0:7899-7999->7899-7999/tcp, 0.0.0.0:8013->8013/tcp, 0.0.0.0:8043->8043/tcp, 0.0.0.0:8080->8080/tcp, 22/tcp, 0.0.0.0:8888->8888/tcp   tools_awx_run_9e820694d57e
+b049a43817b4        memcached:alpine                                   "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds       0.0.0.0:11211->11211/tcp                                                                                                                                           tools_memcached_1
+40de380e3c2e        redis:latest                                       "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds       0.0.0.0:6379->6379/tcp                                                                                                                                             tools_redis_1
+b66a506d3007        postgres:10                                        "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds       0.0.0.0:5432->5432/tcp                                                                                                                                             tools_postgres_1
 ```
 **NOTE**
 
@@ -216,18 +215,23 @@ Using `docker exec`, this will create a session in the running *awx* container,
 If you want to start and use the development environment, you'll first need to bootstrap it by running the following command:
 
 ```bash
-(container)# /bootstrap_development.sh
+(container)# /usr/bin/bootstrap_development.sh
 ```
 
-The above will do all the setup tasks, including running database migrations, so it may take a couple minutes.
+The above will do all the setup tasks, including running database migrations, so it may take a couple minutes. Once it's done it
+will drop you back to the shell.
 
-Now you can start each service individually, or start all services in a pre-configured tmux session like so:
+In order to launch all developer services:
 
 ```bash
-(container)# cd /awx_devel
-(container)# make server
+(container)# /usr/bin/launch_awx.sh
 ```
 
+`launch_awx.sh` also calls `bootstrap_development.sh` so if all you are doing is launching the supervisor to start all services, you don't
+need to call `bootstrap_development.sh` first.
+
+
+
 ### Post Build Steps
 
 Before you can log in and use the system, you will need to create an admin user. Optionally, you may also want to load some demo data.

DATA_MIGRATION.md

@@ -2,96 +2,8 @@
 
 ## Introduction
 
-Upgrades using Django migrations are not expected to work in AWX.  As a result, to upgrade to a new version, it is necessary to export resources from the old AWX node and import them into a freshly-installed node with the new version.  The recommended way to do this is to use the tower-cli send/receive feature.
+Early versions of AWX did not support seamless upgrades between major versions and required the use of a backup and restore tool to perform upgrades.
 
-This tool does __not__ support export/import of the following:
-* Logs/history
-* Credential passwords
-* LDAP/AWX config
+Users who wish to upgrade modern AWX installations should follow the instructions at:
 
-### Install & Configure Tower-CLI
-
-In terminal, pip install tower-cli (if you do not have pip already, install [here](https://pip.pypa.io/en/stable/installing/)):
-```
-$ pip install --upgrade ansible-tower-cli
-```
-
-The AWX host URL, user, and password must be set for the AWX instance to be exported:
-```
-$ tower-cli config host http://<old-awx-host.example.com>
-$ tower-cli config username <user>
-$ tower-cli config password <pass>
-```
-
-For more information on installing tower-cli look [here](http://tower-cli.readthedocs.io/en/latest/quickstart.html).
-
-
-### Export Resources
-
-Export all objects
-
-```$ tower-cli receive --all > assets.json```
-
-
-
-### Teardown Old AWX
-
-Clean up remnants of the old AWX install:
-
-```docker rm -f $(docker ps -aq)```     # remove all old awx containers
-
-```make clean-ui```              # clean up ui artifacts
-
-
-### Install New AWX version
-
-If you are installing AWX as a dev container, pull down the latest code or version you want from GitHub, build
-the image locally, then start the container
-
-```
-git pull                                        # retrieve latest AWX changes from repository
-make docker-compose-build                       # build AWX image
-make docker-compose                             # run container
-```
-For other install methods, refer to the [Install.md](https://github.com/ansible/awx/blob/devel/INSTALL.md). 
- 
-
-### Import Resources
-
-
-Configure tower-cli for your new AWX host as shown earlier.  Import from a JSON file named assets.json
-
-```
-$ tower-cli config host http://<new-awx-host.example.com>
-$ tower-cli config username <user>
-$ tower-cli config password <pass>
-$ tower-cli send assets.json
-```
-
---------------------------------------------------------------------------------
-
-## Additional Info
-
-If you have two running AWX hosts, it is possible to copy all assets from one instance to another
-
-```$ tower-cli receive --tower-host old-awx-host.example.com --all | tower-cli send --tower-host new-awx-host.example.com```
-
-
-
-#### More Granular Exports:
-
-Export all credentials
-
-```$ tower-cli receive --credential all > credentials.json```
-> Note: This exports the credentials with blank strings for passwords and secrets
-
-Export a credential named "My Credential"
-
-```$ tower-cli receive --credential "My Credential"```
-
-#### More Granular Imports:
-
-
-You could import anything except an organization defined in a JSON file named assets.json
-
-```$ tower-cli send --prevent organization assets.json```
+https://github.com/ansible/awx/blob/devel/INSTALL.md#upgrading-from-previous-versions

INSTALL.md

@@ -4,41 +4,47 @@ This document provides a guide for installing AWX.
 
 ## Table of contents
 
-- [Getting started](#getting-started)
-  - [Clone the repo](#clone-the-repo)
-  - [AWX branding](#awx-branding)
-  - [Prerequisites](#prerequisites)
-  - [System Requirements](#system-requirements)
-  - [AWX Tunables](#awx-tunables)
-  - [Choose a deployment platform](#choose-a-deployment-platform)
-  - [Official vs Building Images](#official-vs-building-images)
-- [OpenShift](#openshift)
-  - [Prerequisites](#prerequisites-1)
-    - [Deploying to Minishift](#deploying-to-minishift)
-  - [Pre-build steps](#pre-build-steps)
-  - [PostgreSQL](#postgresql)
-  - [Start the build](#start-the-build)
-  - [Post build](#post-build)
-  - [Accessing AWX](#accessing-awx)
-- [Kubernetes](#kubernetes)
-  - [Prerequisites](#prerequisites-2)
-  - [Pre-build steps](#pre-build-steps-1)
-  - [Configuring Helm](#configuring-helm)
-  - [Start the build](#start-the-build-1)
-  - [Accessing AWX](#accessing-awx-1)
-  - [SSL Termination](#ssl-termination)
-- [Docker Compose](#docker-compose)
-  - [Prerequisites](#prerequisites-3)
-  - [Pre-build steps](#pre-build-steps-2)
-    - [Deploying to a remote host](#deploying-to-a-remote-host)
-    - [Inventory variables](#inventory-variables)
+- [Installing AWX](#installing-awx)
+  * [Getting started](#getting-started)
+    + [Clone the repo](#clone-the-repo)
+    + [AWX branding](#awx-branding)
+    + [Prerequisites](#prerequisites)
+    + [System Requirements](#system-requirements)
+    + [AWX Tunables](#awx-tunables)
+    + [Choose a deployment platform](#choose-a-deployment-platform)
+    + [Official vs Building Images](#official-vs-building-images)
+  * [Upgrading from previous versions](#upgrading-from-previous-versions)
+  * [OpenShift](#openshift)
+    + [Prerequisites](#prerequisites-1)
+    + [Pre-install steps](#pre-install-steps)
+      - [Deploying to Minishift](#deploying-to-minishift)
+      - [PostgreSQL](#postgresql)
+    + [Run the installer](#run-the-installer)
+    + [Post-install](#post-install)
+    + [Accessing AWX](#accessing-awx)
+  * [Kubernetes](#kubernetes)
+    + [Prerequisites](#prerequisites-2)
+    + [Pre-install steps](#pre-install-steps-1)
+    + [Configuring Helm](#configuring-helm)
+    + [Run the installer](#run-the-installer-1)
+    + [Post-install](#post-install-1)
+    + [Accessing AWX](#accessing-awx-1)
+    + [SSL Termination](#ssl-termination)
+  * [Docker-Compose](#docker-compose)
+    + [Prerequisites](#prerequisites-3)
+    + [Pre-install steps](#pre-install-steps-2)
+      - [Deploying to a remote host](#deploying-to-a-remote-host)
+      - [Inventory variables](#inventory-variables)
       - [Docker registry](#docker-registry)
-      - [PostgreSQL](#postgresql-1)
       - [Proxy settings](#proxy-settings)
-  - [Start the build](#start-the-build-2)
-  - [Post build](#post-build-2)
-  - [Accessing AWX](#accessing-awx-2)
+      - [PostgreSQL](#postgresql-1)
+    + [Run the installer](#run-the-installer-2)
+    + [Post-install](#post-install-2)
+    + [Accessing AWX](#accessing-awx-2)
+- [Installing the AWX CLI](#installing-the-awx-cli)
+  * [Building the CLI Documentation](#building-the-cli-documentation)
 
+    
 ## Getting started
 
 ### Clone the repo
@@ -57,7 +63,7 @@ To install the assets, clone the `awx-logos` repo so that it is next to your `aw
 
 Before you can run a deployment, you'll need the following installed in your local environment:
 
-- [Ansible](http://docs.ansible.com/ansible/latest/intro_installation.html) Requires Version 2.4+
+- [Ansible](http://docs.ansible.com/ansible/latest/intro_installation.html) Requires Version 2.8+
 - [Docker](https://docs.docker.com/engine/installation/)
     + A recent version
 - [docker](https://pypi.org/project/docker/) Python module
@@ -114,12 +120,33 @@ If these variables are present then all deployments will use these hosted images
 
 > Multiple versions are provided. `latest` always pulls the most recent. You may also select version numbers at different granularities: 1, 1.0, 1.0.1, 1.0.0.123
 
+
+## Upgrading from previous versions
+
+Upgrading AWX involves rerunning the install playbook. Download a newer release from [https://github.com/ansible/awx/releases](https://github.com/ansible/awx/releases) and re-populate the inventory file with your customized variables.
+
+For convenience, you can create a file called `vars.yml`:
+
+```
+admin_password: 'adminpass'
+pg_password: 'pgpass'
+secret_key: 'mysupersecret'
+```
+
+And pass it to the installer:
+
+```
+$ ansible-playbook -i inventory install.yml -e @vars.yml
+```
+
 ## OpenShift
 
 ### Prerequisites
 
 To complete a deployment to OpenShift, you will obviously need access to an OpenShift cluster. For demo and testing purposes, you can use [Minishift](https://github.com/minishift/minishift) to create a single node cluster running inside a virtual machine.
 
+When using OpenShift for deploying AWX make sure you have correct privileges to add the security context 'privileged', otherwise the installation will fail. The privileged context is needed because of the use of [the bubblewrap tool](https://github.com/containers/bubblewrap) to add an additional layer of security when using containers.
+
 You will also need to have the `oc` command in your PATH. The `install.yml` playbook will call out to `oc` when logging into, and creating objects on the cluster.
 
 The default resource requests per-deployment requires:
@@ -131,9 +158,9 @@ This can be tuned by overriding the variables found in [/installer/roles/kuberne
 
 For more detail on how resource requests are formed see: [https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-compute-resources](https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-compute-resources)
 
-### Pre-build steps
+### Pre-install steps
 
-Before starting the build process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section:
+Before starting the install, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section:
 
 *openshift_host*
 
@@ -195,20 +222,20 @@ By default, AWX will deploy a PostgreSQL pod inside of your cluster. You will ne
 
 If you wish to use an external database, in the inventory file, set the value of `pg_hostname`, and update `pg_username`, `pg_password`, `pg_admin_password`, `pg_database`, and `pg_port` with the connection information. When setting `pg_hostname` the installer will assume you have configured the database in that location and will not launch the postgresql pod.
 
-### Start the build
+### Run the installer
 
-To start the build, you will pass two *extra* variables on the command line. The first is *openshift_password*, which is the password for the *openshift_user*, and the second is *docker_registry_password*, which is the password associated with *docker_registry_username*.
+To start the install, you will pass two *extra* variables on the command line. The first is *openshift_password*, which is the password for the *openshift_user*, and the second is *docker_registry_password*, which is the password associated with *docker_registry_username*.
 
 If you're using the OpenShift internal registry, then you'll pass an access token for the *docker_registry_password* value, rather than a password. The `oc whoami -t` command will generate the required token, as long as you're logged into the cluster via `oc cluster login`.
 
-To start the build and deployment, run the following (docker_registry_password is optional if using official images):
+Run the following command (docker_registry_password is optional if using official images):
 
 ```bash
-# Start the build and deployment
+# Start the install
 $ ansible-playbook -i inventory install.yml -e openshift_password=developer  -e docker_registry_password=$(oc whoami -t)
 ```
 
-### Post build
+### Post-install
 
 After the playbook run completes, check the status of the deployment by running `oc get pods`:
 
@@ -325,9 +352,9 @@ This can be tuned by overriding the variables found in [/installer/roles/kuberne
 
 For more detail on how resource requests are formed see: [https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
 
-### Pre-build steps
+### Pre-install steps
 
-Before starting the build process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section uncommenting when necessary. Make sure the openshift and standalone docker sections are commented out:
+Before starting the install process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section uncommenting when necessary. Make sure the openshift and standalone docker sections are commented out:
 
 *kubernetes_context*
 
@@ -347,7 +374,7 @@ If you want the AWX installer to manage creating the database pod (rather than i
 
 Newer Kubernetes clusters with RBAC enabled will need to make sure a service account is created, make sure to follow the instructions here [https://docs.helm.sh/using_helm/#role-based-access-control](https://docs.helm.sh/using_helm/#role-based-access-control)
 
-### Start the build
+### Run the installer
 
 After making changes to the `inventory` file use `ansible-playbook` to begin the install
 
@@ -355,7 +382,7 @@ After making changes to the `inventory` file use `ansible-playbook` to begin the
 $ ansible-playbook -i inventory install.yml
 ```
 
-### Post build
+### Post-install
 
 After the playbook run completes, check the status of the deployment by running `kubectl get pods --namespace awx` (replace awx with the namespace you used):
 
@@ -403,7 +430,7 @@ Unlike Openshift's `Route` the Kubernetes `Ingress` doesn't yet handle SSL termi
     + This also installs the `docker` Python module, which is incompatible with `docker-py`. If you have previously installed `docker-py`, please uninstall it.
 - [Docker Compose](https://docs.docker.com/compose/install/).
 
-### Pre-build steps
+### Pre-install steps
 
 #### Deploying to a remote host
 
@@ -434,7 +461,7 @@ If you choose to use the official images then the remote host will be the one to
 
 #### Inventory variables
 
-Before starting the build process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section:
+Before starting the install process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section:
 
 *postgres_data_dir*
 
@@ -450,12 +477,16 @@ Before starting the build process, review the [inventory](./installer/inventory)
 
 *ssl_certificate*
 
-> Optionally, provide the path to a file that contains a certificate and its private key.
+> Optionally, provide the path to a file that contains a certificate and its private key. This needs to be a .pem-file
 
 *docker_compose_dir*
 
 > When using docker-compose, the `docker-compose.yml` file will be created there (default `/tmp/awxcompose`).
 
+*custom_venv_dir*
+
+> Adds the custom venv environments from the local host to be passed into the containers at install.
+
 *ca_trust_dir*
 
 > If you're using a non trusted CA, provide a path where the untrusted Certs are stored on your Host.
@@ -476,10 +507,6 @@ If you wish to tag and push built images to a Docker registry, set the following
 
 > Username of the user that will push images to the registry. Defaults to *developer*.
 
-*docker_remove_local_images*
-
-> Due to the way that the docker_image module behaves, images will not be pushed to a remote repository if they are present locally.  Set this to delete local versions of the images that will be pushed to the remote.  This will fail if containers are currently running from those images.
-
 **Note**
 
 > These settings are ignored if using official images
@@ -505,9 +532,9 @@ AWX requires access to a PostgreSQL database, and by default, one will be create
 
 If you wish to use an external database, in the inventory file, set the value of `pg_hostname`, and update `pg_username`, `pg_password`, `pg_admin_password`, `pg_database`, and `pg_port` with the connection information.
 
-### Start the build
+### Run the installer
 
-If you are not pushing images to a Docker registry, start the build by running the following:
+If you are not pushing images to a Docker registry, start the install by running the following:
 
 ```bash
 # Set the working directory to installer
@@ -527,18 +554,9 @@ $ cd installer
 $ ansible-playbook -i inventory -e docker_registry_password=password install.yml
 ```
 
-### Post build
+### Post-install
 
-After the playbook run completes, Docker will report up to 5 running containers. If you chose to use an existing PostgresSQL database, then it will report 4. You can view the running containers using the `docker ps` command, as follows:
-
-```bash
-CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                NAMES
-e240ed8209cd        awx_task:1.0.0.8    "/tini -- /bin/sh ..."   2 minutes ago       Up About a minute   8052/tcp                             awx_task
-1cfd02601690        awx_web:1.0.0.8     "/tini -- /bin/sh ..."   2 minutes ago       Up About a minute   0.0.0.0:443->8052/tcp                 awx_web
-55a552142bcd        memcached:alpine    "docker-entrypoint..."   2 minutes ago       Up 2 minutes        11211/tcp                            memcached
-84011c072aad        rabbitmq:3          "docker-entrypoint..."   2 minutes ago       Up 2 minutes        4369/tcp, 5671-5672/tcp, 25672/tcp   rabbitmq
-97e196120ab3        postgres:9.6        "docker-entrypoint..."   2 minutes ago       Up 2 minutes        5432/tcp                             postgres
-```
+After the playbook run completes, Docker starts a series of containers that provide the services that make up AWX.  You can view the running containers using the `docker ps` command.
 
 If you're deploying using Docker Compose, container names will be prefixed by the name of the folder where the docker-compose.yml file is created (by default, `awx`).
 
@@ -605,13 +623,33 @@ The AWX web server is accessible on the deployment host, using the *host_port* v
 
 You will prompted with a login dialog. The default administrator username is `admin`, and the password is `password`.
 
-### Maintenance using docker-compose
 
-After the installation, maintenance operations with docker-compose can be done by using the  `docker-compose.yml` file created at the location pointed by `docker_compose_dir`.
+# Installing the AWX CLI
 
-Among the possible operations, you may:
+`awx` is the official command-line client for AWX.  It:
 
-- Stop AWX : `docker-compose stop`
-- Upgrade AWX : `docker-compose pull && docker-compose up --force-recreate`
+* Uses naming and structure consistent with the AWX HTTP API
+* Provides consistent output formats with optional machine-parsable formats
+* To the extent possible, auto-detects API versions, available endpoints, and
+  feature support across multiple versions of AWX.
 
-See the [docker-compose documentation](https://docs.docker.com/compose/) for details.
+Potential uses include:
+
+* Configuring and launching jobs/playbooks
+* Checking on the status and output of job runs
+* Managing objects like organizations, users, teams, etc...
+
+The preferred way to install the AWX CLI is through pip directly from GitHub:
+
+    pip install "https://github.com/ansible/awx/archive/$VERSION.tar.gz#egg=awxkit&subdirectory=awxkit"
+    awx --help
+
+...where ``$VERSION`` is the version of AWX you're running.  To see a list of all available releases, visit: https://github.com/ansible/awx/releases
+
+## Building the CLI Documentation
+
+To build the docs, spin up a real AWX server, `pip install sphinx sphinxcontrib-autoprogram`, and run:
+
+    ~ TOWER_HOST=https://awx.example.org TOWER_USERNAME=example TOWER_PASSWORD=secret make clean html
+    ~ cd build/html/ && python -m http.server
+    Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ..

Makefile

@@ -18,7 +18,6 @@ COMPOSE_TAG ?= $(GIT_BRANCH)
 COMPOSE_HOST ?= $(shell hostname)
 
 VENV_BASE ?= /venv
-COLLECTION_VENV ?= /awx_devel/awx_collection_test_venv
 SCL_PREFIX ?=
 CELERY_SCHEDULE_FILE ?= /var/lib/awx/beat.db
 
@@ -26,6 +25,9 @@ DEV_DOCKER_TAG_BASE ?= gcr.io/ansible-tower-engineering
 # Python packages to install only from source (not from binary wheels)
 # Comma separated list
 SRC_ONLY_PKGS ?= cffi,pycparser,psycopg2,twilio
+# These should be upgraded in the AWX and Ansible venv before attempting
+# to install the actual requirements
+VENV_BOOTSTRAP ?= pip==19.3.1 setuptools==41.6.0
 
 # Determine appropriate shasum command
 UNAME_S := $(shell uname -s)
@@ -100,7 +102,7 @@ clean-languages:
 	find . -type f -regex ".*\.mo$$" -delete
 
 # Remove temporary build files, compiled Python files.
-clean: clean-ui clean-api clean-dist
+clean: clean-ui clean-api clean-awxkit clean-dist
 	rm -rf awx/public
 	rm -rf awx/lib/site-packages
 	rm -rf awx/job_status
@@ -116,6 +118,10 @@ clean-api:
 	find . -type d -name "__pycache__" -delete
 	rm -f awx/awx_test.sqlite3*
 	rm -rf requirements/vendor
+	rm -rf awx/projects
+
+clean-awxkit:
+	rm -rf awxkit/*.egg-info awxkit/.tox awxkit/build/*
 
 # convenience target to assert environment variables are defined
 guard-%:
@@ -126,16 +132,16 @@ guard-%:
 
 virtualenv: virtualenv_ansible virtualenv_awx
 
+# virtualenv_* targets do not use --system-site-packages to prevent bugs installing packages
+# but Ansible venvs are expected to have this, so that must be done after venv creation
 virtualenv_ansible:
 	if [ "$(VENV_BASE)" ]; then \
 		if [ ! -d "$(VENV_BASE)" ]; then \
 			mkdir $(VENV_BASE); \
 		fi; \
 		if [ ! -d "$(VENV_BASE)/ansible" ]; then \
-			virtualenv -p python --system-site-packages $(VENV_BASE)/ansible && \
-			$(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --ignore-installed six packaging appdirs && \
-			$(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --ignore-installed setuptools==36.0.1 && \
-			$(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --ignore-installed pip==9.0.1; \
+			virtualenv -p python $(VENV_BASE)/ansible && \
+			$(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) $(VENV_BOOTSTRAP); \
 		fi; \
 	fi
 
@@ -145,36 +151,45 @@ virtualenv_ansible_py3:
 			mkdir $(VENV_BASE); \
 		fi; \
 		if [ ! -d "$(VENV_BASE)/ansible" ]; then \
-			$(PYTHON) -m venv --system-site-packages $(VENV_BASE)/ansible; \
+			virtualenv -p $(PYTHON) $(VENV_BASE)/ansible; \
+			$(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) $(VENV_BOOTSTRAP); \
 		fi; \
 	fi
 
+# flit is needed for offline install of certain packages, specifically ptyprocess
+# it is needed for setup, but not always recognized as a setup dependency
+# similar to pip, setuptools, and wheel, these are all needed here as a bootstrapping issues
 virtualenv_awx:
 	if [ "$(VENV_BASE)" ]; then \
 		if [ ! -d "$(VENV_BASE)" ]; then \
 			mkdir $(VENV_BASE); \
 		fi; \
 		if [ ! -d "$(VENV_BASE)/awx" ]; then \
-			$(PYTHON) -m venv --system-site-packages $(VENV_BASE)/awx; \
-			$(VENV_BASE)/awx/bin/pip install $(PIP_OPTIONS) --ignore-installed docutils==0.14; \
+			virtualenv -p $(PYTHON) $(VENV_BASE)/awx; \
+			$(VENV_BASE)/awx/bin/pip install $(PIP_OPTIONS) $(VENV_BOOTSTRAP); \
 		fi; \
 	fi
 
+# --ignore-install flag is not used because *.txt files should specify exact versions
 requirements_ansible: virtualenv_ansible
 	if [[ "$(PIP_OPTIONS)" == *"--no-index"* ]]; then \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --ignore-installed -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) -r /dev/stdin ; \
 	else \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) --ignore-installed -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) -r /dev/stdin ; \
 	fi
 	$(VENV_BASE)/ansible/bin/pip uninstall --yes -r requirements/requirements_ansible_uninstall.txt
+	# Same effect as using --system-site-packages flag on venv creation
+	rm $(shell ls -d $(VENV_BASE)/ansible/lib/python* | head -n 1)/no-global-site-packages.txt
 
 requirements_ansible_py3: virtualenv_ansible_py3
 	if [[ "$(PIP_OPTIONS)" == *"--no-index"* ]]; then \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) --ignore-installed -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) -r /dev/stdin ; \
 	else \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) --ignore-installed -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) -r /dev/stdin ; \
 	fi
 	$(VENV_BASE)/ansible/bin/pip3 uninstall --yes -r requirements/requirements_ansible_uninstall.txt
+	# Same effect as using --system-site-packages flag on venv creation
+	rm $(shell ls -d $(VENV_BASE)/ansible/lib/python* | head -n 1)/no-global-site-packages.txt
 
 requirements_ansible_dev:
 	if [ "$(VENV_BASE)" ]; then \
@@ -182,13 +197,13 @@ requirements_ansible_dev:
 	fi
 
 # Install third-party requirements needed for AWX's environment.
+# this does not use system site packages intentionally
 requirements_awx: virtualenv_awx
 	if [[ "$(PIP_OPTIONS)" == *"--no-index"* ]]; then \
-	    cat requirements/requirements.txt requirements/requirements_local.txt | $(VENV_BASE)/awx/bin/pip install $(PIP_OPTIONS) --ignore-installed -r /dev/stdin ; \
+	    cat requirements/requirements.txt requirements/requirements_local.txt | $(VENV_BASE)/awx/bin/pip install $(PIP_OPTIONS) -r /dev/stdin ; \
 	else \
-	    cat requirements/requirements.txt requirements/requirements_git.txt | $(VENV_BASE)/awx/bin/pip install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) --ignore-installed -r /dev/stdin ; \
+	    cat requirements/requirements.txt requirements/requirements_git.txt | $(VENV_BASE)/awx/bin/pip install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) -r /dev/stdin ; \
 	fi
-	echo "include-system-site-packages = true" >> $(VENV_BASE)/awx/lib/python$(PYTHON_VERSION)/pyvenv.cfg
 	$(VENV_BASE)/awx/bin/pip uninstall --yes -r requirements/requirements_tower_uninstall.txt
 
 requirements_awx_dev:
@@ -196,7 +211,7 @@ requirements_awx_dev:
 
 requirements: requirements_ansible requirements_awx
 
-requirements_dev: requirements requirements_awx_dev requirements_ansible_dev
+requirements_dev: requirements_awx requirements_ansible_py3 requirements_awx_dev requirements_ansible_dev
 
 requirements_test: requirements
 
@@ -249,28 +264,6 @@ migrate:
 dbchange:
 	$(MANAGEMENT_COMMAND) makemigrations
 
-server_noattach:
-	tmux new-session -d -s awx 'exec make uwsgi'
-	tmux rename-window 'AWX'
-	tmux select-window -t awx:0
-	tmux split-window -v 'exec make dispatcher'
-	tmux new-window 'exec make daphne'
-	tmux select-window -t awx:1
-	tmux rename-window 'WebSockets'
-	tmux split-window -h 'exec make runworker'
-	tmux split-window -v 'exec make nginx'
-	tmux new-window 'exec make receiver'
-	tmux select-window -t awx:2
-	tmux rename-window 'Extra Services'
-	tmux select-window -t awx:0
-
-server: server_noattach
-	tmux -2 attach-session -t awx
-
-# Use with iterm2's native tmux protocol support
-servercc: server_noattach
-	tmux -2 -CC attach-session -t awx
-
 supervisor:
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
@@ -295,18 +288,11 @@ daphne:
 	fi; \
 	daphne -b 127.0.0.1 -p 8051 awx.asgi:channel_layer
 
-runworker:
+wsbroadcast:
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
-	$(PYTHON) manage.py runworker --only-channels websocket.*
-
-# Run the built-in development webserver (by default on http://localhost:8013).
-runserver:
-	@if [ "$(VENV_BASE)" ]; then \
-		. $(VENV_BASE)/awx/bin/activate; \
-	fi; \
-	$(PYTHON) manage.py runserver
+	$(PYTHON) manage.py run_wsbroadcast
 
 # Run to start the background task dispatcher for development.
 dispatcher:
@@ -364,7 +350,7 @@ check: flake8 pep8 # pyflakes pylint
 
 awx-link:
 	cp -R /tmp/awx.egg-info /awx_devel/ || true
-	sed -i "s/placeholder/$(shell git describe --long | sed 's/\./\\./g')/" /awx_devel/awx.egg-info/PKG-INFO
+	sed -i "s/placeholder/$(shell cat VERSION)/" /awx_devel/awx.egg-info/PKG-INFO
 	cp -f /tmp/awx.egg-link /venv/awx/lib/python$(PYTHON_VERSION)/site-packages/awx.egg-link
 
 TEST_DIRS ?= awx/main/tests/unit awx/main/tests/functional awx/conf/tests awx/sso/tests
@@ -378,30 +364,43 @@ test:
 	cd awxkit && $(VENV_BASE)/awx/bin/tox -re py2,py3
 	awx-manage check_migrations --dry-run --check  -n 'vNNN_missing_migration_file'
 
-prepare_collection_venv:
-	rm -rf $(COLLECTION_VENV)
-	mkdir $(COLLECTION_VENV)
-	ln -s /usr/lib/python2.7/site-packages/ansible $(COLLECTION_VENV)/ansible
-	$(VENV_BASE)/awx/bin/pip install --target=$(COLLECTION_VENV) git+https://github.com/ansible/tower-cli.git
-
 COLLECTION_TEST_DIRS ?= awx_collection/test/awx
+COLLECTION_TEST_TARGET ?=
 COLLECTION_PACKAGE ?= awx
 COLLECTION_NAMESPACE ?= awx
+COLLECTION_INSTALL = ~/.ansible/collections/ansible_collections/$(COLLECTION_NAMESPACE)/$(COLLECTION_PACKAGE)
 
 test_collection:
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
-	PYTHONPATH=$(COLLECTION_VENV):/awx_devel/awx_collection:$PYTHONPATH py.test $(COLLECTION_TEST_DIRS)
+	PYTHONPATH=$PYTHONPATH:/usr/lib/python3.6/site-packages py.test $(COLLECTION_TEST_DIRS)
 
 flake8_collection:
 	flake8 awx_collection/  # Different settings, in main exclude list
 
-test_collection_all: prepare_collection_venv test_collection flake8_collection
+test_collection_all: test_collection flake8_collection
+
+# WARNING: symlinking a collection is fundamentally unstable
+# this is for rapid development iteration with playbooks, do not use with other test targets
+symlink_collection:
+	rm -rf $(COLLECTION_INSTALL)
+	mkdir -p ~/.ansible/collections/ansible_collections/$(COLLECTION_NAMESPACE)  # in case it does not exist
+	ln -s $(shell pwd)/awx_collection $(COLLECTION_INSTALL)
 
 build_collection:
 	ansible-playbook -i localhost, awx_collection/template_galaxy.yml -e collection_package=$(COLLECTION_PACKAGE) -e collection_namespace=$(COLLECTION_NAMESPACE) -e collection_version=$(VERSION)
-	ansible-galaxy collection build awx_collection --output-path=awx_collection
+	ansible-galaxy collection build awx_collection --force --output-path=awx_collection
+
+install_collection: build_collection
+	rm -rf $(COLLECTION_INSTALL)
+	ansible-galaxy collection install awx_collection/$(COLLECTION_NAMESPACE)-$(COLLECTION_PACKAGE)-$(VERSION).tar.gz
+
+test_collection_sanity: install_collection
+	cd $(COLLECTION_INSTALL) && ansible-test sanity
+
+test_collection_integration: install_collection
+	cd $(COLLECTION_INSTALL) && ansible-test integration $(COLLECTION_TEST_TARGET)
 
 test_unit:
 	@if [ "$(VENV_BASE)" ]; then \
@@ -609,28 +608,34 @@ docker-auth:
 		echo "$(IMAGE_REPOSITORY_AUTH)" | docker login -u oauth2accesstoken --password-stdin $(IMAGE_REPOSITORY_BASE); \
 	fi;
 
+# This directory is bind-mounted inside of the development container and
+# needs to be pre-created for permissions to be set correctly. Otherwise,
+# Docker will create this directory as root.
+awx/projects:
+	@mkdir -p $@
+
 # Docker isolated rampart
-docker-compose-isolated:
+docker-compose-isolated: awx/projects
 	CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/docker-isolated-override.yml up
 
 # Docker Compose Development environment
-docker-compose: docker-auth
+docker-compose: docker-auth awx/projects
 	CURRENT_UID=$(shell id -u) OS="$(shell docker info | grep 'Operating System')" TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml up --no-recreate awx
 
-docker-compose-cluster: docker-auth
+docker-compose-cluster: docker-auth awx/projects
 	CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose-cluster.yml up
 
-docker-compose-credential-plugins: docker-auth
+docker-compose-credential-plugins: docker-auth awx/projects
 	echo -e "\033[0;31mTo generate a CyberArk Conjur API key: docker exec -it tools_conjur_1 conjurctl account create quick-start\033[0m"
 	CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/docker-credential-plugins-override.yml up --no-recreate awx
 
-docker-compose-test: docker-auth
+docker-compose-test: docker-auth awx/projects
 	cd tools && CURRENT_UID=$(shell id -u) OS="$(shell docker info | grep 'Operating System')" TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /bin/bash
 
-docker-compose-runtest:
+docker-compose-runtest: awx/projects
 	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /start_tests.sh
 
-docker-compose-build-swagger:
+docker-compose-build-swagger: awx/projects
 	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /start_tests.sh swagger
 
 detect-schema-change: genschema
@@ -638,7 +643,7 @@ detect-schema-change: genschema
 	# Ignore differences in whitespace with -b
 	diff -u -b reference-schema.json schema.json
 
-docker-compose-clean:
+docker-compose-clean: awx/projects
 	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm -w /awx_devel --service-ports awx make clean
 	cd tools && TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose rm -sf
 
@@ -647,7 +652,6 @@ docker-compose-build: awx-devel-build
 # Base development image build
 awx-devel-build:
 	docker build -t ansible/awx_devel -f tools/docker-compose/Dockerfile \
-		--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:devel \
 		--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) .
 	docker tag ansible/awx_devel $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
 	#docker push $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
@@ -658,27 +662,25 @@ docker-compose-isolated-build: awx-devel-build
 	docker tag ansible/awx_isolated $(DEV_DOCKER_TAG_BASE)/awx_isolated:$(COMPOSE_TAG)
 	#docker push $(DEV_DOCKER_TAG_BASE)/awx_isolated:$(COMPOSE_TAG)
 
-MACHINE?=default
 docker-clean:
-	eval $$(docker-machine env $(MACHINE))
 	$(foreach container_id,$(shell docker ps -f name=tools_awx -aq),docker stop $(container_id); docker rm -f $(container_id);)
-	-docker images | grep "awx_devel" | awk '{print $$1 ":" $$2}' | xargs docker rmi
+	docker images | grep "awx_devel" | awk '{print $$1 ":" $$2}' | xargs docker rmi
+
+docker-clean-volumes:
+	docker volume rm tools_awx_db
 
 docker-refresh: docker-clean docker-compose
 
 # Docker Development Environment with Elastic Stack Connected
-docker-compose-elk: docker-auth
+docker-compose-elk: docker-auth awx/projects
 	CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/elastic/docker-compose.logstash-link.yml -f tools/elastic/docker-compose.elastic-override.yml up --no-recreate
 
-docker-compose-cluster-elk: docker-auth
+docker-compose-cluster-elk: docker-auth awx/projects
 	TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose-cluster.yml -f tools/elastic/docker-compose.logstash-link-cluster.yml -f tools/elastic/docker-compose.elastic-override.yml up --no-recreate
 
 prometheus:
 	docker run -u0 --net=tools_default --link=`docker ps | egrep -o "tools_awx(_run)?_([^ ]+)?"`:awxweb --volume `pwd`/tools/prometheus:/prometheus --name prometheus -d -p 0.0.0.0:9090:9090 prom/prometheus --web.enable-lifecycle --config.file=/prometheus/prometheus.yml
 
-minishift-dev:
-	ansible-playbook -i localhost, -e devtree_directory=$(CURDIR) tools/clusterdevel/start_minishift_dev.yml
-
 clean-elk:
 	docker stop tools_kibana_1
 	docker stop tools_logstash_1

VERSION

@@ -1 +1 @@
-8.0.0
+11.0.0

awx/__init__.py

@@ -24,31 +24,18 @@ except ImportError: # pragma: no cover
 import hashlib
 
 try:
-    import django
-    from django.db.backends.base import schema
-    from django.db.backends.utils import names_digest
+    import django  # noqa: F401
     HAS_DJANGO = True
 except ImportError:
     HAS_DJANGO = False
+else:
+    from django.db.backends.base import schema
+    from django.db.backends.utils import names_digest
 
 
 if HAS_DJANGO is True:
-    # This line exists to make sure we don't regress on FIPS support if we
-    # upgrade Django; if you're upgrading Django and see this error,
-    # update the version check below, and confirm that FIPS still works.
-    # If operating in a FIPS environment, `hashlib.md5()` will raise a `ValueError`,
-    # but will support the `usedforsecurity` keyword on RHEL and Centos systems.
-
-    # Keep an eye on https://code.djangoproject.com/ticket/28401
-    target_version = '2.2.4'
-    if django.__version__ != target_version:
-        raise RuntimeError(
-            "Django version other than {target} detected: {current}. "
-            "Overriding `names_digest` is known to work for Django {target} "
-            "and may not work in other Django versions.".format(target=target_version,
-                                                                current=django.__version__)
-        )
 
+    # See upgrade blocker note in requirements/README.md
     try:
         names_digest('foo', 'bar', 'baz', length=8)
     except ValueError:
@@ -86,7 +73,14 @@ def oauth2_getattribute(self, attr):
     # Custom method to override
     # oauth2_provider.settings.OAuth2ProviderSettings.__getattribute__
     from django.conf import settings
-    val = settings.OAUTH2_PROVIDER.get(attr)
+    val = None
+    if 'migrate' not in sys.argv:
+        # certain Django OAuth Toolkit migrations actually reference
+        # setting lookups for references to model classes (e.g.,
+        # oauth2_settings.REFRESH_TOKEN_MODEL)
+        # If we're doing an OAuth2 setting lookup *while running* a migration,
+        # don't do our usual "Configure Tower in Tower" database setting lookup
+        val = settings.OAUTH2_PROVIDER.get(attr)
     if val is None:
         val = object.__getattribute__(self, attr)
     return val

awx/api/conf.py

@@ -62,3 +62,15 @@ register(
     category=_('Authentication'),
     category_slug='authentication',
 )
+register(
+    'LOGIN_REDIRECT_OVERRIDE',
+    field_class=fields.CharField,
+    allow_blank=True,
+    required=False,
+    default='',
+    label=_('Login redirect override URL'),
+    help_text=_('URL to which unauthorized users will be redirected to log in. '
+                'If blank, users will be sent to the Tower login page.'),
+    category=_('Authentication'),
+    category_slug='authentication',
+)

awx/api/filters.py

@@ -9,7 +9,7 @@ from functools import reduce
 # Django
 from django.core.exceptions import FieldError, ValidationError
 from django.db import models
-from django.db.models import Q
+from django.db.models import Q, CharField, IntegerField, BooleanField
 from django.db.models.fields import FieldDoesNotExist
 from django.db.models.fields.related import ForeignObjectRel, ManyToManyField, ForeignKey
 from django.contrib.contenttypes.models import ContentType
@@ -63,19 +63,19 @@ class TypeFilterBackend(BaseFilterBackend):
             raise ParseError(*e.args)
 
 
-def get_field_from_path(model, path):
+def get_fields_from_path(model, path):
     '''
     Given a Django ORM lookup path (possibly over multiple models)
-    Returns the last field in the line, and also the revised lookup path
+    Returns the fields in the line, and also the revised lookup path
     ex., given
         model=Organization
         path='project__timeout'
-    returns tuple of field at the end of the line as well as a corrected
-    path, for special cases we do substitutions
-        (<IntegerField for timeout>, 'project__timeout')
+    returns tuple of fields traversed as well and a corrected path,
+    for special cases we do substitutions
+        ([<IntegerField for timeout>], 'project__timeout')
     '''
     # Store of all the fields used to detect repeats
-    field_set = set([])
+    field_list = []
     new_parts = []
     for name in path.split('__'):
         if model is None:
@@ -111,13 +111,24 @@ def get_field_from_path(model, path):
                 raise PermissionDenied(_('Filtering on %s is not allowed.' % name))
             elif getattr(field, '__prevent_search__', False):
                 raise PermissionDenied(_('Filtering on %s is not allowed.' % name))
-        if field in field_set:
+        if field in field_list:
             # Field traversed twice, could create infinite JOINs, DoSing Tower
             raise ParseError(_('Loops not allowed in filters, detected on field {}.').format(field.name))
-        field_set.add(field)
+        field_list.append(field)
         model = getattr(field, 'related_model', None)
 
-    return field, '__'.join(new_parts)
+    return field_list, '__'.join(new_parts)
+
+
+def get_field_from_path(model, path):
+    '''
+    Given a Django ORM lookup path (possibly over multiple models)
+    Returns the last field in the line, and the revised lookup path
+    ex.
+        (<IntegerField for timeout>, 'project__timeout')
+    '''
+    field_list, new_path = get_fields_from_path(model, path)
+    return (field_list[-1], new_path)
 
 
 class FieldLookupBackend(BaseFilterBackend):
@@ -133,7 +144,11 @@ class FieldLookupBackend(BaseFilterBackend):
                          'regex', 'iregex', 'gt', 'gte', 'lt', 'lte', 'in',
                          'isnull', 'search')
 
-    def get_field_from_lookup(self, model, lookup):
+    # A list of fields that we know can be filtered on without the possiblity
+    # of introducing duplicates
+    NO_DUPLICATES_WHITELIST = (CharField, IntegerField, BooleanField)
+
+    def get_fields_from_lookup(self, model, lookup):
 
         if '__' in lookup and lookup.rsplit('__', 1)[-1] in self.SUPPORTED_LOOKUPS:
             path, suffix = lookup.rsplit('__', 1)
@@ -147,11 +162,16 @@ class FieldLookupBackend(BaseFilterBackend):
         # FIXME: Could build up a list of models used across relationships, use
         # those lookups combined with request.user.get_queryset(Model) to make
         # sure user cannot query using objects he could not view.
-        field, new_path = get_field_from_path(model, path)
+        field_list, new_path = get_fields_from_path(model, path)
 
         new_lookup = new_path
         new_lookup = '__'.join([new_path, suffix])
-        return field, new_lookup
+        return field_list, new_lookup
+
+    def get_field_from_lookup(self, model, lookup):
+        '''Method to match return type of single field, if needed.'''
+        field_list, new_lookup = self.get_fields_from_lookup(model, lookup)
+        return (field_list[-1], new_lookup)
 
     def to_python_related(self, value):
         value = force_text(value)
@@ -182,7 +202,10 @@ class FieldLookupBackend(BaseFilterBackend):
         except UnicodeEncodeError:
             raise ValueError("%r is not an allowed field name. Must be ascii encodable." % lookup)
 
-        field, new_lookup = self.get_field_from_lookup(model, lookup)
+        field_list, new_lookup = self.get_fields_from_lookup(model, lookup)
+        field = field_list[-1]
+
+        needs_distinct = (not all(isinstance(f, self.NO_DUPLICATES_WHITELIST) for f in field_list))
 
         # Type names are stored without underscores internally, but are presented and
         # and serialized over the API containing underscores so we remove `_`
@@ -211,10 +234,10 @@ class FieldLookupBackend(BaseFilterBackend):
             for rm_field in related_model._meta.fields:
                 if rm_field.name in ('username', 'first_name', 'last_name', 'email', 'name', 'description', 'playbook'):
                     new_lookups.append('{}__{}__icontains'.format(new_lookup[:-8], rm_field.name))
-            return value, new_lookups
+            return value, new_lookups, needs_distinct
         else:
             value = self.value_to_python_for_field(field, value)
-        return value, new_lookup
+        return value, new_lookup, needs_distinct
 
     def filter_queryset(self, request, queryset, view):
         try:
@@ -225,6 +248,7 @@ class FieldLookupBackend(BaseFilterBackend):
             chain_filters = []
             role_filters = []
             search_filters = {}
+            needs_distinct = False
             # Can only have two values: 'AND', 'OR'
             # If 'AND' is used, an iterm must satisfy all condition to show up in the results.
             # If 'OR' is used, an item just need to satisfy one condition to appear in results.
@@ -256,9 +280,12 @@ class FieldLookupBackend(BaseFilterBackend):
                         search_filter_relation = 'AND'
                         values = reduce(lambda list1, list2: list1 + list2, [i.split(',') for i in values])
                     for value in values:
-                        search_value, new_keys = self.value_to_python(queryset.model, key, force_text(value))
+                        search_value, new_keys, _ = self.value_to_python(queryset.model, key, force_text(value))
                         assert isinstance(new_keys, list)
                         search_filters[search_value] = new_keys
+                    # by definition, search *only* joins across relations,
+                    # so it _always_ needs a .distinct()
+                    needs_distinct = True
                     continue
 
                 # Custom chain__ and or__ filters, mutually exclusive (both can
@@ -282,7 +309,9 @@ class FieldLookupBackend(BaseFilterBackend):
                 for value in values:
                     if q_int:
                         value = int(value)
-                    value, new_key = self.value_to_python(queryset.model, key, value)
+                    value, new_key, distinct = self.value_to_python(queryset.model, key, value)
+                    if distinct:
+                        needs_distinct = True
                     if q_chain:
                         chain_filters.append((q_not, new_key, value))
                     elif q_or:
@@ -332,7 +361,9 @@ class FieldLookupBackend(BaseFilterBackend):
                     else:
                         q = Q(**{k:v})
                     queryset = queryset.filter(q)
-                queryset = queryset.filter(*args).distinct()
+                queryset = queryset.filter(*args)
+                if needs_distinct:
+                    queryset = queryset.distinct()
             return queryset
         except (FieldError, FieldDoesNotExist, ValueError, TypeError) as e:
             raise ParseError(e.args[0])

awx/api/generics.py

@@ -5,10 +5,12 @@
 import inspect
 import logging
 import time
+import uuid
 import urllib.parse
 
 # Django
 from django.conf import settings
+from django.core.cache import cache
 from django.db import connection
 from django.db.models.fields import FieldDoesNotExist
 from django.db.models.fields.related import OneToOneRel
@@ -192,7 +194,7 @@ class APIView(views.APIView):
                 response.data['detail'] += ' To establish a login session, visit /api/login/.'
                 logger.info(status_msg)
             else:
-                logger.warn(status_msg)
+                logger.warning(status_msg)
         response = super(APIView, self).finalize_response(request, response, *args, **kwargs)
         time_started = getattr(self, 'time_started', None)
         response['X-API-Node'] = settings.CLUSTER_HOST_ID
@@ -548,6 +550,15 @@ class SubListCreateAPIView(SubListAPIView, ListCreateAPIView):
         })
         return d
 
+    def get_queryset(self):
+        if hasattr(self, 'parent_key'):
+            # Prefer this filtering because ForeignKey allows us more assumptions
+            parent = self.get_parent_object()
+            self.check_parent_access(parent)
+            qs = self.request.user.get_queryset(self.model)
+            return qs.filter(**{self.parent_key: parent})
+        return super(SubListCreateAPIView, self).get_queryset()
+
     def create(self, request, *args, **kwargs):
         # If the object ID was not specified, it probably doesn't exist in the
         # DB yet. We want to see if we can create it.  The URL may choose to
@@ -574,7 +585,7 @@ class SubListCreateAPIView(SubListAPIView, ListCreateAPIView):
                             status=status.HTTP_400_BAD_REQUEST)
 
         # Verify we have permission to add the object as given.
-        if not request.user.can_access(self.model, 'add', serializer.initial_data):
+        if not request.user.can_access(self.model, 'add', serializer.validated_data):
             raise PermissionDenied()
 
         # save the object through the serializer, reload and returned the saved
@@ -964,6 +975,11 @@ class CopyAPIView(GenericAPIView):
         if hasattr(new_obj, 'admin_role') and request.user not in new_obj.admin_role.members.all():
             new_obj.admin_role.members.add(request.user)
         if sub_objs:
+            # store the copied object dict into memcached, because it's
+            # often too large for postgres' notification bus
+            # (which has a default maximum message size of 8k)
+            key = 'deep-copy-{}'.format(str(uuid.uuid4()))
+            cache.set(key, sub_objs, timeout=3600)
             permission_check_func = None
             if hasattr(type(self), 'deep_copy_permission_check_func'):
                 permission_check_func = (
@@ -971,7 +987,7 @@ class CopyAPIView(GenericAPIView):
                 )
             trigger_delayed_deep_copy(
                 self.model.__module__, self.model.__name__,
-                obj.pk, new_obj.pk, request.user.pk, sub_objs,
+                obj.pk, new_obj.pk, request.user.pk, key,
                 permission_check_func=permission_check_func
             )
         serializer = self._get_copy_return_serializer(new_obj)

awx/api/metadata.py

@@ -2,6 +2,7 @@
 # All Rights Reserved.
 
 from collections import OrderedDict
+from uuid import UUID
 
 # Django
 from django.core.exceptions import PermissionDenied
@@ -20,6 +21,7 @@ from rest_framework.fields import JSONField as DRFJSONField
 from rest_framework.request import clone_request
 
 # AWX
+from awx.api.fields import ChoiceNullField
 from awx.main.fields import JSONField, ImplicitRoleField
 from awx.main.models import InventorySource, NotificationTemplate
 from awx.main.scheduler.kubernetes import PodManager
@@ -59,7 +61,8 @@ class Metadata(metadata.SimpleMetadata):
                 'type': _('Data type for this {}.'),
                 'url': _('URL for this {}.'),
                 'related': _('Data structure with URLs of related resources.'),
-                'summary_fields': _('Data structure with name/description for related resources.'),
+                'summary_fields': _('Data structure with name/description for related resources.  '
+                                    'The output for some objects may be limited for performance reasons.'),
                 'created': _('Timestamp when this {} was created.'),
                 'modified': _('Timestamp when this {} was last modified.'),
             }
@@ -84,6 +87,8 @@ class Metadata(metadata.SimpleMetadata):
         # FIXME: Still isn't showing all default values?
         try:
             default = field.get_default()
+            if type(default) is UUID:
+                default = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
             if field.field_name == 'TOWER_URL_BASE' and default == 'https://towerhost':
                 default = '{}://{}'.format(self.request.scheme, self.request.get_host())
             field_info['default'] = default
@@ -96,7 +101,15 @@ class Metadata(metadata.SimpleMetadata):
             field_info['children'] = self.get_serializer_info(field)
 
         if not isinstance(field, (RelatedField, ManyRelatedField)) and hasattr(field, 'choices'):
-            field_info['choices'] = [(choice_value, choice_name) for choice_value, choice_name in field.choices.items()]
+            choices = [
+                (choice_value, choice_name) for choice_value, choice_name in field.choices.items()
+            ]
+            if not any(choice in ('', None) for choice, _ in choices):
+                if field.allow_blank:
+                    choices = [("", "---------")] + choices
+                if field.allow_null and not isinstance(field, ChoiceNullField):
+                    choices = [(None, "---------")] + choices
+            field_info['choices'] = choices
 
         # Indicate if a field is write-only.
         if getattr(field, 'write_only', False):
@@ -158,9 +171,16 @@ class Metadata(metadata.SimpleMetadata):
             isinstance(field, JSONField) or
             isinstance(model_field, JSONField) or
             isinstance(field, DRFJSONField) or
-            isinstance(getattr(field, 'model_field', None), JSONField)
+            isinstance(getattr(field, 'model_field', None), JSONField) or
+            field.field_name == 'credential_passwords'
         ):
             field_info['type'] = 'json'
+        elif (
+            isinstance(field, ManyRelatedField) and
+            field.field_name == 'credentials'
+            # launch-time credentials
+        ):
+            field_info['type'] = 'list_of_ids'
         elif isinstance(model_field, BooleanField):
             field_info['type'] = 'boolean'
 

awx/api/serializers.py

@@ -72,6 +72,7 @@ from awx.main.utils import (
     prefetch_page_capabilities, get_external_account, truncate_stdout,
 )
 from awx.main.utils.filters import SmartFilter
+from awx.main.utils.named_url_graph import reset_counters
 from awx.main.redact import UriCleaner, REPLACE_STR
 
 from awx.main.validators import vars_validate_or_raise
@@ -98,26 +99,19 @@ SUMMARIZABLE_FK_FIELDS = {
                                            'total_hosts',
                                            'hosts_with_active_failures',
                                            'total_groups',
-                                           'groups_with_active_failures',
                                            'has_inventory_sources',
                                            'total_inventory_sources',
                                            'inventory_sources_with_failures',
                                            'organization_id',
                                            'kind',
                                            'insights_credential_id',),
-    'host': DEFAULT_SUMMARY_FIELDS + ('has_active_failures',
-                                      'has_inventory_sources'),
-    'group': DEFAULT_SUMMARY_FIELDS + ('has_active_failures',
-                                       'total_hosts',
-                                       'hosts_with_active_failures',
-                                       'total_groups',
-                                       'groups_with_active_failures',
-                                       'has_inventory_sources'),
+    'host': DEFAULT_SUMMARY_FIELDS,
+    'group': DEFAULT_SUMMARY_FIELDS,
     'project': DEFAULT_SUMMARY_FIELDS + ('status', 'scm_type'),
     'source_project': DEFAULT_SUMMARY_FIELDS + ('status', 'scm_type'),
     'project_update': DEFAULT_SUMMARY_FIELDS + ('status', 'failed',),
     'credential': DEFAULT_SUMMARY_FIELDS + ('kind', 'cloud', 'kubernetes', 'credential_type_id'),
-    'job': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'elapsed', 'type'),
+    'job': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'elapsed', 'type', 'canceled_on'),
     'job_template': DEFAULT_SUMMARY_FIELDS,
     'workflow_job_template': DEFAULT_SUMMARY_FIELDS,
     'workflow_job': DEFAULT_SUMMARY_FIELDS,
@@ -125,7 +119,7 @@ SUMMARIZABLE_FK_FIELDS = {
     'workflow_approval': DEFAULT_SUMMARY_FIELDS + ('timeout',),
     'schedule': DEFAULT_SUMMARY_FIELDS + ('next_run',),
     'unified_job_template': DEFAULT_SUMMARY_FIELDS + ('unified_job_type',),
-    'last_job': DEFAULT_SUMMARY_FIELDS + ('finished', 'status', 'failed', 'license_error'),
+    'last_job': DEFAULT_SUMMARY_FIELDS + ('finished', 'status', 'failed', 'license_error', 'canceled_on'),
     'last_job_host_summary': DEFAULT_SUMMARY_FIELDS + ('failed',),
     'last_update': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'),
     'current_update': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'),
@@ -139,8 +133,9 @@ SUMMARIZABLE_FK_FIELDS = {
     'insights_credential': DEFAULT_SUMMARY_FIELDS,
     'source_credential': DEFAULT_SUMMARY_FIELDS + ('kind', 'cloud', 'credential_type_id'),
     'target_credential': DEFAULT_SUMMARY_FIELDS + ('kind', 'cloud', 'credential_type_id'),
-    'webhook_credential': DEFAULT_SUMMARY_FIELDS,
+    'webhook_credential': DEFAULT_SUMMARY_FIELDS + ('kind', 'cloud', 'credential_type_id'),
     'approved_or_denied_by': ('id', 'username', 'first_name', 'last_name'),
+    'credential_type': DEFAULT_SUMMARY_FIELDS,
 }
 
 
@@ -353,6 +348,7 @@ class BaseSerializer(serializers.ModelSerializer, metaclass=BaseSerializerMetacl
 
     def _generate_named_url(self, url_path, obj, node):
         url_units = url_path.split('/')
+        reset_counters()
         named_url = node.generate_named_url(obj)
         url_units[4] = named_url
         return '/'.join(url_units)
@@ -648,7 +644,7 @@ class UnifiedJobTemplateSerializer(BaseSerializer):
     _capabilities_prefetch = [
         'admin', 'execute',
         {'copy': ['jobtemplate.project.use', 'jobtemplate.inventory.use',
-                  'workflowjobtemplate.organization.workflow_admin']}
+                  'organization.workflow_admin']}
     ]
 
     class Meta:
@@ -718,7 +714,7 @@ class UnifiedJobSerializer(BaseSerializer):
     class Meta:
         model = UnifiedJob
         fields = ('*', 'unified_job_template', 'launch_type', 'status',
-                  'failed', 'started', 'finished', 'elapsed', 'job_args',
+                  'failed', 'started', 'finished', 'canceled_on', 'elapsed', 'job_args',
                   'job_cwd', 'job_env', 'job_explanation',
                   'execution_node', 'controller_node',
                   'result_traceback', 'event_processing_finished')
@@ -890,6 +886,9 @@ class UserSerializer(BaseSerializer):
         fields = ('*', '-name', '-description', '-modified',
                   'username', 'first_name', 'last_name',
                   'email', 'is_superuser', 'is_system_auditor', 'password', 'ldap_dn', 'last_login', 'external_account')
+        extra_kwargs = {
+            'last_login': {'read_only': True}
+        }
 
     def to_representation(self, obj):
         ret = super(UserSerializer, self).to_representation(obj)
@@ -1252,6 +1251,7 @@ class OrganizationSerializer(BaseSerializer):
         res.update(dict(
             projects    = self.reverse('api:organization_projects_list',       kwargs={'pk': obj.pk}),
             inventories = self.reverse('api:organization_inventories_list',    kwargs={'pk': obj.pk}),
+            job_templates = self.reverse('api:organization_job_templates_list', kwargs={'pk': obj.pk}),
             workflow_job_templates = self.reverse('api:organization_workflow_job_templates_list', kwargs={'pk': obj.pk}),
             users       = self.reverse('api:organization_users_list',          kwargs={'pk': obj.pk}),
             admins      = self.reverse('api:organization_admins_list',         kwargs={'pk': obj.pk}),
@@ -1280,6 +1280,14 @@ class OrganizationSerializer(BaseSerializer):
                     'job_templates': 0, 'admins': 0, 'projects': 0}
             else:
                 summary_dict['related_field_counts'] = counts_dict[obj.id]
+
+        # Organization participation roles (admin, member) can't be assigned
+        # to a team. This provides a hint to the ui so it can know to not
+        # display these roles for team role selection.
+        for key in ('admin_role', 'member_role',):
+            if key in summary_dict.get('object_roles', {}):
+                summary_dict['object_roles'][key]['user_only'] = True
+
         return summary_dict
 
     def validate(self, attrs):
@@ -1393,12 +1401,6 @@ class ProjectSerializer(UnifiedJobTemplateSerializer, ProjectOptionsSerializer):
         def get_field_from_model_or_attrs(fd):
             return attrs.get(fd, self.instance and getattr(self.instance, fd) or None)
 
-        organization = None
-        if 'organization' in attrs:
-            organization = attrs['organization']
-        elif self.instance:
-            organization = self.instance.organization
-
         if 'allow_override' in attrs and self.instance:
             # case where user is turning off this project setting
             if self.instance.allow_override and not attrs['allow_override']:
@@ -1414,11 +1416,7 @@ class ProjectSerializer(UnifiedJobTemplateSerializer, ProjectOptionsSerializer):
                             ' '.join([str(pk) for pk in used_by])
                         )})
 
-        view = self.context.get('view', None)
-        if not organization and not view.request.user.is_superuser:
-            # Only allow super users to create orgless projects
-            raise serializers.ValidationError(_('Organization is missing'))
-        elif get_field_from_model_or_attrs('scm_type') == '':
+        if get_field_from_model_or_attrs('scm_type') == '':
             for fd in ('scm_update_on_launch', 'scm_delete_on_update', 'scm_clean'):
                 if get_field_from_model_or_attrs(fd):
                     raise serializers.ValidationError({fd: _('Update options must be set to false for manual projects.')})
@@ -1472,7 +1470,7 @@ class ProjectUpdateSerializer(UnifiedJobSerializer, ProjectOptionsSerializer):
 
     class Meta:
         model = ProjectUpdate
-        fields = ('*', 'project', 'job_type', '-controller_node')
+        fields = ('*', 'project', 'job_type', 'job_tags', '-controller_node')
 
     def get_related(self, obj):
         res = super(ProjectUpdateSerializer, self).get_related(obj)
@@ -1548,20 +1546,15 @@ class InventorySerializer(BaseSerializerWithVariables):
         'admin', 'adhoc',
         {'copy': 'organization.inventory_admin'}
     ]
-    groups_with_active_failures = serializers.IntegerField(
-        read_only=True,
-        min_value=0,
-        help_text=_('This field has been deprecated and will be removed in a future release')
-    )
 
 
     class Meta:
         model = Inventory
         fields = ('*', 'organization', 'kind', 'host_filter', 'variables', 'has_active_failures',
                   'total_hosts', 'hosts_with_active_failures', 'total_groups',
-                  'groups_with_active_failures', 'has_inventory_sources',
-                  'total_inventory_sources', 'inventory_sources_with_failures',
-                  'insights_credential', 'pending_deletion',)
+                  'has_inventory_sources', 'total_inventory_sources',
+                  'inventory_sources_with_failures', 'insights_credential',
+                  'pending_deletion',)
 
     def get_related(self, obj):
         res = super(InventorySerializer, self).get_related(obj)
@@ -1611,7 +1604,7 @@ class InventorySerializer(BaseSerializerWithVariables):
                         })
                 SmartFilter().query_from_string(host_filter)
             except RuntimeError as e:
-                raise models.base.ValidationError(e)
+                raise models.base.ValidationError(str(e))
         return host_filter
 
     def validate(self, attrs):
@@ -1643,6 +1636,9 @@ class HostSerializer(BaseSerializerWithVariables):
     show_capabilities = ['edit', 'delete']
     capabilities_prefetch = ['inventory.admin']
 
+    has_active_failures = serializers.SerializerMethodField()
+    has_inventory_sources = serializers.SerializerMethodField()
+
     class Meta:
         model = Host
         fields = ('*', 'inventory', 'enabled', 'instance_id', 'variables',
@@ -1756,6 +1752,14 @@ class HostSerializer(BaseSerializerWithVariables):
             ret['last_job_host_summary'] = None
         return ret
 
+    def get_has_active_failures(self, obj):
+        return bool(
+            obj.last_job_host_summary and obj.last_job_host_summary.failed
+        )
+
+    def get_has_inventory_sources(self, obj):
+        return obj.inventory_sources.exists()
+
 
 class AnsibleFactsSerializer(BaseSerializer):
     class Meta:
@@ -1768,17 +1772,10 @@ class AnsibleFactsSerializer(BaseSerializer):
 class GroupSerializer(BaseSerializerWithVariables):
     show_capabilities = ['copy', 'edit', 'delete']
     capabilities_prefetch = ['inventory.admin', 'inventory.adhoc']
-    groups_with_active_failures = serializers.IntegerField(
-        read_only=True,
-        min_value=0,
-        help_text=_('This field has been deprecated and will be removed in a future release')
-    )
 
     class Meta:
         model = Group
-        fields = ('*', 'inventory', 'variables', 'has_active_failures',
-                  'total_hosts', 'hosts_with_active_failures', 'total_groups',
-                  'groups_with_active_failures', 'has_inventory_sources')
+        fields = ('*', 'inventory', 'variables')
 
     def build_relational_field(self, field_name, relation_info):
         field_class, field_kwargs = super(GroupSerializer, self).build_relational_field(field_name, relation_info)
@@ -2037,11 +2034,6 @@ class InventorySourceSerializer(UnifiedJobTemplateSerializer, InventorySourceOpt
             res['credentials'] = self.reverse('api:inventory_source_credentials_list', kwargs={'pk': obj.pk})
         return res
 
-    def get_group(self, obj):  # TODO: remove in 3.3
-        if obj.deprecated_group:
-            return obj.deprecated_group.id
-        return None
-
     def build_relational_field(self, field_name, relation_info):
         field_class, field_kwargs = super(InventorySourceSerializer, self).build_relational_field(field_name, relation_info)
         # SCM Project and inventory are read-only unless creating a new inventory.
@@ -2122,7 +2114,13 @@ class InventorySourceSerializer(UnifiedJobTemplateSerializer, InventorySourceOpt
         def get_field_from_model_or_attrs(fd):
             return attrs.get(fd, self.instance and getattr(self.instance, fd) or None)
 
-        if get_field_from_model_or_attrs('source') != 'scm':
+        if get_field_from_model_or_attrs('source') == 'scm':
+            if (('source' in attrs or 'source_project' in attrs) and
+                    get_field_from_model_or_attrs('source_project') is None):
+                raise serializers.ValidationError(
+                    {"source_project": _("Project required for scm type sources.")}
+                )
+        else:
             redundant_scm_fields = list(filter(
                 lambda x: attrs.get(x, None),
                 ['source_project', 'source_path', 'update_on_project_update']
@@ -2456,12 +2454,18 @@ class CredentialTypeSerializer(BaseSerializer):
             raise PermissionDenied(
                 detail=_("Modifications not allowed for managed credential types")
             )
+
+        old_inputs = {}
+        if self.instance:
+            old_inputs = copy.deepcopy(self.instance.inputs)
+
+        ret = super(CredentialTypeSerializer, self).validate(attrs)
+
         if self.instance and self.instance.credentials.exists():
-            if 'inputs' in attrs and attrs['inputs'] != self.instance.inputs:
+            if 'inputs' in attrs and old_inputs != self.instance.inputs:
                 raise PermissionDenied(
                     detail= _("Modifications to inputs are not allowed for credential types that are in use")
                 )
-        ret = super(CredentialTypeSerializer, self).validate(attrs)
 
         if 'kind' in attrs and attrs['kind'] not in ('cloud', 'net'):
             raise serializers.ValidationError({
@@ -2733,7 +2737,8 @@ class JobOptionsSerializer(LabelsListMixin, BaseSerializer):
         fields = ('*', 'job_type', 'inventory', 'project', 'playbook', 'scm_branch',
                   'forks', 'limit', 'verbosity', 'extra_vars', 'job_tags',
                   'force_handlers', 'skip_tags', 'start_at_task', 'timeout',
-                  'use_fact_cache',)
+                  'use_fact_cache', 'organization',)
+        read_only_fields = ('organization',)
 
     def get_related(self, obj):
         res = super(JobOptionsSerializer, self).get_related(obj)
@@ -2748,6 +2753,8 @@ class JobOptionsSerializer(LabelsListMixin, BaseSerializer):
                 res['project'] = self.reverse('api:project_detail', kwargs={'pk': obj.project.pk})
         except ObjectDoesNotExist:
             setattr(obj, 'project', None)
+        if obj.organization_id:
+            res['organization'] = self.reverse('api:organization_detail', kwargs={'pk': obj.organization_id})
         if isinstance(obj, UnifiedJobTemplate):
             res['extra_credentials'] = self.reverse(
                 'api:job_template_extra_credentials_list',
@@ -2816,7 +2823,7 @@ class JobTemplateMixin(object):
         # .only('id', 'status', 'finished', 'polymorphic_ctype_id')
         optimized_qs = uj_qs.non_polymorphic()
         return [{
-            'id': x.id, 'status': x.status, 'finished': x.finished,
+            'id': x.id, 'status': x.status, 'finished': x.finished, 'canceled_on': x.canceled_on,
             # Make type consistent with API top-level key, for instance workflow_job
             'type': x.get_real_instance_class()._meta.verbose_name.replace(' ', '_')
         } for x in optimized_qs[:10]]
@@ -2894,6 +2901,10 @@ class JobTemplateSerializer(JobTemplateMixin, UnifiedJobTemplateSerializer, JobO
         )
         if obj.host_config_key:
             res['callback'] = self.reverse('api:job_template_callback', kwargs={'pk': obj.pk})
+        if obj.organization_id:
+            res['organization'] = self.reverse('api:organization_detail',   kwargs={'pk': obj.organization_id})
+        if obj.webhook_credential_id:
+            res['webhook_credential'] = self.reverse('api:credential_detail', kwargs={'pk': obj.webhook_credential_id})
         return res
 
     def validate(self, attrs):
@@ -3199,7 +3210,7 @@ class AdHocCommandSerializer(UnifiedJobSerializer):
             field_kwargs['choices'] = module_name_choices
             field_kwargs['required'] = bool(not module_name_default)
             field_kwargs['default'] = module_name_default or serializers.empty
-            field_kwargs['allow_blank'] = bool(module_name_default)
+            field_kwargs['allow_blank'] = False
             field_kwargs.pop('max_length', None)
         return field_class, field_kwargs
 
@@ -3384,6 +3395,8 @@ class WorkflowJobTemplateSerializer(JobTemplateMixin, LabelsListMixin, UnifiedJo
         )
         if obj.organization:
             res['organization'] = self.reverse('api:organization_detail',   kwargs={'pk': obj.organization.pk})
+        if obj.webhook_credential_id:
+            res['webhook_credential'] = self.reverse('api:credential_detail', kwargs={'pk': obj.webhook_credential_id})
         return res
 
     def validate_extra_vars(self, value):
@@ -3598,9 +3611,11 @@ class LaunchConfigurationBaseSerializer(BaseSerializer):
         elif self.instance:
             ujt = self.instance.unified_job_template
         if ujt is None:
-            if 'workflow_job_template' in attrs:
-                return {'workflow_job_template': attrs['workflow_job_template']}
-            return {}
+            ret = {}
+            for fd in ('workflow_job_template', 'identifier'):
+                if fd in attrs:
+                    ret[fd] = attrs[fd]
+            return ret
 
         # build additional field survey_passwords to track redacted variables
         password_dict = {}
@@ -3678,7 +3693,8 @@ class WorkflowJobTemplateNodeSerializer(LaunchConfigurationBaseSerializer):
     class Meta:
         model = WorkflowJobTemplateNode
         fields = ('*', 'workflow_job_template', '-name', '-description', 'id', 'url', 'related',
-                  'unified_job_template', 'success_nodes', 'failure_nodes', 'always_nodes',)
+                  'unified_job_template', 'success_nodes', 'failure_nodes', 'always_nodes', 'all_parents_must_converge',
+                  'identifier',)
 
     def get_related(self, obj):
         res = super(WorkflowJobTemplateNodeSerializer, self).get_related(obj)
@@ -3718,7 +3734,7 @@ class WorkflowJobNodeSerializer(LaunchConfigurationBaseSerializer):
         model = WorkflowJobNode
         fields = ('*', 'job', 'workflow_job', '-name', '-description', 'id', 'url', 'related',
                   'unified_job_template', 'success_nodes', 'failure_nodes', 'always_nodes',
-                  'do_not_run',)
+                  'all_parents_must_converge', 'do_not_run', 'identifier')
 
     def get_related(self, obj):
         res = super(WorkflowJobNodeSerializer, self).get_related(obj)
@@ -3826,7 +3842,7 @@ class JobEventSerializer(BaseSerializer):
         model = JobEvent
         fields = ('*', '-name', '-description', 'job', 'event', 'counter',
                   'event_display', 'event_data', 'event_level', 'failed',
-                  'changed', 'uuid', 'parent_uuid', 'host', 'host_name', 'parent',
+                  'changed', 'uuid', 'parent_uuid', 'host', 'host_name',
                   'playbook', 'play', 'task', 'role', 'stdout', 'start_line', 'end_line',
                   'verbosity')
 
@@ -3835,13 +3851,9 @@ class JobEventSerializer(BaseSerializer):
         res.update(dict(
             job = self.reverse('api:job_detail', kwargs={'pk': obj.job_id}),
         ))
-        if obj.parent_id:
-            res['parent'] = self.reverse('api:job_event_detail', kwargs={'pk': obj.parent_id})
         res['children'] = self.reverse('api:job_event_children_list', kwargs={'pk': obj.pk})
         if obj.host_id:
             res['host'] = self.reverse('api:host_detail', kwargs={'pk': obj.host_id})
-        if obj.hosts.exists():
-            res['hosts'] = self.reverse('api:job_event_hosts_list', kwargs={'pk': obj.pk})
         return res
 
     def get_summary_fields(self, obj):
@@ -3867,26 +3879,6 @@ class JobEventSerializer(BaseSerializer):
         return data
 
 
-class JobEventWebSocketSerializer(JobEventSerializer):
-    created = serializers.SerializerMethodField()
-    modified = serializers.SerializerMethodField()
-    event_name = serializers.CharField(source='event')
-    group_name = serializers.SerializerMethodField()
-
-    class Meta:
-        model = JobEvent
-        fields = ('*', 'event_name', 'group_name',)
-
-    def get_created(self, obj):
-        return obj.created.isoformat()
-
-    def get_modified(self, obj):
-        return obj.modified.isoformat()
-
-    def get_group_name(self, obj):
-        return 'job_events'
-
-
 class ProjectUpdateEventSerializer(JobEventSerializer):
     stdout = serializers.SerializerMethodField()
     event_data = serializers.SerializerMethodField()
@@ -3918,26 +3910,6 @@ class ProjectUpdateEventSerializer(JobEventSerializer):
             return {}
 
 
-class ProjectUpdateEventWebSocketSerializer(ProjectUpdateEventSerializer):
-    created = serializers.SerializerMethodField()
-    modified = serializers.SerializerMethodField()
-    event_name = serializers.CharField(source='event')
-    group_name = serializers.SerializerMethodField()
-
-    class Meta:
-        model = ProjectUpdateEvent
-        fields = ('*', 'event_name', 'group_name',)
-
-    def get_created(self, obj):
-        return obj.created.isoformat()
-
-    def get_modified(self, obj):
-        return obj.modified.isoformat()
-
-    def get_group_name(self, obj):
-        return 'project_update_events'
-
-
 class AdHocCommandEventSerializer(BaseSerializer):
 
     event_display = serializers.CharField(source='get_event_display', read_only=True)
@@ -3969,26 +3941,6 @@ class AdHocCommandEventSerializer(BaseSerializer):
         return data
 
 
-class AdHocCommandEventWebSocketSerializer(AdHocCommandEventSerializer):
-    created = serializers.SerializerMethodField()
-    modified = serializers.SerializerMethodField()
-    event_name = serializers.CharField(source='event')
-    group_name = serializers.SerializerMethodField()
-
-    class Meta:
-        model = AdHocCommandEvent
-        fields = ('*', 'event_name', 'group_name',)
-
-    def get_created(self, obj):
-        return obj.created.isoformat()
-
-    def get_modified(self, obj):
-        return obj.modified.isoformat()
-
-    def get_group_name(self, obj):
-        return 'ad_hoc_command_events'
-
-
 class InventoryUpdateEventSerializer(AdHocCommandEventSerializer):
 
     class Meta:
@@ -4004,26 +3956,6 @@ class InventoryUpdateEventSerializer(AdHocCommandEventSerializer):
         return res
 
 
-class InventoryUpdateEventWebSocketSerializer(InventoryUpdateEventSerializer):
-    created = serializers.SerializerMethodField()
-    modified = serializers.SerializerMethodField()
-    event_name = serializers.CharField(source='event')
-    group_name = serializers.SerializerMethodField()
-
-    class Meta:
-        model = InventoryUpdateEvent
-        fields = ('*', 'event_name', 'group_name',)
-
-    def get_created(self, obj):
-        return obj.created.isoformat()
-
-    def get_modified(self, obj):
-        return obj.modified.isoformat()
-
-    def get_group_name(self, obj):
-        return 'inventory_update_events'
-
-
 class SystemJobEventSerializer(AdHocCommandEventSerializer):
 
     class Meta:
@@ -4039,26 +3971,6 @@ class SystemJobEventSerializer(AdHocCommandEventSerializer):
         return res
 
 
-class SystemJobEventWebSocketSerializer(SystemJobEventSerializer):
-    created = serializers.SerializerMethodField()
-    modified = serializers.SerializerMethodField()
-    event_name = serializers.CharField(source='event')
-    group_name = serializers.SerializerMethodField()
-
-    class Meta:
-        model = SystemJobEvent
-        fields = ('*', 'event_name', 'group_name',)
-
-    def get_created(self, obj):
-        return obj.created.isoformat()
-
-    def get_modified(self, obj):
-        return obj.modified.isoformat()
-
-    def get_group_name(self, obj):
-        return 'system_job_events'
-
-
 class JobLaunchSerializer(BaseSerializer):
 
     # Representational fields
@@ -4153,6 +4065,13 @@ class JobLaunchSerializer(BaseSerializer):
             **attrs)
         self._ignored_fields = rejected
 
+        # Basic validation - cannot run a playbook without a playbook
+        if not template.project:
+            errors['project'] = _("A project is required to run a job.")
+        elif template.project.status in ('error', 'failed'):
+            errors['playbook'] = _("Missing a revision to run due to failed project update.")
+
+        # cannot run a playbook without an inventory
         if template.inventory and template.inventory.pending_deletion is True:
             errors['inventory'] = _("The inventory associated with this Job Template is being deleted.")
         elif 'inventory' in accepted and accepted['inventory'].pending_deletion:
@@ -4338,13 +4257,30 @@ class NotificationTemplateSerializer(BaseSerializer):
         error_list = []
         collected_messages = []
 
+        def check_messages(messages):
+            for message_type in messages:
+                if message_type not in ('message', 'body'):
+                    error_list.append(_("Message type '{}' invalid, must be either 'message' or 'body'").format(message_type))
+                    continue
+                message = messages[message_type]
+                if message is None:
+                    continue
+                if not isinstance(message, str):
+                    error_list.append(_("Expected string for '{}', found {}, ").format(message_type, type(message)))
+                    continue
+                if message_type == 'message':
+                    if '\n' in message:
+                        error_list.append(_("Messages cannot contain newlines (found newline in {} event)".format(event)))
+                        continue
+                collected_messages.append(message)
+
         # Validate structure / content types
         if not isinstance(messages, dict):
             error_list.append(_("Expected dict for 'messages' field, found {}".format(type(messages))))
         else:
             for event in messages:
-                if event not in ['started', 'success', 'error']:
-                    error_list.append(_("Event '{}' invalid, must be one of 'started', 'success', or 'error'").format(event))
+                if event not in ('started', 'success', 'error', 'workflow_approval'):
+                    error_list.append(_("Event '{}' invalid, must be one of 'started', 'success', 'error', or 'workflow_approval'").format(event))
                     continue
                 event_messages = messages[event]
                 if event_messages is None:
@@ -4352,21 +4288,21 @@ class NotificationTemplateSerializer(BaseSerializer):
                 if not isinstance(event_messages, dict):
                     error_list.append(_("Expected dict for event '{}', found {}").format(event, type(event_messages)))
                     continue
-                for message_type in event_messages:
-                    if message_type not in ['message', 'body']:
-                        error_list.append(_("Message type '{}' invalid, must be either 'message' or 'body'").format(message_type))
-                        continue
-                    message = event_messages[message_type]
-                    if message is None:
-                        continue
-                    if not isinstance(message, str):
-                        error_list.append(_("Expected string for '{}', found {}, ").format(message_type, type(message)))
-                        continue
-                    if message_type == 'message':
-                        if '\n' in message:
-                            error_list.append(_("Messages cannot contain newlines (found newline in {} event)".format(event)))
+                if event == 'workflow_approval':
+                    for subevent in event_messages:
+                        if subevent not in ('running', 'approved', 'timed_out', 'denied'):
+                            error_list.append(_("Workflow Approval event '{}' invalid, must be one of "
+                                                "'running', 'approved', 'timed_out', or 'denied'").format(subevent))
                             continue
-                    collected_messages.append(message)
+                        subevent_messages = event_messages[subevent]
+                        if subevent_messages is None:
+                            continue
+                        if not isinstance(subevent_messages, dict):
+                            error_list.append(_("Expected dict for workflow approval event '{}', found {}").format(subevent, type(subevent_messages)))
+                            continue
+                        check_messages(subevent_messages)
+                else:
+                    check_messages(event_messages)
 
         # Subclass to return name of undefined field
         class DescriptiveUndefined(StrictUndefined):
@@ -4497,8 +4433,18 @@ class NotificationSerializer(BaseSerializer):
                   'notification_type', 'recipients', 'subject', 'body')
 
     def get_body(self, obj):
-        if obj.notification_type == 'webhook' and 'body' in obj.body:
-            return obj.body['body']
+        if obj.notification_type in ('webhook', 'pagerduty'):
+            if isinstance(obj.body, dict):
+                if 'body' in obj.body:
+                    return obj.body['body']
+            elif isinstance(obj.body, str):
+                # attempt to load json string
+                try:
+                    potential_body = json.loads(obj.body)
+                    if isinstance(potential_body, dict):
+                        return potential_body
+                except json.JSONDecodeError:
+                    pass
         return obj.body
 
     def get_related(self, obj):
@@ -4590,6 +4536,8 @@ class SchedulePreviewSerializer(BaseSerializer):
         try:
             Schedule.rrulestr(rrule_value)
         except Exception as e:
+            import traceback
+            logger.error(traceback.format_exc())
             raise serializers.ValidationError(_("rrule parsing failed validation: {}").format(e))
         return value
 
@@ -4631,6 +4579,10 @@ class ScheduleSerializer(LaunchConfigurationBaseSerializer, SchedulePreviewSeria
 
     def get_summary_fields(self, obj):
         summary_fields = super(ScheduleSerializer, self).get_summary_fields(obj)
+
+        if isinstance(obj.unified_job_template, SystemJobTemplate):
+            summary_fields['unified_job_template']['job_type'] = obj.unified_job_template.job_type
+
         if 'inventory' in summary_fields:
             return summary_fields
 
@@ -4774,6 +4726,18 @@ class InstanceGroupSerializer(BaseSerializer):
                 raise serializers.ValidationError(_('Isolated instances may not be added or removed from instances groups via the API.'))
             if self.instance and self.instance.controller_id is not None:
                 raise serializers.ValidationError(_('Isolated instance group membership may not be managed via the API.'))
+        if value and self.instance and self.instance.is_containerized:
+            raise serializers.ValidationError(_('Containerized instances may not be managed via the API'))
+        return value
+
+    def validate_policy_instance_percentage(self, value):
+        if value and self.instance and self.instance.is_containerized:
+            raise serializers.ValidationError(_('Containerized instances may not be managed via the API'))
+        return value
+
+    def validate_policy_instance_minimum(self, value):
+        if value and self.instance and self.instance.is_containerized:
+            raise serializers.ValidationError(_('Containerized instances may not be managed via the API'))
         return value
 
     def validate_name(self, value):

awx/api/templates/api/inventory_source_cancel.md

@@ -1,7 +1,7 @@
 # Cancel Inventory Update
 
 Make a GET request to this resource to determine if the inventory update can be
-cancelled.  The response will include the following field:
+canceled.  The response will include the following field:
 
 * `can_cancel`: Indicates whether this update can be canceled (boolean,
   read-only)

awx/api/templates/api/job_cancel.md

@@ -1,7 +1,7 @@
 {% ifmeth GET %}
-# Determine if a Job can be cancelled
+# Determine if a Job can be canceled
 
-Make a GET request to this resource to determine if the job can be cancelled.
+Make a GET request to this resource to determine if the job can be canceled.
 The response will include the following field:
 
 * `can_cancel`: Indicates whether this job can be canceled (boolean, read-only)

awx/api/templates/api/project_update_cancel.md

@@ -1,7 +1,7 @@
 # Cancel Project Update
 
 Make a GET request to this resource to determine if the project update can be
-cancelled.  The response will include the following field:
+canceled.  The response will include the following field:
 
 * `can_cancel`: Indicates whether this update can be canceled (boolean,
   read-only)

awx/api/templates/api/setting_logging_test.md

@@ -1 +1,2 @@
 # Test Logging Configuration
+

awx/api/urls/oauth2_root.py

@@ -1,11 +1,15 @@
 # Copyright (c) 2017 Ansible, Inc.
 # All Rights Reserved.
+from datetime import timedelta
 
+from django.utils.timezone import now
+from django.conf import settings
 from django.conf.urls import url
 
 from oauthlib import oauth2
 from oauth2_provider import views
 
+from awx.main.models import RefreshToken
 from awx.api.views import (
     ApiOAuthAuthorizationRootView,
 )
@@ -14,6 +18,21 @@ from awx.api.views import (
 class TokenView(views.TokenView):
 
     def create_token_response(self, request):
+        # Django OAuth2 Toolkit has a bug whereby refresh tokens are *never*
+        # properly expired (ugh):
+        #
+        # https://github.com/jazzband/django-oauth-toolkit/issues/746
+        #
+        # This code detects and auto-expires them on refresh grant
+        # requests.
+        if request.POST.get('grant_type') == 'refresh_token' and 'refresh_token' in request.POST:
+            refresh_token = RefreshToken.objects.filter(
+                token=request.POST['refresh_token']
+            ).first()
+            if refresh_token:
+                expire_seconds = settings.OAUTH2_PROVIDER.get('REFRESH_TOKEN_EXPIRE_SECONDS', 0)
+                if refresh_token.created + timedelta(seconds=expire_seconds) < now():
+                    return request.build_absolute_uri(), {}, 'The refresh token has expired.', '403'
         try:
             return super(TokenView, self).create_token_response(request)
         except oauth2.AccessDeniedError as e:

awx/api/urls/organization.py

@@ -10,6 +10,7 @@ from awx.api.views import (
     OrganizationAdminsList,
     OrganizationInventoriesList,
     OrganizationProjectsList,
+    OrganizationJobTemplatesList,
     OrganizationWorkflowJobTemplatesList,
     OrganizationTeamsList,
     OrganizationCredentialList,
@@ -33,6 +34,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/admins/$', OrganizationAdminsList.as_view(), name='organization_admins_list'),
     url(r'^(?P<pk>[0-9]+)/inventories/$', OrganizationInventoriesList.as_view(), name='organization_inventories_list'),
     url(r'^(?P<pk>[0-9]+)/projects/$', OrganizationProjectsList.as_view(), name='organization_projects_list'),
+    url(r'^(?P<pk>[0-9]+)/job_templates/$', OrganizationJobTemplatesList.as_view(), name='organization_job_templates_list'),
     url(r'^(?P<pk>[0-9]+)/workflow_job_templates/$', OrganizationWorkflowJobTemplatesList.as_view(), name='organization_workflow_job_templates_list'),
     url(r'^(?P<pk>[0-9]+)/teams/$', OrganizationTeamsList.as_view(), name='organization_teams_list'),
     url(r'^(?P<pk>[0-9]+)/credentials/$', OrganizationCredentialList.as_view(), name='organization_credential_list'),

awx/api/urls/urls.py

@@ -34,7 +34,9 @@ from awx.api.views import (
     OAuth2ApplicationDetail,
 )
 
-from awx.api.views.metrics import MetricsView
+from awx.api.views.metrics import (
+    MetricsView,
+)
 
 from .organization import urls as organization_urls
 from .user import urls as user_urls

awx/api/views/__init__.py

@@ -72,17 +72,17 @@ from awx.api.generics import (
     SubListDestroyAPIView
 )
 from awx.api.versioning import reverse
-from awx.conf.license import get_license
 from awx.main import models
 from awx.main.utils import (
     camelcase_to_underscore,
     extract_ansible_vars,
-    get_awx_version,
+    get_awx_http_client_headers,
     get_object_or_400,
     getattrd,
     get_pk_from_dict,
     schedule_task_manager,
-    ignore_inventory_computed_fields
+    ignore_inventory_computed_fields,
+    set_environ
 )
 from awx.main.utils.encryption import encrypt_value
 from awx.main.utils.filters import SmartFilter
@@ -102,7 +102,7 @@ from awx.main.scheduler.dag_workflow import WorkflowDAG
 from awx.api.views.mixin import (
     ControlledByScmMixin, InstanceGroupMembershipMixin,
     OrganizationCountsMixin, RelatedJobsPreventDeleteMixin,
-    UnifiedJobDeletionMixin,
+    UnifiedJobDeletionMixin, NoTruncateMixin,
 )
 from awx.api.views.organization import ( # noqa
     OrganizationList,
@@ -111,6 +111,7 @@ from awx.api.views.organization import ( # noqa
     OrganizationUsersList,
     OrganizationAdminsList,
     OrganizationProjectsList,
+    OrganizationJobTemplatesList,
     OrganizationWorkflowJobTemplatesList,
     OrganizationTeamsList,
     OrganizationActivityStreamList,
@@ -205,20 +206,15 @@ class DashboardView(APIView):
                                             'failed': ec2_inventory_failed.count()}
 
         user_groups = get_user_queryset(request.user, models.Group)
-        groups_job_failed = (
-            models.Group.objects.filter(hosts_with_active_failures__gt=0) | models.Group.objects.filter(groups_with_active_failures__gt=0)
-        ).count()
         groups_inventory_failed = models.Group.objects.filter(inventory_sources__last_job_failed=True).count()
         data['groups'] = {'url': reverse('api:group_list', request=request),
-                          'failures_url': reverse('api:group_list', request=request) + "?has_active_failures=True",
                           'total': user_groups.count(),
-                          'job_failed': groups_job_failed,
                           'inventory_failed': groups_inventory_failed}
 
         user_hosts = get_user_queryset(request.user, models.Host)
-        user_hosts_failed = user_hosts.filter(has_active_failures=True)
+        user_hosts_failed = user_hosts.filter(last_job_host_summary__failed=True)
         data['hosts'] = {'url': reverse('api:host_list', request=request),
-                         'failures_url': reverse('api:host_list', request=request) + "?has_active_failures=True",
+                         'failures_url': reverse('api:host_list', request=request) + "?last_job_host_summary__failed=True",
                          'total': user_hosts.count(),
                          'failed': user_hosts_failed.count()}
 
@@ -383,6 +379,13 @@ class InstanceGroupDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAP
     serializer_class = serializers.InstanceGroupSerializer
     permission_classes = (InstanceGroupTowerPermission,)
 
+    def update_raw_data(self, data):
+        if self.get_object().is_containerized:
+            data.pop('policy_instance_percentage', None)
+            data.pop('policy_instance_minimum', None)
+            data.pop('policy_instance_list', None)
+        return super(InstanceGroupDetail, self).update_raw_data(data)
+
     def destroy(self, request, *args, **kwargs):
         instance = self.get_object()
         if instance.controller is not None:
@@ -568,6 +571,7 @@ class TeamUsersList(BaseUsersList):
     serializer_class = serializers.UserSerializer
     parent_model = models.Team
     relationship = 'member_role.members'
+    ordering = ('username',)
 
 
 class TeamRolesList(SubListAttachDetachAPIView):
@@ -904,6 +908,7 @@ class UserList(ListCreateAPIView):
     model = models.User
     serializer_class = serializers.UserSerializer
     permission_classes = (UserPermission,)
+    ordering = ('username',)
 
 
 class UserMeList(ListAPIView):
@@ -911,6 +916,7 @@ class UserMeList(ListAPIView):
     model = models.User
     serializer_class = serializers.UserSerializer
     name = _('Me')
+    ordering = ('username',)
 
     def get_queryset(self):
         return self.model.objects.filter(pk=self.request.user.pk)
@@ -1086,7 +1092,7 @@ class UserRolesList(SubListAttachDetachAPIView):
 
         credential_content_type = ContentType.objects.get_for_model(models.Credential)
         if role.content_type == credential_content_type:
-            if role.content_object.organization and user not in role.content_object.organization.member_role:
+            if 'disassociate' not in request.data and role.content_object.organization and user not in role.content_object.organization.member_role:
                 data = dict(msg=_("You cannot grant credential access to a user not in the credentials' organization"))
                 return Response(data, status=status.HTTP_400_BAD_REQUEST)
 
@@ -1254,6 +1260,7 @@ class CredentialOwnerUsersList(SubListAPIView):
     serializer_class = serializers.UserSerializer
     parent_model = models.Credential
     relationship = 'admin_role.members'
+    ordering = ('username',)
 
 
 class CredentialOwnerTeamsList(SubListAPIView):
@@ -1375,6 +1382,7 @@ class CredentialExternalTest(SubDetailAPIView):
 
     model = models.Credential
     serializer_class = serializers.EmptySerializer
+    obj_permission_type = 'use'
 
     def post(self, request, *args, **kwargs):
         obj = self.get_object()
@@ -1600,7 +1608,8 @@ class HostInsights(GenericAPIView):
 
     def _call_insights_api(self, url, session, headers):
         try:
-            res = session.get(url, headers=headers, timeout=120)
+            with set_environ(**settings.AWX_TASK_ENV):
+                res = session.get(url, headers=headers, timeout=120)
         except requests.exceptions.SSLError:
             raise BadGateway(_('SSLError while trying to connect to {}').format(url))
         except requests.exceptions.Timeout:
@@ -1632,18 +1641,6 @@ class HostInsights(GenericAPIView):
 
         return session
 
-    def _get_headers(self):
-        license = get_license(show_key=False).get('license_type', 'UNLICENSED')
-        headers = {
-            'Content-Type': 'application/json',
-            'User-Agent': '{} {} ({})'.format(
-                'AWX' if license == 'open' else 'Red Hat Ansible Tower',
-                get_awx_version(),
-                license
-            )
-        }
-
-        return headers
 
     def _get_platform_info(self, host, session, headers):
         url = '{}/api/inventory/v1/hosts?insights_id={}'.format(
@@ -1710,7 +1707,7 @@ class HostInsights(GenericAPIView):
         username = cred.get_input('username', default='')
         password = cred.get_input('password', default='')
         session = self._get_session(username, password)
-        headers = self._get_headers()
+        headers = get_awx_http_client_headers()
 
         data = self._get_insights(host, session, headers)
         return Response(data, status=status.HTTP_200_OK)
@@ -2136,13 +2133,22 @@ class InventorySourceHostsList(HostRelatedSearchMixin, SubListDestroyAPIView):
     def perform_list_destroy(self, instance_list):
         inv_source = self.get_parent_object()
         with ignore_inventory_computed_fields():
-            # Activity stream doesn't record disassociation here anyway
-            # no signals-related reason to not bulk-delete
-            models.Host.groups.through.objects.filter(
-                host__inventory_sources=inv_source
-            ).delete()
-            r = super(InventorySourceHostsList, self).perform_list_destroy(instance_list)
-        update_inventory_computed_fields.delay(inv_source.inventory_id, True)
+            if not settings.ACTIVITY_STREAM_ENABLED_FOR_INVENTORY_SYNC:
+                from awx.main.signals import disable_activity_stream
+                with disable_activity_stream():
+                    # job host summary deletion necessary to avoid deadlock
+                    models.JobHostSummary.objects.filter(host__inventory_sources=inv_source).update(host=None)
+                    models.Host.objects.filter(inventory_sources=inv_source).delete()
+                    r = super(InventorySourceHostsList, self).perform_list_destroy([])
+            else:
+                # Advance delete of group-host memberships to prevent deadlock
+                # Activity stream doesn't record disassociation here anyway
+                # no signals-related reason to not bulk-delete
+                models.Host.groups.through.objects.filter(
+                    host__inventory_sources=inv_source
+                ).delete()
+                r = super(InventorySourceHostsList, self).perform_list_destroy(instance_list)
+        update_inventory_computed_fields.delay(inv_source.inventory_id)
         return r
 
 
@@ -2157,12 +2163,19 @@ class InventorySourceGroupsList(SubListDestroyAPIView):
     def perform_list_destroy(self, instance_list):
         inv_source = self.get_parent_object()
         with ignore_inventory_computed_fields():
-            # Same arguments for bulk delete as with host list
-            models.Group.hosts.through.objects.filter(
-                group__inventory_sources=inv_source
-            ).delete()
-            r = super(InventorySourceGroupsList, self).perform_list_destroy(instance_list)
-        update_inventory_computed_fields.delay(inv_source.inventory_id, True)
+            if not settings.ACTIVITY_STREAM_ENABLED_FOR_INVENTORY_SYNC:
+                from awx.main.signals import disable_activity_stream
+                with disable_activity_stream():
+                    models.Group.objects.filter(inventory_sources=inv_source).delete()
+                    r = super(InventorySourceGroupsList, self).perform_list_destroy([])
+            else:
+                # Advance delete of group-host memberships to prevent deadlock
+                # Same arguments for bulk delete as with host list
+                models.Group.hosts.through.objects.filter(
+                    group__inventory_sources=inv_source
+                ).delete()
+                r = super(InventorySourceGroupsList, self).perform_list_destroy(instance_list)
+        update_inventory_computed_fields.delay(inv_source.inventory_id)
         return r
 
 
@@ -2534,7 +2547,7 @@ class JobTemplateSurveySpec(GenericAPIView):
                 if not isinstance(val, allow_types):
                     return Response(dict(error=_("'{field_name}' in survey question {idx} expected to be {type_label}.").format(
                         field_name=field_name, type_label=type_label, **context
-                    )))
+                    )), status=status.HTTP_400_BAD_REQUEST)
             if survey_item['variable'] in variable_set:
                 return Response(dict(error=_("'variable' '%(item)s' duplicated in survey question %(survey)s.") % {
                     'item': survey_item['variable'], 'survey': str(idx)}), status=status.HTTP_400_BAD_REQUEST)
@@ -2549,7 +2562,7 @@ class JobTemplateSurveySpec(GenericAPIView):
                     "'{survey_item[type]}' in survey question {idx} is not one of '{allowed_types}' allowed question types."
                 ).format(
                     allowed_types=', '.join(JobTemplateSurveySpec.ALLOWED_TYPES.keys()), **context
-                )))
+                )), status=status.HTTP_400_BAD_REQUEST)
             if 'default' in survey_item and survey_item['default'] != '':
                 if not isinstance(survey_item['default'], JobTemplateSurveySpec.ALLOWED_TYPES[qtype]):
                     type_label = 'string'
@@ -2567,7 +2580,7 @@ class JobTemplateSurveySpec(GenericAPIView):
                     if survey_item[key] is not None and (not isinstance(survey_item[key], int)):
                         return Response(dict(error=_(
                             "The {min_or_max} limit in survey question {idx} expected to be integer."
-                        ).format(min_or_max=key, **context)))
+                        ).format(min_or_max=key, **context)), status=status.HTTP_400_BAD_REQUEST)
             # if it's a multiselect or multiple choice, it must have coices listed
             # choices and defualts must come in as strings seperated by /n characters.
             if qtype == 'multiselect' or qtype == 'multiplechoice':
@@ -2577,7 +2590,7 @@ class JobTemplateSurveySpec(GenericAPIView):
                 else:
                     return Response(dict(error=_(
                         "Survey question {idx} of type {survey_item[type]} must specify choices.".format(**context)
-                    )))
+                    )), status=status.HTTP_400_BAD_REQUEST)
                 # If there is a default string split it out removing extra /n characters.
                 # Note: There can still be extra newline characters added in the API, these are sanitized out using .strip()
                 if 'default' in survey_item:
@@ -2591,11 +2604,11 @@ class JobTemplateSurveySpec(GenericAPIView):
                         if len(list_of_defaults) > 1:
                             return Response(dict(error=_(
                                 "Multiple Choice (Single Select) can only have one default value.".format(**context)
-                            )))
+                            )), status=status.HTTP_400_BAD_REQUEST)
                     if any(item not in survey_item['choices'] for item in list_of_defaults):
                         return Response(dict(error=_(
                             "Default choice must be answered from the choices listed.".format(**context)
-                        )))
+                        )), status=status.HTTP_400_BAD_REQUEST)
 
             # Process encryption substitution
             if ("default" in survey_item and isinstance(survey_item['default'], str) and
@@ -3253,7 +3266,7 @@ class WorkflowJobRelaunch(GenericAPIView):
             jt = obj.job_template
             if not jt:
                 raise ParseError(_('Cannot relaunch slice workflow job orphaned from job template.'))
-            elif not jt.inventory or min(jt.inventory.hosts.count(), jt.job_slice_count) != obj.workflow_nodes.count():
+            elif not obj.inventory or min(obj.inventory.hosts.count(), jt.job_slice_count) != obj.workflow_nodes.count():
                 raise ParseError(_('Cannot relaunch sliced workflow job after slice count has changed.'))
         new_workflow_job = obj.create_relaunch_workflow_job()
         new_workflow_job.signal_start()
@@ -3762,18 +3775,12 @@ class JobHostSummaryDetail(RetrieveAPIView):
     serializer_class = serializers.JobHostSummarySerializer
 
 
-class JobEventList(ListAPIView):
+class JobEventList(NoTruncateMixin, ListAPIView):
 
     model = models.JobEvent
     serializer_class = serializers.JobEventSerializer
     search_fields = ('stdout',)
 
-    def get_serializer_context(self):
-        context = super().get_serializer_context()
-        if self.request.query_params.get('no_truncate'):
-            context.update(no_truncate=True)
-        return context
-
 
 class JobEventDetail(RetrieveAPIView):
 
@@ -3786,7 +3793,7 @@ class JobEventDetail(RetrieveAPIView):
         return context
 
 
-class JobEventChildrenList(SubListAPIView):
+class JobEventChildrenList(NoTruncateMixin, SubListAPIView):
 
     model = models.JobEvent
     serializer_class = serializers.JobEventSerializer
@@ -3810,8 +3817,14 @@ class JobEventHostsList(HostRelatedSearchMixin, SubListAPIView):
     relationship = 'hosts'
     name = _('Job Event Hosts List')
 
+    def get_queryset(self):
+        parent_event = self.get_parent_object()
+        self.check_parent_access(parent_event)
+        qs = self.request.user.get_queryset(self.model).filter(job_events_as_primary_host=parent_event)
+        return qs
 
-class BaseJobEventsList(SubListAPIView):
+
+class BaseJobEventsList(NoTruncateMixin, SubListAPIView):
 
     model = models.JobEvent
     serializer_class = serializers.JobEventSerializer
@@ -3832,8 +3845,7 @@ class HostJobEventsList(BaseJobEventsList):
     def get_queryset(self):
         parent_obj = self.get_parent_object()
         self.check_parent_access(parent_obj)
-        qs = self.request.user.get_queryset(self.model).filter(
-            Q(host=parent_obj) | Q(hosts=parent_obj)).distinct()
+        qs = self.request.user.get_queryset(self.model).filter(host=parent_obj)
         return qs
 
 
@@ -3849,9 +3861,7 @@ class JobJobEventsList(BaseJobEventsList):
     def get_queryset(self):
         job = self.get_parent_object()
         self.check_parent_access(job)
-        qs = job.job_events
-        qs = qs.select_related('host')
-        qs = qs.prefetch_related('hosts', 'children')
+        qs = job.job_events.select_related('host').order_by('start_line')
         return qs.all()
 
 
@@ -4007,18 +4017,12 @@ class AdHocCommandRelaunch(GenericAPIView):
             return Response(data, status=status.HTTP_201_CREATED, headers=headers)
 
 
-class AdHocCommandEventList(ListAPIView):
+class AdHocCommandEventList(NoTruncateMixin, ListAPIView):
 
     model = models.AdHocCommandEvent
     serializer_class = serializers.AdHocCommandEventSerializer
     search_fields = ('stdout',)
 
-    def get_serializer_context(self):
-        context = super().get_serializer_context()
-        if self.request.query_params.get('no_truncate'):
-            context.update(no_truncate=True)
-        return context
-
 
 class AdHocCommandEventDetail(RetrieveAPIView):
 
@@ -4031,7 +4035,7 @@ class AdHocCommandEventDetail(RetrieveAPIView):
         return context
 
 
-class BaseAdHocCommandEventsList(SubListAPIView):
+class BaseAdHocCommandEventsList(NoTruncateMixin, SubListAPIView):
 
     model = models.AdHocCommandEvent
     serializer_class = serializers.AdHocCommandEventSerializer
@@ -4297,8 +4301,15 @@ class NotificationTemplateTest(GenericAPIView):
 
     def post(self, request, *args, **kwargs):
         obj = self.get_object()
-        notification = obj.generate_notification("Tower Notification Test {} {}".format(obj.id, settings.TOWER_URL_BASE),
-                                                 {"body": "Ansible Tower Test Notification {} {}".format(obj.id, settings.TOWER_URL_BASE)})
+        msg = "Tower Notification Test {} {}".format(obj.id, settings.TOWER_URL_BASE)
+        if obj.notification_type in ('email', 'pagerduty'):
+            body = "Ansible Tower Test Notification {} {}".format(obj.id, settings.TOWER_URL_BASE)
+        elif obj.notification_type in ('webhook', 'grafana'):
+            body = '{{"body": "Ansible Tower Test Notification {} {}"}}'.format(obj.id, settings.TOWER_URL_BASE)
+        else:
+            body = {"body": "Ansible Tower Test Notification {} {}".format(obj.id, settings.TOWER_URL_BASE)}
+        notification = obj.generate_notification(msg, body)
+
         if not notification:
             return Response({}, status=status.HTTP_400_BAD_REQUEST)
         else:
@@ -4404,7 +4415,7 @@ class RoleUsersList(SubListAttachDetachAPIView):
 
         credential_content_type = ContentType.objects.get_for_model(models.Credential)
         if role.content_type == credential_content_type:
-            if role.content_object.organization and user not in role.content_object.organization.member_role:
+            if 'disassociate' not in request.data and role.content_object.organization and user not in role.content_object.organization.member_role:
                 data = dict(msg=_("You cannot grant credential access to a user not in the credentials' organization"))
                 return Response(data, status=status.HTTP_400_BAD_REQUEST)
 

awx/api/views/metrics.py

@@ -39,3 +39,4 @@ class MetricsView(APIView):
         if (request.user.is_superuser or request.user.is_system_auditor):
             return Response(metrics().decode('UTF-8'))
         raise PermissionDenied()
+

awx/api/views/mixin.py

@@ -4,10 +4,7 @@
 import dateutil
 import logging
 
-from django.db.models import (
-    Count,
-    F,
-)
+from django.db.models import Count
 from django.db import transaction
 from django.shortcuts import get_object_or_404
 from django.utils.timezone import now
@@ -175,28 +172,18 @@ class OrganizationCountsMixin(object):
 
         inv_qs = Inventory.accessible_objects(self.request.user, 'read_role')
         project_qs = Project.accessible_objects(self.request.user, 'read_role')
+        jt_qs = JobTemplate.accessible_objects(self.request.user, 'read_role')
 
         # Produce counts of Foreign Key relationships
-        db_results['inventories'] = inv_qs\
-            .values('organization').annotate(Count('organization')).order_by('organization')
+        db_results['inventories'] = inv_qs.values('organization').annotate(Count('organization')).order_by('organization')
 
         db_results['teams'] = Team.accessible_objects(
             self.request.user, 'read_role').values('organization').annotate(
             Count('organization')).order_by('organization')
 
-        JT_project_reference = 'project__organization'
-        JT_inventory_reference = 'inventory__organization'
-        db_results['job_templates_project'] = JobTemplate.accessible_objects(
-            self.request.user, 'read_role').exclude(
-            project__organization=F(JT_inventory_reference)).values(JT_project_reference).annotate(
-            Count(JT_project_reference)).order_by(JT_project_reference)
+        db_results['job_templates'] = jt_qs.values('organization').annotate(Count('organization')).order_by('organization')
 
-        db_results['job_templates_inventory'] = JobTemplate.accessible_objects(
-            self.request.user, 'read_role').values(JT_inventory_reference).annotate(
-            Count(JT_inventory_reference)).order_by(JT_inventory_reference)
-
-        db_results['projects'] = project_qs\
-            .values('organization').annotate(Count('organization')).order_by('organization')
+        db_results['projects'] = project_qs.values('organization').annotate(Count('organization')).order_by('organization')
 
         # Other members and admins of organization are always viewable
         db_results['users'] = org_qs.annotate(
@@ -212,11 +199,7 @@ class OrganizationCountsMixin(object):
                 'admins': 0, 'projects': 0}
 
         for res, count_qs in db_results.items():
-            if res == 'job_templates_project':
-                org_reference = JT_project_reference
-            elif res == 'job_templates_inventory':
-                org_reference = JT_inventory_reference
-            elif res == 'users':
+            if res == 'users':
                 org_reference = 'id'
             else:
                 org_reference = 'organization'
@@ -229,14 +212,6 @@ class OrganizationCountsMixin(object):
                         continue
                     count_context[org_id][res] = entry['%s__count' % org_reference]
 
-        # Combine the counts for job templates by project and inventory
-        for org in org_id_list:
-            org_id = org['id']
-            count_context[org_id]['job_templates'] = 0
-            for related_path in ['job_templates_project', 'job_templates_inventory']:
-                if related_path in count_context[org_id]:
-                    count_context[org_id]['job_templates'] += count_context[org_id].pop(related_path)
-
         full_context['related_field_counts'] = count_context
 
         return full_context
@@ -270,3 +245,11 @@ class ControlledByScmMixin(object):
         obj = super(ControlledByScmMixin, self).get_parent_object()
         self._reset_inv_src_rev(obj)
         return obj
+
+
+class NoTruncateMixin(object):
+    def get_serializer_context(self):
+        context = super().get_serializer_context()
+        if self.request.query_params.get('no_truncate'):
+            context.update(no_truncate=True)
+        return context

awx/api/views/organization.py

@@ -20,7 +20,7 @@ from awx.main.models import (
     Role,
     User,
     Team,
-    InstanceGroup,
+    InstanceGroup
 )
 from awx.api.generics import (
     ListCreateAPIView,
@@ -28,6 +28,7 @@ from awx.api.generics import (
     SubListAPIView,
     SubListCreateAttachDetachAPIView,
     SubListAttachDetachAPIView,
+    SubListCreateAPIView,
     ResourceAccessList,
     BaseUsersList,
 )
@@ -35,14 +36,13 @@ from awx.api.generics import (
 from awx.api.serializers import (
     OrganizationSerializer,
     InventorySerializer,
-    ProjectSerializer,
     UserSerializer,
     TeamSerializer,
     ActivityStreamSerializer,
     RoleSerializer,
     NotificationTemplateSerializer,
-    WorkflowJobTemplateSerializer,
     InstanceGroupSerializer,
+    ProjectSerializer, JobTemplateSerializer, WorkflowJobTemplateSerializer
 )
 from awx.api.views.mixin import (
     RelatedJobsPreventDeleteMixin,
@@ -94,7 +94,7 @@ class OrganizationDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPI
         org_counts['projects'] = Project.accessible_objects(**access_kwargs).filter(
             organization__id=org_id).count()
         org_counts['job_templates'] = JobTemplate.accessible_objects(**access_kwargs).filter(
-            project__organization__id=org_id).count()
+            organization__id=org_id).count()
 
         full_context['related_field_counts'] = {}
         full_context['related_field_counts'][org_id] = org_counts
@@ -128,21 +128,27 @@ class OrganizationAdminsList(BaseUsersList):
     ordering = ('username',)
 
 
-class OrganizationProjectsList(SubListCreateAttachDetachAPIView):
+class OrganizationProjectsList(SubListCreateAPIView):
 
     model = Project
     serializer_class = ProjectSerializer
     parent_model = Organization
-    relationship = 'projects'
     parent_key = 'organization'
 
 
-class OrganizationWorkflowJobTemplatesList(SubListCreateAttachDetachAPIView):
+class OrganizationJobTemplatesList(SubListCreateAPIView):
+
+    model = JobTemplate
+    serializer_class = JobTemplateSerializer
+    parent_model = Organization
+    parent_key = 'organization'
+
+
+class OrganizationWorkflowJobTemplatesList(SubListCreateAPIView):
 
     model = WorkflowJobTemplate
     serializer_class = WorkflowJobTemplateSerializer
     parent_model = Organization
-    relationship = 'workflows'
     parent_key = 'organization'
 
 

awx/api/views/root.py

@@ -20,6 +20,7 @@ from rest_framework import status
 import requests
 
 from awx.api.generics import APIView
+from awx.conf.registry import settings_registry
 from awx.main.ha import is_ha_environment
 from awx.main.utils import (
     get_awx_version,
@@ -37,6 +38,7 @@ from awx.main.models import (
     InstanceGroup,
     JobTemplate,
 )
+from awx.main.utils import set_environ
 
 logger = logging.getLogger('awx.api.views.root')
 
@@ -60,6 +62,7 @@ class ApiRootView(APIView):
         data['oauth2'] = drf_reverse('api:oauth_authorization_root_view')
         data['custom_logo'] = settings.CUSTOM_LOGO
         data['custom_login_info'] = settings.CUSTOM_LOGIN_INFO
+        data['login_redirect_override'] = settings.LOGIN_REDIRECT_OVERRIDE
         return Response(data)
 
 
@@ -189,7 +192,8 @@ class ApiV2SubscriptionView(APIView):
             data['rh_password'] = settings.REDHAT_PASSWORD
         try:
             user, pw = data.get('rh_username'), data.get('rh_password')
-            validated = get_licenser().validate_rh(user, pw)
+            with set_environ(**settings.AWX_TASK_ENV):
+                validated = get_licenser().validate_rh(user, pw)
             if user:
                 settings.REDHAT_USERNAME = data['rh_username']
             if pw:
@@ -201,10 +205,15 @@ class ApiV2SubscriptionView(APIView):
                 getattr(getattr(exc, 'response', None), 'status_code', None) == 401
             ):
                 msg = _("The provided credentials are invalid (HTTP 401).")
-            if isinstance(exc, (ValueError, OSError)) and exc.args:
+            elif isinstance(exc, requests.exceptions.ProxyError):
+                msg = _("Unable to connect to proxy server.")
+            elif isinstance(exc, requests.exceptions.ConnectionError):
+                msg = _("Could not connect to subscription service.")
+            elif isinstance(exc, (ValueError, OSError)) and exc.args:
                 msg = exc.args[0]
-            logger.exception(smart_text(u"Invalid license submitted."),
-                             extra=dict(actor=request.user.username))
+            else:
+                logger.exception(smart_text(u"Invalid license submitted."),
+                                 extra=dict(actor=request.user.username))
             return Response({"error": msg}, status=status.HTTP_400_BAD_REQUEST)
 
         return Response(validated)
@@ -301,7 +310,8 @@ class ApiV2ConfigView(APIView):
         # If the license is valid, write it to the database.
         if license_data_validated['valid_key']:
             settings.LICENSE = license_data
-            settings.TOWER_URL_BASE = "{}://{}".format(request.scheme, request.get_host())
+            if not settings_registry.is_setting_read_only('TOWER_URL_BASE'):
+                settings.TOWER_URL_BASE = "{}://{}".format(request.scheme, request.get_host())
             return Response(license_data_validated)
 
         logger.warning(smart_text(u"Invalid license submitted."),

awx/api/views/webhooks.py

@@ -1,6 +1,5 @@
 from hashlib import sha1
 import hmac
-import json
 import logging
 import urllib.parse
 
@@ -151,13 +150,13 @@ class WebhookReceiverBase(APIView):
                 'webhook_credential': obj.webhook_credential,
                 'webhook_guid': event_guid,
             },
-            'extra_vars': json.dumps({
+            'extra_vars': {
                 'tower_webhook_event_type': event_type,
                 'tower_webhook_event_guid': event_guid,
                 'tower_webhook_event_ref': event_ref,
                 'tower_webhook_status_api': status_api,
                 'tower_webhook_payload': request.data,
-            })
+            }
         }
 
         new_job = obj.create_unified_job(**kwargs)

awx/asgi.py

@@ -2,14 +2,15 @@
 # All Rights Reserved.
 import os
 import logging
+import django
 from awx import __version__ as tower_version
 
 # Prepare the AWX environment.
 from awx import prepare_env, MODE
 prepare_env() # NOQA
 
-from django.core.wsgi import get_wsgi_application  # NOQA
-from channels.asgi import get_channel_layer
+from channels.routing import get_default_application
+
 
 """
 ASGI config for AWX project.
@@ -32,6 +33,5 @@ if MODE == 'production':
 
 
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "awx.settings")
-
-
-channel_layer = get_channel_layer()
+django.setup()
+channel_layer = get_default_application()

awx/conf/fields.py

@@ -1,16 +1,17 @@
 # Python
 import os
+import re
 import logging
 import urllib.parse as urlparse
 from collections import OrderedDict
 
 # Django
-from django.core.validators import URLValidator
+from django.core.validators import URLValidator, _lazy_re_compile
 from django.utils.translation import ugettext_lazy as _
 
 # Django REST Framework
 from rest_framework.fields import (  # noqa
-    BooleanField, CharField, ChoiceField, DictField, EmailField,
+    BooleanField, CharField, ChoiceField, DictField, DateTimeField, EmailField,
     IntegerField, ListField, NullBooleanField
 )
 
@@ -118,17 +119,42 @@ class StringListPathField(StringListField):
 
 
 class URLField(CharField):
+    # these lines set up a custom regex that allow numbers in the
+    # top-level domain
+    tld_re = (
+        r'\.'                                              # dot
+        r'(?!-)'                                           # can't start with a dash
+        r'(?:[a-z' + URLValidator.ul + r'0-9' + '-]{2,63}' # domain label, this line was changed from the original URLValidator
+        r'|xn--[a-z0-9]{1,59})'                            # or punycode label
+        r'(?<!-)'                                          # can't end with a dash
+        r'\.?'                                             # may have a trailing dot
+    )
+
+    host_re = '(' + URLValidator.hostname_re + URLValidator.domain_re + tld_re + '|localhost)'
+
+    regex = _lazy_re_compile(
+        r'^(?:[a-z0-9\.\-\+]*)://'  # scheme is validated separately
+        r'(?:[^\s:@/]+(?::[^\s:@/]*)?@)?'  # user:pass authentication
+        r'(?:' + URLValidator.ipv4_re + '|' + URLValidator.ipv6_re + '|' + host_re + ')'
+        r'(?::\d{2,5})?'  # port
+        r'(?:[/?#][^\s]*)?'  # resource path
+        r'\Z', re.IGNORECASE)
 
     def __init__(self, **kwargs):
         schemes = kwargs.pop('schemes', None)
         regex = kwargs.pop('regex', None)
         self.allow_plain_hostname = kwargs.pop('allow_plain_hostname', False)
+        self.allow_numbers_in_top_level_domain = kwargs.pop('allow_numbers_in_top_level_domain', True)
         super(URLField, self).__init__(**kwargs)
         validator_kwargs = dict(message=_('Enter a valid URL'))
         if schemes is not None:
             validator_kwargs['schemes'] = schemes
         if regex is not None:
             validator_kwargs['regex'] = regex
+        if self.allow_numbers_in_top_level_domain and regex is None:
+            # default behavior is to allow numbers in the top level domain
+            # if a custom regex isn't provided
+            validator_kwargs['regex'] = URLField.regex
         self.validators.append(URLValidator(**validator_kwargs))
 
     def to_representation(self, value):

awx/conf/license.py

@@ -1,13 +1,12 @@
 # Copyright (c) 2016 Ansible, Inc.
 # All Rights Reserved.
 
-# Tower
-from awx.main.utils.common import get_licenser
 
 __all__ = ['get_license']
 
 
 def _get_validated_license_data():
+    from awx.main.utils.common import get_licenser
     return get_licenser().validate()
 
 

awx/conf/migrations/0004_v320_reencrypt.py

@@ -2,7 +2,6 @@
 from __future__ import unicode_literals
 
 from django.db import migrations
-from awx.conf.migrations import _reencrypt
 
 
 class Migration(migrations.Migration):
@@ -12,5 +11,8 @@ class Migration(migrations.Migration):
     ]
 
     operations = [
-        migrations.RunPython(_reencrypt.replace_aesecb_fernet),
+        # This list is intentionally empty.
+        # Tower 3.2 included several data migrations that are no longer
+        # necessary (this list is now empty because Tower 3.2 is past EOL and
+        # cannot be directly upgraded to modern versions of Tower)
     ]

awx/conf/migrations/_reencrypt.py

@@ -1,30 +1,13 @@
 import base64
 import hashlib
 
-from django.utils.encoding import smart_str
-
 from cryptography.hazmat.backends import default_backend
 from cryptography.hazmat.primitives.ciphers import Cipher
 from cryptography.hazmat.primitives.ciphers.algorithms import AES
 from cryptography.hazmat.primitives.ciphers.modes import ECB
 
-from awx.conf import settings_registry
 
-
-__all__ = ['replace_aesecb_fernet', 'get_encryption_key', 'encrypt_field',
-           'decrypt_value', 'decrypt_value', 'should_decrypt_field']
-
-
-def replace_aesecb_fernet(apps, schema_editor):
-    from awx.main.utils.encryption import encrypt_field
-    Setting = apps.get_model('conf', 'Setting')
-
-    for setting in Setting.objects.filter().order_by('pk'):
-        if settings_registry.is_setting_encrypted(setting.key):
-            if should_decrypt_field(setting.value):
-                setting.value = decrypt_field(setting, 'value')
-                setting.value = encrypt_field(setting, 'value')
-            setting.save()
+__all__ = ['get_encryption_key', 'decrypt_field']
 
 
 def get_encryption_key(field_name, pk=None):
@@ -76,38 +59,3 @@ def decrypt_field(instance, field_name, subfield=None):
     key = get_encryption_key(field_name, getattr(instance, 'pk', None))
 
     return decrypt_value(key, value)
-
-
-def encrypt_field(instance, field_name, ask=False, subfield=None, skip_utf8=False):
-    '''
-    Return content of the given instance and field name encrypted.
-    '''
-    value = getattr(instance, field_name)
-    if isinstance(value, dict) and subfield is not None:
-        value = value[subfield]
-    if not value or value.startswith('$encrypted$') or (ask and value == 'ASK'):
-        return value
-    if skip_utf8:
-        utf8 = False
-    else:
-        utf8 = type(value) == str
-    value = smart_str(value)
-    key = get_encryption_key(field_name, getattr(instance, 'pk', None))
-    encryptor = Cipher(AES(key), ECB(), default_backend()).encryptor()
-    block_size = 16
-    while len(value) % block_size != 0:
-        value += '\x00'
-    encrypted = encryptor.update(value) + encryptor.finalize()
-    b64data = base64.b64encode(encrypted)
-    tokens = ['$encrypted', 'AES', b64data]
-    if utf8:
-        # If the value to encrypt is utf-8, we need to add a marker so we
-        # know to decode the data when it's decrypted later
-        tokens.insert(1, 'UTF8')
-    return '$'.join(tokens)
-
-
-def should_decrypt_field(value):
-    if hasattr(value, 'startswith'):
-        return value.startswith('$encrypted$') and '$AESCBC$' not in value
-    return False

awx/conf/settings.py

@@ -1,14 +1,9 @@
 # Python
-from collections import namedtuple
 import contextlib
 import logging
-import re
 import sys
 import threading
 import time
-import traceback
-import urllib.parse
-from io import StringIO
 
 # Django
 from django.conf import LazySettings
@@ -60,15 +55,6 @@ SETTING_CACHE_DEFAULTS = True
 __all__ = ['SettingsWrapper', 'get_settings_to_cache', 'SETTING_CACHE_NOTSET']
 
 
-def normalize_broker_url(value):
-    parts = value.rsplit('@', 1)
-    match = re.search('(amqp://[^:]+:)(.*)', parts[0])
-    if match:
-        prefix, password = match.group(1), match.group(2)
-        parts[0] = prefix + urllib.parse.quote(password)
-    return '@'.join(parts)
-
-
 @contextlib.contextmanager
 def _ctit_db_wrapper(trans_safe=False):
     '''
@@ -89,42 +75,11 @@ def _ctit_db_wrapper(trans_safe=False):
                     transaction.set_rollback(False)
         yield
     except DBError:
-        # We want the _full_ traceback with the context
-        # First we get the current call stack, which constitutes the "top",
-        # it has the context up to the point where the context manager is used
-        top_stack = StringIO()
-        traceback.print_stack(file=top_stack)
-        top_lines = top_stack.getvalue().strip('\n').split('\n')
-        top_stack.close()
-        # Get "bottom" stack from the local error that happened
-        # inside of the "with" block this wraps
-        exc_type, exc_value, exc_traceback = sys.exc_info()
-        bottom_stack = StringIO()
-        traceback.print_tb(exc_traceback, file=bottom_stack)
-        bottom_lines = bottom_stack.getvalue().strip('\n').split('\n')
-        # Glue together top and bottom where overlap is found
-        bottom_cutoff = 0
-        for i, line in enumerate(bottom_lines):
-            if line in top_lines:
-                # start of overlapping section, take overlap from bottom
-                top_lines = top_lines[:top_lines.index(line)]
-                bottom_cutoff = i
-                break
-        bottom_lines = bottom_lines[bottom_cutoff:]
-        tb_lines = top_lines + bottom_lines
-
-        tb_string = '\n'.join(
-            ['Traceback (most recent call last):'] +
-            tb_lines +
-            ['{}: {}'.format(exc_type.__name__, str(exc_value))]
-        )
-        bottom_stack.close()
-        # Log the combined stack
         if trans_safe:
-            if 'check_migrations' not in sys.argv:
-                logger.debug('Database settings are not available, using defaults, error:\n{}'.format(tb_string))
+            if 'migrate' not in sys.argv and 'check_migrations' not in sys.argv:
+                logger.exception('Database settings are not available, using defaults.')
         else:
-            logger.debug('Error modifying something related to database settings.\n{}'.format(tb_string))
+            logger.exception('Error modifying something related to database settings.')
     finally:
         if trans_safe and is_atomic and rollback_set:
             transaction.set_rollback(rollback_set)
@@ -136,6 +91,15 @@ def filter_sensitive(registry, key, value):
     return value
 
 
+class TransientSetting(object):
+
+    __slots__ = ('pk', 'value')
+
+    def __init__(self, pk, value):
+        self.pk = pk
+        self.value = value
+
+
 class EncryptedCacheProxy(object):
 
     def __init__(self, cache, registry, encrypter=None, decrypter=None):
@@ -163,7 +127,6 @@ class EncryptedCacheProxy(object):
     def get(self, key, **kwargs):
         value = self.cache.get(key, **kwargs)
         value = self._handle_encryption(self.decrypter, key, value)
-        logger.debug('cache get(%r, %r) -> %r', key, empty, filter_sensitive(self.registry, key, value))
         return value
 
     def set(self, key, value, log=True, **kwargs):
@@ -186,8 +149,6 @@ class EncryptedCacheProxy(object):
             self.set(key, value, log=False, **kwargs)
 
     def _handle_encryption(self, method, key, value):
-        TransientSetting = namedtuple('TransientSetting', ['pk', 'value'])
-
         if value is not empty and self.registry.is_setting_encrypted(key):
             # If the setting exists in the database, we'll use its primary key
             # as part of the AES key when encrypting/decrypting
@@ -443,16 +404,7 @@ class SettingsWrapper(UserSettingsHolder):
                 value = self._get_local(name)
         if value is not empty:
             return value
-        value = self._get_default(name)
-        # sometimes users specify RabbitMQ passwords that contain
-        # unescaped : and @ characters that confused urlparse, e.g.,
-        # amqp://guest:a@ns:ibl3#@localhost:5672//
-        #
-        # detect these scenarios, and automatically escape the user's
-        # password so it just works
-        if name == 'BROKER_URL':
-            value = normalize_broker_url(value)
-        return value
+        return self._get_default(name)
 
     def _set_local(self, name, value):
         field = self.registry.get_setting_field(name)

awx/conf/tests/functional/test_api.py

@@ -325,17 +325,3 @@ def test_setting_singleton_delete_no_read_only_fields(api_request, dummy_setting
         )
         assert response.data['FOO_BAR'] == 23
 
-
-@pytest.mark.django_db
-def test_setting_logging_test(api_request):
-    with mock.patch('awx.conf.views.AWXProxyHandler.perform_test') as mock_func:
-        api_request(
-            'post',
-            reverse('api:setting_logging_test'),
-            data={'LOG_AGGREGATOR_HOST': 'http://foobar', 'LOG_AGGREGATOR_TYPE': 'logstash'}
-        )
-        call = mock_func.call_args_list[0]
-        args, kwargs = call
-        given_settings = kwargs['custom_settings']
-        assert given_settings.LOG_AGGREGATOR_HOST == 'http://foobar'
-        assert given_settings.LOG_AGGREGATOR_TYPE == 'logstash'

awx/conf/tests/unit/test_fields.py

@@ -1,7 +1,7 @@
 import pytest
 
 from rest_framework.fields import ValidationError
-from awx.conf.fields import StringListBooleanField, StringListPathField, ListTuplesField
+from awx.conf.fields import StringListBooleanField, StringListPathField, ListTuplesField, URLField
 
 
 class TestStringListBooleanField():
@@ -62,7 +62,7 @@ class TestListTuplesField():
     FIELD_VALUES = [
         ([('a', 'b'), ('abc', '123')], [("a", "b"), ("abc", "123")]),
     ]
-    
+
     FIELD_VALUES_INVALID = [
         ("abc", type("abc")),
         ([('a', 'b', 'c'), ('abc', '123', '456')], type(('a',))),
@@ -130,3 +130,25 @@ class TestStringListPathField():
             field.to_internal_value([value])
         assert e.value.detail[0] == "{} is not a valid path choice.".format(value)
 
+
+class TestURLField():
+    regex = "^https://www.example.org$"
+
+    @pytest.mark.parametrize("url,schemes,regex, allow_numbers_in_top_level_domain, expect_no_error",[
+        ("ldap://www.example.org42", "ldap", None, True, True),
+        ("https://www.example.org42", "https", None, False, False),
+        ("https://www.example.org", None,  regex, None, True),
+        ("https://www.example3.org", None, regex, None, False),
+        ("ftp://www.example.org", "https", None, None, False)
+    ])
+    def test_urls(self, url, schemes, regex, allow_numbers_in_top_level_domain, expect_no_error):
+        kwargs = {}
+        kwargs.setdefault("allow_numbers_in_top_level_domain", allow_numbers_in_top_level_domain)
+        kwargs.setdefault("schemes", schemes)
+        kwargs.setdefault("regex", regex)
+        field = URLField(**kwargs)
+        if expect_no_error:
+            field.run_validators(url)
+        else:
+            with pytest.raises(ValidationError):
+                field.run_validators(url)

awx/conf/views.py

@@ -3,7 +3,11 @@
 
 # Python
 import collections
+import logging
+import subprocess
 import sys
+import socket
+from socket import SHUT_RDWR
 
 # Django
 from django.conf import settings
@@ -11,7 +15,7 @@ from django.http import Http404
 from django.utils.translation import ugettext_lazy as _
 
 # Django REST Framework
-from rest_framework.exceptions import PermissionDenied, ValidationError
+from rest_framework.exceptions import PermissionDenied
 from rest_framework.response import Response
 from rest_framework import serializers
 from rest_framework import status
@@ -26,7 +30,6 @@ from awx.api.generics import (
 from awx.api.permissions import IsSuperUser
 from awx.api.versioning import reverse
 from awx.main.utils import camelcase_to_underscore
-from awx.main.utils.handlers import AWXProxyHandler, LoggingConnectivityException
 from awx.main.tasks import handle_setting_changes
 from awx.conf.models import Setting
 from awx.conf.serializers import SettingCategorySerializer, SettingSingletonSerializer
@@ -161,40 +164,47 @@ class SettingLoggingTest(GenericAPIView):
     filter_backends = []
 
     def post(self, request, *args, **kwargs):
-        defaults = dict()
-        for key in settings_registry.get_registered_settings(category_slug='logging'):
-            try:
-                defaults[key] = settings_registry.get_setting_field(key).get_default()
-            except serializers.SkipField:
-                defaults[key] = None
-        obj = type('Settings', (object,), defaults)()
-        serializer = self.get_serializer(obj, data=request.data)
-        serializer.is_valid(raise_exception=True)
-        # Special validation specific to logging test.
-        errors = {}
-        for key in ['LOG_AGGREGATOR_TYPE', 'LOG_AGGREGATOR_HOST']:
-            if not request.data.get(key, ''):
-                errors[key] = 'This field is required.'
-        if errors:
-            raise ValidationError(errors)
-
-        if request.data.get('LOG_AGGREGATOR_PASSWORD', '').startswith('$encrypted$'):
-            serializer.validated_data['LOG_AGGREGATOR_PASSWORD'] = getattr(
-                settings, 'LOG_AGGREGATOR_PASSWORD', ''
-            )
+        # Error if logging is not enabled
+        enabled = getattr(settings, 'LOG_AGGREGATOR_ENABLED', False)
+        if not enabled:
+            return Response({'error': 'Logging not enabled'}, status=status.HTTP_409_CONFLICT)
+        
+        # Send test message to configured logger based on db settings
+        logging.getLogger('awx').error('AWX Connection Test Message')
+        
+        hostname = getattr(settings, 'LOG_AGGREGATOR_HOST', None)
+        protocol = getattr(settings, 'LOG_AGGREGATOR_PROTOCOL', None)
 
         try:
-            class MockSettings:
-                pass
-            mock_settings = MockSettings()
-            for k, v in serializer.validated_data.items():
-                setattr(mock_settings, k, v)
-            AWXProxyHandler().perform_test(custom_settings=mock_settings)
-            if mock_settings.LOG_AGGREGATOR_PROTOCOL.upper() == 'UDP':
-                return Response(status=status.HTTP_201_CREATED)
-        except LoggingConnectivityException as e:
-            return Response({'error': str(e)}, status=status.HTTP_500_INTERNAL_SERVER_ERROR)
-        return Response(status=status.HTTP_200_OK)
+            subprocess.check_output(
+                ['rsyslogd', '-N1', '-f', '/var/lib/awx/rsyslog/rsyslog.conf'],
+                stderr=subprocess.STDOUT
+            )
+        except subprocess.CalledProcessError as exc:
+            return Response({'error': exc.output}, status=status.HTTP_400_BAD_REQUEST)
+        
+        # Check to ensure port is open at host
+        if protocol in ['udp', 'tcp']:
+            port = getattr(settings, 'LOG_AGGREGATOR_PORT', None)
+            # Error if port is not set when using UDP/TCP
+            if not port:
+                return Response({'error': 'Port required for ' + protocol}, status=status.HTTP_400_BAD_REQUEST)
+        else:
+            # if http/https by this point, domain is reacheable
+            return Response(status=status.HTTP_202_ACCEPTED)
+
+        if protocol == 'udp':
+            s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        else:
+            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        try:
+            s.settimeout(.5)
+            s.connect((hostname, int(port)))
+            s.shutdown(SHUT_RDWR)
+            s.close()
+            return Response(status=status.HTTP_202_ACCEPTED)
+        except Exception as e:
+            return Response({'error': str(e)}, status=status.HTTP_400_BAD_REQUEST)
 
 
 # Create view functions for all of the class-based views to simplify inclusion

awx/main/access.py

@@ -11,7 +11,6 @@ from functools import reduce
 from django.conf import settings
 from django.db.models import Q, Prefetch
 from django.contrib.auth.models import User
-from django.contrib.contenttypes.models import ContentType
 from django.utils.translation import ugettext_lazy as _
 from django.core.exceptions import ObjectDoesNotExist
 
@@ -307,7 +306,7 @@ class BaseAccess(object):
 
         return True  # User has access to both, permission check passed
 
-    def check_license(self, add_host_name=None, feature=None, check_expiration=True):
+    def check_license(self, add_host_name=None, feature=None, check_expiration=True, quiet=False):
         validation_info = get_licenser().validate()
         if validation_info.get('license_type', 'UNLICENSED') == 'open':
             return
@@ -317,8 +316,10 @@ class BaseAccess(object):
             validation_info['time_remaining'] = 99999999
             validation_info['grace_period_remaining'] = 99999999
 
-        report_violation = lambda message: logger.error(message)
-
+        if quiet:
+            report_violation = lambda message: None
+        else:
+            report_violation = lambda message: logger.warning(message)
         if (
             validation_info.get('trial', False) is True or
             validation_info['instance_count'] == 10  # basic 10 license
@@ -403,14 +404,6 @@ class BaseAccess(object):
                 # Cannot copy manual project without errors
                 user_capabilities[display_method] = False
                 continue
-            elif display_method in ['start', 'schedule'] and isinstance(obj, Group):  # TODO: remove in 3.3
-                try:
-                    if obj.deprecated_inventory_source and not obj.deprecated_inventory_source._can_update():
-                        user_capabilities[display_method] = False
-                        continue
-                except Group.deprecated_inventory_source.RelatedObjectDoesNotExist:
-                    user_capabilities[display_method] = False
-                    continue
             elif display_method in ['start', 'schedule'] and isinstance(obj, (Project)):
                 if obj.scm_type == '':
                     user_capabilities[display_method] = False
@@ -465,7 +458,7 @@ class BaseAccess(object):
                 else:
                     relationship = 'members'
                 return access_method(obj, parent_obj, relationship, skip_sub_obj_read_check=True, data={})
-        except (ParseError, ObjectDoesNotExist):
+        except (ParseError, ObjectDoesNotExist, PermissionDenied):
             return False
         return False
 
@@ -648,8 +641,8 @@ class UserAccess(BaseAccess):
                 # in these cases only superusers can modify orphan users
                 return False
             return not obj.roles.all().exclude(
-                content_type=ContentType.objects.get_for_model(User)
-            ).filter(ancestors__in=self.user.roles.all()).exists()
+                ancestors__in=self.user.roles.all()
+            ).exists()
         else:
             return self.is_all_org_admin(obj)
 
@@ -787,7 +780,6 @@ class OrganizationAccess(NotificationAttachMixin, BaseAccess):
         return self.user in obj.admin_role
 
     def can_delete(self, obj):
-        self.check_license(check_expiration=False)
         is_change_possible = self.can_change(obj, None)
         if not is_change_possible:
             return False
@@ -907,7 +899,7 @@ class HostAccess(BaseAccess):
     model = Host
     select_related = ('created_by', 'modified_by', 'inventory',
                       'last_job__job_template', 'last_job_host_summary__job',)
-    prefetch_related = ('groups',)
+    prefetch_related = ('groups', 'inventory_sources')
 
     def filtered_queryset(self):
         return self.model.objects.filter(inventory__in=Inventory.accessible_pk_qs(self.user, 'read_role'))
@@ -1409,7 +1401,7 @@ class JobTemplateAccess(NotificationAttachMixin, BaseAccess):
     '''
 
     model = JobTemplate
-    select_related = ('created_by', 'modified_by', 'inventory', 'project',
+    select_related = ('created_by', 'modified_by', 'inventory', 'project', 'organization',
                       'next_schedule',)
     prefetch_related = (
         'instance_groups',
@@ -1433,16 +1425,11 @@ class JobTemplateAccess(NotificationAttachMixin, BaseAccess):
         Users who are able to create deploy jobs can also run normal and check (dry run) jobs.
         '''
         if not data:  # So the browseable API will work
-            return (
-                Project.accessible_objects(self.user, 'use_role').exists() or
-                Inventory.accessible_objects(self.user, 'use_role').exists())
+            return Project.accessible_objects(self.user, 'use_role').exists()
 
         # if reference_obj is provided, determine if it can be copied
         reference_obj = data.get('reference_obj', None)
 
-        if 'survey_enabled' in data and data['survey_enabled']:
-            self.check_license(feature='surveys')
-
         if self.user.is_superuser:
             return True
 
@@ -1502,22 +1489,23 @@ class JobTemplateAccess(NotificationAttachMixin, BaseAccess):
         return self.user in obj.execute_role
 
     def can_change(self, obj, data):
-        data_for_change = data
         if self.user not in obj.admin_role and not self.user.is_superuser:
             return False
-        if data is not None:
-            data = dict(data)
+        if data is None:
+            return True
 
-            if self.changes_are_non_sensitive(obj, data):
-                if 'survey_enabled' in data and obj.survey_enabled != data['survey_enabled'] and data['survey_enabled']:
-                    self.check_license(feature='surveys')
-                return True
+        data = dict(data)
 
-            for required_field in ('inventory', 'project'):
-                required_obj = getattr(obj, required_field, None)
-                if required_field not in data_for_change and required_obj is not None:
-                    data_for_change[required_field] = required_obj.pk
-        return self.can_read(obj) and (self.can_add(data_for_change) if data is not None else True)
+        if self.changes_are_non_sensitive(obj, data):
+            return True
+
+        for required_field, cls in (('inventory', Inventory), ('project', Project)):
+            is_mandatory = True
+            if not getattr(obj, '{}_id'.format(required_field)):
+                is_mandatory = False
+            if not self.check_related(required_field, cls, data, obj=obj, role_field='use_role', mandatory=is_mandatory):
+                return False
+        return True
 
     def changes_are_non_sensitive(self, obj, data):
         '''
@@ -1552,9 +1540,9 @@ class JobTemplateAccess(NotificationAttachMixin, BaseAccess):
     @check_superuser
     def can_attach(self, obj, sub_obj, relationship, data, skip_sub_obj_read_check=False):
         if relationship == "instance_groups":
-            if not obj.project.organization:
+            if not obj.organization:
                 return False
-            return self.user.can_access(type(sub_obj), "read", sub_obj) and self.user in obj.project.organization.admin_role
+            return self.user.can_access(type(sub_obj), "read", sub_obj) and self.user in obj.organization.admin_role
         if relationship == 'credentials' and isinstance(sub_obj, Credential):
             return self.user in obj.admin_role and self.user in sub_obj.use_role
         return super(JobTemplateAccess, self).can_attach(
@@ -1585,6 +1573,7 @@ class JobAccess(BaseAccess):
     select_related = ('created_by', 'modified_by', 'job_template', 'inventory',
                       'project', 'project_update',)
     prefetch_related = (
+        'organization',
         'unified_job_template',
         'instance_group',
         'credentials__credential_type',
@@ -1605,42 +1594,19 @@ class JobAccess(BaseAccess):
 
         return qs.filter(
             Q(job_template__in=JobTemplate.accessible_objects(self.user, 'read_role')) |
-            Q(inventory__organization__in=org_access_qs) |
-            Q(project__organization__in=org_access_qs)).distinct()
-
-    def related_orgs(self, obj):
-        orgs = []
-        if obj.inventory and obj.inventory.organization:
-            orgs.append(obj.inventory.organization)
-        if obj.project and obj.project.organization and obj.project.organization not in orgs:
-            orgs.append(obj.project.organization)
-        return orgs
-
-    def org_access(self, obj, role_types=['admin_role']):
-        orgs = self.related_orgs(obj)
-        for org in orgs:
-            for role_type in role_types:
-                role = getattr(org, role_type)
-                if self.user in role:
-                    return True
-        return False
+            Q(organization__in=org_access_qs)).distinct()
 
     def can_add(self, data, validate_license=True):
-        if validate_license:
-            self.check_license()
-
-        if not data:  # So the browseable API will work
-            return True
-        return self.user.is_superuser
+        raise NotImplementedError('Direct job creation not possible in v2 API')
 
     def can_change(self, obj, data):
-        return (obj.status == 'new' and
-                self.can_read(obj) and
-                self.can_add(data, validate_license=False))
+        raise NotImplementedError('Direct job editing not supported in v2 API')
 
     @check_superuser
     def can_delete(self, obj):
-        return self.org_access(obj)
+        if not obj.organization:
+            return False
+        return self.user in obj.organization.admin_role
 
     def can_start(self, obj, validate_license=True):
         if validate_license:
@@ -1660,45 +1626,32 @@ class JobAccess(BaseAccess):
         except JobLaunchConfig.DoesNotExist:
             config = None
 
+        # Standard permissions model
+        if obj.job_template and (self.user not in obj.job_template.execute_role):
+            return False
+
         # Check if JT execute access (and related prompts) is sufficient
-        if obj.job_template is not None:
-            if config is None:
-                prompts_access = False
-            elif not config.has_user_prompts(obj.job_template):
-                prompts_access = True
-            elif obj.created_by_id != self.user.pk and vars_are_encrypted(config.extra_data):
-                prompts_access = False
-                if self.save_messages:
-                    self.messages['detail'] = _('Job was launched with secret prompts provided by another user.')
-            else:
-                prompts_access = (
-                    JobLaunchConfigAccess(self.user).can_add({'reference_obj': config}) and
-                    not config.has_unprompted(obj.job_template)
-                )
-            jt_access = self.user in obj.job_template.execute_role
-            if prompts_access and jt_access:
+        if config and obj.job_template:
+            if not config.has_user_prompts(obj.job_template):
                 return True
-            elif not jt_access:
-                return False
+            elif obj.created_by_id != self.user.pk and vars_are_encrypted(config.extra_data):
+                # never allowed, not even for org admins
+                raise PermissionDenied(_('Job was launched with secret prompts provided by another user.'))
+            elif not config.has_unprompted(obj.job_template):
+                if JobLaunchConfigAccess(self.user).can_add({'reference_obj': config}):
+                    return True
 
-        org_access = bool(obj.inventory) and self.user in obj.inventory.organization.inventory_admin_role
-        project_access = obj.project is None or self.user in obj.project.admin_role
-        credential_access = all([self.user in cred.use_role for cred in obj.credentials.all()])
+        # Standard permissions model without job template involved
+        if obj.organization and self.user in obj.organization.execute_role:
+            return True
+        elif not (obj.job_template or obj.organization):
+            raise PermissionDenied(_('Job has been orphaned from its job template and organization.'))
+        elif obj.job_template and config is not None:
+            raise PermissionDenied(_('Job was launched with prompted fields you do not have access to.'))
+        elif obj.job_template and config is None:
+            raise PermissionDenied(_('Job was launched with unknown prompted fields. Organization admin permissions required.'))
 
-        # job can be relaunched if user could make an equivalent JT
-        ret = org_access and credential_access and project_access
-        if not ret and self.save_messages and not self.messages:
-            if not obj.job_template:
-                pretext = _('Job has been orphaned from its job template.')
-            elif config is None:
-                pretext = _('Job was launched with unknown prompted fields.')
-            else:
-                pretext = _('Job was launched with prompted fields.')
-            if credential_access:
-                self.messages['detail'] = '{} {}'.format(pretext, _(' Organization level permissions required.'))
-            else:
-                self.messages['detail'] = '{} {}'.format(pretext, _(' You do not have permission to related resources.'))
-        return ret
+        return False
 
     def get_method_capability(self, method, obj, parent_obj):
         if method == 'start':
@@ -1711,10 +1664,16 @@ class JobAccess(BaseAccess):
     def can_cancel(self, obj):
         if not obj.can_cancel:
             return False
-        # Delete access allows org admins to stop running jobs
-        if self.user == obj.created_by or self.can_delete(obj):
+        # Users may always cancel their own jobs
+        if self.user == obj.created_by:
             return True
-        return obj.job_template is not None and self.user in obj.job_template.admin_role
+        # Users with direct admin to JT may cancel jobs started by anyone
+        if obj.job_template and self.user in obj.job_template.admin_role:
+            return True
+        # If orphaned, allow org JT admins to stop running jobs
+        if not obj.job_template and obj.organization and self.user in obj.organization.job_template_admin_role:
+            return True
+        return False
 
 
 class SystemJobTemplateAccess(BaseAccess):
@@ -1949,11 +1908,11 @@ class WorkflowJobNodeAccess(BaseAccess):
 # TODO: notification attachments?
 class WorkflowJobTemplateAccess(NotificationAttachMixin, BaseAccess):
     '''
-    I can only see/manage Workflow Job Templates if I'm a super user
+    I can see/manage Workflow Job Templates based on object roles
     '''
 
     model = WorkflowJobTemplate
-    select_related = ('created_by', 'modified_by', 'next_schedule',
+    select_related = ('created_by', 'modified_by', 'organization', 'next_schedule',
                       'admin_role', 'execute_role', 'read_role',)
 
     def filtered_queryset(self):
@@ -1971,10 +1930,6 @@ class WorkflowJobTemplateAccess(NotificationAttachMixin, BaseAccess):
         if not data:  # So the browseable API will work
             return Organization.accessible_objects(self.user, 'workflow_admin_role').exists()
 
-        # will check this if surveys are added to WFJT
-        if 'survey_enabled' in data and data['survey_enabled']:
-            self.check_license(feature='surveys')
-
         return (
             self.check_related('organization', Organization, data, role_field='workflow_admin_role', mandatory=True) and
             self.check_related('inventory', Inventory, data, role_field='use_role')
@@ -2043,7 +1998,7 @@ class WorkflowJobAccess(BaseAccess):
        I can also cancel it if I started it
     '''
     model = WorkflowJob
-    select_related = ('created_by', 'modified_by',)
+    select_related = ('created_by', 'modified_by', 'organization',)
 
     def filtered_queryset(self):
         return WorkflowJob.objects.filter(
@@ -2098,23 +2053,20 @@ class WorkflowJobAccess(BaseAccess):
                 self.messages['detail'] = _('Workflow Job was launched with unknown prompts.')
             return False
 
+        # execute permission to WFJT is mandatory for any relaunch
+        if self.user not in template.execute_role:
+            return False
+
         # Check if access to prompts to prevent relaunch
         if config.prompts_dict():
             if obj.created_by_id != self.user.pk and vars_are_encrypted(config.extra_data):
-                if self.save_messages:
-                    self.messages['detail'] = _('Job was launched with secret prompts provided by another user.')
-                return False
+                raise PermissionDenied(_("Job was launched with secret prompts provided by another user."))
             if not JobLaunchConfigAccess(self.user).can_add({'reference_obj': config}):
-                if self.save_messages:
-                    self.messages['detail'] = _('Job was launched with prompts you lack access to.')
-                return False
+                raise PermissionDenied(_('Job was launched with prompts you lack access to.'))
             if config.has_unprompted(template):
-                if self.save_messages:
-                    self.messages['detail'] = _('Job was launched with prompts no longer accepted.')
-                return False
+                raise PermissionDenied(_('Job was launched with prompts no longer accepted.'))
 
-        # execute permission to WFJT is mandatory for any relaunch
-        return (self.user in template.execute_role)
+        return True  # passed config checks
 
     def can_recreate(self, obj):
         node_qs = obj.workflow_job_nodes.all().prefetch_related('inventory', 'credentials', 'unified_job_template')
@@ -2248,7 +2200,7 @@ class JobEventAccess(BaseAccess):
     '''
 
     model = JobEvent
-    prefetch_related = ('hosts', 'job__job_template', 'host',)
+    prefetch_related = ('job__job_template', 'host',)
 
     def filtered_queryset(self):
         return self.model.objects.filter(
@@ -2340,6 +2292,7 @@ class UnifiedJobTemplateAccess(BaseAccess):
     prefetch_related = (
         'last_job',
         'current_job',
+        'organization',
         'credentials__credential_type',
         Prefetch('labels', queryset=Label.objects.all().order_by('name')),
     )
@@ -2379,6 +2332,7 @@ class UnifiedJobAccess(BaseAccess):
     prefetch_related = (
         'created_by',
         'modified_by',
+        'organization',
         'unified_job_node__workflow_job',
         'unified_job_template',
         'instance_group',
@@ -2409,8 +2363,7 @@ class UnifiedJobAccess(BaseAccess):
             Q(unified_job_template_id__in=UnifiedJobTemplate.accessible_pk_qs(self.user, 'read_role')) |
             Q(inventoryupdate__inventory_source__inventory__id__in=inv_pk_qs) |
             Q(adhoccommand__inventory__id__in=inv_pk_qs) |
-            Q(job__inventory__organization__in=org_auditor_qs) |
-            Q(job__project__organization__in=org_auditor_qs)
+            Q(organization__in=org_auditor_qs)
         )
         return qs
 
@@ -2437,6 +2390,9 @@ class ScheduleAccess(BaseAccess):
     def can_add(self, data):
         if not JobLaunchConfigAccess(self.user).can_add(data):
             return False
+        if not data:
+            return Role.objects.filter(role_field__in=['update_role', 'execute_role'], ancestors__in=self.user.roles.all()).exists()
+
         return self.check_related('unified_job_template', UnifiedJobTemplate, data, role_field='execute_role', mandatory=True)
 
     @check_superuser

awx/main/analytics/broadcast_websocket.py

@@ -0,0 +1,170 @@
+import datetime
+import asyncio
+import logging
+import aioredis
+import redis
+import re
+
+from prometheus_client import (
+    generate_latest,
+    Gauge,
+    Counter,
+    Enum,
+    CollectorRegistry,
+    parser,
+)
+
+from django.conf import settings
+
+
+BROADCAST_WEBSOCKET_REDIS_KEY_NAME = 'broadcast_websocket_stats'
+
+
+logger = logging.getLogger('awx.main.analytics.broadcast_websocket')
+
+
+def dt_to_seconds(dt):
+    return int((dt - datetime.datetime(1970,1,1)).total_seconds())
+
+
+def now_seconds():
+    return dt_to_seconds(datetime.datetime.now())
+
+
+def safe_name(s):
+    # Replace all non alpha-numeric characters with _
+    return re.sub('[^0-9a-zA-Z]+', '_', s)
+
+
+# Second granularity; Per-minute
+class FixedSlidingWindow():
+    def __init__(self, start_time=None):
+        self.buckets = dict()
+        self.start_time = start_time or now_seconds()
+
+    def cleanup(self, now_bucket=None):
+        now_bucket = now_bucket or now_seconds()
+        if self.start_time + 60 <= now_bucket:
+            self.start_time = now_bucket + 60 + 1
+
+            # Delete old entries
+            for k in list(self.buckets.keys()):
+                if k < self.start_time:
+                    del self.buckets[k]
+
+    def record(self, ts=None):
+        ts = ts or datetime.datetime.now()
+        now_bucket = int((ts - datetime.datetime(1970,1,1)).total_seconds())
+
+        val = self.buckets.get(now_bucket, 0)
+        self.buckets[now_bucket] = val + 1
+
+        self.cleanup(now_bucket)
+
+    def render(self):
+        self.cleanup()
+        return sum(self.buckets.values()) or 0
+
+
+class BroadcastWebsocketStatsManager():
+    def __init__(self, event_loop, local_hostname):
+        self._local_hostname = local_hostname
+
+        self._event_loop = event_loop
+        self._stats = dict()
+        self._redis_key = BROADCAST_WEBSOCKET_REDIS_KEY_NAME
+
+    def new_remote_host_stats(self, remote_hostname):
+        self._stats[remote_hostname] = BroadcastWebsocketStats(self._local_hostname,
+                                                               remote_hostname)
+        return self._stats[remote_hostname]
+
+    def delete_remote_host_stats(self, remote_hostname):
+        del self._stats[remote_hostname]
+
+    async def run_loop(self):
+        try:
+            redis_conn = await aioredis.create_redis_pool(settings.BROKER_URL)
+            while True:
+                stats_data_str = ''.join(stat.serialize() for stat in self._stats.values())
+                await redis_conn.set(self._redis_key, stats_data_str)
+
+                await asyncio.sleep(settings.BROADCAST_WEBSOCKET_STATS_POLL_RATE_SECONDS)
+        except Exception as e:
+            logger.warn(e)
+            await asyncio.sleep(settings.BROADCAST_WEBSOCKET_STATS_POLL_RATE_SECONDS)
+            self.start()
+
+    def start(self):
+        self.async_task = self._event_loop.create_task(self.run_loop())
+        return self.async_task
+
+    @classmethod
+    def get_stats_sync(cls):
+        '''
+        Stringified verion of all the stats
+        '''
+        redis_conn = redis.Redis.from_url(settings.BROKER_URL)
+        stats_str = redis_conn.get(BROADCAST_WEBSOCKET_REDIS_KEY_NAME) or b''
+        return parser.text_string_to_metric_families(stats_str.decode('UTF-8'))
+
+
+class BroadcastWebsocketStats():
+    def __init__(self, local_hostname, remote_hostname):
+        self._local_hostname = local_hostname
+        self._remote_hostname = remote_hostname
+        self._registry = CollectorRegistry()
+
+        # TODO: More robust replacement
+        self.name = safe_name(self._local_hostname)
+        self.remote_name = safe_name(self._remote_hostname)
+
+        self._messages_received_total = Counter(f'awx_{self.remote_name}_messages_received_total',
+                                                'Number of messages received, to be forwarded, by the broadcast websocket system',
+                                                registry=self._registry)
+        self._messages_received = Gauge(f'awx_{self.remote_name}_messages_received',
+                                        'Number forwarded messages received by the broadcast websocket system, for the duration of the current connection',
+                                        registry=self._registry)
+        self._connection = Enum(f'awx_{self.remote_name}_connection',
+                                'Websocket broadcast connection',
+                                states=['disconnected', 'connected'],
+                                registry=self._registry)
+        self._connection.state('disconnected')
+        self._connection_start = Gauge(f'awx_{self.remote_name}_connection_start',
+                                       'Time the connection was established',
+                                       registry=self._registry)
+
+        self._messages_received_per_minute = Gauge(f'awx_{self.remote_name}_messages_received_per_minute',
+                                                   'Messages received per minute',
+                                                   registry=self._registry)
+        self._internal_messages_received_per_minute = FixedSlidingWindow()
+
+    def unregister(self):
+        self._registry.unregister(f'awx_{self.remote_name}_messages_received')
+        self._registry.unregister(f'awx_{self.remote_name}_connection')
+
+    def record_message_received(self):
+        self._internal_messages_received_per_minute.record()
+        self._messages_received.inc()
+        self._messages_received_total.inc()
+
+    def record_connection_established(self):
+        self._connection.state('connected')
+        self._connection_start.set_to_current_time()
+        self._messages_received.set(0)
+
+    def record_connection_lost(self):
+        self._connection.state('disconnected')
+
+    def get_connection_duration(self):
+        return (datetime.datetime.now() - self._connection_established_ts).total_seconds()
+
+    def render(self):
+        msgs_per_min = self._internal_messages_received_per_minute.render()
+        self._messages_received_per_minute.set(msgs_per_min)
+
+    def serialize(self):
+        self.render()
+
+        registry_data = generate_latest(self._registry).decode('UTF-8')
+        return registry_data

awx/main/analytics/collectors.py

@@ -31,7 +31,7 @@ data _since_ the last report date - i.e., new data in the last 24 hours)
 '''
 
 
-@register('config', '1.0')
+@register('config', '1.1')
 def config(since):
     license_info = get_license(show_key=False)
     install_type = 'traditional'
@@ -52,7 +52,8 @@ def config(since):
         'tower_version': get_awx_version(),
         'ansible_version': get_ansible_version(),
         'license_type': license_info.get('license_type', 'UNLICENSED'),
-        'free_instances': license_info.get('free instances', 0),
+        'free_instances': license_info.get('free_instances', 0),
+        'total_licensed_instances': license_info.get('instance_count', 0),
         'license_expiry': license_info.get('time_remaining', 0),
         'pendo_tracking': settings.PENDO_TRACKING_STATE,
         'authentication_backends': settings.AUTHENTICATION_BACKENDS,
@@ -121,22 +122,27 @@ def cred_type_counts(since):
     return counts
     
     
-@register('inventory_counts', '1.0')
+@register('inventory_counts', '1.2')
 def inventory_counts(since):
     counts = {}
     for inv in models.Inventory.objects.filter(kind='').annotate(num_sources=Count('inventory_sources', distinct=True), 
                                                                  num_hosts=Count('hosts', distinct=True)).only('id', 'name', 'kind'):
+        source_list = []
+        for source in inv.inventory_sources.filter().annotate(num_hosts=Count('hosts', distinct=True)).values('name','source', 'num_hosts'):
+            source_list.append(source)
         counts[inv.id] = {'name': inv.name,
                           'kind': inv.kind,
                           'hosts': inv.num_hosts,
-                          'sources': inv.num_sources
+                          'sources': inv.num_sources,
+                          'source_list': source_list
                           }
 
     for smart_inv in models.Inventory.objects.filter(kind='smart'):
         counts[smart_inv.id] = {'name': smart_inv.name,
                                 'kind': smart_inv.kind,
-                                'num_hosts': smart_inv.hosts.count(),
-                                'num_sources': smart_inv.inventory_sources.count()
+                                'hosts': smart_inv.hosts.count(),
+                                'sources': 0,
+                                'source_list': []
                                 }
     return counts
 
@@ -166,6 +172,8 @@ def instance_info(since, include_hostnames=False):
     instances = models.Instance.objects.values_list('hostname').values(
         'uuid', 'version', 'capacity', 'cpu', 'memory', 'managed_by_policy', 'hostname', 'last_isolated_check', 'enabled')
     for instance in instances:
+        consumed_capacity = sum(x.task_impact for x in models.UnifiedJob.objects.filter(execution_node=instance['hostname'],
+                                status__in=('running', 'waiting')))
         instance_info = {
             'uuid': instance['uuid'],
             'version': instance['version'],
@@ -174,7 +182,9 @@ def instance_info(since, include_hostnames=False):
             'memory': instance['memory'],
             'managed_by_policy': instance['managed_by_policy'],
             'last_isolated_check': _get_isolated_datetime(instance['last_isolated_check']),
-            'enabled': instance['enabled']
+            'enabled': instance['enabled'],
+            'consumed_capacity': consumed_capacity,
+            'remaining_capacity': instance['capacity'] - consumed_capacity
         }
         if include_hostnames is True:
             instance_info['hostname'] = instance['hostname']
@@ -217,7 +227,7 @@ def query_info(since, collection_type):
 
 
 # Copies Job Events from db to a .csv to be shipped
-@table_version('events_table.csv', '1.0')
+@table_version('events_table.csv', '1.1')
 @table_version('unified_jobs_table.csv', '1.0')
 @table_version('unified_job_template_table.csv', '1.0')
 def copy_tables(since, full_path):
@@ -244,6 +254,11 @@ def copy_tables(since, full_path):
                               main_jobevent.job_id, 
                               main_jobevent.host_id, 
                               main_jobevent.host_name
+                              , CAST(main_jobevent.event_data::json->>'start' AS TIMESTAMP WITH TIME ZONE) AS start,
+                              CAST(main_jobevent.event_data::json->>'end' AS TIMESTAMP WITH TIME ZONE) AS end,
+                              main_jobevent.event_data::json->'duration' AS duration,
+                              main_jobevent.event_data::json->'res'->'warnings' AS warnings,
+                              main_jobevent.event_data::json->'res'->'deprecations' AS deprecations
                               FROM main_jobevent 
                               WHERE main_jobevent.created > {}
                               ORDER BY main_jobevent.id ASC) TO STDOUT WITH CSV HEADER'''.format(since.strftime("'%Y-%m-%d %H:%M:%S'"))
@@ -252,7 +267,7 @@ def copy_tables(since, full_path):
     unified_job_query = '''COPY (SELECT main_unifiedjob.id,
                                  main_unifiedjob.polymorphic_ctype_id,
                                  django_content_type.model,
-                                 main_project.organization_id,
+                                 main_unifiedjob.organization_id,
                                  main_organization.name as organization_name,
                                  main_unifiedjob.created,  
                                  main_unifiedjob.name,  
@@ -270,12 +285,10 @@ def copy_tables(since, full_path):
                                  main_unifiedjob.job_explanation, 
                                  main_unifiedjob.instance_group_id
                                  FROM main_unifiedjob
-                                 JOIN main_job ON main_unifiedjob.id = main_job.unifiedjob_ptr_id
                                  JOIN django_content_type ON main_unifiedjob.polymorphic_ctype_id = django_content_type.id
-                                 JOIN main_project ON main_project.unifiedjobtemplate_ptr_id = main_job.project_id
-                                 JOIN main_organization ON main_organization.id = main_project.organization_id
-                                 WHERE main_unifiedjob.created > {} 
-                                 AND main_unifiedjob.launch_type != 'sync'
+                                 LEFT JOIN main_organization ON main_organization.id = main_unifiedjob.organization_id
+                                 WHERE (main_unifiedjob.created > {0} OR main_unifiedjob.finished > {0})
+                                       AND main_unifiedjob.launch_type != 'sync'
                                  ORDER BY main_unifiedjob.id ASC) TO STDOUT WITH CSV HEADER'''.format(since.strftime("'%Y-%m-%d %H:%M:%S'"))    
     _copy_table(table='unified_jobs', query=unified_job_query, path=full_path)
 

awx/main/analytics/core.py

@@ -15,6 +15,7 @@ from awx.conf.license import get_license
 from awx.main.models import Job
 from awx.main.access import access_registry
 from awx.main.models.ha import TowerAnalyticsState
+from awx.main.utils import get_awx_http_client_headers, set_environ
 
 
 __all__ = ['register', 'gather', 'ship', 'table_version']
@@ -133,13 +134,17 @@ def gather(dest=None, module=None, collection_type='scheduled'):
         settings.SYSTEM_UUID,
         run_now.strftime('%Y-%m-%d-%H%M%S%z')
     ])
-    tgz = shutil.make_archive(
-        os.path.join(os.path.dirname(dest), tarname),
-        'gztar',
-        dest
-    )
-    shutil.rmtree(dest)
-    return tgz
+    try:
+        tgz = shutil.make_archive(
+            os.path.join(os.path.dirname(dest), tarname),
+            'gztar',
+            dest
+        )
+        return tgz
+    except Exception:
+        logger.exception("Failed to write analytics archive file")
+    finally: 
+        shutil.rmtree(dest)
 
 
 def ship(path):
@@ -165,11 +170,16 @@ def ship(path):
             return logger.error('REDHAT_PASSWORD is not set')
         with open(path, 'rb') as f:
             files = {'file': (os.path.basename(path), f, settings.INSIGHTS_AGENT_MIME)}
-            response = requests.post(url, 
-                                     files=files,
-                                     verify="/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem",
-                                     auth=(rh_user, rh_password),
-                                     timeout=(31, 31))
+            s = requests.Session()
+            s.headers = get_awx_http_client_headers()
+            s.headers.pop('Content-Type')
+            with set_environ(**settings.AWX_TASK_ENV):
+                response = s.post(url,
+                                  files=files,
+                                  verify="/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem",
+                                  auth=(rh_user, rh_password),
+                                  headers=s.headers,
+                                  timeout=(31, 31))
             if response.status_code != 202:
                 return logger.exception('Upload failed with status {}, {}'.format(response.status_code,
                                                                                   response.text))

awx/main/analytics/metrics.py

@@ -46,6 +46,8 @@ INSTANCE_MEMORY = Gauge('awx_instance_memory', 'RAM (Kb) on each node in a Tower
 INSTANCE_INFO = Info('awx_instance', 'Info about each node in a Tower system', ['hostname', 'instance_uuid',])
 INSTANCE_LAUNCH_TYPE = Gauge('awx_instance_launch_type_total', 'Type of Job launched', ['node', 'launch_type',])
 INSTANCE_STATUS = Gauge('awx_instance_status_total', 'Status of Job launched', ['node', 'status',])
+INSTANCE_CONSUMED_CAPACITY = Gauge('awx_instance_consumed_capacity', 'Consumed capacity of each node in a Tower system', ['hostname', 'instance_uuid',])
+INSTANCE_REMAINING_CAPACITY = Gauge('awx_instance_remaining_capacity', 'Remaining capacity of each node in a Tower system', ['hostname', 'instance_uuid',])
 
 LICENSE_INSTANCE_TOTAL = Gauge('awx_license_instance_total', 'Total number of managed hosts provided by your license')
 LICENSE_INSTANCE_FREE = Gauge('awx_license_instance_free', 'Number of remaining managed hosts provided by your license')
@@ -104,6 +106,8 @@ def metrics():
         INSTANCE_CAPACITY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['capacity'])
         INSTANCE_CPU.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['cpu'])
         INSTANCE_MEMORY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['memory'])
+        INSTANCE_CONSUMED_CAPACITY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['consumed_capacity'])
+        INSTANCE_REMAINING_CAPACITY.labels(hostname=hostname, instance_uuid=uuid).set(instance_data[uuid]['remaining_capacity'])
         INSTANCE_INFO.labels(hostname=hostname, instance_uuid=uuid).info({
             'enabled': str(instance_data[uuid]['enabled']),
             'last_isolated_check': getattr(instance_data[uuid], 'last_isolated_check', 'None'),

awx/main/conf.py

@@ -54,15 +54,6 @@ register(
     category_slug='system',
 )
 
-register(
-    'TOWER_ADMIN_ALERTS',
-    field_class=fields.BooleanField,
-    label=_('Enable Administrator Alerts'),
-    help_text=_('Email Admin users for system events that may require attention.'),
-    category=_('System'),
-    category_slug='system',
-)
-
 register(
     'TOWER_URL_BASE',
     field_class=fields.URLField,
@@ -513,6 +504,27 @@ register(
     category_slug='jobs'
 )
 
+register(
+    'PUBLIC_GALAXY_ENABLED',
+    field_class=fields.BooleanField,
+    default=True,
+    label=_('Allow Access to Public Galaxy'),
+    help_text=_('Allow or deny access to the public Ansible Galaxy during project updates.'),
+    category=_('Jobs'),
+    category_slug='jobs'
+)
+
+register(
+    'GALAXY_IGNORE_CERTS',
+    field_class=fields.BooleanField,
+    default=False,
+    label=_('Ignore Ansible Galaxy SSL Certificate Verification'),
+    help_text=_('If set to true, certificate validation will not be done when'
+                'installing content from any Galaxy server.'),
+    category=_('Jobs'),
+    category_slug='jobs'
+)
+
 register(
     'STDOUT_MAX_BYTES_DISPLAY',
     field_class=fields.IntegerField,
@@ -604,6 +616,18 @@ register(
     category_slug='jobs',
 )
 
+register(
+    'MAX_FORKS',
+    field_class=fields.IntegerField,
+    allow_null=False,
+    default=200,
+    label=_('Maximum number of forks per job.'),
+    help_text=_('Saving a Job Template with more than this number of forks will result in an error. '
+                'When set to 0, no limit is applied.'),
+    category=_('Jobs'),
+    category_slug='jobs',
+)
+
 register(
     'LOG_AGGREGATOR_HOST',
     field_class=fields.CharField,
@@ -643,7 +667,7 @@ register(
     allow_blank=True,
     default='',
     label=_('Logging Aggregator Username'),
-    help_text=_('Username for external log aggregator (if required).'),
+    help_text=_('Username for external log aggregator (if required; HTTP/s only).'),
     category=_('Logging'),
     category_slug='logging',
     required=False,
@@ -655,7 +679,7 @@ register(
     default='',
     encrypted=True,
     label=_('Logging Aggregator Password/Token'),
-    help_text=_('Password or authentication token for external log aggregator (if required).'),
+    help_text=_('Password or authentication token for external log aggregator (if required; HTTP/s only).'),
     category=_('Logging'),
     category_slug='logging',
     required=False,
@@ -775,6 +799,28 @@ register(
 )
 
 
+register(
+    'AUTOMATION_ANALYTICS_LAST_GATHER',
+    field_class=fields.DateTimeField,
+    label=_('Last gather date for Automation Analytics.'),
+    allow_null=True,
+    category=_('System'),
+    category_slug='system'
+)
+
+
+register(
+    'AUTOMATION_ANALYTICS_GATHER_INTERVAL',
+    field_class=fields.IntegerField,
+    label=_('Automation Analytics Gather Interval'),
+    help_text=_('Interval (in seconds) between data gathering.'),
+    default=14400,	# every 4 hours
+    min_value=1800,	# every 30 minutes
+    category=_('System'),
+    category_slug='system'
+)
+
+
 def logging_validate(serializer, attrs):
     if not serializer.instance or \
             not hasattr(serializer.instance, 'LOG_AGGREGATOR_HOST') or \
@@ -799,10 +845,7 @@ def galaxy_validate(serializer, attrs):
     to save settings which obviously break all project updates.
     """
     prefix = 'PRIMARY_GALAXY_'
-
-    from awx.main.constants import GALAXY_SERVER_FIELDS
-    if not any('{}{}'.format(prefix, subfield.upper()) in attrs for subfield in GALAXY_SERVER_FIELDS):
-        return attrs
+    errors = {}
 
     def _new_value(setting_name):
         if setting_name in attrs:
@@ -811,10 +854,22 @@ def galaxy_validate(serializer, attrs):
             return ''
         return getattr(serializer.instance, setting_name, '')
 
+    if not _new_value('PRIMARY_GALAXY_URL'):
+        if _new_value('PUBLIC_GALAXY_ENABLED') is False:
+            msg = _('A URL for Primary Galaxy must be defined before disabling public Galaxy.')
+            # put error in both keys because UI has trouble with errors in toggles
+            for key in ('PRIMARY_GALAXY_URL', 'PUBLIC_GALAXY_ENABLED'):
+                errors.setdefault(key, [])
+                errors[key].append(msg)
+            raise serializers.ValidationError(errors)
+
+    from awx.main.constants import GALAXY_SERVER_FIELDS
+    if not any('{}{}'.format(prefix, subfield.upper()) in attrs for subfield in GALAXY_SERVER_FIELDS):
+        return attrs
+
     galaxy_data = {}
     for subfield in GALAXY_SERVER_FIELDS:
         galaxy_data[subfield] = _new_value('{}{}'.format(prefix, subfield.upper()))
-    errors = {}
     if not galaxy_data['url']:
         for k, v in galaxy_data.items():
             if v:

awx/main/constants.py

@@ -38,7 +38,7 @@ ENV_BLACKLIST = frozenset((
     'AD_HOC_COMMAND_ID', 'REST_API_URL', 'REST_API_TOKEN', 'MAX_EVENT_RES',
     'CALLBACK_QUEUE', 'CALLBACK_CONNECTION', 'CACHE',
     'JOB_CALLBACK_DEBUG', 'INVENTORY_HOSTVARS',
-    'AWX_HOST', 'PROJECT_REVISION'
+    'AWX_HOST', 'PROJECT_REVISION', 'SUPERVISOR_WEB_CONFIG_PATH'
 ))
 
 # loggers that may be called in process of emitting a log

awx/main/consumers.py

@@ -1,97 +1,241 @@
 import json
 import logging
+import time
+import hmac
+import asyncio
 
-from channels import Group
-from channels.auth import channel_session_user_from_http, channel_session_user
-
-from django.utils.encoding import smart_str
-from django.http.cookie import parse_cookie
 from django.core.serializers.json import DjangoJSONEncoder
+from django.conf import settings
+from django.utils.encoding import force_bytes
+from django.contrib.auth.models import User
+
+from channels.generic.websocket import AsyncJsonWebsocketConsumer
+from channels.layers import get_channel_layer
+from channels.db import database_sync_to_async
 
 
 logger = logging.getLogger('awx.main.consumers')
 XRF_KEY = '_auth_user_xrf'
 
 
-def discard_groups(message):
-    if 'groups' in message.channel_session:
-        for group in message.channel_session['groups']:
-            Group(group).discard(message.reply_channel)
+class WebsocketSecretAuthHelper:
+    """
+    Middlewareish for websockets to verify node websocket broadcast interconnect.
+
+    Note: The "ish" is due to the channels routing interface. Routing occurs 
+    _after_ authentication; making it hard to apply this auth to _only_ a subset of
+    websocket endpoints.
+    """
+
+    @classmethod
+    def construct_secret(cls):
+        nonce_serialized = f"{int(time.time())}"
+        payload_dict = {
+            'secret': settings.BROADCAST_WEBSOCKET_SECRET,
+            'nonce': nonce_serialized
+        }
+        payload_serialized = json.dumps(payload_dict)
+
+        secret_serialized = hmac.new(force_bytes(settings.BROADCAST_WEBSOCKET_SECRET),
+                                     msg=force_bytes(payload_serialized),
+                                     digestmod='sha256').hexdigest()
+
+        return 'HMAC-SHA256 {}:{}'.format(nonce_serialized, secret_serialized)
 
 
-@channel_session_user_from_http
-def ws_connect(message):
-    headers = dict(message.content.get('headers', ''))
-    message.reply_channel.send({"accept": True})
-    message.content['method'] = 'FAKE'
-    if message.user.is_authenticated:
-        message.reply_channel.send(
-            {"text": json.dumps({"accept": True, "user": message.user.id})}
-        )
-        # store the valid CSRF token from the cookie so we can compare it later
-        # on ws_receive
-        cookie_token = parse_cookie(
-            smart_str(headers.get(b'cookie'))
-        ).get('csrftoken')
-        if cookie_token:
-            message.channel_session[XRF_KEY] = cookie_token
-    else:
-        logger.error("Request user is not authenticated to use websocket.")
-        message.reply_channel.send({"close": True})
-    return None
+    @classmethod
+    def verify_secret(cls, s, nonce_tolerance=300):
+        try:
+            (prefix, payload) = s.split(' ')
+            if prefix != 'HMAC-SHA256':
+                raise ValueError('Unsupported encryption algorithm')
+            (nonce_parsed, secret_parsed) = payload.split(':')
+        except Exception:
+            raise ValueError("Failed to parse secret")
+
+        try:
+            payload_expected = {
+                'secret': settings.BROADCAST_WEBSOCKET_SECRET,
+                'nonce': nonce_parsed,
+            }
+            payload_serialized = json.dumps(payload_expected)
+        except Exception:
+            raise ValueError("Failed to create hash to compare to secret.")
+
+        secret_serialized = hmac.new(force_bytes(settings.BROADCAST_WEBSOCKET_SECRET),
+                                     msg=force_bytes(payload_serialized),
+                                     digestmod='sha256').hexdigest()
+
+        if secret_serialized != secret_parsed:
+            raise ValueError("Invalid secret")
+
+        # Avoid timing attack and check the nonce after all the heavy lifting
+        now = int(time.time())
+        nonce_parsed = int(nonce_parsed)
+        nonce_diff = now - nonce_parsed
+        if abs(nonce_diff) > nonce_tolerance:
+            logger.warn(f"Potential replay attack or machine(s) time out of sync by {nonce_diff} seconds.")
+            raise ValueError("Potential replay attack or machine(s) time out of sync by {nonce_diff} seconds.")
+
+        return True
+
+    @classmethod
+    def is_authorized(cls, scope):
+        secret = ''
+        for k, v in scope['headers']:
+            if k.decode("utf-8") == 'secret':
+                secret = v.decode("utf-8")
+                break
+        WebsocketSecretAuthHelper.verify_secret(secret)
 
 
-@channel_session_user
-def ws_disconnect(message):
-    discard_groups(message)
+class BroadcastConsumer(AsyncJsonWebsocketConsumer):
+
+    async def connect(self):
+        try:
+            WebsocketSecretAuthHelper.is_authorized(self.scope)
+        except Exception:
+            # TODO: log ip of connected client
+            logger.warn("Broadcast client failed to authorize for reason.")
+            await self.close()
+            return
+
+        # TODO: log ip of connected client
+        logger.info(f"Broadcast client connected.")
+        await self.accept()
+        await self.channel_layer.group_add(settings.BROADCAST_WEBSOCKET_GROUP_NAME, self.channel_name)
+
+    async def disconnect(self, code):
+        # TODO: log ip of disconnected client
+        logger.info("Client disconnected")
+
+    async def internal_message(self, event):
+        await self.send(event['text'])
 
 
-@channel_session_user
-def ws_receive(message):
-    from awx.main.access import consumer_access
-    user = message.user
-    raw_data = message.content['text']
-    data = json.loads(raw_data)
+class EventConsumer(AsyncJsonWebsocketConsumer):
+    async def connect(self):
+        user = self.scope['user']
+        if user and not user.is_anonymous:
+            await self.accept()
+            await self.send_json({"accept": True, "user": user.id})
+            # store the valid CSRF token from the cookie so we can compare it later
+            # on ws_receive
+            cookie_token = self.scope['cookies'].get('csrftoken')
+            if cookie_token:
+                self.scope['session'][XRF_KEY] = cookie_token
+        else:
+            logger.error("Request user is not authenticated to use websocket.")
+            # TODO: Carry over from channels 1 implementation
+            # We should never .accept() the client and close without sending a close message
+            await self.accept()
+            await self.send_json({"close": True})
+            await self.close()
 
-    xrftoken = data.get('xrftoken')
-    if (
-        not xrftoken or
-        XRF_KEY not in message.channel_session or
-        xrftoken != message.channel_session[XRF_KEY]
-    ):
-        logger.error(
-            "access denied to channel, XRF mismatch for {}".format(user.username)
-        )
-        message.reply_channel.send({
-            "text": json.dumps({"error": "access denied to channel"})
-        })
-        return
+    @database_sync_to_async
+    def user_can_see_object_id(self, user_access, oid):
+        # At this point user is a channels.auth.UserLazyObject object
+        # This causes problems with our generic role permissions checking.
+        # Specifically, type(user) != User
+        # Therefore, get the "real" User objects from the database before
+        # calling the access permission methods
+        user_access.user = User.objects.get(id=user_access.user.id)
+        res = user_access.get_queryset().filter(pk=oid).exists()
+        return res
 
-    if 'groups' in data:
-        discard_groups(message)
-        groups = data['groups']
-        current_groups = set(message.channel_session.pop('groups') if 'groups' in message.channel_session else [])
-        for group_name,v in groups.items():
-            if type(v) is list:
-                for oid in v:
-                    name = '{}-{}'.format(group_name, oid)
-                    access_cls = consumer_access(group_name)
-                    if access_cls is not None:
-                        user_access = access_cls(user)
-                        if not user_access.get_queryset().filter(pk=oid).exists():
-                            message.reply_channel.send({"text": json.dumps(
-                                {"error": "access denied to channel {0} for resource id {1}".format(group_name, oid)})})
-                            continue
-                    current_groups.add(name)
-                    Group(name).add(message.reply_channel)
-            else:
-                current_groups.add(group_name)
-                Group(group_name).add(message.reply_channel)
-        message.channel_session['groups'] = list(current_groups)
+    async def receive_json(self, data):
+        from awx.main.access import consumer_access
+        user = self.scope['user']
+        xrftoken = data.get('xrftoken')
+        if (
+            not xrftoken or
+            XRF_KEY not in self.scope["session"] or
+            xrftoken != self.scope["session"][XRF_KEY]
+        ):
+            logger.error(f"access denied to channel, XRF mismatch for {user.username}")
+            await self.send_json({"error": "access denied to channel"})
+            return
+
+        if 'groups' in data:
+            groups = data['groups']
+            new_groups = set()
+            current_groups = set(self.scope['session'].pop('groups') if 'groups' in self.scope['session'] else [])
+            for group_name,v in groups.items():
+                if type(v) is list:
+                    for oid in v:
+                        name = '{}-{}'.format(group_name, oid)
+                        access_cls = consumer_access(group_name)
+                        if access_cls is not None:
+                            user_access = access_cls(user)
+                            if not await self.user_can_see_object_id(user_access, oid):
+                                await self.send_json({"error": "access denied to channel {0} for resource id {1}".format(group_name, oid)})
+                                continue
+                        new_groups.add(name)
+                else:
+                    await self.send_json({"error": "access denied to channel"})
+                    logger.error(f"groups must be a list, not {groups}")
+                    return
+
+            old_groups = current_groups - new_groups
+            for group_name in old_groups:
+                await self.channel_layer.group_discard(
+                    group_name,
+                    self.channel_name,
+                )
+
+            new_groups_exclusive = new_groups - current_groups
+            for group_name in new_groups_exclusive:
+                await self.channel_layer.group_add(
+                    group_name,
+                    self.channel_name
+                )
+            logger.debug(f"Channel {self.channel_name} left groups {old_groups} and joined {new_groups_exclusive}")
+            self.scope['session']['groups'] = new_groups
+            await self.send_json({
+                "groups_current": list(new_groups),
+                "groups_left": list(old_groups),
+                "groups_joined": list(new_groups_exclusive)
+            })
+
+    async def internal_message(self, event):
+        await self.send(event['text'])
+
+
+def run_sync(func):
+    event_loop = asyncio.new_event_loop()
+    event_loop.run_until_complete(func)
+    event_loop.close()
+
+
+def _dump_payload(payload):
+    try:
+        return json.dumps(payload, cls=DjangoJSONEncoder)
+    except ValueError:
+        logger.error("Invalid payload to emit")
+        return None
 
 
 def emit_channel_notification(group, payload):
-    try:
-        Group(group).send({"text": json.dumps(payload, cls=DjangoJSONEncoder)})
-    except ValueError:
-        logger.error("Invalid payload emitting channel {} on topic: {}".format(group, payload))
+    from awx.main.wsbroadcast import wrap_broadcast_msg # noqa
+
+    payload_dumped = _dump_payload(payload)
+    if payload_dumped is None:
+        return
+
+    channel_layer = get_channel_layer()
+
+    run_sync(channel_layer.group_send(
+        group,
+        {
+            "type": "internal.message",
+            "text": payload_dumped
+        },
+    ))
+
+    run_sync(channel_layer.group_send(
+        settings.BROADCAST_WEBSOCKET_GROUP_NAME,
+        {
+            "type": "internal.message",
+            "text": wrap_broadcast_msg(group, payload_dumped),
+        },
+    ))

awx/main/credential_plugins/aim.py

@@ -43,7 +43,7 @@ aim_inputs = {
         'id': 'object_query',
         'label': _('Object Query'),
         'type': 'string',
-        'help_text': _('Lookup query for the object. Ex: "Safe=TestSafe;Object=testAccountName123"'),
+        'help_text': _('Lookup query for the object. Ex: Safe=TestSafe;Object=testAccountName123'),
     }, {
         'id': 'object_query_format',
         'label': _('Object Query Format'),

awx/main/credential_plugins/azure_kv.py

@@ -3,6 +3,16 @@ from .plugin import CredentialPlugin
 from django.utils.translation import ugettext_lazy as _
 from azure.keyvault import KeyVaultClient, KeyVaultAuthentication
 from azure.common.credentials import ServicePrincipalCredentials
+from msrestazure import azure_cloud
+
+
+# https://github.com/Azure/msrestazure-for-python/blob/master/msrestazure/azure_cloud.py
+clouds = [
+    vars(azure_cloud)[n]
+    for n in dir(azure_cloud)
+    if n.startswith("AZURE_") and n.endswith("_CLOUD")
+]
+default_cloud = vars(azure_cloud)["AZURE_PUBLIC_CLOUD"]
 
 
 azure_keyvault_inputs = {
@@ -24,6 +34,12 @@ azure_keyvault_inputs = {
         'id': 'tenant',
         'label': _('Tenant ID'),
         'type': 'string'
+    }, {
+        'id': 'cloud_name',
+        'label': _('Cloud Environment'),
+        'help_text': _('Specify which azure cloud environment to use.'),
+        'choices': list(set([default_cloud.name] + [c.name for c in clouds])),
+        'default': default_cloud.name
     }],
     'metadata': [{
         'id': 'secret_field',
@@ -42,6 +58,7 @@ azure_keyvault_inputs = {
 
 def azure_keyvault_backend(**kwargs):
     url = kwargs['url']
+    [cloud] = [c for c in clouds if c.name == kwargs.get('cloud_name', default_cloud.name)]
 
     def auth_callback(server, resource, scope):
         credentials = ServicePrincipalCredentials(
@@ -49,7 +66,7 @@ def azure_keyvault_backend(**kwargs):
             client_id = kwargs['client'],
             secret = kwargs['secret'],
             tenant = kwargs['tenant'],
-            resource = "https://vault.azure.net",
+            resource = f"https://{cloud.suffixes.keyvault_dns.split('.', 1).pop()}",
         )
         token = credentials.token
         return token['token_type'], token['access_token']

awx/main/db/profiled_pg/base.py

@@ -64,7 +64,7 @@ class RecordedQueryLog(object):
             if not os.path.isdir(self.dest):
                 os.makedirs(self.dest)
             progname = ' '.join(sys.argv)
-            for match in ('uwsgi', 'dispatcher', 'callback_receiver', 'runworker'):
+            for match in ('uwsgi', 'dispatcher', 'callback_receiver', 'wsbroadcast'):
                 if match in progname:
                     progname = match
                     break

awx/main/dispatch/__init__.py

@@ -1,5 +1,62 @@
+import psycopg2
+import select
+
+from contextlib import contextmanager
+
 from django.conf import settings
 
 
+NOT_READY = ([], [], [])
+
+
 def get_local_queuename():
     return settings.CLUSTER_HOST_ID
+
+
+class PubSub(object):
+    def __init__(self, conn):
+        assert conn.autocommit, "Connection must be in autocommit mode."
+        self.conn = conn
+
+    def listen(self, channel):
+        with self.conn.cursor() as cur:
+            cur.execute('LISTEN "%s";' % channel)
+
+    def unlisten(self, channel):
+        with self.conn.cursor() as cur:
+            cur.execute('UNLISTEN "%s";' % channel)
+
+    def notify(self, channel, payload):
+        with self.conn.cursor() as cur:
+            cur.execute('SELECT pg_notify(%s, %s);', (channel, payload))
+
+    def events(self, select_timeout=5, yield_timeouts=False):
+        while True:
+            if select.select([self.conn], [], [], select_timeout) == NOT_READY:
+                if yield_timeouts:
+                    yield None
+            else:
+                self.conn.poll()
+                while self.conn.notifies:
+                    yield self.conn.notifies.pop(0)
+
+    def close(self):
+        self.conn.close()
+
+
+@contextmanager
+def pg_bus_conn():
+    conf = settings.DATABASES['default']
+    conn = psycopg2.connect(dbname=conf['NAME'],
+                            host=conf['HOST'],
+                            user=conf['USER'],
+                            password=conf['PASSWORD'],
+                            port=conf['PORT'],
+                            **conf.get("OPTIONS", {}))
+    # Django connection.cursor().connection doesn't have autocommit=True on
+    conn.set_session(autocommit=True)
+    pubsub = PubSub(conn)
+    yield pubsub
+    conn.close()
+
+

awx/main/dispatch/control.py

@@ -1,11 +1,10 @@
 import logging
-import socket
-
-from django.conf import settings
+import uuid
+import json
 
 from awx.main.dispatch import get_local_queuename
-from awx.main.dispatch.kombu import Connection
-from kombu import Queue, Exchange, Producer, Consumer
+
+from . import pg_bus_conn
 
 logger = logging.getLogger('awx.main.dispatch')
 
@@ -20,15 +19,6 @@ class Control(object):
             raise RuntimeError('{} must be in {}'.format(service, self.services))
         self.service = service
         self.queuename = host or get_local_queuename()
-        self.queue = Queue(self.queuename, Exchange(self.queuename), routing_key=self.queuename)
-
-    def publish(self, msg, conn, **kwargs):
-        producer = Producer(
-            exchange=self.queue.exchange,
-            channel=conn,
-            routing_key=self.queuename
-        )
-        producer.publish(msg, expiration=5, **kwargs)
 
     def status(self, *args, **kwargs):
         return self.control_with_reply('status', *args, **kwargs)
@@ -36,24 +26,28 @@ class Control(object):
     def running(self, *args, **kwargs):
         return self.control_with_reply('running', *args, **kwargs)
 
+    @classmethod
+    def generate_reply_queue_name(cls):
+        return f"reply_to_{str(uuid.uuid4()).replace('-','_')}"
+
     def control_with_reply(self, command, timeout=5):
         logger.warn('checking {} {} for {}'.format(self.service, command, self.queuename))
-        reply_queue = Queue(name="amq.rabbitmq.reply-to")
+        reply_queue = Control.generate_reply_queue_name()
         self.result = None
-        with Connection(settings.BROKER_URL) as conn:
-            with Consumer(conn, reply_queue, callbacks=[self.process_message], no_ack=True):
-                self.publish({'control': command}, conn, reply_to='amq.rabbitmq.reply-to')
-                try:
-                    conn.drain_events(timeout=timeout)
-                except socket.timeout:
-                    logger.error('{} did not reply within {}s'.format(self.service, timeout))
-                    raise
-        return self.result
+
+        with pg_bus_conn() as conn:
+            conn.listen(reply_queue)
+            conn.notify(self.queuename,
+                        json.dumps({'control': command, 'reply_to': reply_queue}))
+
+            for reply in conn.events(select_timeout=timeout, yield_timeouts=True):
+                if reply is None:
+                    logger.error(f'{self.service} did not reply within {timeout}s')
+                    raise RuntimeError("{self.service} did not reply within {timeout}s")
+                break
+
+        return json.loads(reply.payload)
 
     def control(self, msg, **kwargs):
-        with Connection(settings.BROKER_URL) as conn:
-            self.publish(msg, conn)
-
-    def process_message(self, body, message):
-        self.result = body
-        message.ack()
+        with pg_bus_conn() as conn:
+            conn.notify(self.queuename, json.dumps(msg))

awx/main/dispatch/kombu.py

@@ -1,42 +0,0 @@
-from amqp.exceptions import PreconditionFailed
-from django.conf import settings
-from kombu.connection import Connection as KombuConnection
-from kombu.transport import pyamqp
-
-import logging
-
-logger = logging.getLogger('awx.main.dispatch')
-
-
-__all__ = ['Connection']
-
-
-class Connection(KombuConnection):
-
-    def __init__(self, *args, **kwargs):
-        super(Connection, self).__init__(*args, **kwargs)
-        class _Channel(pyamqp.Channel):
-
-            def queue_declare(self, queue, *args, **kwargs):
-                kwargs['durable'] = settings.BROKER_DURABILITY
-                try:
-                    return super(_Channel, self).queue_declare(queue, *args, **kwargs)
-                except PreconditionFailed as e:
-                    if "inequivalent arg 'durable'" in getattr(e, 'reply_text', None):
-                        logger.error(
-                            'queue {} durability is not {}, deleting and recreating'.format(
-
-                                queue,
-                                kwargs['durable']
-                            )
-                        )
-                        self.queue_delete(queue)
-                    return super(_Channel, self).queue_declare(queue, *args, **kwargs)
-
-        class _Connection(pyamqp.Connection):
-            Channel = _Channel
-
-        class _Transport(pyamqp.Transport):
-            Connection = _Connection
-
-        self.transport_cls = _Transport

awx/main/dispatch/periodic.py

@@ -0,0 +1,56 @@
+import logging
+import os
+import time
+from multiprocessing import Process
+
+from django.conf import settings
+from django.db import connections
+from schedule import Scheduler
+
+from awx.main.dispatch.worker import TaskWorker
+
+logger = logging.getLogger('awx.main.dispatch.periodic')
+
+
+class Scheduler(Scheduler):
+
+    def run_continuously(self):
+        idle_seconds = max(
+            1,
+            min(self.jobs).period.total_seconds() / 2
+        )
+
+        def run():
+            ppid = os.getppid()
+            logger.warn(f'periodic beat started')
+            while True:
+                if os.getppid() != ppid:
+                    # if the parent PID changes, this process has been orphaned
+                    # via e.g., segfault or sigkill, we should exit too
+                    pid = os.getpid()
+                    logger.warn(f'periodic beat exiting gracefully pid:{pid}')
+                    raise SystemExit()
+                try:
+                    for conn in connections.all():
+                        # If the database connection has a hiccup, re-establish a new
+                        # connection
+                        conn.close_if_unusable_or_obsolete()
+                    self.run_pending()
+                except Exception:
+                    logger.exception(
+                        'encountered an error while scheduling periodic tasks'
+                    )
+                time.sleep(idle_seconds)
+
+        process = Process(target=run)
+        process.daemon = True
+        process.start()
+
+
+def run_continuously():
+    scheduler = Scheduler()
+    for task in settings.CELERYBEAT_SCHEDULE.values():
+        apply_async = TaskWorker.resolve_callable(task['task']).apply_async
+        total_seconds = task['schedule'].total_seconds()
+        scheduler.every(total_seconds).seconds.do(apply_async)
+    scheduler.run_continuously()

awx/main/dispatch/pool.py

@@ -1,7 +1,9 @@
 import logging
 import os
 import random
+import signal
 import sys
+import time
 import traceback
 from uuid import uuid4
 
@@ -72,9 +74,6 @@ class PoolWorker(object):
             if not body.get('uuid'):
                 body['uuid'] = str(uuid4())
             uuid = body['uuid']
-        logger.debug('delivered {} to worker[{}] qsize {}'.format(
-            uuid, self.pid, self.qsize
-        ))
         self.managed_tasks[uuid] = body
         self.queue.put(body, block=True, timeout=5)
         self.messages_sent += 1
@@ -123,8 +122,16 @@ class PoolWorker(object):
         # if any tasks were finished, removed them from the managed tasks for
         # this worker
         for uuid in finished:
-            self.messages_finished += 1
-            del self.managed_tasks[uuid]
+            try:
+                del self.managed_tasks[uuid]
+                self.messages_finished += 1
+            except KeyError:
+                # ansible _sometimes_ appears to send events w/ duplicate UUIDs;
+                # UUIDs for ansible events are *not* actually globally unique
+                # when this occurs, it's _fine_ to ignore this KeyError because
+                # the purpose of self.managed_tasks is to just track internal
+                # state of which events are *currently* being processed.
+                logger.warn('Event UUID {} appears to be have been duplicated.'.format(uuid))
 
     @property
     def current_task(self):
@@ -239,7 +246,7 @@ class WorkerPool(object):
             ' qsize={{ w.managed_tasks|length }}'
             ' rss={{ w.mb }}MB'
             '{% for task in w.managed_tasks.values() %}'
-            '\n     - {% if loop.index0 == 0 %}running {% else %}queued {% endif %}'
+            '\n     - {% if loop.index0 == 0 %}running {% if "age" in task %}for: {{ "%.1f" % task["age"] }}s {% endif %}{% else %}queued {% endif %}'
             '{{ task["uuid"] }} '
             '{% if "task" in task %}'
             '{{ task["task"].rsplit(".", 1)[-1] }}'
@@ -269,7 +276,7 @@ class WorkerPool(object):
                 logger.warn("could not write to queue %s" % preferred_queue)
                 logger.warn("detail: {}".format(tb))
             write_attempt_order.append(preferred_queue)
-        logger.warn("could not write payload to any queue, attempted order: {}".format(write_attempt_order))
+        logger.error("could not write payload to any queue, attempted order: {}".format(write_attempt_order))
         return None
 
     def stop(self, signum):
@@ -360,6 +367,26 @@ class AutoscalePool(WorkerPool):
                 logger.warn('scaling down worker pid:{}'.format(w.pid))
                 w.quit()
                 self.workers.remove(w)
+            if w.alive:
+                # if we discover a task manager invocation that's been running
+                # too long, reap it (because otherwise it'll just hold the postgres
+                # advisory lock forever); the goal of this code is to discover
+                # deadlocks or other serious issues in the task manager that cause
+                # the task manager to never do more work
+                current_task = w.current_task
+                if current_task and isinstance(current_task, dict):
+                    if current_task.get('task', '').endswith('tasks.run_task_manager'):
+                        if 'started' not in current_task:
+                            w.managed_tasks[
+                                current_task['uuid']
+                            ]['started'] = time.time()
+                        age = time.time() - current_task['started']
+                        w.managed_tasks[current_task['uuid']]['age'] = age
+                        if age > (60 * 5):
+                            logger.error(
+                                f'run_task_manager has held the advisory lock for >5m, sending SIGTERM to {w.pid}'
+                            )  # noqa
+                            os.kill(w.pid, signal.SIGTERM)
 
         for m in orphaned:
             # if all the workers are dead, spawn at least one

awx/main/dispatch/publish.py

@@ -1,12 +1,12 @@
 import inspect
 import logging
 import sys
+import json
 from uuid import uuid4
 
 from django.conf import settings
-from kombu import Exchange, Producer
 
-from awx.main.dispatch.kombu import Connection
+from . import pg_bus_conn
 
 logger = logging.getLogger('awx.main.dispatch')
 
@@ -39,24 +39,22 @@ class task:
     add.apply_async([1, 1])
     Adder.apply_async([1, 1])
 
-    # Tasks can also define a specific target queue or exchange type:
+    # Tasks can also define a specific target queue or use the special fan-out queue tower_broadcast:
 
     @task(queue='slow-tasks')
     def snooze():
         time.sleep(10)
 
-    @task(queue='tower_broadcast', exchange_type='fanout')
+    @task(queue='tower_broadcast')
     def announce():
         print("Run this everywhere!")
     """
 
-    def __init__(self, queue=None, exchange_type=None):
+    def __init__(self, queue=None):
         self.queue = queue
-        self.exchange_type = exchange_type
 
     def __call__(self, fn=None):
         queue = self.queue
-        exchange_type = self.exchange_type
 
         class PublisherMixin(object):
 
@@ -73,9 +71,12 @@ class task:
                 kwargs = kwargs or {}
                 queue = (
                     queue or
-                    getattr(cls.queue, 'im_func', cls.queue) or
-                    settings.CELERY_DEFAULT_QUEUE
+                    getattr(cls.queue, 'im_func', cls.queue)
                 )
+                if not queue:
+                    msg = f'{cls.name}: Queue value required and may not be None'
+                    logger.error(msg)
+                    raise ValueError(msg)
                 obj = {
                     'uuid': task_id,
                     'args': args,
@@ -86,21 +87,8 @@ class task:
                 if callable(queue):
                     queue = queue()
                 if not settings.IS_TESTING(sys.argv):
-                    with Connection(settings.BROKER_URL) as conn:
-                        exchange = Exchange(queue, type=exchange_type or 'direct')
-                        producer = Producer(conn)
-                        logger.debug('publish {}({}, queue={})'.format(
-                            cls.name,
-                            task_id,
-                            queue
-                        ))
-                        producer.publish(obj,
-                                         serializer='json',
-                                         compression='bzip2',
-                                         exchange=exchange,
-                                         declare=[exchange],
-                                         delivery_mode="persistent",
-                                         routing_key=queue)
+                    with pg_bus_conn() as conn:
+                        conn.notify(queue, json.dumps(obj))
                 return (obj, queue)
 
         # If the object we're wrapping *is* a class (e.g., RunJob), return

awx/main/dispatch/worker/__init__.py

@@ -1,3 +1,3 @@
-from .base import AWXConsumer, BaseWorker  # noqa
+from .base import AWXConsumerRedis, AWXConsumerPG, BaseWorker  # noqa
 from .callback import CallbackBrokerWorker  # noqa
 from .task import TaskWorker  # noqa

awx/main/dispatch/worker/base.py

@@ -5,14 +5,17 @@ import os
 import logging
 import signal
 import sys
+import redis
+import json
+import psycopg2
 from uuid import UUID
 from queue import Empty as QueueEmpty
 
 from django import db
-from kombu import Producer
-from kombu.mixins import ConsumerMixin
+from django.conf import settings
 
 from awx.main.dispatch.pool import WorkerPool
+from awx.main.dispatch import pg_bus_conn
 
 if 'run_callback_receiver' in sys.argv:
     logger = logging.getLogger('awx.main.commands.run_callback_receiver')
@@ -37,10 +40,11 @@ class WorkerSignalHandler:
         self.kill_now = True
 
 
-class AWXConsumer(ConsumerMixin):
+class AWXConsumerBase(object):
+    def __init__(self, name, worker, queues=[], pool=None):
+        self.should_stop = False
 
-    def __init__(self, name, connection, worker, queues=[], pool=None):
-        self.connection = connection
+        self.name = name
         self.total_messages = 0
         self.queues = queues
         self.worker = worker
@@ -49,25 +53,15 @@ class AWXConsumer(ConsumerMixin):
             self.pool = WorkerPool()
         self.pool.init_workers(self.worker.work_loop)
 
-    def get_consumers(self, Consumer, channel):
-        logger.debug(self.listening_on)
-        return [Consumer(queues=self.queues, accept=['json'],
-                         callbacks=[self.process_task])]
-
     @property
     def listening_on(self):
-        return 'listening on {}'.format([
-            '{} [{}]'.format(q.name, q.exchange.type) for q in self.queues
-        ])
+        return f'listening on {self.queues}'
 
-    def control(self, body, message):
+    def control(self, body):
         logger.warn(body)
         control = body.get('control')
         if control in ('status', 'running'):
-            producer = Producer(
-                channel=self.connection,
-                routing_key=message.properties['reply_to']
-            )
+            reply_queue = body['reply_to']
             if control == 'status':
                 msg = '\n'.join([self.listening_on, self.pool.debug()])
             elif control == 'running':
@@ -75,20 +69,21 @@ class AWXConsumer(ConsumerMixin):
                 for worker in self.pool.workers:
                     worker.calculate_managed_tasks()
                     msg.extend(worker.managed_tasks.keys())
-            producer.publish(msg)
+
+            with pg_bus_conn() as conn:
+                conn.notify(reply_queue, json.dumps(msg))
         elif control == 'reload':
             for worker in self.pool.workers:
                 worker.quit()
         else:
             logger.error('unrecognized control message: {}'.format(control))
-        message.ack()
 
-    def process_task(self, body, message):
+    def process_task(self, body):
         if 'control' in body:
             try:
-                return self.control(body, message)
+                return self.control(body)
             except Exception:
-                logger.exception("Exception handling control message:")
+                logger.exception(f"Exception handling control message: {body}")
                 return
         if len(self.pool):
             if "uuid" in body and body['uuid']:
@@ -102,23 +97,68 @@ class AWXConsumer(ConsumerMixin):
             queue = 0
         self.pool.write(queue, body)
         self.total_messages += 1
-        message.ack()
 
     def run(self, *args, **kwargs):
         signal.signal(signal.SIGINT, self.stop)
         signal.signal(signal.SIGTERM, self.stop)
-        self.worker.on_start()
-        super(AWXConsumer, self).run(*args, **kwargs)
+
+        # Child should implement other things here
 
     def stop(self, signum, frame):
-        self.should_stop = True  # this makes the kombu mixin stop consuming
+        self.should_stop = True
         logger.warn('received {}, stopping'.format(signame(signum)))
         self.worker.on_stop()
         raise SystemExit()
 
 
+class AWXConsumerRedis(AWXConsumerBase):
+    def run(self, *args, **kwargs):
+        super(AWXConsumerRedis, self).run(*args, **kwargs)
+        self.worker.on_start()
+
+        queue = redis.Redis.from_url(settings.BROKER_URL)
+        while True:
+            try:
+                res = queue.blpop(self.queues)
+                res = json.loads(res[1])
+                self.process_task(res)
+            except redis.exceptions.RedisError:
+                logger.exception(f"encountered an error communicating with redis")
+            except (json.JSONDecodeError, KeyError):
+                logger.exception(f"failed to decode JSON message from redis")
+            if self.should_stop:
+                return
+
+
+class AWXConsumerPG(AWXConsumerBase):
+    def run(self, *args, **kwargs):
+        super(AWXConsumerPG, self).run(*args, **kwargs)
+
+        logger.warn(f"Running worker {self.name} listening to queues {self.queues}")
+        init = False
+
+        while True:
+            try:
+                with pg_bus_conn() as conn:
+                    for queue in self.queues:
+                        conn.listen(queue)
+                    if init is False:
+                        self.worker.on_start()
+                        init = True
+                    for e in conn.events():
+                        self.process_task(json.loads(e.payload))
+                    if self.should_stop:
+                        return
+            except psycopg2.InterfaceError:
+                logger.warn("Stale Postgres message bus connection, reconnecting")
+                continue
+
+
 class BaseWorker(object):
 
+    def read(self, queue):
+        return queue.get(block=True, timeout=1)
+
     def work_loop(self, queue, finished, idx, *args):
         ppid = os.getppid()
         signal_handler = WorkerSignalHandler()
@@ -128,7 +168,7 @@ class BaseWorker(object):
             if os.getppid() != ppid:
                 break
             try:
-                body = queue.get(block=True, timeout=1)
+                body = self.read(queue)
                 if body == 'QUIT':
                     break
             except QueueEmpty:
@@ -145,7 +185,6 @@ class BaseWorker(object):
             finally:
                 if 'uuid' in body:
                     uuid = body['uuid']
-                    logger.debug('task {} is finished'.format(uuid))
                     finished.put(uuid)
         logger.warn('worker exiting gracefully pid:{}'.format(os.getpid()))
 

awx/main/dispatch/worker/callback.py

@@ -1,19 +1,33 @@
+import cProfile
 import logging
+import os
+import pstats
+import signal
+import tempfile
 import time
 import traceback
+from queue import Empty as QueueEmpty
 
 from django.conf import settings
+from django.utils.timezone import now as tz_now
 from django.db import DatabaseError, OperationalError, connection as django_connection
-from django.db.utils import InterfaceError, InternalError
+from django.db.utils import InterfaceError, InternalError, IntegrityError
 
 from awx.main.consumers import emit_channel_notification
 from awx.main.models import (JobEvent, AdHocCommandEvent, ProjectUpdateEvent,
-                             InventoryUpdateEvent, SystemJobEvent, UnifiedJob)
+                             InventoryUpdateEvent, SystemJobEvent, UnifiedJob,
+                             Job)
+from awx.main.tasks import handle_success_and_failure_notifications
+from awx.main.models.events import emit_event_detail
 
 from .base import BaseWorker
 
 logger = logging.getLogger('awx.main.commands.run_callback_receiver')
 
+# the number of seconds to buffer events in memory before flushing
+# using JobEvent.objects.bulk_create()
+BUFFER_SECONDS = .1
+
 
 class CallbackBrokerWorker(BaseWorker):
     '''
@@ -25,90 +39,129 @@ class CallbackBrokerWorker(BaseWorker):
     '''
 
     MAX_RETRIES = 2
+    prof = None
+
+    def __init__(self):
+        self.buff = {}
+
+    def read(self, queue):
+        try:
+            return queue.get(block=True, timeout=BUFFER_SECONDS)
+        except QueueEmpty:
+            return {'event': 'FLUSH'}
+
+    def toggle_profiling(self, *args):
+        if self.prof:
+            self.prof.disable()
+            filename = f'callback-{os.getpid()}.pstats'
+            filepath = os.path.join(tempfile.gettempdir(), filename)
+            with open(filepath, 'w') as f:
+                pstats.Stats(self.prof, stream=f).sort_stats('cumulative').print_stats()
+            pstats.Stats(self.prof).dump_stats(filepath + '.raw')
+            self.prof = False
+            logger.error(f'profiling is disabled, wrote {filepath}')
+        else:
+            self.prof = cProfile.Profile()
+            self.prof.enable()
+            logger.error('profiling is enabled')
+
+    def work_loop(self, *args, **kw):
+        if settings.AWX_CALLBACK_PROFILE:
+            signal.signal(signal.SIGUSR1, self.toggle_profiling)
+        return super(CallbackBrokerWorker, self).work_loop(*args, **kw)
+
+    def flush(self, force=False):
+        now = tz_now()
+        if (
+            force or
+            any([len(events) >= 1000 for events in self.buff.values()])
+        ):
+            for cls, events in self.buff.items():
+                logger.debug(f'{cls.__name__}.objects.bulk_create({len(events)})')
+                for e in events:
+                    if not e.created:
+                        e.created = now
+                    e.modified = now
+                try:
+                    cls.objects.bulk_create(events)
+                except Exception as exc:
+                    # if an exception occurs, we should re-attempt to save the
+                    # events one-by-one, because something in the list is
+                    # broken/stale (e.g., an IntegrityError on a specific event)
+                    for e in events:
+                        try:
+                            if (
+                                isinstance(exc, IntegrityError),
+                                getattr(e, 'host_id', '')
+                            ):
+                                # this is one potential IntegrityError we can
+                                # work around - if the host disappears before
+                                # the event can be processed
+                                e.host_id = None
+                            e.save()
+                        except Exception:
+                            logger.exception('Database Error Saving Job Event')
+                for e in events:
+                    emit_event_detail(e)
+            self.buff = {}
 
     def perform_work(self, body):
         try:
-            event_map = {
-                'job_id': JobEvent,
-                'ad_hoc_command_id': AdHocCommandEvent,
-                'project_update_id': ProjectUpdateEvent,
-                'inventory_update_id': InventoryUpdateEvent,
-                'system_job_id': SystemJobEvent,
-            }
+            flush = body.get('event') == 'FLUSH'
+            if not flush:
+                event_map = {
+                    'job_id': JobEvent,
+                    'ad_hoc_command_id': AdHocCommandEvent,
+                    'project_update_id': ProjectUpdateEvent,
+                    'inventory_update_id': InventoryUpdateEvent,
+                    'system_job_id': SystemJobEvent,
+                }
 
-            if not any([key in body for key in event_map]):
-                raise Exception('Payload does not have a job identifier')
-
-            def _save_event_data():
+                job_identifier = 'unknown job'
                 for key, cls in event_map.items():
                     if key in body:
-                        cls.create_from_data(**body)
+                        job_identifier = body[key]
+                        break
 
-            job_identifier = 'unknown job'
-            job_key = 'unknown'
-            for key in event_map.keys():
-                if key in body:
-                    job_identifier = body[key]
-                    job_key = key
-                    break
-
-            if settings.DEBUG:
-                from pygments import highlight
-                from pygments.lexers import PythonLexer
-                from pygments.formatters import Terminal256Formatter
-                from pprint import pformat
                 if body.get('event') == 'EOF':
-                    event_thing = 'EOF event'
-                else:
-                    event_thing = 'event {}'.format(body.get('counter', 'unknown'))
-                logger.info('Callback worker received {} for {} {}'.format(
-                    event_thing, job_key[:-len('_id')], job_identifier
-                ))
-                logger.debug('Body: {}'.format(
-                    highlight(pformat(body, width=160), PythonLexer(), Terminal256Formatter(style='friendly'))
-                )[:1024 * 4])
+                    try:
+                        final_counter = body.get('final_counter', 0)
+                        logger.info('Event processing is finished for Job {}, sending notifications'.format(job_identifier))
+                        # EOF events are sent when stdout for the running task is
+                        # closed. don't actually persist them to the database; we
+                        # just use them to report `summary` websocket events as an
+                        # approximation for when a job is "done"
+                        emit_channel_notification(
+                            'jobs-summary',
+                            dict(group_name='jobs', unified_job_id=job_identifier, final_counter=final_counter)
+                        )
+                        # Additionally, when we've processed all events, we should
+                        # have all the data we need to send out success/failure
+                        # notification templates
+                        uj = UnifiedJob.objects.get(pk=job_identifier)
 
-            if body.get('event') == 'EOF':
-                try:
-                    final_counter = body.get('final_counter', 0)
-                    logger.info('Event processing is finished for Job {}, sending notifications'.format(job_identifier))
-                    # EOF events are sent when stdout for the running task is
-                    # closed. don't actually persist them to the database; we
-                    # just use them to report `summary` websocket events as an
-                    # approximation for when a job is "done"
-                    emit_channel_notification(
-                        'jobs-summary',
-                        dict(group_name='jobs', unified_job_id=job_identifier, final_counter=final_counter)
-                    )
-                    # Additionally, when we've processed all events, we should
-                    # have all the data we need to send out success/failure
-                    # notification templates
-                    uj = UnifiedJob.objects.get(pk=job_identifier)
-                    if hasattr(uj, 'send_notification_templates'):
-                        retries = 0
-                        while retries < 5:
-                            if uj.finished:
-                                uj.send_notification_templates('succeeded' if uj.status == 'successful' else 'failed')
-                                break
-                            else:
-                                # wait a few seconds to avoid a race where the
-                                # events are persisted _before_ the UJ.status
-                                # changes from running -> successful
-                                retries += 1
-                                time.sleep(1)
-                                uj = UnifiedJob.objects.get(pk=job_identifier)
-                except Exception:
-                    logger.exception('Worker failed to emit notifications: Job {}'.format(job_identifier))
-                return
+                        if isinstance(uj, Job):
+                            # *actual playbooks* send their success/failure
+                            # notifications in response to the playbook_on_stats
+                            # event handling code in main.models.events
+                            pass
+                        elif hasattr(uj, 'send_notification_templates'):
+                            handle_success_and_failure_notifications.apply_async([uj.id])
+                    except Exception:
+                        logger.exception('Worker failed to emit notifications: Job {}'.format(job_identifier))
+                    return
+
+                event = cls.create_from_data(**body)
+                self.buff.setdefault(cls, []).append(event)
 
             retries = 0
             while retries <= self.MAX_RETRIES:
                 try:
-                    _save_event_data()
+                    self.flush(force=flush)
                     break
                 except (OperationalError, InterfaceError, InternalError):
                     if retries >= self.MAX_RETRIES:
-                        logger.exception('Worker could not re-establish database connectivity, giving up on event for Job {}'.format(job_identifier))
+                        logger.exception('Worker could not re-establish database connectivity, giving up on one or more events.')
                         return
                     delay = 60 * retries
                     logger.exception('Database Error Saving Job Event, retry #{i} in {delay} seconds:'.format(
@@ -119,7 +172,7 @@ class CallbackBrokerWorker(BaseWorker):
                     time.sleep(delay)
                     retries += 1
                 except DatabaseError:
-                    logger.exception('Database Error Saving Job Event for Job {}'.format(job_identifier))
+                    logger.exception('Database Error Saving Job Event')
                     break
         except Exception as exc:
             tb = traceback.format_exc()

awx/main/dispatch/worker/task.py

@@ -4,6 +4,7 @@ import importlib
 import sys
 import traceback
 
+from kubernetes.config import kube_config
 
 from awx.main.tasks import dispatch_startup, inform_cluster_of_shutdown
 
@@ -107,6 +108,14 @@ class TaskWorker(BaseWorker):
             for callback in body.get('errbacks', []) or []:
                 callback['uuid'] = body['uuid']
                 self.perform_work(callback)
+        finally:
+            # It's frustrating that we have to do this, but the python k8s
+            # client leaves behind cacert files in /tmp, so we must clean up
+            # the tmpdir per-dispatcher process every time a new task comes in
+            try:
+                kube_config._cleanup_temp_files()
+            except Exception:
+                logger.exception('failed to cleanup k8s client tmp files')
 
         for callback in body.get('callbacks', []) or []:
             callback['uuid'] = body['uuid']

awx/main/fields.py

@@ -56,7 +56,8 @@ from awx.main import utils
 
 __all__ = ['AutoOneToOneField', 'ImplicitRoleField', 'JSONField',
            'SmartFilterField', 'OrderedManyToManyField',
-           'update_role_parentage_for_instance', 'is_implicit_parent']
+           'update_role_parentage_for_instance',
+           'is_implicit_parent']
 
 
 # Provide a (better) custom error message for enum jsonschema validation
@@ -140,8 +141,9 @@ def resolve_role_field(obj, field):
         return []
 
     if len(field_components) == 1:
-        role_cls = str(utils.get_current_apps().get_model('main', 'Role'))
-        if not str(type(obj)) == role_cls:
+        # use extremely generous duck typing to accomidate all possible forms
+        # of the model that may be used during various migrations
+        if obj._meta.model_name != 'role' or obj._meta.app_label != 'main':
             raise Exception(smart_text('{} refers to a {}, not a Role'.format(field, type(obj))))
         ret.append(obj.id)
     else:
@@ -197,18 +199,27 @@ def update_role_parentage_for_instance(instance):
     updates the parents listing for all the roles
     of a given instance if they have changed
     '''
+    parents_removed = set()
+    parents_added = set()
     for implicit_role_field in getattr(instance.__class__, '__implicit_role_fields'):
         cur_role = getattr(instance, implicit_role_field.name)
         original_parents = set(json.loads(cur_role.implicit_parents))
         new_parents = implicit_role_field._resolve_parent_roles(instance)
-        cur_role.parents.remove(*list(original_parents - new_parents))
-        cur_role.parents.add(*list(new_parents - original_parents))
+        removals = original_parents - new_parents
+        if removals:
+            cur_role.parents.remove(*list(removals))
+            parents_removed.add(cur_role.pk)
+        additions = new_parents - original_parents
+        if additions:
+            cur_role.parents.add(*list(additions))
+            parents_added.add(cur_role.pk)
         new_parents_list = list(new_parents)
         new_parents_list.sort()
         new_parents_json = json.dumps(new_parents_list)
         if cur_role.implicit_parents != new_parents_json:
             cur_role.implicit_parents = new_parents_json
-            cur_role.save()
+            cur_role.save(update_fields=['implicit_parents'])
+    return (parents_added, parents_removed)
 
 
 class ImplicitRoleDescriptor(ForwardManyToOneDescriptor):
@@ -256,20 +267,18 @@ class ImplicitRoleField(models.ForeignKey):
             field_names = [field_names]
 
         for field_name in field_names:
-            # Handle the OR syntax for role parents
-            if type(field_name) == tuple:
-                continue
-
-            if type(field_name) == bytes:
-                field_name = field_name.decode('utf-8')
 
             if field_name.startswith('singleton:'):
                 continue
 
             field_name, sep, field_attr = field_name.partition('.')
-            field = getattr(cls, field_name)
+            # Non existent fields will occur if ever a parent model is
+            # moved inside a migration, needed for job_template_organization_field
+            # migration in particular
+            # consistency is assured by unit test awx.main.tests.functional
+            field = getattr(cls, field_name, None)
 
-            if type(field) is ReverseManyToOneDescriptor or \
+            if field and type(field) is ReverseManyToOneDescriptor or \
                type(field) is ManyToManyDescriptor:
 
                 if '.' in field_attr:

awx/main/isolated/manager.py

@@ -6,6 +6,7 @@ import stat
 import tempfile
 import time
 import logging
+import yaml
 
 from django.conf import settings
 import ansible_runner
@@ -14,7 +15,6 @@ import awx
 from awx.main.utils import (
     get_system_task_capacity
 )
-from awx.main.queue import CallbackQueueDispatcher
 
 logger = logging.getLogger('awx.isolated.manager')
 playbook_logger = logging.getLogger('awx.isolated.manager.playbooks')
@@ -31,15 +31,16 @@ def set_pythonpath(venv_libdir, env):
 
 class IsolatedManager(object):
 
-    def __init__(self, cancelled_callback=None, check_callback=None, pod_manager=None):
+    def __init__(self, event_handler, canceled_callback=None, check_callback=None, pod_manager=None):
         """
-        :param cancelled_callback:  a callable - which returns `True` or `False`
+        :param event_handler: a callable used to persist event data from isolated nodes
+        :param canceled_callback:  a callable - which returns `True` or `False`
                                     - signifying if the job has been prematurely
-                                      cancelled
+                                      canceled
         """
-        self.cancelled_callback = cancelled_callback
+        self.event_handler = event_handler
+        self.canceled_callback = canceled_callback
         self.check_callback = check_callback
-        self.idle_timeout = max(60, 2 * settings.AWX_ISOLATED_CONNECTION_TIMEOUT)
         self.started_at = None
         self.captured_command_artifact = False
         self.instance = None
@@ -48,10 +49,17 @@ class IsolatedManager(object):
     def build_inventory(self, hosts):
         if self.instance and self.instance.is_containerized:
             inventory = {'all': {'hosts': {}}}
+            fd, path = tempfile.mkstemp(
+                prefix='.kubeconfig', dir=self.private_data_dir
+            )
+            with open(path, 'wb') as temp:
+                temp.write(yaml.dump(self.pod_manager.kube_config).encode())
+                temp.flush()
+                os.chmod(temp.name, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR)
             for host in hosts:
                 inventory['all']['hosts'][host] = {
                     "ansible_connection": "kubectl",
-                    "ansible_kubectl_config": self.pod_manager.kube_config
+                    "ansible_kubectl_config": path,
                 }
         else:
             inventory = '\n'.join([
@@ -98,9 +106,8 @@ class IsolatedManager(object):
             'envvars': env,
             'finished_callback': finished_callback,
             'verbosity': verbosity,
-            'cancel_callback': self.cancelled_callback,
+            'cancel_callback': self.canceled_callback,
             'settings': {
-                'idle_timeout': self.idle_timeout,
                 'job_timeout': settings.AWX_ISOLATED_LAUNCH_TIMEOUT,
                 'pexpect_timeout': getattr(settings, 'PEXPECT_TIMEOUT', 5),
                 'suppress_ansible_output': True,
@@ -110,7 +117,7 @@ class IsolatedManager(object):
     def path_to(self, *args):
         return os.path.join(self.private_data_dir, *args)
 
-    def run_management_playbook(self, playbook, private_data_dir, **kw):
+    def run_management_playbook(self, playbook, private_data_dir, idle_timeout=None, **kw):
         iso_dir = tempfile.mkdtemp(
             prefix=playbook,
             dir=private_data_dir
@@ -118,6 +125,10 @@ class IsolatedManager(object):
         params = self.runner_params.copy()
         params['playbook'] = playbook
         params['private_data_dir'] = iso_dir
+        if idle_timeout:
+            params['settings']['idle_timeout'] = idle_timeout
+        else:
+            params['settings'].pop('idle_timeout', None)
         params.update(**kw)
         if all([
             getattr(settings, 'AWX_ISOLATED_KEY_GENERATION', False) is True,
@@ -143,6 +154,8 @@ class IsolatedManager(object):
             '- /artifacts/job_events/*-partial.json.tmp',
             # don't rsync the ssh_key FIFO
             '- /env/ssh_key',
+            # don't rsync kube config files
+            '- .kubeconfig*'
         ]
 
         for filename, data in (
@@ -167,6 +180,7 @@ class IsolatedManager(object):
         logger.debug('Starting job {} on isolated host with `run_isolated.yml` playbook.'.format(self.instance.id))
         runner_obj = self.run_management_playbook('run_isolated.yml',
                                                   self.private_data_dir,
+                                                  idle_timeout=max(60, 2 * settings.AWX_ISOLATED_CONNECTION_TIMEOUT),
                                                   extravars=extravars)
 
         if runner_obj.status == 'failed':
@@ -195,17 +209,16 @@ class IsolatedManager(object):
         status = 'failed'
         rc = None
         last_check = time.time()
-        dispatcher = CallbackQueueDispatcher()
 
         while status == 'failed':
-            canceled = self.cancelled_callback() if self.cancelled_callback else False
+            canceled = self.canceled_callback() if self.canceled_callback else False
             if not canceled and time.time() - last_check < interval:
-                # If the job isn't cancelled, but we haven't waited `interval` seconds, wait longer
+                # If the job isn't canceled, but we haven't waited `interval` seconds, wait longer
                 time.sleep(1)
                 continue
 
             if canceled:
-                logger.warning('Isolated job {} was manually cancelled.'.format(self.instance.id))
+                logger.warning('Isolated job {} was manually canceled.'.format(self.instance.id))
 
             logger.debug('Checking on isolated job {} with `check_isolated.yml`.'.format(self.instance.id))
             runner_obj = self.run_management_playbook('check_isolated.yml',
@@ -225,7 +238,7 @@ class IsolatedManager(object):
                     except json.decoder.JSONDecodeError:  # Just in case it's not fully here yet.
                         pass
 
-            self.consume_events(dispatcher)
+            self.consume_events()
 
             last_check = time.time()
 
@@ -253,19 +266,11 @@ class IsolatedManager(object):
 
         # consume events one last time just to be sure we didn't miss anything
         # in the final sync
-        self.consume_events(dispatcher)
-
-        # emit an EOF event
-        event_data = {
-            'event': 'EOF',
-            'final_counter': len(self.handled_events)
-        }
-        event_data.setdefault(self.event_data_key, self.instance.id)
-        dispatcher.dispatch(event_data)
+        self.consume_events()
 
         return status, rc
 
-    def consume_events(self, dispatcher):
+    def consume_events(self):
         # discover new events and ingest them
         events_path = self.path_to('artifacts', self.ident, 'job_events')
 
@@ -275,7 +280,7 @@ class IsolatedManager(object):
         if os.path.exists(events_path):
             for event in set(os.listdir(events_path)) - self.handled_events:
                 path = os.path.join(events_path, event)
-                if os.path.exists(path):
+                if os.path.exists(path) and os.path.isfile(path):
                     try:
                         event_data = json.load(
                             open(os.path.join(events_path, event), 'r')
@@ -289,16 +294,10 @@ class IsolatedManager(object):
                         # practice
                         # in this scenario, just ignore this event and try it
                         # again on the next sync
-                        pass
-                    event_data.setdefault(self.event_data_key, self.instance.id)
-                    dispatcher.dispatch(event_data)
+                        continue
+                    self.event_handler(event_data)
                     self.handled_events.add(event)
 
-                    # handle artifacts
-                    if event_data.get('event_data', {}).get('artifact_data', {}):
-                        self.instance.artifacts = event_data['event_data']['artifact_data']
-                        self.instance.save(update_fields=['artifacts'])
-
 
     def cleanup(self):
         extravars = {
@@ -357,39 +356,37 @@ class IsolatedManager(object):
                 private_data_dir
             )
 
-            if runner_obj.status == 'successful':
-                for instance in instance_qs:
-                    task_result = {}
-                    try:
-                        task_result = runner_obj.get_fact_cache(instance.hostname)
-                    except Exception:
-                        logger.exception('Failed to read status from isolated instances')
-                    if 'awx_capacity_cpu' in task_result and 'awx_capacity_mem' in task_result:
-                        task_result = {
-                            'cpu': task_result['awx_cpu'],
-                            'mem': task_result['awx_mem'],
-                            'capacity_cpu': task_result['awx_capacity_cpu'],
-                            'capacity_mem': task_result['awx_capacity_mem'],
-                            'version': task_result['awx_capacity_version']
-                        }
-                        IsolatedManager.update_capacity(instance, task_result)
-                        logger.debug('Isolated instance {} successful heartbeat'.format(instance.hostname))
-                    elif instance.capacity == 0:
-                        logger.debug('Isolated instance {} previously marked as lost, could not re-join.'.format(
-                            instance.hostname))
-                    else:
-                        logger.warning('Could not update status of isolated instance {}'.format(instance.hostname))
-                        if instance.is_lost(isolated=True):
-                            instance.capacity = 0
-                            instance.save(update_fields=['capacity'])
-                            logger.error('Isolated instance {} last checked in at {}, marked as lost.'.format(
-                                instance.hostname, instance.modified))
+            for instance in instance_qs:
+                task_result = {}
+                try:
+                    task_result = runner_obj.get_fact_cache(instance.hostname)
+                except Exception:
+                    logger.exception('Failed to read status from isolated instances')
+                if 'awx_capacity_cpu' in task_result and 'awx_capacity_mem' in task_result:
+                    task_result = {
+                        'cpu': task_result['awx_cpu'],
+                        'mem': task_result['awx_mem'],
+                        'capacity_cpu': task_result['awx_capacity_cpu'],
+                        'capacity_mem': task_result['awx_capacity_mem'],
+                        'version': task_result['awx_capacity_version']
+                    }
+                    IsolatedManager.update_capacity(instance, task_result)
+                    logger.debug('Isolated instance {} successful heartbeat'.format(instance.hostname))
+                elif instance.capacity == 0:
+                    logger.debug('Isolated instance {} previously marked as lost, could not re-join.'.format(
+                        instance.hostname))
+                else:
+                    logger.warning('Could not update status of isolated instance {}'.format(instance.hostname))
+                    if instance.is_lost(isolated=True):
+                        instance.capacity = 0
+                        instance.save(update_fields=['capacity'])
+                        logger.error('Isolated instance {} last checked in at {}, marked as lost.'.format(
+                            instance.hostname, instance.modified))
         finally:
             if os.path.exists(private_data_dir):
                 shutil.rmtree(private_data_dir)
 
-    def run(self, instance, private_data_dir, playbook, module, module_args,
-            event_data_key, ident=None):
+    def run(self, instance, private_data_dir, playbook, module, module_args, ident=None):
         """
         Run a job on an isolated host.
 
@@ -400,14 +397,12 @@ class IsolatedManager(object):
         :param playbook:         the playbook to run
         :param module:           the module to run
         :param module_args:      the module args to use
-        :param event_data_key:   e.g., job_id, inventory_id, ...
 
         For a completed job run, this function returns (status, rc),
         representing the status and return code of the isolated
         `ansible-playbook` run.
         """
         self.ident = ident
-        self.event_data_key = event_data_key
         self.instance = instance
         self.private_data_dir = private_data_dir
         self.runner_params = self.build_runner_params(
@@ -418,9 +413,4 @@ class IsolatedManager(object):
         status, rc = self.dispatch(playbook, module, module_args)
         if status == 'successful':
             status, rc = self.check()
-        else:
-            # emit an EOF event
-            event_data = {'event': 'EOF', 'final_counter': 0}
-            event_data.setdefault(self.event_data_key, self.instance.id)
-            CallbackQueueDispatcher().dispatch(event_data)
         return status, rc

awx/main/management/commands/callback_stats.py

@@ -0,0 +1,40 @@
+import time
+import sys
+
+from django.db import connection
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+
+    def handle(self, *args, **options):
+        with connection.cursor() as cursor:
+            start = {}
+            for relation in (
+                'main_jobevent', 'main_inventoryupdateevent',
+                'main_projectupdateevent', 'main_adhoccommandevent'
+            ):
+                cursor.execute(f"SELECT MAX(id) FROM {relation};")
+                start[relation] = cursor.fetchone()[0] or 0
+            clear = False
+            while True:
+                lines = []
+                for relation in (
+                    'main_jobevent', 'main_inventoryupdateevent',
+                    'main_projectupdateevent', 'main_adhoccommandevent'
+                ):
+                    lines.append(relation)
+                    minimum = start[relation]
+                    cursor.execute(
+                        f"SELECT MAX(id) - MIN(id) FROM {relation} WHERE id > {minimum} AND modified > now() - '1 minute'::interval;"
+                    )
+                    events = cursor.fetchone()[0] or 0
+                    lines.append(f'↳  last minute {events}')
+                    lines.append('')
+                if clear:
+                    for i in range(12):
+                        sys.stdout.write('\x1b[1A\x1b[2K')
+                for l in lines:
+                    print(l)
+                clear = True
+                time.sleep(.25)

awx/main/management/commands/cleanup_jobs.py

@@ -16,13 +16,12 @@ from awx.main.models import (
     Job, AdHocCommand, ProjectUpdate, InventoryUpdate,
     SystemJob, WorkflowJob, Notification
 )
-from awx.main.signals import ( # noqa
-    emit_update_inventory_on_created_or_deleted,
-    emit_update_inventory_computed_fields,
+from awx.main.signals import (
     disable_activity_stream,
     disable_computed_fields
 )
-from django.db.models.signals import post_save, post_delete, m2m_changed # noqa
+
+from awx.main.management.commands.deletion import AWXCollector, pre_delete
 
 
 class Command(BaseCommand):
@@ -60,27 +59,37 @@ class Command(BaseCommand):
                             action='store_true', dest='only_workflow_jobs',
                             help='Remove workflow jobs')
 
-    def cleanup_jobs(self):
-        #jobs_qs = Job.objects.exclude(status__in=('pending', 'running'))
-        #jobs_qs = jobs_qs.filter(created__lte=self.cutoff)
-        skipped, deleted = 0, 0
-        jobs = Job.objects.filter(created__lt=self.cutoff)
-        for job in jobs.iterator():
-            job_display = '"%s" (%d host summaries, %d events)' % \
-                          (str(job),
-                           job.job_host_summaries.count(), job.job_events.count())
-            if job.status in ('pending', 'waiting', 'running'):
-                action_text = 'would skip' if self.dry_run else 'skipping'
-                self.logger.debug('%s %s job %s', action_text, job.status, job_display)
-                skipped += 1
-            else:
-                action_text = 'would delete' if self.dry_run else 'deleting'
-                self.logger.info('%s %s', action_text, job_display)
-                if not self.dry_run:
-                    job.delete()
-                deleted += 1
 
-        skipped += Job.objects.filter(created__gte=self.cutoff).count()
+    def cleanup_jobs(self):
+        skipped, deleted = 0, 0
+
+        batch_size = 1000000
+
+        while True:
+            # get queryset for available jobs to remove
+            qs = Job.objects.filter(created__lt=self.cutoff).exclude(status__in=['pending', 'waiting', 'running'])
+            # get pk list for the first N (batch_size) objects
+            pk_list = qs[0:batch_size].values_list('pk')
+            # You cannot delete queries with sql LIMIT set, so we must
+            # create a new query from this pk_list
+            qs_batch = Job.objects.filter(pk__in=pk_list)
+            just_deleted = 0
+            if not self.dry_run:
+                del_query = pre_delete(qs_batch)
+                collector = AWXCollector(del_query.db)
+                collector.collect(del_query)
+                _, models_deleted = collector.delete()
+                if models_deleted:
+                    just_deleted = models_deleted['main.Job']
+                deleted += just_deleted
+            else:
+                just_deleted = 0 # break from loop, this is dry run
+                deleted = qs.count()
+
+            if just_deleted == 0:
+                break
+
+        skipped += (Job.objects.filter(created__gte=self.cutoff) | Job.objects.filter(status__in=['pending', 'waiting', 'running'])).count()
         return skipped, deleted
 
     def cleanup_ad_hoc_commands(self):

awx/main/management/commands/deletion.py

@@ -0,0 +1,177 @@
+from django.contrib.contenttypes.models import ContentType
+from django.db.models.deletion import (
+    DO_NOTHING, Collector, get_candidate_relations_to_delete,
+)
+from collections import Counter, OrderedDict
+from django.db import transaction
+from django.db.models import sql
+
+
+def bulk_related_objects(field, objs, using):
+    # This overrides the method in django.contrib.contenttypes.fields.py
+    """
+    Return all objects related to ``objs`` via this ``GenericRelation``.
+    """
+    return field.remote_field.model._base_manager.db_manager(using).filter(**{
+        "%s__pk" % field.content_type_field_name: ContentType.objects.db_manager(using).get_for_model(
+            field.model, for_concrete_model=field.for_concrete_model).pk,
+        "%s__in" % field.object_id_field_name: list(objs.values_list('pk', flat=True))
+    })
+
+
+def pre_delete(qs):
+    # taken from .delete method in django.db.models.query.py
+    assert qs.query.can_filter(), \
+        "Cannot use 'limit' or 'offset' with delete."
+
+    if qs._fields is not None:
+        raise TypeError("Cannot call delete() after .values() or .values_list()")
+
+    del_query = qs._chain()
+
+    # The delete is actually 2 queries - one to find related objects,
+    # and one to delete. Make sure that the discovery of related
+    # objects is performed on the same database as the deletion.
+    del_query._for_write = True
+
+    # Disable non-supported fields.
+    del_query.query.select_for_update = False
+    del_query.query.select_related = False
+    del_query.query.clear_ordering(force_empty=True)
+    return del_query
+
+
+class AWXCollector(Collector):
+
+    def add(self, objs, source=None, nullable=False, reverse_dependency=False):
+        """
+        Add 'objs' to the collection of objects to be deleted.  If the call is
+        the result of a cascade, 'source' should be the model that caused it,
+        and 'nullable' should be set to True if the relation can be null.
+
+        Return a list of all objects that were not already collected.
+        """
+        if not objs.exists():
+            return objs
+        model = objs.model
+        self.data.setdefault(model, [])
+        self.data[model].append(objs)
+        # Nullable relationships can be ignored -- they are nulled out before
+        # deleting, and therefore do not affect the order in which objects have
+        # to be deleted.
+        if source is not None and not nullable:
+            if reverse_dependency:
+                source, model = model, source
+            self.dependencies.setdefault(
+                source._meta.concrete_model, set()).add(model._meta.concrete_model)
+        return objs
+
+    def add_field_update(self, field, value, objs):
+        """
+        Schedule a field update. 'objs' must be a homogeneous iterable
+        collection of model instances (e.g. a QuerySet).
+        """
+        if not objs.exists():
+            return
+        model = objs.model
+        self.field_updates.setdefault(model, {})
+        self.field_updates[model].setdefault((field, value), [])
+        self.field_updates[model][(field, value)].append(objs)
+
+    def collect(self, objs, source=None, nullable=False, collect_related=True,
+                source_attr=None, reverse_dependency=False, keep_parents=False):
+        """
+        Add 'objs' to the collection of objects to be deleted as well as all
+        parent instances.  'objs' must be a homogeneous iterable collection of
+        model instances (e.g. a QuerySet).  If 'collect_related' is True,
+        related objects will be handled by their respective on_delete handler.
+
+        If the call is the result of a cascade, 'source' should be the model
+        that caused it and 'nullable' should be set to True, if the relation
+        can be null.
+
+        If 'reverse_dependency' is True, 'source' will be deleted before the
+        current model, rather than after. (Needed for cascading to parent
+        models, the one case in which the cascade follows the forwards
+        direction of an FK rather than the reverse direction.)
+
+        If 'keep_parents' is True, data of parent model's will be not deleted.
+        """
+
+        if hasattr(objs, 'polymorphic_disabled'):
+            objs.polymorphic_disabled = True
+
+        if self.can_fast_delete(objs):
+            self.fast_deletes.append(objs)
+            return
+        new_objs = self.add(objs, source, nullable,
+                            reverse_dependency=reverse_dependency)
+        if not new_objs.exists():
+            return
+
+        model = new_objs.model
+
+        if not keep_parents:
+            # Recursively collect concrete model's parent models, but not their
+            # related objects. These will be found by meta.get_fields()
+            concrete_model = model._meta.concrete_model
+            for ptr in concrete_model._meta.parents.keys():
+                if ptr:
+                    parent_objs = ptr.objects.filter(pk__in = new_objs.values_list('pk', flat=True))
+                    self.collect(parent_objs, source=model,
+                                 collect_related=False,
+                                 reverse_dependency=True)
+        if collect_related:
+            parents = model._meta.parents
+            for related in get_candidate_relations_to_delete(model._meta):
+                # Preserve parent reverse relationships if keep_parents=True.
+                if keep_parents and related.model in parents:
+                    continue
+                field = related.field
+                if field.remote_field.on_delete == DO_NOTHING:
+                    continue
+                related_qs = self.related_objects(related, new_objs)
+                if self.can_fast_delete(related_qs, from_field=field):
+                    self.fast_deletes.append(related_qs)
+                elif related_qs:
+                    field.remote_field.on_delete(self, field, related_qs, self.using)
+            for field in model._meta.private_fields:
+                if hasattr(field, 'bulk_related_objects'):
+                    # It's something like generic foreign key.
+                    sub_objs = bulk_related_objects(field, new_objs, self.using)
+                    self.collect(sub_objs, source=model, nullable=True)
+
+    def delete(self):
+        self.sort()
+
+        # collect pk_list before deletion (once things start to delete
+        # queries might not be able to retreive pk list)
+        del_dict = OrderedDict()
+        for model, instances in self.data.items():
+            del_dict.setdefault(model, [])
+            for inst in instances:
+                del_dict[model] += list(inst.values_list('pk', flat=True))
+
+        deleted_counter = Counter()
+
+        with transaction.atomic(using=self.using, savepoint=False):
+
+            # update fields
+            for model, instances_for_fieldvalues in self.field_updates.items():
+                for (field, value), instances in instances_for_fieldvalues.items():
+                    for inst in instances:
+                        query = sql.UpdateQuery(model)
+                        query.update_batch(inst.values_list('pk', flat=True),
+                                           {field.name: value}, self.using)
+            # fast deletes
+            for qs in self.fast_deletes:
+                count = qs._raw_delete(using=self.using)
+                deleted_counter[qs.model._meta.label] += count
+
+            # delete instances
+            for model, pk_list in del_dict.items():
+                query = sql.DeleteQuery(model)
+                count = query.delete_batch(pk_list, self.using)
+                deleted_counter[model._meta.label] += count
+
+            return sum(deleted_counter.values()), dict(deleted_counter)

awx/main/management/commands/deprovision_instance.py

@@ -1,8 +1,6 @@
 # Copyright (c) 2016 Ansible, Inc.
 # All Rights Reserved
 
-import subprocess
-
 from django.db import transaction
 from django.core.management.base import BaseCommand, CommandError
 
@@ -33,18 +31,9 @@ class Command(BaseCommand):
         with advisory_lock('instance_registration_%s' % hostname):
             instance = Instance.objects.filter(hostname=hostname)
             if instance.exists():
-                isolated = instance.first().is_isolated()
                 instance.delete()
                 print("Instance Removed")
-                if isolated:
-                    print('Successfully deprovisioned {}'.format(hostname))
-                else:
-                    result = subprocess.Popen("rabbitmqctl forget_cluster_node rabbitmq@{}".format(hostname), shell=True).wait()
-                    if result != 0:
-                        print("Node deprovisioning may have failed when attempting to "
-                              "remove the RabbitMQ instance {} from the cluster".format(hostname))
-                    else:
-                        print('Successfully deprovisioned {}'.format(hostname))
+                print('Successfully deprovisioned {}'.format(hostname))
                 print('(changed: True)')
             else:
                 print('No instance found matching name {}'.format(hostname))

awx/main/management/commands/inventory_import.py

@@ -28,6 +28,7 @@ from awx.main.models.inventory import (
     Host
 )
 from awx.main.utils.mem_inventory import MemInventory, dict_to_mem_data
+from awx.main.utils.safe_yaml import sanitize_jinja
 
 # other AWX imports
 from awx.main.models.rbac import batch_role_ancestor_rebuilding
@@ -495,12 +496,6 @@ class Command(BaseCommand):
             group_names = all_group_names[offset:(offset + self._batch_size)]
             for group_pk in groups_qs.filter(name__in=group_names).values_list('pk', flat=True):
                 del_group_pks.discard(group_pk)
-        if self.inventory_source.deprecated_group_id in del_group_pks:  # TODO: remove in 3.3
-            logger.warning(
-                'Group "%s" from v1 API is not deleted by overwrite',
-                self.inventory_source.deprecated_group.name
-            )
-            del_group_pks.discard(self.inventory_source.deprecated_group_id)
         # Now delete all remaining groups in batches.
         all_del_pks = sorted(list(del_group_pks))
         for offset in range(0, len(all_del_pks), self._batch_size):
@@ -533,12 +528,6 @@ class Command(BaseCommand):
         # Set of all host pks managed by this inventory source
         all_source_host_pks = self._existing_host_pks()
         for db_group in db_groups.all():
-            if self.inventory_source.deprecated_group_id == db_group.id:  # TODO: remove in 3.3
-                logger.debug(
-                    'Group "%s" from v1 API child group/host connections preserved',
-                    db_group.name
-                )
-                continue
             # Delete child group relationships not present in imported data.
             db_children = db_group.children
             db_children_name_pk_map = dict(db_children.values_list('name', 'pk'))
@@ -795,6 +784,10 @@ class Command(BaseCommand):
             if self.instance_id_var:
                 instance_id = self._get_instance_id(mem_host.variables)
                 host_attrs['instance_id'] = instance_id
+            try:
+                sanitize_jinja(mem_host_name)
+            except ValueError as e:
+                raise ValueError(str(e) + ': {}'.format(mem_host_name))
             db_host = self.inventory.hosts.update_or_create(name=mem_host_name, defaults=host_attrs)[0]
             if enabled is False:
                 logger.debug('Host "%s" added (disabled)', mem_host_name)
@@ -916,11 +909,14 @@ class Command(BaseCommand):
         available_instances = license_info.get('available_instances', 0)
         free_instances = license_info.get('free_instances', 0)
         time_remaining = license_info.get('time_remaining', 0)
+        hard_error = license_info.get('trial', False) is True or license_info['instance_count'] == 10
         new_count = Host.objects.active_count()
-        if time_remaining <= 0 and not license_info.get('demo', False):
-            logger.error(LICENSE_EXPIRED_MESSAGE)
-            if license_info.get('trial', False) is True:
+        if time_remaining <= 0:
+            if hard_error:
+                logger.error(LICENSE_EXPIRED_MESSAGE)
                 raise CommandError("License has expired!")
+            else:
+                logger.warning(LICENSE_EXPIRED_MESSAGE)
         # special check for tower-type inventory sources
         # but only if running the plugin
         TOWER_SOURCE_FILES = ['tower.yml', 'tower.yaml']
@@ -933,15 +929,11 @@ class Command(BaseCommand):
                 'new_count': new_count,
                 'available_instances': available_instances,
             }
-            if license_info.get('demo', False):
-                logger.error(DEMO_LICENSE_MESSAGE % d)
-            else:
+            if hard_error:
                 logger.error(LICENSE_MESSAGE % d)
-            if (
-                license_info.get('trial', False) is True or
-                license_info['instance_count'] == 10  # basic 10 license
-            ):
                 raise CommandError('License count exceeded!')
+            else:
+                logger.warning(LICENSE_MESSAGE % d)
 
     def check_org_host_limit(self):
         license_info = get_licenser().validate()
@@ -1002,12 +994,6 @@ class Command(BaseCommand):
         except re.error:
             raise CommandError('invalid regular expression for --host-filter')
 
-        '''
-        TODO: Remove this deprecation when we remove support for rax.py
-        '''
-        if self.source == "rax.py":
-            logger.info("Rackspace inventory sync is Deprecated in Tower 3.1.0 and support for Rackspace will be removed in a future release.")
-
         begin = time.time()
         self.load_inventory_from_database()
 

awx/main/management/commands/provision_instance.py

@@ -1,6 +1,8 @@
 # Copyright (c) 2015 Ansible, Inc.
 # All Rights Reserved
 
+from uuid import uuid4
+
 from awx.main.models import Instance
 from django.conf import settings
 
@@ -22,6 +24,8 @@ class Command(BaseCommand):
     def add_arguments(self, parser):
         parser.add_argument('--hostname', dest='hostname', type=str,
                             help='Hostname used during provisioning')
+        parser.add_argument('--is-isolated', dest='is_isolated', action='store_true',
+                            help='Specify whether the instance is isolated')
 
     def _register_hostname(self, hostname):
         if not hostname:
@@ -37,7 +41,10 @@ class Command(BaseCommand):
     def handle(self, **options):
         if not options.get('hostname'):
             raise CommandError("Specify `--hostname` to use this command.")
-        self.uuid = settings.SYSTEM_UUID
+        if options['is_isolated']:
+            self.uuid = str(uuid4())
+        else:
+            self.uuid = settings.SYSTEM_UUID
         self.changed = False
         self._register_hostname(options.get('hostname'))
         if self.changed:

awx/main/management/commands/regenerate_secret_key.py

@@ -0,0 +1,129 @@
+import base64
+import json
+import os
+
+from django.core.management.base import BaseCommand
+from django.conf import settings
+from django.db import transaction
+from django.db.models.signals import post_save
+
+from awx.conf import settings_registry
+from awx.conf.models import Setting
+from awx.conf.signals import on_post_save_setting
+from awx.main.models import (
+    UnifiedJob, Credential, NotificationTemplate, Job, JobTemplate, WorkflowJob,
+    WorkflowJobTemplate, OAuth2Application
+)
+from awx.main.utils.encryption import (
+    encrypt_field, decrypt_field, encrypt_value, decrypt_value, get_encryption_key
+)
+
+
+class Command(BaseCommand):
+    """
+    Regenerate a new SECRET_KEY value and re-encrypt every secret in the
+    Tower database.
+    """
+
+    @transaction.atomic
+    def handle(self, **options):
+        self.old_key = settings.SECRET_KEY
+        self.new_key = base64.encodebytes(os.urandom(33)).decode().rstrip()
+        self._notification_templates()
+        self._credentials()
+        self._unified_jobs()
+        self._oauth2_app_secrets()
+        self._settings()
+        self._survey_passwords()
+        return self.new_key
+
+    def _notification_templates(self):
+        for nt in NotificationTemplate.objects.iterator():
+            CLASS_FOR_NOTIFICATION_TYPE = dict([(x[0], x[2]) for x in NotificationTemplate.NOTIFICATION_TYPES])
+            notification_class = CLASS_FOR_NOTIFICATION_TYPE[nt.notification_type]
+            for field in filter(lambda x: notification_class.init_parameters[x]['type'] == "password",
+                                notification_class.init_parameters):
+                nt.notification_configuration[field] = decrypt_field(nt, 'notification_configuration', subfield=field, secret_key=self.old_key)
+                nt.notification_configuration[field] = encrypt_field(nt, 'notification_configuration', subfield=field, secret_key=self.new_key)
+            nt.save()
+
+    def _credentials(self):
+        for credential in Credential.objects.iterator():
+            for field_name in credential.credential_type.secret_fields:
+                if field_name in credential.inputs:
+                    credential.inputs[field_name] = decrypt_field(
+                        credential,
+                        field_name,
+                        secret_key=self.old_key
+                    )
+                    credential.inputs[field_name] = encrypt_field(
+                        credential,
+                        field_name,
+                        secret_key=self.new_key
+                    )
+                credential.save()
+
+    def _unified_jobs(self):
+        for uj in UnifiedJob.objects.iterator():
+            if uj.start_args:
+                uj.start_args = decrypt_field(
+                    uj,
+                    'start_args',
+                    secret_key=self.old_key
+                )
+                uj.start_args = encrypt_field(uj, 'start_args', secret_key=self.new_key)
+                uj.save()
+
+    def _oauth2_app_secrets(self):
+        for app in OAuth2Application.objects.iterator():
+            raw = app.client_secret
+            app.client_secret = raw
+            encrypted = encrypt_value(raw, secret_key=self.new_key)
+            OAuth2Application.objects.filter(pk=app.pk).update(client_secret=encrypted)
+
+    def _settings(self):
+        # don't update memcached, the *actual* value isn't changing
+        post_save.disconnect(on_post_save_setting, sender=Setting)
+        for setting in Setting.objects.filter().order_by('pk'):
+            if settings_registry.is_setting_encrypted(setting.key):
+                setting.value = decrypt_field(setting, 'value', secret_key=self.old_key)
+                setting.value = encrypt_field(setting, 'value', secret_key=self.new_key)
+                setting.save()
+
+    def _survey_passwords(self):
+        for _type in (JobTemplate, WorkflowJobTemplate):
+            for jt in _type.objects.exclude(survey_spec={}):
+                changed = False
+                if jt.survey_spec.get('spec', []):
+                    for field in jt.survey_spec['spec']:
+                        if field.get('type') == 'password' and field.get('default', ''):
+                            raw = decrypt_value(
+                                get_encryption_key('value', None, secret_key=self.old_key),
+                                field['default']
+                            )
+                            field['default'] = encrypt_value(
+                                raw,
+                                pk=None,
+                                secret_key=self.new_key
+                            )
+                            changed = True
+                if changed:
+                    jt.save(update_fields=["survey_spec"])
+
+        for _type in (Job, WorkflowJob):
+            for job in _type.objects.exclude(survey_passwords={}).iterator():
+                changed = False
+                for key in job.survey_passwords:
+                    if key in job.extra_vars:
+                        extra_vars = json.loads(job.extra_vars)
+                        if not extra_vars.get(key):
+                            continue
+                        raw = decrypt_value(
+                            get_encryption_key('value', None, secret_key=self.old_key),
+                            extra_vars[key]
+                        )
+                        extra_vars[key] = encrypt_value(raw, pk=None, secret_key=self.new_key)
+                        job.extra_vars = json.dumps(extra_vars)
+                        changed = True
+                if changed:
+                    job.save(update_fields=["extra_vars"])

awx/main/management/commands/register_queue.py

@@ -6,6 +6,7 @@ from awx.main.utils.pglock import advisory_lock
 from awx.main.models import Instance, InstanceGroup
 
 from django.core.management.base import BaseCommand, CommandError
+from django.db import transaction
 
 
 class InstanceNotFound(Exception):
@@ -31,7 +32,6 @@ class Command(BaseCommand):
 
 
     def get_create_update_instance_group(self, queuename, instance_percent, instance_min):
-        ig = InstanceGroup.objects.filter(name=queuename)
         created = False
         changed = False
 
@@ -98,26 +98,27 @@ class Command(BaseCommand):
         if options.get('hostnames'):
             hostname_list = options.get('hostnames').split(",")
 
-        with advisory_lock('instance_group_registration_{}'.format(queuename)):
-            changed2 = False
-            changed3 = False
-            (ig, created, changed1) = self.get_create_update_instance_group(queuename, inst_per, inst_min)
-            if created:
-                print("Creating instance group {}".format(ig.name))
-            elif not created:
-                print("Instance Group already registered {}".format(ig.name))
+        with advisory_lock('cluster_policy_lock'):
+            with transaction.atomic():
+                changed2 = False
+                changed3 = False
+                (ig, created, changed1) = self.get_create_update_instance_group(queuename, inst_per, inst_min)
+                if created:
+                    print("Creating instance group {}".format(ig.name))
+                elif not created:
+                    print("Instance Group already registered {}".format(ig.name))
 
-            if ctrl:
-                (ig_ctrl, changed2) = self.update_instance_group_controller(ig, ctrl)
-                if changed2:
-                    print("Set controller group {} on {}.".format(ctrl, queuename))
+                if ctrl:
+                    (ig_ctrl, changed2) = self.update_instance_group_controller(ig, ctrl)
+                    if changed2:
+                        print("Set controller group {} on {}.".format(ctrl, queuename))
 
-            try:
-                (instances, changed3) = self.add_instances_to_group(ig, hostname_list)
-                for i in instances:
-                    print("Added instance {} to {}".format(i.hostname, ig.name))
-            except InstanceNotFound as e:
-                instance_not_found_err = e
+                try:
+                    (instances, changed3) = self.add_instances_to_group(ig, hostname_list)
+                    for i in instances:
+                        print("Added instance {} to {}".format(i.hostname, ig.name))
+                except InstanceNotFound as e:
+                    instance_not_found_err = e
 
         if any([changed1, changed2, changed3]):
             print('(changed: True)')

awx/main/management/commands/replay_job_events.py

@@ -9,6 +9,7 @@ import random
 from django.utils import timezone
 from django.core.management.base import BaseCommand
 
+from awx.main.models.events import emit_event_detail
 from awx.main.models import (
     UnifiedJob,
     Job,
@@ -17,14 +18,6 @@ from awx.main.models import (
     InventoryUpdate,
     SystemJob
 )
-from awx.main.consumers import emit_channel_notification
-from awx.api.serializers import (
-    JobEventWebSocketSerializer,
-    AdHocCommandEventWebSocketSerializer,
-    ProjectUpdateEventWebSocketSerializer,
-    InventoryUpdateEventWebSocketSerializer,
-    SystemJobEventWebSocketSerializer
-)
 
 
 class JobStatusLifeCycle():
@@ -96,21 +89,6 @@ class ReplayJobEvents(JobStatusLifeCycle):
             raise RuntimeError("No events for job id {}".format(job.id))
         return job_events, count
 
-    def get_serializer(self, job):
-        if type(job) is Job:
-            return JobEventWebSocketSerializer
-        elif type(job) is AdHocCommand:
-            return AdHocCommandEventWebSocketSerializer
-        elif type(job) is ProjectUpdate:
-            return ProjectUpdateEventWebSocketSerializer
-        elif type(job) is InventoryUpdate:
-            return InventoryUpdateEventWebSocketSerializer
-        elif type(job) is SystemJob:
-            return SystemJobEventWebSocketSerializer
-        else:
-            raise RuntimeError("Job is of type {} and replay is not yet supported.".format(type(job)))
-            sys.exit(1)
-
     def run(self, job_id, speed=1.0, verbosity=0, skip_range=[], random_seed=0, final_status_delay=0, debug=False):
         stats = {
             'events_ontime': {
@@ -136,7 +114,6 @@ class ReplayJobEvents(JobStatusLifeCycle):
         try:
             job = self.get_job(job_id)
             job_events, job_event_count = self.get_job_events(job)
-            serializer = self.get_serializer(job)
         except RuntimeError as e:
             print("{}".format(e.message))
             sys.exit(1)
@@ -162,8 +139,7 @@ class ReplayJobEvents(JobStatusLifeCycle):
                 stats['replay_start'] = self.replay_start
                 je_previous = je_current
 
-            je_serialized = serializer(je_current).data
-            emit_channel_notification('{}-{}'.format(je_serialized['group_name'], job.id), je_serialized)
+            emit_event_detail(je_current)
 
             replay_offset = self.replay_offset(je_previous.created, speed)
             recording_diff = (je_current.created - je_previous.created).total_seconds() * (1.0 / speed)

awx/main/management/commands/run_callback_receiver.py

@@ -3,10 +3,8 @@
 
 from django.conf import settings
 from django.core.management.base import BaseCommand
-from kombu import Exchange, Queue
 
-from awx.main.dispatch.kombu import Connection
-from awx.main.dispatch.worker import AWXConsumer, CallbackBrokerWorker
+from awx.main.dispatch.worker import AWXConsumerRedis, CallbackBrokerWorker
 
 
 class Command(BaseCommand):
@@ -18,23 +16,15 @@ class Command(BaseCommand):
     help = 'Launch the job callback receiver'
 
     def handle(self, *arg, **options):
-        with Connection(settings.BROKER_URL) as conn:
-            consumer = None
-            try:
-                consumer = AWXConsumer(
-                    'callback_receiver',
-                    conn,
-                    CallbackBrokerWorker(),
-                    [
-                        Queue(
-                            settings.CALLBACK_QUEUE,
-                            Exchange(settings.CALLBACK_QUEUE, type='direct'),
-                            routing_key=settings.CALLBACK_QUEUE
-                        )
-                    ]
-                )
-                consumer.run()
-            except KeyboardInterrupt:
-                print('Terminating Callback Receiver')
-                if consumer:
-                    consumer.stop()
+        consumer = None
+        try:
+            consumer = AWXConsumerRedis(
+                'callback_receiver',
+                CallbackBrokerWorker(),
+                queues=[getattr(settings, 'CALLBACK_QUEUE', '')],
+            )
+            consumer.run()
+        except KeyboardInterrupt:
+            print('Terminating Callback Receiver')
+            if consumer:
+                consumer.stop()

awx/main/management/commands/run_dispatcher.py

@@ -1,21 +1,17 @@
 # Copyright (c) 2015 Ansible, Inc.
 # All Rights Reserved.
-import os
 import logging
-from multiprocessing import Process
 
 from django.conf import settings
 from django.core.cache import cache as django_cache
 from django.core.management.base import BaseCommand
-from django.db import connection as django_connection, connections
-from kombu import Exchange, Queue
+from django.db import connection as django_connection
 
-from awx.main.utils.handlers import AWXProxyHandler
 from awx.main.dispatch import get_local_queuename, reaper
 from awx.main.dispatch.control import Control
-from awx.main.dispatch.kombu import Connection
 from awx.main.dispatch.pool import AutoscalePool
-from awx.main.dispatch.worker import AWXConsumer, TaskWorker
+from awx.main.dispatch.worker import AWXConsumerPG, TaskWorker
+from awx.main.dispatch import periodic
 
 logger = logging.getLogger('awx.main.dispatch')
 
@@ -36,71 +32,6 @@ class Command(BaseCommand):
                             help=('cause the dispatcher to recycle all of its worker processes;'
                                   'running jobs will run to completion first'))
 
-    def beat(self):
-        from celery import Celery
-        from celery.beat import PersistentScheduler
-        from celery.apps import beat
-
-        class AWXScheduler(PersistentScheduler):
-
-            def __init__(self, *args, **kwargs):
-                self.ppid = os.getppid()
-                super(AWXScheduler, self).__init__(*args, **kwargs)
-
-            def setup_schedule(self):
-                super(AWXScheduler, self).setup_schedule()
-                self.update_from_dict(settings.CELERYBEAT_SCHEDULE)
-
-            def tick(self, *args, **kwargs):
-                if os.getppid() != self.ppid:
-                    # if the parent PID changes, this process has been orphaned
-                    # via e.g., segfault or sigkill, we should exit too
-                    raise SystemExit()
-                return super(AWXScheduler, self).tick(*args, **kwargs)
-
-            def apply_async(self, entry, producer=None, advance=True, **kwargs):
-                for conn in connections.all():
-                    # If the database connection has a hiccup, re-establish a new
-                    # connection
-                    conn.close_if_unusable_or_obsolete()
-                task = TaskWorker.resolve_callable(entry.task)
-                result, queue = task.apply_async()
-
-                class TaskResult(object):
-                    id = result['uuid']
-
-                return TaskResult()
-
-        sched_file = '/var/lib/awx/beat.db'
-        app = Celery()
-        app.conf.BROKER_URL = settings.BROKER_URL
-        app.conf.CELERY_TASK_RESULT_EXPIRES = False
-
-        # celery in py3 seems to have a bug where the celerybeat schedule
-        # shelve can become corrupted; we've _only_ seen this in Ubuntu and py36
-        # it can be avoided by detecting and removing the corrupted file
-        # at some point, we'll just stop using celerybeat, because it's clearly
-        # buggy, too -_-
-        #
-        # https://github.com/celery/celery/issues/4777
-        sched = AWXScheduler(schedule_filename=sched_file, app=app)
-        try:
-            sched.setup_schedule()
-        except Exception:
-            logger.exception('{} is corrupted, removing.'.format(sched_file))
-            sched._remove_db()
-        finally:
-            try:
-                sched.close()
-            except Exception:
-                logger.exception('{} failed to sync/close'.format(sched_file))
-
-        beat.Beat(
-            30,
-            app,
-            schedule=sched_file, scheduler_cls=AWXScheduler
-        ).run()
-
     def handle(self, *arg, **options):
         if options.get('status'):
             print(Control('dispatcher').status())
@@ -116,42 +47,24 @@ class Command(BaseCommand):
         # for the DB and memcached connections (that way lies race conditions)
         django_connection.close()
         django_cache.close()
-        beat = Process(target=self.beat)
-        beat.daemon = True
-        beat.start()
+
+        # spawn a daemon thread to periodically enqueues scheduled tasks
+        # (like the node heartbeat)
+        periodic.run_continuously()
 
         reaper.reap()
         consumer = None
 
-        # don't ship external logs inside the dispatcher's parent process
-        # this exists to work around a race condition + deadlock bug on fork
-        # in cpython itself:
-        # https://bugs.python.org/issue37429
-        AWXProxyHandler.disable()
-        with Connection(settings.BROKER_URL) as conn:
-            try:
-                bcast = 'tower_broadcast_all'
-                queues = [
-                    Queue(q, Exchange(q), routing_key=q)
-                    for q in (settings.AWX_CELERY_QUEUES_STATIC + [get_local_queuename()])
-                ]
-                queues.append(
-                    Queue(
-                        construct_bcast_queue_name(bcast),
-                        exchange=Exchange(bcast, type='fanout'),
-                        routing_key=bcast,
-                        reply=True
-                    )
-                )
-                consumer = AWXConsumer(
-                    'dispatcher',
-                    conn,
-                    TaskWorker(),
-                    queues,
-                    AutoscalePool(min_workers=4)
-                )
-                consumer.run()
-            except KeyboardInterrupt:
-                logger.debug('Terminating Task Dispatcher')
-                if consumer:
-                    consumer.stop()
+        try:
+            queues = ['tower_broadcast_all', get_local_queuename()]
+            consumer = AWXConsumerPG(
+                'dispatcher',
+                TaskWorker(),
+                queues,
+                AutoscalePool(min_workers=4)
+            )
+            consumer.run()
+        except KeyboardInterrupt:
+            logger.debug('Terminating Task Dispatcher')
+            if consumer:
+                consumer.stop()

awx/main/management/commands/run_wsbroadcast.py

@@ -0,0 +1,134 @@
+# Copyright (c) 2015 Ansible, Inc.
+# All Rights Reserved.
+import logging
+import asyncio
+import datetime
+import re
+import redis
+from datetime import datetime as dt
+
+from django.core.management.base import BaseCommand
+from django.db.models import Q
+
+from awx.main.analytics.broadcast_websocket import (
+    BroadcastWebsocketStatsManager,
+    safe_name,
+)
+from awx.main.wsbroadcast import BroadcastWebsocketManager
+from awx.main.models.ha import Instance
+
+
+logger = logging.getLogger('awx.main.wsbroadcast')
+
+
+class Command(BaseCommand):
+    help = 'Launch the websocket broadcaster'
+
+    def add_arguments(self, parser):
+        parser.add_argument('--status', dest='status', action='store_true',
+                            help='print the internal state of any running broadcast websocket')
+
+    @classmethod
+    def display_len(cls, s):
+        return len(re.sub('\x1b.*?m', '', s))
+
+    @classmethod
+    def _format_lines(cls, host_stats, padding=5):
+        widths = [0 for i in host_stats[0]]
+        for entry in host_stats:
+            for i, e in enumerate(entry):
+                if Command.display_len(e) > widths[i]:
+                    widths[i] = Command.display_len(e)
+        paddings = [padding for i in widths]
+
+        lines = []
+        for entry in host_stats:
+            line = ""
+            for pad, width, value in zip(paddings, widths, entry):
+                if len(value) > Command.display_len(value):
+                    width += len(value) - Command.display_len(value)
+                total_width = width + pad
+                line += f'{value:{total_width}}'
+            lines.append(line)
+        return lines
+
+    @classmethod
+    def get_connection_status(cls, me, hostnames, data):
+        host_stats = [('hostname', 'state', 'start time', 'duration (sec)')]
+        for h in hostnames:
+            connection_color = '91'    # red
+            h_safe = safe_name(h)
+            prefix = f'awx_{h_safe}'
+            connection_state = data.get(f'{prefix}_connection', 'N/A')
+            connection_started = 'N/A'
+            connection_duration = 'N/A'
+            if connection_state is None:
+                connection_state = 'unknown'
+            if connection_state == 'connected':
+                connection_color = '92' # green
+                connection_started = data.get(f'{prefix}_connection_start', 'Error')
+                if connection_started != 'Error':
+                    connection_started = datetime.datetime.fromtimestamp(connection_started)
+                    connection_duration = int((dt.now() - connection_started).total_seconds())
+
+            connection_state = f'\033[{connection_color}m{connection_state}\033[0m'
+
+            host_stats.append((h, connection_state, str(connection_started), str(connection_duration)))
+
+        return host_stats
+
+    @classmethod
+    def get_connection_stats(cls, me, hostnames, data):
+        host_stats = [('hostname', 'total', 'per minute')]
+        for h in hostnames:
+            h_safe = safe_name(h)
+            prefix = f'awx_{h_safe}'
+            messages_total = data.get(f'{prefix}_messages_received', '0')
+            messages_per_minute = data.get(f'{prefix}_messages_received_per_minute', '0')
+
+            host_stats.append((h, str(int(messages_total)), str(int(messages_per_minute))))
+
+        return host_stats
+
+    def handle(self, *arg, **options):
+        if options.get('status'):
+            try:
+                stats_all = BroadcastWebsocketStatsManager.get_stats_sync()
+            except redis.exceptions.ConnectionError as e:
+                print(f"Unable to get Broadcast Websocket Status. Failed to connect to redis {e}")
+                return
+
+            data = {}
+            for family in stats_all:
+                if family.type == 'gauge' and len(family.samples) > 1:
+                    for sample in family.samples:
+                        if sample.value >= 1:
+                            data[family.name] = sample.labels[family.name]
+                            break
+                else:
+                    data[family.name] = family.samples[0].value
+            me = Instance.objects.me()
+            hostnames = [i.hostname for i in Instance.objects.exclude(Q(hostname=me.hostname) | Q(rampart_groups__controller__isnull=False))]
+
+            host_stats = Command.get_connection_status(me, hostnames, data)
+            lines = Command._format_lines(host_stats)
+
+            print(f'Broadcast websocket connection status from "{me.hostname}" to:')
+            print('\n'.join(lines))
+
+            host_stats = Command.get_connection_stats(me, hostnames, data)
+            lines = Command._format_lines(host_stats)
+
+            print(f'\nBroadcast websocket connection stats from "{me.hostname}" to:')
+            print('\n'.join(lines))
+
+            return
+
+        try:
+            broadcast_websocket_mgr = BroadcastWebsocketManager()
+            task = broadcast_websocket_mgr.start()
+
+            loop = asyncio.get_event_loop()
+            loop.run_until_complete(task)
+        except KeyboardInterrupt:
+            logger.debug('Terminating Websocket Broadcaster')

awx/main/managers.py

@@ -3,6 +3,7 @@
 
 import sys
 import logging
+import os
 
 from django.db import models
 from django.conf import settings
@@ -78,8 +79,7 @@ class HostManager(models.Manager):
                 self.core_filters = {}
 
                 qs = qs & q
-                unique_by_name = qs.order_by('name', 'pk').distinct('name')
-                return qs.filter(pk__in=unique_by_name)
+                return qs.order_by('name', 'pk').distinct('name')
         return qs
 
 
@@ -115,7 +115,7 @@ class InstanceManager(models.Manager):
             return node[0]
         raise RuntimeError("No instance found with the current cluster host id")
 
-    def register(self, uuid=None, hostname=None):
+    def register(self, uuid=None, hostname=None, ip_address=None):
         if not uuid:
             uuid = settings.SYSTEM_UUID
         if not hostname:
@@ -123,13 +123,23 @@ class InstanceManager(models.Manager):
         with advisory_lock('instance_registration_%s' % hostname):
             instance = self.filter(hostname=hostname)
             if instance.exists():
-                return (False, instance[0])
-            instance = self.create(uuid=uuid, hostname=hostname, capacity=0)
+                instance = instance.get()
+                if instance.ip_address != ip_address:
+                    instance.ip_address = ip_address
+                    instance.save(update_fields=['ip_address'])
+                    return (True, instance)
+                else:
+                    return (False, instance)
+            instance = self.create(uuid=uuid,
+                                   hostname=hostname,
+                                   ip_address=ip_address,
+                                   capacity=0)
         return (True, instance)
 
     def get_or_register(self):
         if settings.AWX_AUTO_DEPROVISION_INSTANCES:
-            return self.register()
+            pod_ip = os.environ.get('MY_POD_IP')
+            return self.register(ip_address=pod_ip)
         else:
             return (False, self.me())
 

awx/main/middleware.py

@@ -12,23 +12,19 @@ import urllib.parse
 
 from django.conf import settings
 from django.contrib.auth.models import User
-from django.db.models.signals import post_save
 from django.db.migrations.executor import MigrationExecutor
-from django.db import IntegrityError, connection
-from django.utils.functional import curry
+from django.db import connection
 from django.shortcuts import get_object_or_404, redirect
 from django.apps import apps
 from django.utils.deprecation import MiddlewareMixin
 from django.utils.translation import ugettext_lazy as _
 from django.urls import reverse, resolve
 
-from awx.main.models import ActivityStream
 from awx.main.utils.named_url_graph import generate_graph, GraphNode
 from awx.conf import fields, register
 
 
 logger = logging.getLogger('awx.main.middleware')
-analytics_logger = logging.getLogger('awx.analytics.activity_stream')
 perf_logger = logging.getLogger('awx.analytics.performance')
 
 
@@ -62,64 +58,20 @@ class TimingMiddleware(threading.local, MiddlewareMixin):
         with open(filepath, 'w') as f:
             f.write('%s %s\n' % (request.method, request.get_full_path()))
             pstats.Stats(self.prof, stream=f).sort_stats('cumulative').print_stats()
+
+        if settings.AWX_REQUEST_PROFILE_WITH_DOT:
+            from gprof2dot import main as generate_dot
+            raw = os.path.join(self.dest, filename) + '.raw'
+            pstats.Stats(self.prof).dump_stats(raw)
+            generate_dot([
+                '-n', '2.5', '-f', 'pstats', '-o',
+                os.path.join( self.dest, filename).replace('.pstats', '.dot'),
+                raw
+            ])
+            os.remove(raw)
         return filepath
 
 
-class ActivityStreamMiddleware(threading.local, MiddlewareMixin):
-
-    def __init__(self, get_response=None):
-        self.disp_uid = None
-        self.instance_ids = []
-        super().__init__(get_response)
-
-    def process_request(self, request):
-        if hasattr(request, 'user') and request.user.is_authenticated:
-            user = request.user
-        else:
-            user = None
-
-        set_actor = curry(self.set_actor, user)
-        self.disp_uid = str(uuid.uuid1())
-        self.instance_ids = []
-        post_save.connect(set_actor, sender=ActivityStream, dispatch_uid=self.disp_uid, weak=False)
-
-    def process_response(self, request, response):
-        drf_request = getattr(request, 'drf_request', None)
-        drf_user = getattr(drf_request, 'user', None)
-        if self.disp_uid is not None:
-            post_save.disconnect(dispatch_uid=self.disp_uid)
-
-        for instance in ActivityStream.objects.filter(id__in=self.instance_ids):
-            if drf_user and drf_user.id:
-                instance.actor = drf_user
-                try:
-                    instance.save(update_fields=['actor'])
-                    analytics_logger.info('Activity Stream update entry for %s' % str(instance.object1),
-                                          extra=dict(changes=instance.changes, relationship=instance.object_relationship_type,
-                                          actor=drf_user.username, operation=instance.operation,
-                                          object1=instance.object1, object2=instance.object2))
-                except IntegrityError:
-                    logger.debug("Integrity Error saving Activity Stream instance for id : " + str(instance.id))
-            # else:
-            #     obj1_type_actual = instance.object1_type.split(".")[-1]
-            #     if obj1_type_actual in ("InventoryUpdate", "ProjectUpdate", "Job") and instance.id is not None:
-            #         instance.delete()
-
-        self.instance_ids = []
-        return response
-
-    def set_actor(self, user, sender, instance, **kwargs):
-        if sender == ActivityStream:
-            if isinstance(user, User) and instance.actor is None:
-                user = User.objects.filter(id=user.id)
-                if user.exists():
-                    user = user[0]
-                    instance.actor = user
-            else:
-                if instance.id not in self.instance_ids:
-                    self.instance_ids.append(instance.id)
-
-
 class SessionTimeoutMiddleware(MiddlewareMixin):
     """
     Resets the session timeout for both the UI and the actual session for the API
@@ -181,21 +133,41 @@ class URLModificationMiddleware(MiddlewareMixin):
         )
         super().__init__(get_response)
 
-    def _named_url_to_pk(self, node, named_url):
-        kwargs = {}
-        if not node.populate_named_url_query_kwargs(kwargs, named_url):
-            return named_url
-        return str(get_object_or_404(node.model, **kwargs).pk)
+    @staticmethod
+    def _hijack_for_old_jt_name(node, kwargs, named_url):
+        try:
+            int(named_url)
+            return False
+        except ValueError:
+            pass
+        JobTemplate = node.model
+        name = urllib.parse.unquote(named_url)
+        return JobTemplate.objects.filter(name=name).order_by('organization__created').first()
 
-    def _convert_named_url(self, url_path):
+    @classmethod
+    def _named_url_to_pk(cls, node, resource, named_url):
+        kwargs = {}
+        if node.populate_named_url_query_kwargs(kwargs, named_url):
+            return str(get_object_or_404(node.model, **kwargs).pk)
+        if resource == 'job_templates' and '++' not in named_url:
+            # special case for deprecated job template case
+            # will not raise a 404 on its own
+            jt = cls._hijack_for_old_jt_name(node, kwargs, named_url)
+            if jt:
+                return str(jt.pk)
+        return named_url
+
+    @classmethod
+    def _convert_named_url(cls, url_path):
         url_units = url_path.split('/')
         # If the identifier is an empty string, it is always invalid.
         if len(url_units) < 6 or url_units[1] != 'api' or url_units[2] not in ['v2'] or not url_units[4]:
             return url_path
         resource = url_units[3]
         if resource in settings.NAMED_URL_MAPPINGS:
-            url_units[4] = self._named_url_to_pk(settings.NAMED_URL_GRAPH[settings.NAMED_URL_MAPPINGS[resource]],
-                                                 url_units[4])
+            url_units[4] = cls._named_url_to_pk(
+                settings.NAMED_URL_GRAPH[settings.NAMED_URL_MAPPINGS[resource]],
+                resource, url_units[4])
         return '/'.join(url_units)
 
     def process_request(self, request):

awx/main/migrations/0006_v320_release.py

@@ -464,7 +464,7 @@ class Migration(migrations.Migration):
         migrations.AddField(
             model_name='unifiedjob',
             name='instance_group',
-            field=models.ForeignKey(on_delete=models.SET_NULL, default=None, blank=True, to='main.InstanceGroup', help_text='The Rampart/Instance group the job was run under', null=True),
+            field=models.ForeignKey(on_delete=models.SET_NULL, default=None, blank=True, to='main.InstanceGroup', help_text='The Instance group the job was run under', null=True),
         ),
         migrations.AddField(
             model_name='unifiedjobtemplate',

awx/main/migrations/0007_v320_data_migrations.py

@@ -7,12 +7,6 @@ from django.db import migrations, models
 
 # AWX
 from awx.main.migrations import ActivityStreamDisabledMigration
-from awx.main.migrations import _inventory_source as invsrc
-from awx.main.migrations import _migration_utils as migration_utils
-from awx.main.migrations import _reencrypt as reencrypt
-from awx.main.migrations import _scan_jobs as scan_jobs
-from awx.main.migrations import _credentialtypes as credentialtypes
-from awx.main.migrations import _azure_credentials as azurecreds
 import awx.main.fields
 
 
@@ -23,16 +17,8 @@ class Migration(ActivityStreamDisabledMigration):
     ]
 
     operations = [
-        # Inventory Refresh
-        migrations.RunPython(migration_utils.set_current_apps_for_migrations),
-        migrations.RunPython(invsrc.remove_rax_inventory_sources),
-        migrations.RunPython(azurecreds.remove_azure_credentials),
-        migrations.RunPython(invsrc.remove_azure_inventory_sources),
-        migrations.RunPython(invsrc.remove_inventory_source_with_no_inventory_link),
-        migrations.RunPython(invsrc.rename_inventory_sources),
-        migrations.RunPython(reencrypt.replace_aesecb_fernet),
-        migrations.RunPython(scan_jobs.migrate_scan_job_templates),
-
-        migrations.RunPython(credentialtypes.migrate_to_v2_credentials),
-        migrations.RunPython(credentialtypes.migrate_job_credentials),
+        # This list is intentionally empty.
+        # Tower 3.2 included several data migrations that are no longer
+        # necessary (this list is now empty because Tower 3.2 is past EOL and
+        # cannot be directly upgraded to modern versions of Tower)
     ]

awx/main/migrations/0010_v322_add_ovirt4_tower_inventory.py

@@ -15,8 +15,6 @@ class Migration(migrations.Migration):
     ]
 
     operations = [
-        migrations.RunPython(migration_utils.set_current_apps_for_migrations),
-        migrations.RunPython(credentialtypes.create_rhv_tower_credtype),
         migrations.AlterField(
             model_name='inventorysource',
             name='source',

awx/main/migrations/0011_v322_encrypt_survey_passwords.py

@@ -3,8 +3,6 @@ from __future__ import unicode_literals
 
 from django.db import migrations
 from awx.main.migrations import ActivityStreamDisabledMigration
-from awx.main.migrations import _reencrypt as reencrypt
-from awx.main.migrations import _migration_utils as migration_utils
 
 
 class Migration(ActivityStreamDisabledMigration):
@@ -14,6 +12,8 @@ class Migration(ActivityStreamDisabledMigration):
     ]
 
     operations = [
-        migrations.RunPython(migration_utils.set_current_apps_for_migrations),
-        migrations.RunPython(reencrypt.encrypt_survey_passwords),
+        # This list is intentionally empty.
+        # Tower 3.2 included several data migrations that are no longer
+        # necessary (this list is now empty because Tower 3.2 is past EOL and
+        # cannot be directly upgraded to modern versions of Tower)
     ]

awx/main/migrations/0012_v322_update_cred_types.py

@@ -1,10 +1,6 @@
 # -*- coding: utf-8 -*-
 from __future__ import unicode_literals
 
-# AWX
-from awx.main.migrations import _migration_utils as migration_utils
-from awx.main.migrations import _credentialtypes as credentialtypes
-
 from django.db import migrations
 
 
@@ -15,6 +11,8 @@ class Migration(migrations.Migration):
     ]
 
     operations = [
-        migrations.RunPython(migration_utils.set_current_apps_for_migrations),
-        migrations.RunPython(credentialtypes.add_azure_cloud_environment_field),
+        # This list is intentionally empty.
+        # Tower 3.2 included several data migrations that are no longer
+        # necessary (this list is now empty because Tower 3.2 is past EOL and
+        # cannot be directly upgraded to modern versions of Tower)
     ]

awx/main/migrations/0032_v330_polymorphic_delete.py

@@ -16,6 +16,6 @@ class Migration(migrations.Migration):
         migrations.AlterField(
             model_name='unifiedjob',
             name='instance_group',
-            field=models.ForeignKey(blank=True, default=None, help_text='The Rampart/Instance group the job was run under', null=True, on_delete=awx.main.utils.polymorphic.SET_NULL, to='main.InstanceGroup'),
+            field=models.ForeignKey(blank=True, default=None, help_text='The Instance group the job was run under', null=True, on_delete=awx.main.utils.polymorphic.SET_NULL, to='main.InstanceGroup'),
         ),
     ]

awx/main/migrations/0078_v360_clear_sessions_tokens_jt.py

@@ -19,11 +19,11 @@ class Migration(migrations.Migration):
         migrations.AlterField(
             model_name='systemjob',
             name='job_type',
-            field=models.CharField(blank=True, choices=[('cleanup_jobs', 'Remove jobs older than a certain number of days'), ('cleanup_activitystream', 'Remove activity stream entries older than a certain number of days'), ('clearsessions', 'Removes expired browser sessions from the database'), ('cleartokens', 'Removes expired OAuth 2 access tokens and refresh tokens')], default='', max_length=32),
+            field=models.CharField(blank=True, choices=[('cleanup_jobs', 'Remove jobs older than a certain number of days'), ('cleanup_activitystream', 'Remove activity stream entries older than a certain number of days'), ('cleanup_sessions', 'Removes expired browser sessions from the database'), ('cleanup_tokens', 'Removes expired OAuth 2 access tokens and refresh tokens')], default='', max_length=32),
         ),
         migrations.AlterField(
             model_name='systemjobtemplate',
             name='job_type',
-            field=models.CharField(blank=True, choices=[('cleanup_jobs', 'Remove jobs older than a certain number of days'), ('cleanup_activitystream', 'Remove activity stream entries older than a certain number of days'), ('clearsessions', 'Removes expired browser sessions from the database'), ('cleartokens', 'Removes expired OAuth 2 access tokens and refresh tokens')], default='', max_length=32),
+            field=models.CharField(blank=True, choices=[('cleanup_jobs', 'Remove jobs older than a certain number of days'), ('cleanup_activitystream', 'Remove activity stream entries older than a certain number of days'), ('cleanup_sessions', 'Removes expired browser sessions from the database'), ('cleanup_tokens', 'Removes expired OAuth 2 access tokens and refresh tokens')], default='', max_length=32),
         ),
     ]

awx/main/migrations/0099_v361_license_cleanup.py

@@ -0,0 +1,21 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations
+
+
+def _cleanup_license_setting(apps, schema_editor):
+    Setting = apps.get_model('conf', 'Setting')
+    for license in Setting.objects.filter(key='LICENSE').all():
+        for k in ('rh_username', 'rh_password'):
+            license.value.pop(k, None)
+        license.save()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0098_v360_rename_cyberark_aim_credential_type'),
+    ]
+
+    operations = [migrations.RunPython(_cleanup_license_setting)]

awx/main/migrations/0100_v370_projectupdate_job_tags.py

@@ -0,0 +1,18 @@
+# Generated by Django 2.2.4 on 2019-11-01 18:46
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0099_v361_license_cleanup'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='projectupdate',
+            name='job_tags',
+            field=models.CharField(blank=True, default='', help_text='Parts of the project update playbook that will be run.', max_length=1024),
+        ),
+    ]

awx/main/migrations/0101_v370_generate_new_uuids_for_iso_nodes.py

@@ -0,0 +1,26 @@
+# -*- coding: utf-8 -*-
+from uuid import uuid4
+
+from django.db import migrations
+
+
+def _generate_new_uuid_for_iso_nodes(apps, schema_editor):
+    Instance = apps.get_model('main', 'Instance')
+    for instance in Instance.objects.all():
+        # The below code is a copy paste of instance.is_isolated()
+        # We can't call is_isolated because we are using the "old" version
+        # of the Instance definition.
+        if instance.rampart_groups.filter(controller__isnull=False).exists():
+            instance.uuid = str(uuid4())
+            instance.save()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0100_v370_projectupdate_job_tags'),
+    ]
+
+    operations = [
+        migrations.RunPython(_generate_new_uuid_for_iso_nodes)
+    ]

awx/main/migrations/0102_v370_unifiedjob_canceled.py

@@ -0,0 +1,18 @@
+# Generated by Django 2.2.4 on 2019-11-25 20:53
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0101_v370_generate_new_uuids_for_iso_nodes'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='unifiedjob',
+            name='canceled_on',
+            field=models.DateTimeField(db_index=True, default=None, editable=False, help_text='The date and time when the cancel request was sent.', null=True),
+        ),
+    ]

awx/main/migrations/0103_v370_remove_computed_fields.py

@@ -0,0 +1,52 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11.16 on 2019-02-21 17:35
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0102_v370_unifiedjob_canceled'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='group',
+            name='groups_with_active_failures',
+        ),
+        migrations.RemoveField(
+            model_name='group',
+            name='has_active_failures',
+        ),
+        migrations.RemoveField(
+            model_name='group',
+            name='has_inventory_sources',
+        ),
+        migrations.RemoveField(
+            model_name='group',
+            name='hosts_with_active_failures',
+        ),
+        migrations.RemoveField(
+            model_name='group',
+            name='total_groups',
+        ),
+        migrations.RemoveField(
+            model_name='group',
+            name='total_hosts',
+        ),
+        migrations.RemoveField(
+            model_name='host',
+            name='has_active_failures',
+        ),
+        migrations.RemoveField(
+            model_name='host',
+            name='has_inventory_sources',
+        ),
+        migrations.AlterField(
+            model_name='jobhostsummary',
+            name='failed',
+            field=models.BooleanField(db_index=True, default=False, editable=False),
+        ),
+    ]

awx/main/migrations/0104_v370_cleanup_old_scan_jts.py

@@ -0,0 +1,24 @@
+# Generated by Django 2.2.8 on 2020-01-15 20:01
+
+from django.db import migrations, models
+
+
+def cleanup_scan_jts(apps, schema_editor):
+    JobTemplate = apps.get_model('main', 'JobTemplate')
+    JobTemplate.objects.filter(job_type='scan').update(job_type='run')
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0103_v370_remove_computed_fields'),
+    ]
+
+    operations = [
+        migrations.RunPython(cleanup_scan_jts),
+        migrations.AlterField(
+            model_name='jobtemplate',
+            name='job_type',
+            field=models.CharField(choices=[('run', 'Run'), ('check', 'Check')], default='run', max_length=64),
+        ),
+    ]

awx/main/migrations/0105_v370_remove_jobevent_parent_and_hosts.py

@@ -0,0 +1,21 @@
+# Generated by Django 2.2.8 on 2020-01-15 18:01
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0104_v370_cleanup_old_scan_jts'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='jobevent',
+            name='parent',
+        ),
+        migrations.RemoveField(
+            model_name='jobevent',
+            name='hosts',
+        ),
+    ]

awx/main/migrations/0106_v370_remove_inventory_groups_with_active_failures.py

@@ -0,0 +1,17 @@
+# Generated by Django 2.2.8 on 2020-01-27 12:39
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0105_v370_remove_jobevent_parent_and_hosts'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='inventory',
+            name='groups_with_active_failures',
+        ),
+    ]

awx/main/migrations/0107_v370_workflow_convergence_api_toggle.py

@@ -0,0 +1,23 @@
+# Generated by Django 2.2.4 on 2020-01-08 22:11
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0106_v370_remove_inventory_groups_with_active_failures'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='workflowjobnode',
+            name='all_parents_must_converge',
+            field=models.BooleanField(default=False, help_text='If enabled then the node will only run if all of the parent nodes have met the criteria to reach this node'),
+        ),
+        migrations.AddField(
+            model_name='workflowjobtemplatenode',
+            name='all_parents_must_converge',
+            field=models.BooleanField(default=False, help_text='If enabled then the node will only run if all of the parent nodes have met the criteria to reach this node'),
+        ),
+    ]

awx/main/migrations/0108_v370_unifiedjob_dependencies_processed.py

@@ -0,0 +1,18 @@
+# Generated by Django 2.2.8 on 2020-02-06 16:43
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0107_v370_workflow_convergence_api_toggle'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='unifiedjob',
+            name='dependencies_processed',
+            field=models.BooleanField(default=False, editable=False, help_text='If True, the task manager has already processed potential dependencies for this job.'),
+        ),
+    ]

awx/main/migrations/0109_v370_job_template_organization_field.py

@@ -0,0 +1,81 @@
+# Generated by Django 2.2.4 on 2019-08-07 19:56
+
+import awx.main.utils.polymorphic
+import awx.main.fields
+from django.db import migrations, models
+import django.db.models.deletion
+
+from awx.main.migrations._rbac import (
+    rebuild_role_parentage, rebuild_role_hierarchy,
+    migrate_ujt_organization, migrate_ujt_organization_backward,
+    restore_inventory_admins, restore_inventory_admins_backward
+)
+
+
+def rebuild_jt_parents(apps, schema_editor):
+    rebuild_role_parentage(apps, schema_editor, models=('jobtemplate',))
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0108_v370_unifiedjob_dependencies_processed'),
+    ]
+
+    operations = [
+        # backwards parents and ancestors caching
+        migrations.RunPython(migrations.RunPython.noop, rebuild_jt_parents),
+        # add new organization field for JT and all other unified jobs
+        migrations.AddField(
+            model_name='unifiedjob',
+            name='tmp_organization',
+            field=models.ForeignKey(blank=True, help_text='The organization used to determine access to this unified job.', null=True, on_delete=awx.main.utils.polymorphic.SET_NULL, related_name='unifiedjobs', to='main.Organization'),
+        ),
+        migrations.AddField(
+            model_name='unifiedjobtemplate',
+            name='tmp_organization',
+            field=models.ForeignKey(blank=True, help_text='The organization used to determine access to this template.', null=True, on_delete=awx.main.utils.polymorphic.SET_NULL, related_name='unifiedjobtemplates', to='main.Organization'),
+        ),
+        # while new and old fields exist, copy the organization fields
+        migrations.RunPython(migrate_ujt_organization, migrate_ujt_organization_backward),
+        # with data saved, remove old fields
+        migrations.RemoveField(
+            model_name='project',
+            name='organization',
+        ),
+        migrations.RemoveField(
+            model_name='workflowjobtemplate',
+            name='organization',
+        ),
+        # now, without safely rename the new field without conflicts from old field
+        migrations.RenameField(
+            model_name='unifiedjobtemplate',
+            old_name='tmp_organization',
+            new_name='organization',
+        ),
+        migrations.RenameField(
+            model_name='unifiedjob',
+            old_name='tmp_organization',
+            new_name='organization',
+        ),
+        # parentage of job template roles has genuinely changed at this point
+        migrations.AlterField(
+            model_name='jobtemplate',
+            name='admin_role',
+            field=awx.main.fields.ImplicitRoleField(editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role=['organization.job_template_admin_role'], related_name='+', to='main.Role'),
+        ),
+        migrations.AlterField(
+            model_name='jobtemplate',
+            name='execute_role',
+            field=awx.main.fields.ImplicitRoleField(editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role=['admin_role', 'organization.execute_role'], related_name='+', to='main.Role'),
+        ),
+        migrations.AlterField(
+            model_name='jobtemplate',
+            name='read_role',
+            field=awx.main.fields.ImplicitRoleField(editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role=['organization.auditor_role', 'inventory.organization.auditor_role', 'execute_role', 'admin_role'], related_name='+', to='main.Role'),
+        ),
+        # Re-compute the role parents and ancestors caching
+        migrations.RunPython(rebuild_jt_parents, migrations.RunPython.noop),
+        # for all permissions that will be removed, make them explicit
+        migrations.RunPython(restore_inventory_admins, restore_inventory_admins_backward),
+    ]

awx/main/migrations/0110_v370_instance_ip_address.py

@@ -0,0 +1,18 @@
+# Generated by Django 2.2.8 on 2020-02-12 17:55
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0109_v370_job_template_organization_field'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='instance',
+            name='ip_address',
+            field=models.CharField(blank=True, default=None, max_length=50, null=True, unique=True),
+        ),
+    ]

awx/main/migrations/0111_v370_delete_channelgroup.py

@@ -0,0 +1,16 @@
+# Generated by Django 2.2.8 on 2020-02-17 14:50
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0110_v370_instance_ip_address'),
+    ]
+
+    operations = [
+        migrations.DeleteModel(
+            name='ChannelGroup',
+        ),
+    ]