Merge pull request #9169 from rooftopcellist/version-1101

Bump version to 17.0.1 & update changelog Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
Bump version to 17.0.1 & update changelog
2026-02-05 11:34:43 -03:30 · 2021-01-26 18:22:28 +00:00 · 2021-01-26 12:35:14 -05:00 · 2021-01-26 17:16:43 +00:00 · 2021-01-26 15:40:56 +00:00 · 2021-01-26 08:48:08 -06:00
3047 changed files with 125201 additions and 259638 deletions
--- a/.dockerignore
+++ b/.dockerignore
@@ -1,2 +1 @@
-.git
 awx/ui/node_modules
--- a/.github/ISSUE_TEMPLATE/bug_report.md
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -3,6 +3,12 @@ name: "\U0001F41B Bug report"
 about: Create a report to help us improve

 ---
+<!-- Issues are for **concrete, actionable bugs and feature requests** only - if you're just asking for debugging help or technical support, please use:
+
+- http://webchat.freenode.net/?channels=ansible-awx
+- https://groups.google.com/forum/#!forum/awx-project
+
+We have to limit this because of limited volunteer time to respond to issues! -->

 ##### ISSUE TYPE
 - Bug Report
--- a/.github/ISSUE_TEMPLATE/feature_request.md
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -3,6 +3,12 @@ name: "✨ Feature request"
 about: Suggest an idea for this project

 ---
+<!-- Issues are for **concrete, actionable bugs and feature requests** only - if you're just asking for debugging help or technical support, please use:
+
+- http://webchat.freenode.net/?channels=ansible-awx
+- https://groups.google.com/forum/#!forum/awx-project
+
+We have to limit this because of limited volunteer time to respond to issues! -->

 ##### ISSUE TYPE
 - Feature Idea
--- a/.gitignore
+++ b/.gitignore
@@ -29,12 +29,14 @@ awx/ui/client/languages
 awx/ui/templates/ui/index.html
 awx/ui/templates/ui/installing.html
 awx/ui_next/node_modules/
+awx/ui_next/src/locales/
 awx/ui_next/coverage/
-awx/ui_next/build/locales/_build
+awx/ui_next/build
+awx/ui_next/.env.local
+awx/ui_next/instrumented
 rsyslog.pid
-/tower-license
-/tower-license/**
 tools/prometheus/data
+tools/docker-compose/Dockerfile

 # Tower setup playbook testing
 setup/test/roles/postgresql
@@ -138,9 +140,12 @@ use_dev_supervisor.txt
 # Ansible module tests
 /awx_collection_test_venv/
 /awx_collection/*.tar.gz
-/awx_collection/galaxy.yml
 /sanity/
+/awx_collection_build/

 .idea/*
 *.unison.tmp
 *.#
+/tools/docker-compose/overrides/
+/awx/ui_next/.ui-built
+/Dockerfile
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,189 @@

 This is a list of high-level changes for each release of AWX. A full list of commits can be found at `https://github.com/ansible/awx/releases/tag/<version>`.

+# 17.0.1 (January 26, 2021)
+- Fixed pgdocker directory permissions issue with Local Docker installer: https://github.com/ansible/awx/pull/9152
+- Fixed a bug in the UI which caused toggle settings to not be changed when clicked: https://github.com/ansible/awx/pull/9093
+
+# 17.0.0 (January 22, 2021)
+- AWX now requires PostgreSQL 12 by default: https://github.com/ansible/awx/pull/8943
+  **Note:** users who encounter permissions errors at upgrade time should `chown -R ~/.awx/pgdocker` to ensure it's owned by the user running the install playbook
+- Added support for region name for OpenStack inventory: https://github.com/ansible/awx/issues/5080
+- Added the ability to chain undefined attributes in custom notification templates: https://github.com/ansible/awx/issues/8677
+- Dramatically simplified the `image_build` role: https://github.com/ansible/awx/pull/8980
+- Fixed a bug which can cause schema migrations to fail at install time: https://github.com/ansible/awx/issues/9077
+- Fixed a bug which caused the `is_superuser` user property to be out of date in certain circumstances: https://github.com/ansible/awx/pull/8833
+- Fixed a bug which sometimes results in race conditions on setting access: https://github.com/ansible/awx/pull/8580
+- Fixed a bug which sometimes causes an unexpected delay in stdout for some playbooks: https://github.com/ansible/awx/issues/9085
+- (UI) Added support for credential password prompting on job launch: https://github.com/ansible/awx/pull/9028
+- (UI) Added the ability to configure LDAP settings in the UI: https://github.com/ansible/awx/issues/8291
+- (UI) Added a sync button to the Project detail view: https://github.com/ansible/awx/issues/8847
+- (UI) Added a form for configuring Google Outh 2.0 settings: https://github.com/ansible/awx/pull/8762
+- (UI) Added searchable keys and related keys to the Credentials list: https://github.com/ansible/awx/issues/8603
+- (UI) Added support for advanced search and copying to Notification Templates: https://github.com/ansible/awx/issues/7879
+- (UI) Added support for prompting on workflow nodes: https://github.com/ansible/awx/issues/5913
+- (UI) Added support for session timeouts: https://github.com/ansible/awx/pull/8250
+- (UI) Fixed a bug that broke websocket streaming for the insecure ws:// protocol: https://github.com/ansible/awx/pull/8877
+- (UI) Fixed a bug in the user interface when a translation for the browser's preferred locale isn't available: https://github.com/ansible/awx/issues/8884
+- (UI) Fixed bug where navigating from one survey question form directly to another wasn't reloading the form: https://github.com/ansible/awx/issues/7522
+- (UI) Fixed a bug which can cause an uncaught error while launching a Job Template: https://github.com/ansible/awx/issues/8936
+- Updated autobahn to address CVE-2020-35678
+
+## 16.0.0 (December 10, 2020)
+- AWX now ships with a reimagined user interface.  **Please read this before upgrading:** https://groups.google.com/g/awx-project/c/KuT5Ao92HWo
+- Removed support for syncing inventory from Red Hat CloudForms - https://github.com/ansible/awx/commit/0b701b3b2
+- Removed support for Mercurial-based project updates - https://github.com/ansible/awx/issues/7932
+- Upgraded NodeJS to actively maintained LTS 14.15.1 - https://github.com/ansible/awx/pull/8766
+- Added Git-LFS to the default image build - https://github.com/ansible/awx/pull/8700
+- Added the ability to specify `metadata.labels` in the podspec for container groups - https://github.com/ansible/awx/issues/8486
+- Added support for Kubernetes pod annotations - https://github.com/ansible/awx/pull/8434
+- Added the ability to label the web container in local Docker installs - https://github.com/ansible/awx/pull/8449
+- Added additional metadata (as an extra var) to playbook runs to report the SCM branch name - https://github.com/ansible/awx/pull/8433
+- Fixed a bug that caused k8s installations to fail due to an incorrect Helm repo - https://github.com/ansible/awx/issues/8715
+- Fixed a bug that prevented certain Workflow Approval resources from being deleted - https://github.com/ansible/awx/pull/8612
+- Fixed a bug that prevented the deletion of inventories stuck in "pending deletion" state - https://github.com/ansible/awx/issues/8525
+- Fixed a display bug in webhook notifications with certain unicode characters - https://github.com/ansible/awx/issues/7400
+- Improved support for exporting dependent objects (Inventory Hosts and Groups) in the `awx export` CLI tool - https://github.com/ansible/awx/commit/607bc0788
+
+## 15.0.1 (October 20, 2020)
+- Added several optimizations to improve performance for a variety of high-load simultaneous job launch use cases https://github.com/ansible/awx/pull/8403
+- Added the ability to source roles and collections from requirements.yaml files (not just requirements.yml) - https://github.com/ansible/awx/issues/4540
+- awx.awx collection modules now provide a clearer error message for incompatible versions of awxkit - https://github.com/ansible/awx/issues/8127
+- Fixed a bug in notification messages that contain certain unicode characters - https://github.com/ansible/awx/issues/7400
+- Fixed a bug that prevents the deletion of Workflow Approval records - https://github.com/ansible/awx/issues/8305
+- Fixed a bug that broke the selection of webhook credentials - https://github.com/ansible/awx/issues/7892
+- Fixed a bug which can cause confusing behavior for social auth logins across distinct browser tabs - https://github.com/ansible/awx/issues/8154
+- Fixed several bugs in the output of Workflow Job Templates using the `awx export` tool - https://github.com/ansible/awx/issues/7798 https://github.com/ansible/awx/pull/7847
+- Fixed a race condition that can lead to missing hosts when running parallel inventory syncs - https://github.com/ansible/awx/issues/5571
+- Fixed an HTTP 500 error when certain LDAP group parameters aren't properly set - https://github.com/ansible/awx/issues/7622
+- Updated a few dependencies in response to several CVEs:
+    * CVE-2020-7720
+    * CVE-2020-7743
+    * CVE-2020-7676
+
+## 15.0.0 (September 30, 2020)
+- Added improved support for fetching Ansible collections from private Galaxy content sources (such as https://github.com/ansible/galaxy_ng) - https://github.com/ansible/awx/issues/7813
+  **Note:** as part of this change, new Organizations created in the AWX API will _no longer_ automatically synchronize roles and collections from galaxy.ansible.com by default.  More details on this change can be found at:  https://github.com/ansible/awx/issues/8341#issuecomment-707310633
+- AWX now utilizes a version of certifi that auto-discovers certificates in the system certificate store - https://github.com/ansible/awx/pull/8242
+- Added support for arbitrary custom inventory plugin configuration: https://github.com/ansible/awx/issues/5150
+- Added an optional setting to disable the auto-creation of organizations and teams on successful SAML login. - https://github.com/ansible/awx/pull/8069
+- Added a number of optimizations to AWX's callback receiver to improve the speed of stdout processing for simultaneous playbooks runs - https://github.com/ansible/awx/pull/8193 https://github.com/ansible/awx/pull/8191
+- Added the ability to use `!include` and `!import` constructors when constructing YAML for use with the AWX CLI - https://github.com/ansible/awx/issues/8135
+- Fixed a bug that prevented certain users from being able to edit approval nodes in Workflows - https://github.com/ansible/awx/pull/8253
+- Fixed a bug that broke password prompting for credentials in certain cases - https://github.com/ansible/awx/issues/8202
+- Fixed a bug which can cause PostgreSQL deadlocks when running many parallel playbooks against large shared inventories - https://github.com/ansible/awx/issues/8145
+- Fixed a bug which can cause delays in AWX's task manager when large numbers of simultaneous jobs are scheduled - https://github.com/ansible/awx/issues/7655
+- Fixed a bug which can cause certain scheduled jobs - those that run every X minute(s) or hour(s) - to fail to run at the proper time - https://github.com/ansible/awx/issues/8071
+- Fixed a performance issue for playbooks that store large amounts of data using the `set_stats` module - https://github.com/ansible/awx/issues/8006
+- Fixed a bug related to AWX's handling of the auth_path argument for the HashiVault KeyValue credential plugin - https://github.com/ansible/awx/pull/7991
+- Fixed a bug that broke support for Remote Archive SCM Type project syncs on platforms that utilize Python2 - https://github.com/ansible/awx/pull/8057
+- Updated to the latest version of Django Rest Framework to address CVE-2020-25626
+- Updated to the latest version of Django to address CVE-2020-24583 and CVE-2020-24584
+- Updated to the latest verson of channels_redis to address a bug that slowly causes Daphne processes to leak memory over time - https://github.com/django/channels_redis/issues/212
+
+## 14.1.0 (Aug 25, 2020)
+- AWX images can now be built on ARM64 - https://github.com/ansible/awx/pull/7607
+- Added the Remote Archive SCM Type to support using immutable artifacts and releases (such as tarballs and zip files) as projects - https://github.com/ansible/awx/issues/7954
+- Deprecated official support for Mercurial-based project updates - https://github.com/ansible/awx/issues/7932
+- Added resource import/export support to the official AWX collection - https://github.com/ansible/awx/issues/7329
+- Added the ability to import YAML-based resources (instead of just JSON) when using the AWX CLI - https://github.com/ansible/awx/pull/7808
+- Users upgrading from older versions of AWX may encounter an issue that causes their postgres container to restart in a loop (https://github.com/ansible/awx/issues/7854) - if you encounter this, bring your containers down and then back up (e.g., `docker-compose down && docker-compose up -d`) after upgrading to 14.1.0.
+- Updated the AWX CLI to export labels associated with Workflow Job Templates - https://github.com/ansible/awx/pull/7847
+- Updated to the latest python-ldap to address a bug - https://github.com/ansible/awx/issues/7868
+- Upgraded git-python to fix a bug that caused workflows to sometimes fail - https://github.com/ansible/awx/issues/6119
+- Worked around a bug in the channels_redis library that slowly causes Daphne processes to leak memory over time - https://github.com/django/channels_redis/issues/212
+- Fixed a bug in the AWX CLI that prevented Workflow nodes from importing properly - https://github.com/ansible/awx/issues/7793
+- Fixed a bug in the awx.awx collection release process that templated the wrong version - https://github.com/ansible/awx/issues/7870
+- Fixed a bug that caused errors rendering stdout that contained UTF-16 surrogate pairs - https://github.com/ansible/awx/pull/7918
+
+## 14.0.0 (Aug 6, 2020)
+- As part of our commitment to inclusivity in open source, we recently took some time to audit AWX's source code and user interface and replace certain terminology with more inclusive language.  Strictly speaking, this isn't a bug or a feature, but we think it's important and worth calling attention to:
+    * https://github.com/ansible/awx/commit/78229f58715fbfbf88177e54031f532543b57acc
+    * https://www.redhat.com/en/blog/making-open-source-more-inclusive-eradicating-problematic-language
+- Installing roles and collections via requirements.yml as part of Project Updates now requires at least Ansible 2.9 - https://github.com/ansible/awx/issues/7769
+- Deprecated the use of the `PRIMARY_GALAXY_USERNAME` and `PRIMARY_GALAXY_PASSWORD` settings. We recommend using tokens to access Galaxy or Automation Hub.
+- Added local caching for downloaded roles and collections so they are not re-downloaded on nodes where they are up to date with the project - https://github.com/ansible/awx/issues/5518
+- Added the ability to associate K8S/OpenShift credentials to Job Template for playbook interaction with the `community.kubernetes` collection - https://github.com/ansible/awx/issues/5735
+- Added the ability to include HTML in the Custom Login Info presented on the login page - https://github.com/ansible/awx/issues/7600
+- Fixed https://access.redhat.com/security/cve/cve-2020-14327 - Server-side request forgery on credentials
+- Fixed https://access.redhat.com/security/cve/cve-2020-14328 - Server-side request forgery on webhooks
+- Fixed https://access.redhat.com/security/cve/cve-2020-14329 - Sensitive data exposure on labels
+- Fixed https://access.redhat.com/security/cve/cve-2020-14337 - Named URLs allow for testing the presence or absence of objects
+- Fixed a number of bugs in the user interface related to an upgrade of jQuery:
+     * https://github.com/ansible/awx/issues/7530
+     * https://github.com/ansible/awx/issues/7546
+     * https://github.com/ansible/awx/issues/7534
+     * https://github.com/ansible/awx/issues/7606
+- Fixed a bug that caused the `-f yaml` flag of the AWX CLI to not print properly formatted YAML - https://github.com/ansible/awx/issues/7795
+- Fixed a bug in the installer that caused errors when `docker_registry_password` was set - https://github.com/ansible/awx/issues/7695
+- Fixed a permissions error that prevented certain users from starting AWX services - https://github.com/ansible/awx/issues/7545
+- Fixed a bug that allows superusers to run unsafe Jinja code when defining custom Credential Types - https://github.com/ansible/awx/pull/7584/
+- Fixed a bug that prevented users from creating (or editing) custom Credential Types containing boolean fields - https://github.com/ansible/awx/issues/7483
+- Fixed a bug that prevented users with postgres usernames containing uppercase letters from restoring backups succesfully - https://github.com/ansible/awx/pull/7519
+- Fixed a bug which allowed the creation (in the Tower API) of Groups and Hosts with the same name - https://github.com/ansible/awx/issues/4680
+
+## 13.0.0 (Jun 23, 2020)
+- Added import and export commands to the official AWX CLI, replacing send and receive from the old tower-cli (https://github.com/ansible/awx/pull/6125).
+- Removed scripts as a means of running inventory updates of built-in types (https://github.com/ansible/awx/pull/6911)
+- Ansible 2.8 is now partially unsupported; some inventory source types are known to no longer work.
+- Fixed an issue where the vmware inventory source ssl_verify source variable was not recognized (https://github.com/ansible/awx/pull/7360)
+- Fixed a bug that caused redis' listen socket to have too-permissive file permissions (https://github.com/ansible/awx/pull/7317)
+- Fixed a bug that caused rsyslogd's configuration file to have world-readable file permissions, potentially leaking secrets (CVE-2020-10782)
+
+## 12.0.0 (Jun 9, 2020)
+- Removed memcached as a dependency of AWX (https://github.com/ansible/awx/pull/7240)
+- Moved to a single container image build instead of separate awx_web and awx_task images. The container image is just `awx` (https://github.com/ansible/awx/pull/7228)
+- Official AWX container image builds now use a two-stage container build process that notably reduces the size of our published images (https://github.com/ansible/awx/pull/7017)
+- Removed support for HipChat notifications ([EoL announcement](https://www.atlassian.com/partnerships/slack/faq#faq-98b17ca3-247f-423b-9a78-70a91681eff0)); all previously-created HipChat notification templates will be deleted due to this removal.
+- Fixed a bug which broke AWX installations with oc version 4.3 (https://github.com/ansible/awx/pull/6948/)
+- Fixed a performance issue that caused notable delay of stdout processing for playbooks run against large numbers of hosts (https://github.com/ansible/awx/issues/6991)
+- Fixed a bug that caused CyberArk AIM credential plugin looks to hang forever in some environments (https://github.com/ansible/awx/issues/6986)
+- Fixed a bug that caused ANY/ALL converage settings not to properly save when editing approval nodes in the UI (https://github.com/ansible/awx/issues/6998)
+- Fixed a bug that broke support for the satellite6_group_prefix source variable (https://github.com/ansible/awx/issues/7031)
+- Fixed a bug that prevented changes to workflow node convergence settings when approval nodes were in use (https://github.com/ansible/awx/issues/7063)
+- Fixed a bug that caused notifications to fail on newer version of Mattermost (https://github.com/ansible/awx/issues/7264)
+- Fixed a bug (by upgrading to 0.8.1 of the foreman collection) that prevented host_filters from working properly with Foreman-based inventory (https://github.com/ansible/awx/issues/7225)
+- Fixed a bug that prevented the usage of the Conjur credential plugin with secrets that contain spaces (https://github.com/ansible/awx/issues/7191)
+- Fixed a bug in awx-manage run_wsbroadcast --status in kubernetes (https://github.com/ansible/awx/pull/7009)
+- Fixed a bug that broke notification toggles for system jobs in the UI (https://github.com/ansible/awx/pull/7042)
+- Fixed a bug that broke local pip installs of awxkit (https://github.com/ansible/awx/issues/7107)
+- Fixed a bug that prevented PagerDuty notifications from sending for workflow job template approvals (https://github.com/ansible/awx/issues/7094)
+- Fixed a bug that broke external log aggregation support for URL paths that include the = character (such as the tokens for SumoLogic) (https://github.com/ansible/awx/issues/7139)
+- Fixed a bug that prevented organization admins from removing labels from workflow job templates (https://github.com/ansible/awx/pull/7143)
+
+## 11.2.0 (Apr 29, 2020)
+
+- Inventory updates now use collection-based plugins by default (in Ansible 2.9+):
+    - amazon.aws.aws_ec2
+    - community.vmware.vmware_vm_inventory
+    - azure.azcollection.azure_rm
+    - google.cloud.gcp_compute
+    - theforeman.foreman.foreman
+    - openstack.cloud.openstack
+    - ovirt.ovirt_collection.ovirt
+    - awx.awx.tower
+- Added support for Approle and LDAP/AD mechanisms to the Hashicorp Vault credential plugin (https://github.com/ansible/awx/issues/5076)
+- Added Project (Domain Name) support for the OpenStack Keystone v3 API (https://github.com/ansible/awx/issues/6831)
+- Added a new setting for raising log verbosity for rsyslogd (https://github.com/ansible/awx/pull/6818)
+- Added the ability to monitor stdout in the CLI for running jobs and workflow jobs (https://github.com/ansible/awx/issues/6165)
+- Fixed a bug which prevented the AWX CLI from properly installing with newer versions of pip (https://github.com/ansible/awx/issues/6870)
+- Fixed a bug which broke AWX's external logging support when configured with HTTPS endpoints that utilize self-signed certificates (https://github.com/ansible/awx/issues/6851)
+- Fixed a local docker installer bug that mistakenly attempted to upgrade PostgreSQL when an external pg_hostname is specified (https://github.com/ansible/awx/pull/5398)
+- Fixed a race condition that caused task container crashes when pods are quickly brought down and back up (https://github.com/ansible/awx/issues/6750)
+- Fixed a bug that caused 404 errors when attempting to view the second page of the workflow approvals view (https://github.com/ansible/awx/issues/6803)
+- Fixed a bug that prevented the use of ANSIBLE_SSH_ARGS for ad-hoc-commands (https://github.com/ansible/awx/pull/6811)
+- Fixed a bug that broke AWX installs/upgrades on Red Hat OpenShift (https://github.com/ansible/awx/issues/6791)
+
+
+## 11.1.0 (Apr 22, 2020)
+- Changed rsyslogd to persist queued events to disk (to prevent a risk of out-of-memory errors) (https://github.com/ansible/awx/issues/6746)
+- Added the ability to configure the destination and maximum disk size of rsyslogd spool (in the event of a log aggregator outage) (https://github.com/ansible/awx/pull/6763)
+- Added the ability to discover playbooks in project clones from symlinked directories (https://github.com/ansible/awx/pull/6773)
+- Fixed a bug that caused certain log aggregator settings to break logging integration (https://github.com/ansible/awx/issues/6760)
+- Fixed a bug that caused playbook execution in container groups to sometimes unexpectedly deadlock (https://github.com/ansible/awx/issues/6692)
+- Improved stability of the new redis clustering implementation (https://github.com/ansible/awx/pull/6739 https://github.com/ansible/awx/pull/6720)
+- Improved stability of the new rsyslogd-based logging implementation (https://github.com/ansible/awx/pull/6796)
+
 ## 11.0.0 (Apr 16, 2020)
 - As of AWX 11.0.0, Kubernetes-based deployments use a Deployment rather than a StatefulSet.
 - Reimplemented external logging support using rsyslogd to improve reliability and address a number of issues (https://github.com/ansible/awx/issues/5155)
@@ -70,7 +253,7 @@ This is a list of high-level changes for each release of AWX. A full list of com
 - Updated the bundled version of openstacksdk to address a known issue https://github.com/ansible/awx/issues/5821
 - Updated the bundled vmware_inventory plugin to the latest version to address a bug https://github.com/ansible/awx/pull/5668
 - Fixed a bug that can cause inventory updates to fail to properly save their output when run within a workflow https://github.com/ansible/awx/pull/5666
- Removed a number of pre-computed fields from the Host and Group models to improve AWX performance.  As part of this change, inventory group UIs throughout the interface no longer display status icons https://github.com/ansible/awx/pull/5448 
+- Removed a number of pre-computed fields from the Host and Group models to improve AWX performance.  As part of this change, inventory group UIs throughout the interface no longer display status icons https://github.com/ansible/awx/pull/5448

 ## 9.1.1 (Jan 14, 2020)

--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -80,12 +80,12 @@ For Linux platforms, refer to the following from Docker:
 If you're not using Docker for Mac, or Docker for Windows, you may need, or choose to, install the Docker compose Python module separately, in which case you'll need to run the following:

 ```bash
-(host)$ pip install docker-compose
+(host)$ pip3 install docker-compose
 ```

 #### Frontend Development

-See [the ui development documentation](awx/ui/README.md).
+See [the ui development documentation](awx/ui_next/CONTRIBUTING.md).


 ### Build the environment
@@ -157,9 +157,8 @@ If you start a second terminal session, you can take a look at the running conta
 $ docker ps
 CONTAINER ID        IMAGE                                              COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                              NAMES
 44251b476f98        gcr.io/ansible-tower-engineering/awx_devel:devel   "/entrypoint.sh /bin…"   27 seconds ago      Up 23 seconds       0.0.0.0:6899->6899/tcp, 0.0.0.0:7899-7999->7899-7999/tcp, 0.0.0.0:8013->8013/tcp, 0.0.0.0:8043->8043/tcp, 0.0.0.0:8080->8080/tcp, 22/tcp, 0.0.0.0:8888->8888/tcp   tools_awx_run_9e820694d57e
-b049a43817b4        memcached:alpine                                   "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds       0.0.0.0:11211->11211/tcp                                                                                                                                           tools_memcached_1
-40de380e3c2e        redis:latest                                       "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds       0.0.0.0:6379->6379/tcp                                                                                                                                             tools_redis_1
-b66a506d3007        postgres:10                                        "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds       0.0.0.0:5432->5432/tcp                                                                                                                                             tools_postgres_1
+40de380e3c2e        redis:latest                                       "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds
+b66a506d3007        postgres:12                                        "docker-entrypoint.s…"   28 seconds ago      Up 26 seconds       0.0.0.0:5432->5432/tcp                                                                                                                                             tools_postgres_1
 ```
 **NOTE**

--- a/INSTALL.md
+++ b/INSTALL.md
@@ -10,7 +10,6 @@ This document provides a guide for installing AWX.
    + [AWX branding](#awx-branding)
    + [Prerequisites](#prerequisites)
    + [System Requirements](#system-requirements)
-    + [AWX Tunables](#awx-tunables)
    + [Choose a deployment platform](#choose-a-deployment-platform)
    + [Official vs Building Images](#official-vs-building-images)
  * [Upgrading from previous versions](#upgrading-from-previous-versions)
@@ -44,14 +43,24 @@ This document provides a guide for installing AWX.
 - [Installing the AWX CLI](#installing-the-awx-cli)
  * [Building the CLI Documentation](#building-the-cli-documentation)

-    
+
 ## Getting started

 ### Clone the repo

-If you have not already done so, you will need to clone, or create a local copy, of the [AWX repo](https://github.com/ansible/awx). For more on how to clone the repo, view [git clone help](https://git-scm.com/docs/git-clone).
+If you have not already done so, you will need to clone, or create a local copy, of the [AWX repo](https://github.com/ansible/awx). We generally recommend that you view the releases page:

-Once you have a local copy, run commands within the root of the project tree.
+https://github.com/ansible/awx/releases
+
+...and clone the latest stable release, e.g.,
+
+`git clone -b x.y.z https://github.com/ansible/awx.git`
+
+Please note that deploying from `HEAD` (or the latest commit) is **not** stable, and that if you want to do this, you should proceed at your own risk (also, see the section #official-vs-building-images for building your own image).
+
+For more on how to clone the repo, view [git clone help](https://git-scm.com/docs/git-clone).
+
+Once you have a local copy, run the commands in the following sections from the root of the project tree.

 ### AWX branding

@@ -69,10 +78,15 @@ Before you can run a deployment, you'll need the following installed in your loc
 - [docker](https://pypi.org/project/docker/) Python module
    + This is incompatible with `docker-py`. If you have previously installed `docker-py`, please uninstall it.
    + We use this module instead of `docker-py` because it is what the `docker-compose` Python module requires.
+- [community.general.docker_image collection](https://docs.ansible.com/ansible/latest/collections/community/general/docker_image_module.html)
+    + This is only required if you are using Ansible >= 2.10
 - [GNU Make](https://www.gnu.org/software/make/)
 - [Git](https://git-scm.com/) Requires Version 1.8.4+
- [Node 10.x LTS version](https://nodejs.org/en/download/)
+- Python 3.6+
+- [Node 14.x LTS version](https://nodejs.org/en/download/)
+    + This is only required if you're [building your own container images](#official-vs-building-images) with `use_container_for_build=false`
 - [NPM 6.x LTS](https://docs.npmjs.com/)
+    + This is only required if you're [building your own container images](#official-vs-building-images) with `use_container_for_build=false`

 ### System Requirements

@@ -82,11 +96,7 @@ The system that runs the AWX service will need to satisfy the following requirem
 - At least 2 cpu cores
 - At least 20GB of space
 - Running Docker, Openshift, or Kubernetes
- If you choose to use an external PostgreSQL database, please note that the minimum version is 9.6+.
-
-### AWX Tunables
-
-**TODO** add tunable bits
+- If you choose to use an external PostgreSQL database, please note that the minimum version is 10+.

 ### Choose a deployment platform

@@ -101,7 +111,7 @@ In the sections below, you'll find deployment details and instructions for each

 ### Official vs Building Images

-When installing AWX you have the option of building your own images or using the images provided on DockerHub (see [awx_web](https://hub.docker.com/r/ansible/awx_web/) and [awx_task](https://hub.docker.com/r/ansible/awx_task/))
+When installing AWX you have the option of building your own image or using the image provided on DockerHub (see [awx](https://hub.docker.com/r/ansible/awx/))

 This is controlled by the following variables in the `inventory` file

@@ -114,12 +124,16 @@ If these variables are present then all deployments will use these hosted images

 *dockerhub_base*

-> The base location on DockerHub where the images are hosted (by default this pulls container images named `ansible/awx_web:tag` and `ansible/awx_task:tag`)
+> The base location on DockerHub where the images are hosted (by default this pulls a container image named `ansible/awx:tag`)

 *dockerhub_version*

 > Multiple versions are provided. `latest` always pulls the most recent. You may also select version numbers at different granularities: 1, 1.0, 1.0.1, 1.0.0.123

+*use_container_for_build*
+
+> Use a local distribution build container image for building the AWX package. This is helpful if you don't want to bother installing the build-time dependencies as it is taken care of already.
+

 ## Upgrading from previous versions

@@ -143,7 +157,7 @@ $ ansible-playbook -i inventory install.yml -e @vars.yml

 ### Prerequisites

-To complete a deployment to OpenShift, you will obviously need access to an OpenShift cluster. For demo and testing purposes, you can use [Minishift](https://github.com/minishift/minishift) to create a single node cluster running inside a virtual machine.
+To complete a deployment to OpenShift, you will need access to an OpenShift cluster. For demo and testing purposes, you can use [Minishift](https://github.com/minishift/minishift) to create a single node cluster running inside a virtual machine.

 When using OpenShift for deploying AWX make sure you have correct privileges to add the security context 'privileged', otherwise the installation will fail. The privileged context is needed because of the use of [the bubblewrap tool](https://github.com/containers/bubblewrap) to add an additional layer of security when using containers.

@@ -339,7 +353,7 @@ Once you access the AWX server, you will be prompted with a login dialog. The de
 A Kubernetes deployment will require you to have access to a Kubernetes cluster as well as the following tools:

 - [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- [helm](https://docs.helm.sh/using_helm/#quickstart-guide)
+- [helm](https://helm.sh/docs/intro/quickstart/)

 The installation program will reference `kubectl` directly. `helm` is only necessary if you are letting the installer configure PostgreSQL for you.

@@ -370,9 +384,11 @@ Before starting the install process, review the [inventory](./installer/inventor

 ### Configuring Helm

-If you want the AWX installer to manage creating the database pod (rather than installing and configuring postgres on your own). Then you will need to have a working `helm` installation, you can find details here: [https://docs.helm.sh/using_helm/#quickstart-guide](https://docs.helm.sh/using_helm/#quickstart-guide).
+If you want the AWX installer to manage creating the database pod (rather than installing and configuring postgres on your own). Then you will need to have a working `helm` installation, you can find details here: [https://helm.sh/docs/intro/quickstart/](https://helm.sh/docs/intro/quickstart/).

-Newer Kubernetes clusters with RBAC enabled will need to make sure a service account is created, make sure to follow the instructions here [https://docs.helm.sh/using_helm/#role-based-access-control](https://docs.helm.sh/using_helm/#role-based-access-control)
+You do not need to create a [Persistent Volume Claim](https://docs.openshift.org/latest/dev_guide/persistent_volumes.html) as Helm does it for you. However, an existing one may be used by setting the `pg_persistence_existingclaim` variable.
+
+Newer Kubernetes clusters with RBAC enabled will need to make sure a service account is created, make sure to follow the instructions here [https://helm.sh/docs/topics/rbac/](https://helm.sh/docs/topics/rbac/)

 ### Run the installer

@@ -469,11 +485,11 @@ Before starting the install process, review the [inventory](./installer/inventor

 *host_port*

-> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container. Defaults to *80*.
+> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container. If undefined no port will be exposed. Defaults to *80*.

 *host_port_ssl*

-> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container for SSL support. Defaults to *443*, only works if you also set `ssl_certificate` (see below).
+> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container for SSL support. If undefined no port will be exposed. Defaults to *443*, only works if you also set `ssl_certificate` (see below).

 *ssl_certificate*

@@ -481,7 +497,7 @@ Before starting the install process, review the [inventory](./installer/inventor

 *docker_compose_dir*

-> When using docker-compose, the `docker-compose.yml` file will be created there (default `/tmp/awxcompose`).
+> When using docker-compose, the `docker-compose.yml` file will be created there (default `~/.awx/awxcompose`).

 *custom_venv_dir*

@@ -563,7 +579,7 @@ If you're deploying using Docker Compose, container names will be prefixed by th
 Immediately after the containers start, the *awx_task* container will perform required setup tasks, including database migrations. These tasks need to complete before the web interface can be accessed. To monitor the progress, you can follow the container's STDOUT by running the following:

 ```bash
-# Tail the the awx_task log
+# Tail the awx_task log
 $ docker logs -f awx_task
 ```

@@ -639,17 +655,16 @@ Potential uses include:
 * Checking on the status and output of job runs
 * Managing objects like organizations, users, teams, etc...

-The preferred way to install the AWX CLI is through pip directly from GitHub:
+The preferred way to install the AWX CLI is through pip directly from PyPI:

-    pip install "https://github.com/ansible/awx/archive/$VERSION.tar.gz#egg=awxkit&subdirectory=awxkit"
+    pip3 install awxkit
    awx --help

-...where ``$VERSION`` is the version of AWX you're running.  To see a list of all available releases, visit: https://github.com/ansible/awx/releases
-
 ## Building the CLI Documentation

-To build the docs, spin up a real AWX server, `pip install sphinx sphinxcontrib-autoprogram`, and run:
+To build the docs, spin up a real AWX server, `pip3 install sphinx sphinxcontrib-autoprogram`, and run:

+    ~ cd awxkit/awxkit/cli/docs
    ~ TOWER_HOST=https://awx.example.org TOWER_USERNAME=example TOWER_PASSWORD=secret make clean html
    ~ cd build/html/ && python -m http.server
    Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ..
--- a/ISSUES.md
+++ b/ISSUES.md
@@ -31,7 +31,7 @@ If your issue isn't considered high priority, then please be patient as it may t

 `state:needs_info` The issue needs more information. This could be more debug output, more specifics out the system such as version information. Any detail that is currently preventing this issue from moving forward. This should be considered a blocked state.

-`state:needs_review` The the issue/pull request needs to be reviewed by other maintainers and contributors. This is usually used when there is a question out to another maintainer or when a person is less familar with an area of the code base the issue is for.
+`state:needs_review` The issue/pull request needs to be reviewed by other maintainers and contributors. This is usually used when there is a question out to another maintainer or when a person is less familar with an area of the code base the issue is for.

 `state:needs_revision` More commonly used on pull requests, this state represents that there are changes that are being waited on.

--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -4,12 +4,13 @@ recursive-include awx *.mo
 recursive-include awx/static *
 recursive-include awx/templates *.html
 recursive-include awx/api/templates *.md *.html
-recursive-include awx/ui/templates *.html
-recursive-include awx/ui/static *
+recursive-include awx/ui_next/build *.html
+recursive-include awx/ui_next/build *
 recursive-include awx/playbooks *.yml
 recursive-include awx/lib/site-packages *
 recursive-include awx/plugins *.ps1
 recursive-include requirements *.txt
+recursive-include requirements *.yml
 recursive-include config *
 recursive-include docs/licenses *
 recursive-exclude awx devonly.py*
--- a/214
+++ b/214
@@ -6,25 +6,28 @@ PACKER ?= packer
 PACKER_BUILD_OPTS ?= -var 'official=$(OFFICIAL)' -var 'aw_repo_url=$(AW_REPO_URL)'
 NODE ?= node
 NPM_BIN ?= npm
+CHROMIUM_BIN=/tmp/chrome-linux/chrome
 DEPS_SCRIPT ?= packaging/bundle/deps.py
 GIT_BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD)
 MANAGEMENT_COMMAND ?= awx-manage
 IMAGE_REPOSITORY_AUTH ?=
 IMAGE_REPOSITORY_BASE ?= https://gcr.io
 VERSION := $(shell cat VERSION)
+PYCURL_SSL_LIBRARY ?= openssl

 # NOTE: This defaults the container image version to the branch that's active
 COMPOSE_TAG ?= $(GIT_BRANCH)
 COMPOSE_HOST ?= $(shell hostname)

-VENV_BASE ?= /venv
+VENV_BASE ?= /var/lib/awx/venv/
+COLLECTION_BASE ?= /var/lib/awx/vendor/awx_ansible_collections
 SCL_PREFIX ?=
 CELERY_SCHEDULE_FILE ?= /var/lib/awx/beat.db

 DEV_DOCKER_TAG_BASE ?= gcr.io/ansible-tower-engineering
 # Python packages to install only from source (not from binary wheels)
 # Comma separated list
-SRC_ONLY_PKGS ?= cffi,pycparser,psycopg2,twilio
+SRC_ONLY_PKGS ?= cffi,pycparser,psycopg2,twilio,pycurl
 # These should be upgraded in the AWX and Ansible venv before attempting
 # to install the actual requirements
 VENV_BOOTSTRAP ?= pip==19.3.1 setuptools==41.6.0
@@ -54,11 +57,6 @@ WHEEL_COMMAND ?= bdist_wheel
 SDIST_TAR_FILE ?= $(SDIST_TAR_NAME).tar.gz
 WHEEL_FILE ?= $(WHEEL_NAME)-py2-none-any.whl

-# UI flag files
-UI_DEPS_FLAG_FILE = awx/ui/.deps_built
-UI_RELEASE_DEPS_FLAG_FILE = awx/ui/.release_deps_built
-UI_RELEASE_FLAG_FILE = awx/ui/.release_built
-
 I18N_FLAG_FILE = .i18n_built

 .PHONY: awx-link clean clean-tmp clean-venv requirements requirements_dev \
@@ -68,21 +66,6 @@ I18N_FLAG_FILE = .i18n_built
 	ui-docker-machine ui-docker ui-release ui-devel \
 	ui-test ui-deps ui-test-ci VERSION

-# remove ui build artifacts
-clean-ui: clean-languages
-	rm -rf awx/ui/static/
-	rm -rf awx/ui/node_modules/
-	rm -rf awx/ui/test/unit/reports/
-	rm -rf awx/ui/test/spec/reports/
-	rm -rf awx/ui/test/e2e/reports/
-	rm -rf awx/ui/client/languages/
-	rm -rf awx/ui_next/node_modules/
-	rm -rf awx/ui_next/coverage/
-	rm -rf awx/ui_next/build/locales/_build/
-	rm -f $(UI_DEPS_FLAG_FILE)
-	rm -f $(UI_RELEASE_DEPS_FLAG_FILE)
-	rm -f $(UI_RELEASE_FLAG_FILE)
-
 clean-tmp:
 	rm -rf tmp/

@@ -173,9 +156,9 @@ virtualenv_awx:
 # --ignore-install flag is not used because *.txt files should specify exact versions
 requirements_ansible: virtualenv_ansible
 	if [[ "$(PIP_OPTIONS)" == *"--no-index"* ]]; then \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | PYCURL_SSL_LIBRARY=$(PYCURL_SSL_LIBRARY) $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) -r /dev/stdin ; \
 	else \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | PYCURL_SSL_LIBRARY=$(PYCURL_SSL_LIBRARY) $(VENV_BASE)/ansible/bin/pip install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) -r /dev/stdin ; \
 	fi
 	$(VENV_BASE)/ansible/bin/pip uninstall --yes -r requirements/requirements_ansible_uninstall.txt
 	# Same effect as using --system-site-packages flag on venv creation
@@ -183,9 +166,9 @@ requirements_ansible: virtualenv_ansible

 requirements_ansible_py3: virtualenv_ansible_py3
 	if [[ "$(PIP_OPTIONS)" == *"--no-index"* ]]; then \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_local.txt | PYCURL_SSL_LIBRARY=$(PYCURL_SSL_LIBRARY) $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) -r /dev/stdin ; \
 	else \
-	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) -r /dev/stdin ; \
+	    cat requirements/requirements_ansible.txt requirements/requirements_ansible_git.txt | PYCURL_SSL_LIBRARY=$(PYCURL_SSL_LIBRARY) $(VENV_BASE)/ansible/bin/pip3 install $(PIP_OPTIONS) --no-binary $(SRC_ONLY_PKGS) -r /dev/stdin ; \
 	fi
 	$(VENV_BASE)/ansible/bin/pip3 uninstall --yes -r requirements/requirements_ansible_uninstall.txt
 	# Same effect as using --system-site-packages flag on venv creation
@@ -209,7 +192,15 @@ requirements_awx: virtualenv_awx
 requirements_awx_dev:
 	$(VENV_BASE)/awx/bin/pip install -r requirements/requirements_dev.txt

-requirements: requirements_ansible requirements_awx
+requirements_collections:
+	mkdir -p $(COLLECTION_BASE)
+	n=0; \
+	until [ "$$n" -ge 5 ]; do \
+	    ansible-galaxy collection install -r requirements/collections_requirements.yml -p $(COLLECTION_BASE) && break; \
+	    n=$$((n+1)); \
+	done
+
+requirements: requirements_ansible requirements_awx requirements_collections

 requirements_dev: requirements_awx requirements_ansible_py3 requirements_awx_dev requirements_ansible_dev

@@ -280,7 +271,7 @@ uwsgi: collectstatic
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
-    uwsgi -b 32768 --socket 127.0.0.1:8050 --module=awx.wsgi:application --home=/venv/awx --chdir=/awx_devel/ --vacuum --processes=5 --harakiri=120 --master --no-orphans --py-autoreload 1 --max-requests=1000 --stats /tmp/stats.socket --lazy-apps --logformat "%(addr) %(method) %(uri) - %(proto) %(status)" --hook-accepting1="exec:supervisorctl restart tower-processes:awx-dispatcher tower-processes:awx-receiver"
+    uwsgi -b 32768 --socket 127.0.0.1:8050 --module=awx.wsgi:application --home=/var/lib/awx/venv/awx --chdir=/awx_devel/ --vacuum --processes=5 --harakiri=120 --master --no-orphans --py-autoreload 1 --max-requests=1000 --stats /tmp/stats.socket --lazy-apps --logformat "%(addr) %(method) %(uri) - %(proto) %(status)" --hook-accepting1="exec:supervisorctl restart tower-processes:awx-dispatcher tower-processes:awx-receiver"

 daphne:
 	@if [ "$(VENV_BASE)" ]; then \
@@ -349,20 +340,20 @@ swagger: reports
 check: flake8 pep8 # pyflakes pylint

 awx-link:
-	cp -R /tmp/awx.egg-info /awx_devel/ || true
-	sed -i "s/placeholder/$(shell cat VERSION)/" /awx_devel/awx.egg-info/PKG-INFO
-	cp -f /tmp/awx.egg-link /venv/awx/lib/python$(PYTHON_VERSION)/site-packages/awx.egg-link
+	[ -d "/awx_devel/awx.egg-info" ] || python3 /awx_devel/setup.py egg_info_dev
+	cp -f /tmp/awx.egg-link /var/lib/awx/venv/awx/lib/python$(PYTHON_VERSION)/site-packages/awx.egg-link

 TEST_DIRS ?= awx/main/tests/unit awx/main/tests/functional awx/conf/tests awx/sso/tests

 # Run all API unit tests.
 test:
-	@if [ "$(VENV_BASE)" ]; then \
+	if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
 	PYTHONDONTWRITEBYTECODE=1 py.test -p no:cacheprovider -n auto $(TEST_DIRS)
-	cd awxkit && $(VENV_BASE)/awx/bin/tox -re py2,py3
-	awx-manage check_migrations --dry-run --check  -n 'vNNN_missing_migration_file'
+	cmp VERSION awxkit/VERSION || "VERSION and awxkit/VERSION *must* match"
+	cd awxkit && $(VENV_BASE)/awx/bin/tox -re py3
+	awx-manage check_migrations --dry-run --check  -n 'missing_migration_file'

 COLLECTION_TEST_DIRS ?= awx_collection/test/awx
 COLLECTION_TEST_TARGET ?=
@@ -371,10 +362,15 @@ COLLECTION_NAMESPACE ?= awx
 COLLECTION_INSTALL = ~/.ansible/collections/ansible_collections/$(COLLECTION_NAMESPACE)/$(COLLECTION_PACKAGE)

 test_collection:
-	@if [ "$(VENV_BASE)" ]; then \
+	rm -f $(shell ls -d $(VENV_BASE)/awx/lib/python* | head -n 1)/no-global-site-packages.txt
+	if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
-	PYTHONPATH=$PYTHONPATH:/usr/lib/python3.6/site-packages py.test $(COLLECTION_TEST_DIRS)
+	py.test $(COLLECTION_TEST_DIRS) -v
+	# The python path needs to be modified so that the tests can find Ansible within the container
+	# First we will use anything expility set as PYTHONPATH
+	# Second we will load any libraries out of the virtualenv (if it's unspecified that should be ok because python should not load out of an empty directory)
+	# Finally we will add the system path so that the tests can find the ansible libraries

 flake8_collection:
 	flake8 awx_collection/  # Different settings, in main exclude list
@@ -389,12 +385,12 @@ symlink_collection:
 	ln -s $(shell pwd)/awx_collection $(COLLECTION_INSTALL)

 build_collection:
-	ansible-playbook -i localhost, awx_collection/template_galaxy.yml -e collection_package=$(COLLECTION_PACKAGE) -e collection_namespace=$(COLLECTION_NAMESPACE) -e collection_version=$(VERSION)
-	ansible-galaxy collection build awx_collection --force --output-path=awx_collection
+	ansible-playbook -i localhost, awx_collection/tools/template_galaxy.yml -e collection_package=$(COLLECTION_PACKAGE) -e collection_namespace=$(COLLECTION_NAMESPACE) -e collection_version=$(VERSION) -e '{"awx_template_version":false}'
+	ansible-galaxy collection build awx_collection_build --force --output-path=awx_collection_build

 install_collection: build_collection
 	rm -rf $(COLLECTION_INSTALL)
-	ansible-galaxy collection install awx_collection/$(COLLECTION_NAMESPACE)-$(COLLECTION_PACKAGE)-$(VERSION).tar.gz
+	ansible-galaxy collection install awx_collection_build/$(COLLECTION_NAMESPACE)-$(COLLECTION_PACKAGE)-$(VERSION).tar.gz

 test_collection_sanity: install_collection
 	cd $(COLLECTION_INSTALL) && ansible-test sanity
@@ -464,114 +460,47 @@ else
 	@echo No PO files
 endif

-# generate UI .pot
-pot: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run pot
-
-# generate django .pot .po
-LANG = "en-us"
-messages:
-	@if [ "$(VENV_BASE)" ]; then \
-		. $(VENV_BASE)/awx/bin/activate; \
-	fi; \
-	$(PYTHON) manage.py makemessages -l $(LANG) --keep-pot
-
-# generate l10n .json .mo
-languages: $(I18N_FLAG_FILE)
-
-$(I18N_FLAG_FILE): $(UI_RELEASE_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run languages
-	$(PYTHON) tools/scripts/compilemessages.py
-	touch $(I18N_FLAG_FILE)
-
-# End l10n TASKS
-# --------------------------------------
-
-# UI RELEASE TASKS
-# --------------------------------------
-ui-release: $(UI_RELEASE_FLAG_FILE)
-
-$(UI_RELEASE_FLAG_FILE): $(I18N_FLAG_FILE) $(UI_RELEASE_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run build-release
-	touch $(UI_RELEASE_FLAG_FILE)
-
-$(UI_RELEASE_DEPS_FLAG_FILE):
-	PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 $(NPM_BIN) --unsafe-perm --prefix awx/ui ci --no-save awx/ui
-	touch $(UI_RELEASE_DEPS_FLAG_FILE)
-
-# END UI RELEASE TASKS
-# --------------------------------------

 # UI TASKS
 # --------------------------------------
-ui-deps: $(UI_DEPS_FLAG_FILE)

-$(UI_DEPS_FLAG_FILE):
-	@if [ -f ${UI_RELEASE_DEPS_FLAG_FILE} ]; then \
-		rm -rf awx/ui/node_modules; \
-		rm -f ${UI_RELEASE_DEPS_FLAG_FILE}; \
-	fi; \
-	$(NPM_BIN) --unsafe-perm --prefix awx/ui ci --no-save awx/ui
-	touch $(UI_DEPS_FLAG_FILE)
+UI_BUILD_FLAG_FILE = awx/ui_next/.ui-built

-ui-docker-machine: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run ui-docker-machine -- $(MAKEFLAGS)
+clean-ui:
+	rm -rf node_modules
+	rm -rf awx/ui_next/node_modules
+	rm -rf awx/ui_next/build
+	rm -rf awx/ui_next/src/locales/_build
+	rm -rf $(UI_BUILD_FLAG_FILE)
+	git checkout awx/ui_next/src/locales

-# Native docker. Builds UI and raises BrowserSync & filesystem polling.
-ui-docker: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run ui-docker -- $(MAKEFLAGS)
+awx/ui_next/node_modules:
+	$(NPM_BIN) --prefix awx/ui_next --loglevel warn --ignore-scripts install

-# Builds UI with development UI without raising browser-sync or filesystem polling.
-ui-devel: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run build-devel -- $(MAKEFLAGS)
+$(UI_BUILD_FLAG_FILE):
+	$(NPM_BIN) --prefix awx/ui_next --loglevel warn run extract-strings
+	$(NPM_BIN) --prefix awx/ui_next --loglevel warn run compile-strings
+	$(NPM_BIN) --prefix awx/ui_next --loglevel warn run build
+	git checkout awx/ui_next/src/locales
+	mkdir -p awx/public/static/css
+	mkdir -p awx/public/static/js
+	mkdir -p awx/public/static/media
+	cp -r awx/ui_next/build/static/css/* awx/public/static/css
+	cp -r awx/ui_next/build/static/js/* awx/public/static/js
+	cp -r awx/ui_next/build/static/media/* awx/public/static/media
+	touch $@

-ui-test: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run test
+ui-release: awx/ui_next/node_modules $(UI_BUILD_FLAG_FILE)

-ui-lint: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) run --prefix awx/ui jshint
-	$(NPM_BIN) run --prefix awx/ui lint
+ui-devel: awx/ui_next/node_modules
+	@$(MAKE) -B $(UI_BUILD_FLAG_FILE)

-# A standard go-to target for API developers to use building the frontend
-ui: clean-ui ui-devel
-
-ui-test-ci: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) --prefix awx/ui run test:ci
-	$(NPM_BIN) --prefix awx/ui run unit
-
-jshint: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) run --prefix awx/ui jshint
-	$(NPM_BIN) run --prefix awx/ui lint
-
-ui-zuul-lint-and-test: $(UI_DEPS_FLAG_FILE)
-	$(NPM_BIN) run --prefix awx/ui jshint
-	$(NPM_BIN) run --prefix awx/ui lint
-	$(NPM_BIN) --prefix awx/ui run test:ci
-	$(NPM_BIN) --prefix awx/ui run unit
-
-# END UI TASKS
-# --------------------------------------
-
-# UI NEXT TASKS
-# --------------------------------------
-
-ui-next-lint:
-	$(NPM_BIN) --prefix awx/ui_next install
-	$(NPM_BIN) run --prefix awx/ui_next lint
-	$(NPM_BIN) run --prefix awx/ui_next prettier-check
-
-ui-next-test:
-	$(NPM_BIN) --prefix awx/ui_next install
-	$(NPM_BIN) run --prefix awx/ui_next test
-
-ui-next-zuul-lint-and-test:
+ui-zuul-lint-and-test:
 	$(NPM_BIN) --prefix awx/ui_next install
 	$(NPM_BIN) run --prefix awx/ui_next lint
 	$(NPM_BIN) run --prefix awx/ui_next prettier-check
 	$(NPM_BIN) run --prefix awx/ui_next test

-# END UI NEXT TASKS
-# --------------------------------------

 # Build a pip-installable package into dist/ with a timestamped version number.
 dev_build:
@@ -618,9 +547,11 @@ awx/projects:
 docker-compose-isolated: awx/projects
 	CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/docker-isolated-override.yml up

+COMPOSE_UP_OPTS ?=
+
 # Docker Compose Development environment
 docker-compose: docker-auth awx/projects
-	CURRENT_UID=$(shell id -u) OS="$(shell docker info | grep 'Operating System')" TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml up --no-recreate awx
+	CURRENT_UID=$(shell id -u) OS="$(shell docker info | grep 'Operating System')" TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml $(COMPOSE_UP_OPTS) up --no-recreate awx

 docker-compose-cluster: docker-auth awx/projects
 	CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose-cluster.yml up
@@ -636,7 +567,7 @@ docker-compose-runtest: awx/projects
 	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /start_tests.sh

 docker-compose-build-swagger: awx/projects
-	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /start_tests.sh swagger
+	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports --no-deps awx /start_tests.sh swagger

 detect-schema-change: genschema
 	curl https://s3.amazonaws.com/awx-public-ci-files/schema.json -o reference-schema.json
@@ -644,20 +575,18 @@ detect-schema-change: genschema
 	diff -u -b reference-schema.json schema.json

 docker-compose-clean: awx/projects
-	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm -w /awx_devel --service-ports awx make clean
 	cd tools && TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose rm -sf

-docker-compose-build: awx-devel-build
-
 # Base development image build
-awx-devel-build:
+docker-compose-build:
+	ansible localhost -m template -a "src=installer/roles/image_build/templates/Dockerfile.j2 dest=tools/docker-compose/Dockerfile" -e build_dev=True
 	docker build -t ansible/awx_devel -f tools/docker-compose/Dockerfile \
 		--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) .
 	docker tag ansible/awx_devel $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
 	#docker push $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)

 # For use when developing on "isolated" AWX deployments
-docker-compose-isolated-build: awx-devel-build
+docker-compose-isolated-build: docker-compose-build
 	docker build -t ansible/awx_isolated -f tools/docker-isolated/Dockerfile .
 	docker tag ansible/awx_isolated $(DEV_DOCKER_TAG_BASE)/awx_isolated:$(COMPOSE_TAG)
 	#docker push $(DEV_DOCKER_TAG_BASE)/awx_isolated:$(COMPOSE_TAG)
@@ -666,7 +595,7 @@ docker-clean:
 	$(foreach container_id,$(shell docker ps -f name=tools_awx -aq),docker stop $(container_id); docker rm -f $(container_id);)
 	docker images | grep "awx_devel" | awk '{print $$1 ":" $$2}' | xargs docker rmi

-docker-clean-volumes:
+docker-clean-volumes: docker-compose-clean
 	docker volume rm tools_awx_db

 docker-refresh: docker-clean docker-compose
@@ -690,7 +619,10 @@ clean-elk:
 	docker rm tools_kibana_1

 psql-container:
-	docker run -it --net tools_default --rm postgres:10 sh -c 'exec psql -h "postgres" -p "5432" -U postgres'
+	docker run -it --net tools_default --rm postgres:12 sh -c 'exec psql -h "postgres" -p "5432" -U postgres'

 VERSION:
 	@echo "awx: $(VERSION)"
+
+Dockerfile: installer/roles/image_build/templates/Dockerfile.j2
+	ansible localhost -m template -a "src=installer/roles/image_build/templates/Dockerfile.j2 dest=Dockerfile"
--- a/README.md
+++ b/README.md
@@ -1,7 +1,5 @@
 [![Gated by Zuul](https://zuul-ci.org/gated.svg)](https://ansible.softwarefactory-project.io/zuul/status)

-<img src="https://raw.githubusercontent.com/ansible/awx-logos/master/awx/ui/client/assets/logo-login.svg?sanitize=true" width=200 alt="AWX" />
-
 AWX provides a web-based user interface, REST API, and task engine built on top of [Ansible](https://github.com/ansible/ansible). It is the upstream project for [Tower](https://www.ansible.com/tower), a commercial derivative of AWX.  

 To install AWX, please view the [Install guide](./INSTALL.md).
@@ -16,20 +14,20 @@ Contributing
 ------------

 - Refer to the [Contributing guide](./CONTRIBUTING.md) to get started developing, testing, and building AWX.
- All code submissions are done through pull requests against the `devel` branch.
- All contributors must use git commit --signoff for any commit to be merged, and agree that usage of --signoff constitutes agreement with the terms of [DCO 1.1](./DCO_1_1.md)
- Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason.
- If submitting a large code change, it's a good idea to join the `#ansible-awx` channel on irc.freenode.net, and talk about what you would like to do or add first. This not only helps everyone know what's going on, it also helps save time and effort, if the community decides some changes are needed.
+- All code submissions are made through pull requests against the `devel` branch.
+- All contributors must use git commit --signoff for any commit to be merged and agree that usage of --signoff constitutes agreement with the terms of [DCO 1.1](./DCO_1_1.md)
+- Take care to make sure no merge commits are in the submission, and use `git rebase` vs. `git merge` for this reason.
+- If submitting a large code change, it's a good idea to join the `#ansible-awx` channel on irc.freenode.net and talk about what you would like to do or add first. This not only helps everyone know what's going on, but it also helps save time and effort if the community decides some changes are needed.

 Reporting Issues
 ----------------

-If you're experiencing a problem that you feel is a bug in AWX, or have ideas for how to improve AWX, we encourage you to open an issue, and share your feedback. But before opening a new issue, we ask that you please take a look at our [Issues guide](./ISSUES.md).
+If you're experiencing a problem that you feel is a bug in AWX or have ideas for improving AWX, we encourage you to open an issue and share your feedback. But before opening a new issue, we ask that you please take a look at our [Issues guide](./ISSUES.md).

 Code of Conduct
 ---------------

-We ask all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions, or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)   
+We ask all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)   

 Get Involved
 ------------
@@ -43,4 +41,3 @@ License
 -------

 [Apache v2](./LICENSE.md)
-
--- a/2
+++ b/2
@@ -1 +1 @@
-11.0.0
+17.0.1
--- a/awx/init.py
+++ b/awx/init.py
@@ -30,6 +30,7 @@ except ImportError:
    HAS_DJANGO = False
 else:
    from django.db.backends.base import schema
+    from django.db.models import indexes
    from django.db.backends.utils import names_digest


@@ -50,6 +51,7 @@ if HAS_DJANGO is True:
            return h.hexdigest()[:length]

        schema.names_digest = names_digest
+        indexes.names_digest = names_digest


 def find_commands(management_dir):
--- a/awx/api/conf.py
+++ b/awx/api/conf.py
@@ -16,6 +16,7 @@ register(
    help_text=_('Number of seconds that a user is inactive before they will need to login again.'),
    category=_('Authentication'),
    category_slug='authentication',
+    unit=_('seconds'),
 )
 register(
    'SESSIONS_PER_USER',
@@ -49,6 +50,7 @@ register(
                'in the number of seconds.'),
    category=_('Authentication'),
    category_slug='authentication',
+    unit=_('seconds'),
 )
 register(
    'ALLOW_OAUTH2_FOR_EXTERNAL_USERS',
--- a/awx/api/filters.py
+++ b/awx/api/filters.py
@@ -146,7 +146,7 @@ class FieldLookupBackend(BaseFilterBackend):

    # A list of fields that we know can be filtered on without the possiblity
    # of introducing duplicates
-    NO_DUPLICATES_WHITELIST = (CharField, IntegerField, BooleanField)
+    NO_DUPLICATES_ALLOW_LIST = (CharField, IntegerField, BooleanField)

    def get_fields_from_lookup(self, model, lookup):

@@ -205,7 +205,7 @@ class FieldLookupBackend(BaseFilterBackend):
        field_list, new_lookup = self.get_fields_from_lookup(model, lookup)
        field = field_list[-1]

-        needs_distinct = (not all(isinstance(f, self.NO_DUPLICATES_WHITELIST) for f in field_list))
+        needs_distinct = (not all(isinstance(f, self.NO_DUPLICATES_ALLOW_LIST) for f in field_list))

        # Type names are stored without underscores internally, but are presented and
        # and serialized over the API containing underscores so we remove `_`
@@ -257,6 +257,11 @@ class FieldLookupBackend(BaseFilterBackend):
                if key in self.RESERVED_NAMES:
                    continue

+                # HACK: make `created` available via API for the Django User ORM model
+                # so it keep compatiblity with other objects which exposes the `created` attr.
+                if queryset.model._meta.object_name == 'User' and key.startswith('created'):
+                    key = key.replace('created', 'date_joined')
+
                # HACK: Make job event filtering by host name mostly work even
                # when not capturing job event hosts M2M.
                if queryset.model._meta.object_name == 'JobEvent' and key.startswith('hosts__name'):
--- a/awx/api/generics.py
+++ b/awx/api/generics.py
@@ -45,9 +45,11 @@ from awx.main.utils import (
    get_search_fields,
    getattrd,
    get_object_or_400,
-    decrypt_field
+    decrypt_field,
+    get_awx_version,
 )
 from awx.main.utils.db import get_all_field_names
+from awx.main.views import ApiErrorView
 from awx.api.serializers import ResourceAccessListElementSerializer, CopySerializer, UserSerializer
 from awx.api.versioning import URLPathVersioning
 from awx.api.metadata import SublistAttachDetatchMetadata, Metadata
@@ -156,11 +158,11 @@ class APIView(views.APIView):
            self.queries_before = len(connection.queries)

        # If there are any custom headers in REMOTE_HOST_HEADERS, make sure
-        # they respect the proxy whitelist
+        # they respect the allowed proxy list
        if all([
-            settings.PROXY_IP_WHITELIST,
-            request.environ.get('REMOTE_ADDR') not in settings.PROXY_IP_WHITELIST,
-            request.environ.get('REMOTE_HOST') not in settings.PROXY_IP_WHITELIST
+            settings.PROXY_IP_ALLOWED_LIST,
+            request.environ.get('REMOTE_ADDR') not in settings.PROXY_IP_ALLOWED_LIST,
+            request.environ.get('REMOTE_HOST') not in settings.PROXY_IP_ALLOWED_LIST
        ]):
            for custom_header in settings.REMOTE_HOST_HEADERS:
                if custom_header.startswith('HTTP_'):
@@ -185,6 +187,30 @@ class APIView(views.APIView):
        '''
        Log warning for 400 requests.  Add header with elapsed time.
        '''
+        from awx.main.utils import get_licenser
+        from awx.main.utils.licensing import OpenLicense
+        #
+        # If the URL was rewritten, and we get a 404, we should entirely
+        # replace the view in the request context with an ApiErrorView()
+        # Without this change, there will be subtle differences in the BrowseableAPIRenderer
+        #
+        # These differences could provide contextual clues which would allow
+        # anonymous users to determine if usernames were valid or not
+        # (e.g., if an anonymous user visited `/api/v2/users/valid/`, and got a 404,
+        # but also saw that the page heading said "User Detail", they might notice
+        # that's a difference in behavior from a request to `/api/v2/users/not-valid/`, which
+        # would show a page header of "Not Found").  Changing the view here
+        # guarantees that the rendered response will look exactly like the response
+        # when you visit a URL that has no matching URL paths in `awx.api.urls`.
+        #
+        if response.status_code == 404 and 'awx.named_url_rewritten' in request.environ:
+            self.headers.pop('Allow', None)
+            response = super(APIView, self).finalize_response(request, response, *args, **kwargs)
+            view = ApiErrorView()
+            setattr(view, 'request', request)
+            response.renderer_context['view'] = view
+            return response
+
        if response.status_code >= 400:
            status_msg = "status %s received by user %s attempting to access %s from %s" % \
                         (response.status_code, request.user, request.path, request.META.get('REMOTE_ADDR', None))
@@ -197,6 +223,9 @@ class APIView(views.APIView):
                logger.warning(status_msg)
        response = super(APIView, self).finalize_response(request, response, *args, **kwargs)
        time_started = getattr(self, 'time_started', None)
+        response['X-API-Product-Version'] = get_awx_version()
+        response['X-API-Product-Name'] = 'AWX' if isinstance(get_licenser(), OpenLicense) else 'Red Hat Ansible Tower'
+        
        response['X-API-Node'] = settings.CLUSTER_HOST_ID
        if time_started:
            time_elapsed = time.time() - self.time_started
@@ -832,7 +861,7 @@ class CopyAPIView(GenericAPIView):

    @staticmethod
    def _decrypt_model_field_if_needed(obj, field_name, field_val):
-        if field_name in getattr(type(obj), 'REENCRYPTION_BLACKLIST_AT_COPY', []):
+        if field_name in getattr(type(obj), 'REENCRYPTION_BLOCKLIST_AT_COPY', []):
            return field_val
        if isinstance(obj, Credential) and field_name == 'inputs':
            for secret in obj.credential_type.secret_fields:
@@ -878,7 +907,7 @@ class CopyAPIView(GenericAPIView):
                field_val = getattr(obj, field.name)
            except AttributeError:
                continue
-            # Adjust copy blacklist fields here.
+            # Adjust copy blocked fields here.
            if field.name in fields_to_discard or field.name in [
                'id', 'pk', 'polymorphic_ctype', 'unifiedjobtemplate_ptr', 'created_by', 'modified_by'
            ] or field.name.endswith('_role'):
@@ -975,7 +1004,7 @@ class CopyAPIView(GenericAPIView):
        if hasattr(new_obj, 'admin_role') and request.user not in new_obj.admin_role.members.all():
            new_obj.admin_role.members.add(request.user)
        if sub_objs:
-            # store the copied object dict into memcached, because it's
+            # store the copied object dict into cache, because it's
            # often too large for postgres' notification bus
            # (which has a default maximum message size of 8k)
            key = 'deep-copy-{}'.format(str(uuid.uuid4()))
--- a/awx/api/metadata.py
+++ b/awx/api/metadata.py
@@ -23,7 +23,7 @@ from rest_framework.request import clone_request
 # AWX
 from awx.api.fields import ChoiceNullField
 from awx.main.fields import JSONField, ImplicitRoleField
-from awx.main.models import InventorySource, NotificationTemplate
+from awx.main.models import NotificationTemplate
 from awx.main.scheduler.kubernetes import PodManager


@@ -39,7 +39,7 @@ class Metadata(metadata.SimpleMetadata):
            'min_length', 'max_length',
            'min_value', 'max_value',
            'category', 'category_slug',
-            'defined_in_file'
+            'defined_in_file', 'unit',
        ]

        for attr in text_attrs:
@@ -115,19 +115,6 @@ class Metadata(metadata.SimpleMetadata):
        if getattr(field, 'write_only', False):
            field_info['write_only'] = True

-        # Special handling of inventory source_region choices that vary based on
-        # selected inventory source.
-        if field.field_name == 'source_regions':
-            for cp in ('azure_rm', 'ec2', 'gce'):
-                get_regions = getattr(InventorySource, 'get_%s_region_choices' % cp)
-                field_info['%s_region_choices' % cp] = get_regions()
-
-        # Special handling of group_by choices for EC2.
-        if field.field_name == 'group_by':
-            for cp in ('ec2',):
-                get_group_by_choices = getattr(InventorySource, 'get_%s_group_by_choices' % cp)
-                field_info['%s_group_by_choices' % cp] = get_group_by_choices()
-
        # Special handling of notification configuration where the required properties
        # are conditional on the type selected.
        if field.field_name == 'notification_configuration':
--- a/awx/api/renderers.py
+++ b/awx/api/renderers.py
@@ -7,6 +7,24 @@ from prometheus_client.parser import text_string_to_metric_families
 # Django REST Framework
 from rest_framework import renderers
 from rest_framework.request import override_method
+from rest_framework.utils import encoders
+
+
+class SurrogateEncoder(encoders.JSONEncoder):
+
+    def encode(self, obj):
+        ret = super(SurrogateEncoder, self).encode(obj)
+        try:
+            ret.encode()
+        except UnicodeEncodeError as e:
+            if 'surrogates not allowed' in e.reason:
+                ret = ret.encode('utf-8', 'replace').decode()
+        return ret
+
+
+class DefaultJSONRenderer(renderers.JSONRenderer):
+
+    encoder_class = SurrogateEncoder


 class BrowsableAPIRenderer(renderers.BrowsableAPIRenderer):
--- a/awx/api/serializers.py
+++ b/awx/api/serializers.py
@@ -126,7 +126,7 @@ SUMMARIZABLE_FK_FIELDS = {
    'current_job': DEFAULT_SUMMARY_FIELDS + ('status', 'failed', 'license_error'),
    'inventory_source': ('source', 'last_updated', 'status'),
    'custom_inventory_script': DEFAULT_SUMMARY_FIELDS,
-    'source_script': ('name', 'description'),
+    'source_script': DEFAULT_SUMMARY_FIELDS,
    'role': ('id', 'role_field'),
    'notification_template': DEFAULT_SUMMARY_FIELDS,
    'instance_group': ('id', 'name', 'controller_id', 'is_containerized'),
@@ -453,7 +453,7 @@ class BaseSerializer(serializers.ModelSerializer, metaclass=BaseSerializerMetacl
                if 'capability_map' not in self.context:
                    if hasattr(self, 'polymorphic_base'):
                        model = self.polymorphic_base.Meta.model
-                        prefetch_list = self.polymorphic_base._capabilities_prefetch
+                        prefetch_list = self.polymorphic_base.capabilities_prefetch
                    else:
                        model = self.Meta.model
                        prefetch_list = self.capabilities_prefetch
@@ -640,12 +640,9 @@ class EmptySerializer(serializers.Serializer):


 class UnifiedJobTemplateSerializer(BaseSerializer):
-    # As a base serializer, the capabilities prefetch is not used directly
-    _capabilities_prefetch = [
-        'admin', 'execute',
-        {'copy': ['jobtemplate.project.use', 'jobtemplate.inventory.use',
-                  'organization.workflow_admin']}
-    ]
+    # As a base serializer, the capabilities prefetch is not used directly,
+    # instead they are derived from the Workflow Job Template Serializer and the Job Template Serializer, respectively.
+    capabilities_prefetch = []

    class Meta:
        model = UnifiedJobTemplate
@@ -695,7 +692,7 @@ class UnifiedJobTemplateSerializer(BaseSerializer):
                serializer.polymorphic_base = self
                # capabilities prefetch is only valid for these models
                if isinstance(obj, (JobTemplate, WorkflowJobTemplate)):
-                    serializer.capabilities_prefetch = self._capabilities_prefetch
+                    serializer.capabilities_prefetch = serializer_class.capabilities_prefetch
                else:
                    serializer.capabilities_prefetch = None
            return serializer.to_representation(obj)
@@ -806,7 +803,9 @@ class UnifiedJobSerializer(BaseSerializer):
                td = now() - obj.started
                ret['elapsed'] = (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10 ** 6) / (10 ** 6 * 1.0)
            ret['elapsed'] = float(ret['elapsed'])
-
+        # Because this string is saved in the db in the source language,
+        # it must be marked for translation after it is pulled from the db, not when set
+        ret['job_explanation'] = _(obj.job_explanation)
        return ret


@@ -1267,6 +1266,7 @@ class OrganizationSerializer(BaseSerializer):
            object_roles = self.reverse('api:organization_object_roles_list', kwargs={'pk': obj.pk}),
            access_list = self.reverse('api:organization_access_list', kwargs={'pk': obj.pk}),
            instance_groups = self.reverse('api:organization_instance_groups_list', kwargs={'pk': obj.pk}),
+            galaxy_credentials = self.reverse('api:organization_galaxy_credentials_list', kwargs={'pk': obj.pk}),
        ))
        return res

@@ -1330,10 +1330,14 @@ class ProjectOptionsSerializer(BaseSerializer):
            scm_type = attrs.get('scm_type', u'') or u''
        if self.instance and not scm_type:
            valid_local_paths.append(self.instance.local_path)
+        if self.instance and scm_type and "local_path" in attrs and self.instance.local_path != attrs['local_path']:
+            errors['local_path'] = _(f'Cannot change local_path for {scm_type}-based projects')
        if scm_type:
            attrs.pop('local_path', None)
        if 'local_path' in attrs and attrs['local_path'] not in valid_local_paths:
            errors['local_path'] = _('This path is already being used by another manual project.')
+        if attrs.get('scm_branch') and scm_type == 'archive':
+            errors['scm_branch'] = _('SCM branch cannot be used with archive projects.')
        if attrs.get('scm_refspec') and scm_type != 'git':
            errors['scm_refspec'] = _('SCM refspec can only be used with git projects.')

@@ -1695,9 +1699,13 @@ class HostSerializer(BaseSerializerWithVariables):
        d.setdefault('recent_jobs', [{
            'id': j.job.id,
            'name': j.job.job_template.name if j.job.job_template is not None else "",
+            'type': j.job.job_type_name,
            'status': j.job.status,
            'finished': j.job.finished,
-        } for j in obj.job_host_summaries.select_related('job__job_template').order_by('-created')[:5]])
+        } for j in obj.job_host_summaries.select_related('job__job_template').order_by('-created').defer(
+            'job__extra_vars',
+            'job__artifacts',
+        )[:5]])
        return d

    def _get_host_port_from_name(self, name):
@@ -1729,6 +1737,7 @@ class HostSerializer(BaseSerializerWithVariables):

    def validate(self, attrs):
        name = force_text(attrs.get('name', self.instance and self.instance.name or ''))
+        inventory = attrs.get('inventory', self.instance and self.instance.inventory or '')
        host, port = self._get_host_port_from_name(name)

        if port:
@@ -1737,6 +1746,8 @@ class HostSerializer(BaseSerializerWithVariables):
            vars_dict = parse_yaml_or_json(variables)
            vars_dict['ansible_ssh_port'] = port
            attrs['variables'] = json.dumps(vars_dict)
+        if Group.objects.filter(name=name, inventory=inventory).exists():
+            raise serializers.ValidationError(_('A Group with that name already exists.'))

        return super(HostSerializer, self).validate(attrs)

@@ -1803,6 +1814,13 @@ class GroupSerializer(BaseSerializerWithVariables):
            res['inventory'] = self.reverse('api:inventory_detail', kwargs={'pk': obj.inventory.pk})
        return res

+    def validate(self, attrs):
+        name = force_text(attrs.get('name', self.instance and self.instance.name or ''))
+        inventory = attrs.get('inventory', self.instance and self.instance.inventory or '')
+        if Host.objects.filter(name=name, inventory=inventory).exists():
+            raise serializers.ValidationError(_('A Host with that name already exists.'))
+        return super(GroupSerializer, self).validate(attrs)
+
    def validate_name(self, value):
        if value in ('all', '_meta'):
            raise serializers.ValidationError(_('Invalid group name.'))
@@ -1919,7 +1937,7 @@ class InventorySourceOptionsSerializer(BaseSerializer):

    class Meta:
        fields = ('*', 'source', 'source_path', 'source_script', 'source_vars', 'credential',
-                  'source_regions', 'instance_filters', 'group_by', 'overwrite', 'overwrite_vars',
+                  'enabled_var', 'enabled_value', 'host_filter', 'overwrite', 'overwrite_vars',
                  'custom_virtualenv', 'timeout', 'verbosity')

    def get_related(self, obj):
@@ -1934,12 +1952,12 @@ class InventorySourceOptionsSerializer(BaseSerializer):
    def validate_source_vars(self, value):
        ret = vars_validate_or_raise(value)
        for env_k in parse_yaml_or_json(value):
-            if env_k in settings.INV_ENV_VARIABLE_BLACKLIST:
+            if env_k in settings.INV_ENV_VARIABLE_BLOCKED:
                raise serializers.ValidationError(_("`{}` is a prohibited environment variable".format(env_k)))
        return ret

    def validate(self, attrs):
-        # TODO: Validate source, validate source_regions
+        # TODO: Validate source
        errors = {}

        source = attrs.get('source', self.instance and self.instance.source or '')
@@ -2306,6 +2324,7 @@ class RoleSerializer(BaseSerializer):
            content_model = obj.content_type.model_class()
            ret['summary_fields']['resource_type'] = get_type_for_model(content_model)
            ret['summary_fields']['resource_type_display_name'] = content_model._meta.verbose_name.title()
+            ret['summary_fields']['resource_id'] = obj.object_id

        return ret

@@ -2517,10 +2536,11 @@ class CredentialTypeSerializer(BaseSerializer):
 class CredentialSerializer(BaseSerializer):
    show_capabilities = ['edit', 'delete', 'copy', 'use']
    capabilities_prefetch = ['admin', 'use']
+    managed_by_tower = serializers.ReadOnlyField()

    class Meta:
        model = Credential
-        fields = ('*', 'organization', 'credential_type', 'inputs', 'kind', 'cloud', 'kubernetes')
+        fields = ('*', 'organization', 'credential_type', 'managed_by_tower', 'inputs', 'kind', 'cloud', 'kubernetes')
        extra_kwargs = {
            'credential_type': {
                'label': _('Credential Type'),
@@ -2584,6 +2604,13 @@ class CredentialSerializer(BaseSerializer):

        return summary_dict

+    def validate(self, attrs):
+        if self.instance and self.instance.managed_by_tower:
+            raise PermissionDenied(
+                detail=_("Modifications not allowed for managed credentials")
+            )
+        return super(CredentialSerializer, self).validate(attrs)
+
    def get_validation_exclusions(self, obj=None):
        ret = super(CredentialSerializer, self).get_validation_exclusions(obj)
        for field in ('credential_type', 'inputs'):
@@ -2591,6 +2618,17 @@ class CredentialSerializer(BaseSerializer):
                ret.remove(field)
        return ret

+    def validate_organization(self, org):
+        if (
+            self.instance and
+            self.instance.credential_type.kind == 'galaxy' and
+            org is None
+        ):
+            raise serializers.ValidationError(_(
+                "Galaxy credentials must be owned by an Organization."
+            ))
+        return org
+
    def validate_credential_type(self, credential_type):
        if self.instance and credential_type.pk != self.instance.credential_type.pk:
            for related_objects in (
@@ -2641,12 +2679,29 @@ class CredentialSerializerCreate(CredentialSerializer):
                    owner_fields.add(field)
                else:
                    attrs.pop(field)
+
        if not owner_fields:
            raise serializers.ValidationError({"detail": _("Missing 'user', 'team', or 'organization'.")})

+        if len(owner_fields) > 1:
+            received = ", ".join(sorted(owner_fields))
+            raise serializers.ValidationError({"detail": _(
+                "Only one of 'user', 'team', or 'organization' should be provided, "
+                "received {} fields.".format(received)
+            )})
+
        if attrs.get('team'):
            attrs['organization'] = attrs['team'].organization

+        if (
+            'credential_type' in attrs and
+            attrs['credential_type'].kind == 'galaxy' and
+            list(owner_fields) != ['organization']
+        ):
+            raise serializers.ValidationError({"organization": _(
+                "Galaxy credentials must be owned by an Organization."
+            )})
+
        return super(CredentialSerializerCreate, self).validate(attrs)

    def create(self, validated_data):
@@ -2756,16 +2811,11 @@ class JobOptionsSerializer(LabelsListMixin, BaseSerializer):
        if obj.organization_id:
            res['organization'] = self.reverse('api:organization_detail', kwargs={'pk': obj.organization_id})
        if isinstance(obj, UnifiedJobTemplate):
-            res['extra_credentials'] = self.reverse(
-                'api:job_template_extra_credentials_list',
-                kwargs={'pk': obj.pk}
-            )
            res['credentials'] = self.reverse(
                'api:job_template_credentials_list',
                kwargs={'pk': obj.pk}
            )
        elif isinstance(obj, UnifiedJob):
-            res['extra_credentials'] = self.reverse('api:job_extra_credentials_list', kwargs={'pk': obj.pk})
            res['credentials'] = self.reverse('api:job_credentials_list', kwargs={'pk': obj.pk})

        return res
@@ -2825,7 +2875,7 @@ class JobTemplateMixin(object):
        return [{
            'id': x.id, 'status': x.status, 'finished': x.finished, 'canceled_on': x.canceled_on,
            # Make type consistent with API top-level key, for instance workflow_job
-            'type': x.get_real_instance_class()._meta.verbose_name.replace(' ', '_')
+            'type': x.job_type_name
        } for x in optimized_qs[:10]]

    def get_summary_fields(self, obj):
@@ -2934,7 +2984,6 @@ class JobTemplateSerializer(JobTemplateMixin, UnifiedJobTemplateSerializer, JobO
        summary_fields = super(JobTemplateSerializer, self).get_summary_fields(obj)
        all_creds = []
        # Organize credential data into multitude of deprecated fields
-        extra_creds = []
        if obj.pk:
            for cred in obj.credentials.all():
                summarized_cred = {
@@ -2945,10 +2994,6 @@ class JobTemplateSerializer(JobTemplateMixin, UnifiedJobTemplateSerializer, JobO
                    'cloud': cred.credential_type.kind == 'cloud'
                }
                all_creds.append(summarized_cred)
-                if cred.credential_type.kind in ('cloud', 'net'):
-                    extra_creds.append(summarized_cred)
-        if self.is_detail_view:
-            summary_fields['extra_credentials'] = extra_creds
        summary_fields['credentials'] = all_creds
        return summary_fields

@@ -3023,7 +3068,6 @@ class JobSerializer(UnifiedJobSerializer, JobOptionsSerializer):
        summary_fields = super(JobSerializer, self).get_summary_fields(obj)
        all_creds = []
        # Organize credential data into multitude of deprecated fields
-        extra_creds = []
        if obj.pk:
            for cred in obj.credentials.all():
                summarized_cred = {
@@ -3034,10 +3078,6 @@ class JobSerializer(UnifiedJobSerializer, JobOptionsSerializer):
                    'cloud': cred.credential_type.kind == 'cloud'
                }
                all_creds.append(summarized_cred)
-                if cred.credential_type.kind in ('cloud', 'net'):
-                    extra_creds.append(summarized_cred)
-        if self.is_detail_view:
-            summary_fields['extra_credentials'] = extra_creds
        summary_fields['credentials'] = all_creds
        return summary_fields

@@ -3397,6 +3437,12 @@ class WorkflowJobTemplateSerializer(JobTemplateMixin, LabelsListMixin, UnifiedJo
            res['organization'] = self.reverse('api:organization_detail',   kwargs={'pk': obj.organization.pk})
        if obj.webhook_credential_id:
            res['webhook_credential'] = self.reverse('api:credential_detail', kwargs={'pk': obj.webhook_credential_id})
+        if obj.inventory_id:
+            res['inventory'] = self.reverse(
+                'api:inventory_detail', kwargs={
+                    'pk': obj.inventory_id
+                }
+            )
        return res

    def validate_extra_vars(self, value):
@@ -3612,7 +3658,7 @@ class LaunchConfigurationBaseSerializer(BaseSerializer):
            ujt = self.instance.unified_job_template
        if ujt is None:
            ret = {}
-            for fd in ('workflow_job_template', 'identifier'):
+            for fd in ('workflow_job_template', 'identifier', 'all_parents_must_converge'):
                if fd in attrs:
                    ret[fd] = attrs[fd]
            return ret
@@ -3668,7 +3714,7 @@ class LaunchConfigurationBaseSerializer(BaseSerializer):
                        attrs.get('survey_passwords', {}).pop(key, None)
                    else:
                        errors.setdefault('extra_vars', []).append(
-                            _('"$encrypted$ is a reserved keyword, may not be used for {var_name}."'.format(key))
+                            _('"$encrypted$ is a reserved keyword, may not be used for {}."'.format(key))
                        )

        # Launch configs call extra_vars extra_data for historical reasons
@@ -3899,15 +3945,23 @@ class ProjectUpdateEventSerializer(JobEventSerializer):
        return UriCleaner.remove_sensitive(obj.stdout)

    def get_event_data(self, obj):
-        try:
-            return json.loads(
-                UriCleaner.remove_sensitive(
-                    json.dumps(obj.event_data)
+        # the project update playbook uses the git or svn modules
+        # to clone repositories, and those modules are prone to printing
+        # raw SCM URLs in their stdout (which *could* contain passwords)
+        # attempt to detect and filter HTTP basic auth passwords in the stdout
+        # of these types of events
+        if obj.event_data.get('task_action') in ('git', 'svn'):
+            try:
+                return json.loads(
+                    UriCleaner.remove_sensitive(
+                        json.dumps(obj.event_data)
+                    )
                )
-            )
-        except Exception:
-            logger.exception("Failed to sanitize event_data")
-            return {}
+            except Exception:
+                logger.exception("Failed to sanitize event_data")
+                return {}
+        else:
+            return obj.event_data


 class AdHocCommandEventSerializer(BaseSerializer):
@@ -4085,7 +4139,8 @@ class JobLaunchSerializer(BaseSerializer):
                errors.setdefault('credentials', []).append(_(
                    'Cannot assign multiple {} credentials.'
                ).format(cred.unique_hash(display=True)))
-            if cred.credential_type.kind not in ('ssh', 'vault', 'cloud', 'net'):
+            if cred.credential_type.kind not in ('ssh', 'vault', 'cloud',
+                                                 'net', 'kubernetes'):
                errors.setdefault('credentials', []).append(_(
                    'Cannot assign a Credential of kind `{}`'
                ).format(cred.credential_type.kind))
@@ -4107,7 +4162,10 @@ class JobLaunchSerializer(BaseSerializer):
        # verify that credentials (either provided or existing) don't
        # require launch-time passwords that have not been provided
        if 'credentials' in accepted:
-            launch_credentials = accepted['credentials']
+            launch_credentials = Credential.unique_dict(
+                list(template_credentials.all()) +
+                list(accepted['credentials'])
+            ).values()
        else:
            launch_credentials = template_credentials
        passwords = attrs.get('credential_passwords', {})  # get from original attrs
@@ -4649,6 +4707,8 @@ class InstanceSerializer(BaseSerializer):

 class InstanceGroupSerializer(BaseSerializer):

+    show_capabilities = ['edit', 'delete']
+
    committed_capacity = serializers.SerializerMethodField()
    consumed_capacity = serializers.SerializerMethodField()
    percent_capacity_remaining = serializers.SerializerMethodField()
--- a/awx/api/templates/api/_schedule_detail.md
+++ b/awx/api/templates/api/_schedule_detail.md
@@ -4,7 +4,6 @@ The following lists the expected format and details of our rrules:
 * DTSTART is expected to be in UTC
 * INTERVAL is required
 * SECONDLY is not supported
-* TZID is not supported
 * RRULE must precede the rule statements
 * BYDAY is supported but not BYDAY with a numerical prefix
 * BYYEARDAY and BYWEEKNO are not supported
--- a/awx/api/templates/api/dashboard_jobs_graph_view.md
+++ b/awx/api/templates/api/dashboard_jobs_graph_view.md
@@ -8,7 +8,7 @@ The `period` of the data can be adjusted with:

    ?period=month

-Where `month` can be replaced with `week`, or `day`.  `month` is the default.
+Where `month` can be replaced with `week`, `two_weeks`, or `day`.  `month` is the default.

 The type of job can be filtered with:

--- a/awx/api/urls/organization.py
+++ b/awx/api/urls/organization.py
@@ -21,6 +21,7 @@ from awx.api.views import (
    OrganizationNotificationTemplatesSuccessList,
    OrganizationNotificationTemplatesApprovalList,
    OrganizationInstanceGroupsList,
+    OrganizationGalaxyCredentialsList,
    OrganizationObjectRolesList,
    OrganizationAccessList,
    OrganizationApplicationList,
@@ -49,6 +50,7 @@ urls = [
    url(r'^(?P<pk>[0-9]+)/notification_templates_approvals/$', OrganizationNotificationTemplatesApprovalList.as_view(),
        name='organization_notification_templates_approvals_list'),
    url(r'^(?P<pk>[0-9]+)/instance_groups/$', OrganizationInstanceGroupsList.as_view(), name='organization_instance_groups_list'),
+    url(r'^(?P<pk>[0-9]+)/galaxy_credentials/$', OrganizationGalaxyCredentialsList.as_view(), name='organization_galaxy_credentials_list'),
    url(r'^(?P<pk>[0-9]+)/object_roles/$', OrganizationObjectRolesList.as_view(), name='organization_object_roles_list'),
    url(r'^(?P<pk>[0-9]+)/access_list/$', OrganizationAccessList.as_view(), name='organization_access_list'),
    url(r'^(?P<pk>[0-9]+)/applications/$', OrganizationApplicationList.as_view(), name='organization_applications_list'),
--- a/awx/api/urls/urls.py
+++ b/awx/api/urls/urls.py
@@ -15,6 +15,7 @@ from awx.api.views import (
    ApiV2PingView,
    ApiV2ConfigView,
    ApiV2SubscriptionView,
+    ApiV2AttachView,
    AuthView,
    UserMeList,
    DashboardView,
@@ -23,9 +24,7 @@ from awx.api.views import (
    UnifiedJobList,
    HostAnsibleFactsDetail,
    JobCredentialsList,
-    JobExtraCredentialsList,
    JobTemplateCredentialsList,
-    JobTemplateExtraCredentialsList,
    SchedulePreview,
    ScheduleZoneInfo,
    OAuth2ApplicationList,
@@ -83,9 +82,7 @@ v2_urls = [
    url(r'^credential_types/', include(credential_type_urls)),
    url(r'^credential_input_sources/', include(credential_input_source_urls)),
    url(r'^hosts/(?P<pk>[0-9]+)/ansible_facts/$', HostAnsibleFactsDetail.as_view(), name='host_ansible_facts_detail'),
-    url(r'^jobs/(?P<pk>[0-9]+)/extra_credentials/$', JobExtraCredentialsList.as_view(), name='job_extra_credentials_list'),
    url(r'^jobs/(?P<pk>[0-9]+)/credentials/$', JobCredentialsList.as_view(), name='job_credentials_list'),
-    url(r'^job_templates/(?P<pk>[0-9]+)/extra_credentials/$', JobTemplateExtraCredentialsList.as_view(), name='job_template_extra_credentials_list'),
    url(r'^job_templates/(?P<pk>[0-9]+)/credentials/$', JobTemplateCredentialsList.as_view(), name='job_template_credentials_list'),
    url(r'^schedules/preview/$', SchedulePreview.as_view(), name='schedule_rrule'),
    url(r'^schedules/zoneinfo/$', ScheduleZoneInfo.as_view(), name='schedule_zoneinfo'),
@@ -98,6 +95,7 @@ v2_urls = [
    url(r'^ping/$', ApiV2PingView.as_view(), name='api_v2_ping_view'),
    url(r'^config/$', ApiV2ConfigView.as_view(), name='api_v2_config_view'),
    url(r'^config/subscriptions/$', ApiV2SubscriptionView.as_view(), name='api_v2_subscription_view'),
+    url(r'^config/attach/$', ApiV2AttachView.as_view(), name='api_v2_attach_view'),
    url(r'^auth/$', AuthView.as_view()),
    url(r'^me/$', UserMeList.as_view(), name='user_me_list'),
    url(r'^dashboard/$', DashboardView.as_view(), name='dashboard_view'),
--- a/awx/api/views/init.py
+++ b/awx/api/views/init.py
@@ -12,7 +12,9 @@ import socket
 import sys
 import time
 from base64 import b64encode
-from collections import OrderedDict, Iterable
+from collections import OrderedDict
+
+from urllib3.exceptions import ConnectTimeoutError


 # Django
@@ -122,6 +124,7 @@ from awx.api.views.organization import ( # noqa
    OrganizationNotificationTemplatesSuccessList,
    OrganizationNotificationTemplatesApprovalList,
    OrganizationInstanceGroupsList,
+    OrganizationGalaxyCredentialsList,
    OrganizationAccessList,
    OrganizationObjectRolesList,
 )
@@ -150,6 +153,7 @@ from awx.api.views.root import ( # noqa
    ApiV2PingView,
    ApiV2ConfigView,
    ApiV2SubscriptionView,
+    ApiV2AttachView,
 )
 from awx.api.views.webhooks import ( # noqa
    WebhookKeyView,
@@ -171,6 +175,15 @@ def api_exception_handler(exc, context):
        exc = ParseError(exc.args[0])
    if isinstance(context['view'], UnifiedJobStdout):
        context['view'].renderer_classes = [renderers.BrowsableAPIRenderer, JSONRenderer]
+    if isinstance(exc, APIException):
+        req = context['request']._request
+        if 'awx.named_url_rewritten' in req.environ and not str(getattr(exc, 'status_code', 0)).startswith('2'):
+            # if the URL was rewritten, and it's not a 2xx level status code,
+            # revert the request.path to its original value to avoid leaking
+            # any context about the existance of resources
+            req.path = req.environ['awx.named_url_rewritten']
+            if exc.status_code == 403:
+                exc = NotFound(detail=_('Not found.'))
    return exception_handler(exc, context)


@@ -229,8 +242,8 @@ class DashboardView(APIView):
        git_failed_projects = git_projects.filter(last_job_failed=True)
        svn_projects = user_projects.filter(scm_type='svn')
        svn_failed_projects = svn_projects.filter(last_job_failed=True)
-        hg_projects = user_projects.filter(scm_type='hg')
-        hg_failed_projects = hg_projects.filter(last_job_failed=True)
+        archive_projects = user_projects.filter(scm_type='archive')
+        archive_failed_projects = archive_projects.filter(last_job_failed=True)
        data['scm_types'] = {}
        data['scm_types']['git'] = {'url': reverse('api:project_list', request=request) + "?scm_type=git",
                                    'label': 'Git',
@@ -242,11 +255,11 @@ class DashboardView(APIView):
                                    'failures_url': reverse('api:project_list', request=request) + "?scm_type=svn&last_job_failed=True",
                                    'total': svn_projects.count(),
                                    'failed': svn_failed_projects.count()}
-        data['scm_types']['hg'] = {'url': reverse('api:project_list', request=request) + "?scm_type=hg",
-                                   'label': 'Mercurial',
-                                   'failures_url': reverse('api:project_list', request=request) + "?scm_type=hg&last_job_failed=True",
-                                   'total': hg_projects.count(),
-                                   'failed': hg_failed_projects.count()}
+        data['scm_types']['archive'] = {'url': reverse('api:project_list', request=request) + "?scm_type=archive",
+                                        'label': 'Remote Archive',
+                                        'failures_url': reverse('api:project_list', request=request) + "?scm_type=archive&last_job_failed=True",
+                                        'total': archive_projects.count(),
+                                        'failed': archive_failed_projects.count()}

        user_list = get_user_queryset(request.user, models.User)
        team_list = get_user_queryset(request.user, models.Team)
@@ -297,6 +310,9 @@ class DashboardJobsGraphView(APIView):
        if period == 'month':
            end_date = start_date - dateutil.relativedelta.relativedelta(months=1)
            interval = 'days'
+        elif period == 'two_weeks':
+            end_date = start_date - dateutil.relativedelta.relativedelta(weeks=2)
+            interval = 'days'
        elif period == 'week':
            end_date = start_date - dateutil.relativedelta.relativedelta(weeks=1)
            interval = 'days'
@@ -1337,6 +1353,13 @@ class CredentialDetail(RetrieveUpdateDestroyAPIView):
    model = models.Credential
    serializer_class = serializers.CredentialSerializer

+    def destroy(self, request, *args, **kwargs):
+        instance = self.get_object()
+        if instance.managed_by_tower:
+            raise PermissionDenied(detail=_("Deletion not allowed for managed credentials"))
+        return super(CredentialDetail, self).destroy(request, *args, **kwargs)
+
+

 class CredentialActivityStreamList(SubListAPIView):

@@ -1397,10 +1420,18 @@ class CredentialExternalTest(SubDetailAPIView):
            obj.credential_type.plugin.backend(**backend_kwargs)
            return Response({}, status=status.HTTP_202_ACCEPTED)
        except requests.exceptions.HTTPError as exc:
-            message = 'HTTP {}\n{}'.format(exc.response.status_code, exc.response.text)
+            message = 'HTTP {}'.format(exc.response.status_code)
            return Response({'inputs': message}, status=status.HTTP_400_BAD_REQUEST)
        except Exception as exc:
-            return Response({'inputs': str(exc)}, status=status.HTTP_400_BAD_REQUEST)
+            message = exc.__class__.__name__
+            args = getattr(exc, 'args', [])
+            for a in args:
+                if isinstance(
+                    getattr(a, 'reason', None),
+                    ConnectTimeoutError
+                ):
+                    message = str(a.reason)
+            return Response({'inputs': message}, status=status.HTTP_400_BAD_REQUEST)


 class CredentialInputSourceDetail(RetrieveUpdateDestroyAPIView):
@@ -1449,10 +1480,18 @@ class CredentialTypeExternalTest(SubDetailAPIView):
            obj.plugin.backend(**backend_kwargs)
            return Response({}, status=status.HTTP_202_ACCEPTED)
        except requests.exceptions.HTTPError as exc:
-            message = 'HTTP {}\n{}'.format(exc.response.status_code, exc.response.text)
+            message = 'HTTP {}'.format(exc.response.status_code)
            return Response({'inputs': message}, status=status.HTTP_400_BAD_REQUEST)
        except Exception as exc:
-            return Response({'inputs': str(exc)}, status=status.HTTP_400_BAD_REQUEST)
+            message = exc.__class__.__name__
+            args = getattr(exc, 'args', [])
+            for a in args:
+                if isinstance(
+                    getattr(a, 'reason', None),
+                    ConnectTimeoutError
+                ):
+                    message = str(a.reason)
+            return Response({'inputs': message}, status=status.HTTP_400_BAD_REQUEST)


 class HostRelatedSearchMixin(object):
@@ -2337,70 +2376,24 @@ class JobTemplateLaunch(RetrieveAPIView):
        old field structure to launch endpoint
        TODO: delete this method with future API version changes
        '''
-        ignored_fields = {}
        modern_data = data.copy()

        id_fd = '{}_id'.format('inventory')
        if 'inventory' not in modern_data and id_fd in modern_data:
            modern_data['inventory'] = modern_data[id_fd]

-        # Automatically convert legacy launch credential arguments into a list of `.credentials`
-        if 'credentials' in modern_data and 'extra_credentials' in modern_data:
-            raise ParseError({"error": _(
-                "'credentials' cannot be used in combination with 'extra_credentials'."
-            )})
-
-        if 'extra_credentials' in modern_data:
-            # make a list of the current credentials
-            existing_credentials = obj.credentials.all()
-            template_credentials = list(existing_credentials)  # save copy of existing
-            new_credentials = []
-            if 'extra_credentials' in modern_data:
-                existing_credentials = [
-                    cred for cred in existing_credentials
-                    if cred.credential_type.kind not in ('cloud', 'net')
-                ]
-                prompted_value = modern_data.pop('extra_credentials')
-
-                # validate type, since these are not covered by a serializer
-                if not isinstance(prompted_value, Iterable):
-                    msg = _(
-                        "Incorrect type. Expected a list received {}."
-                    ).format(prompted_value.__class__.__name__)
-                    raise ParseError({'extra_credentials': [msg], 'credentials': [msg]})
-
-                # add the deprecated credential specified in the request
-                if not isinstance(prompted_value, Iterable) or isinstance(prompted_value, str):
-                    prompted_value = [prompted_value]
-
-                # If user gave extra_credentials, special case to use exactly
-                # the given list without merging with JT credentials
-                if prompted_value:
-                    obj._deprecated_credential_launch = True  # signal to not merge credentials
-                new_credentials.extend(prompted_value)
-
-            # combine the list of "new" and the filtered list of "old"
-            new_credentials.extend([cred.pk for cred in existing_credentials])
-            if new_credentials:
-                # If provided list doesn't contain the pre-existing credentials
-                # defined on the template, add them back here
-                for cred_obj in template_credentials:
-                    if cred_obj.pk not in new_credentials:
-                        new_credentials.append(cred_obj.pk)
-                modern_data['credentials'] = new_credentials
-
        # credential passwords were historically provided as top-level attributes
        if 'credential_passwords' not in modern_data:
            modern_data['credential_passwords'] = data.copy()

-        return (modern_data, ignored_fields)
+        return modern_data


    def post(self, request, *args, **kwargs):
        obj = self.get_object()

        try:
-            modern_data, ignored_fields = self.modernize_launch_payload(
+            modern_data = self.modernize_launch_payload(
                data=request.data, obj=obj
            )
        except ParseError as exc:
@@ -2410,8 +2403,6 @@ class JobTemplateLaunch(RetrieveAPIView):
        if not serializer.is_valid():
            return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

-        ignored_fields.update(serializer._ignored_fields)
-
        if not request.user.can_access(models.JobLaunchConfig, 'add', serializer.validated_data, template=obj):
            raise PermissionDenied()

@@ -2427,11 +2418,11 @@ class JobTemplateLaunch(RetrieveAPIView):
            data = OrderedDict()
            if isinstance(new_job, models.WorkflowJob):
                data['workflow_job'] = new_job.id
-                data['ignored_fields'] = self.sanitize_for_response(ignored_fields)
+                data['ignored_fields'] = self.sanitize_for_response(serializer._ignored_fields)
                data.update(serializers.WorkflowJobSerializer(new_job, context=self.get_serializer_context()).to_representation(new_job))
            else:
                data['job'] = new_job.id
-                data['ignored_fields'] = self.sanitize_for_response(ignored_fields)
+                data['ignored_fields'] = self.sanitize_for_response(serializer._ignored_fields)
                data.update(serializers.JobSerializer(new_job, context=self.get_serializer_context()).to_representation(new_job))
            headers = {'Location': new_job.get_absolute_url(request)}
            return Response(data, status=status.HTTP_201_CREATED, headers=headers)
@@ -2705,28 +2696,12 @@ class JobTemplateCredentialsList(SubListCreateAttachDetachAPIView):
            return {"error": _("Cannot assign multiple {credential_type} credentials.").format(
                credential_type=sub.unique_hash(display=True))}
        kind = sub.credential_type.kind
-        if kind not in ('ssh', 'vault', 'cloud', 'net'):
+        if kind not in ('ssh', 'vault', 'cloud', 'net', 'kubernetes'):
            return {'error': _('Cannot assign a Credential of kind `{}`.').format(kind)}

        return super(JobTemplateCredentialsList, self).is_valid_relation(parent, sub, created)


-class JobTemplateExtraCredentialsList(JobTemplateCredentialsList):
-
-    deprecated = True
-
-    def get_queryset(self):
-        sublist_qs = super(JobTemplateExtraCredentialsList, self).get_queryset()
-        sublist_qs = sublist_qs.filter(credential_type__kind__in=['cloud', 'net'])
-        return sublist_qs
-
-    def is_valid_relation(self, parent, sub, created=False):
-        valid = super(JobTemplateExtraCredentialsList, self).is_valid_relation(parent, sub, created)
-        if sub.credential_type.kind not in ('cloud', 'net'):
-            return {'error': _('Extra credentials must be network or cloud.')}
-        return valid
-
-
 class JobTemplateLabelList(DeleteLastUnattachLabelMixin, SubListCreateAttachDetachAPIView):

    model = models.Label
@@ -3065,7 +3040,7 @@ class WorkflowJobTemplateNodeCreateApproval(RetrieveAPIView):
            approval_template,
            context=self.get_serializer_context()
        ).data
-        return Response(data, status=status.HTTP_200_OK)
+        return Response(data, status=status.HTTP_201_CREATED)

    def check_permissions(self, request):
        obj = self.get_object().workflow_job_template
@@ -3543,16 +3518,6 @@ class JobCredentialsList(SubListAPIView):
    relationship = 'credentials'


-class JobExtraCredentialsList(JobCredentialsList):
-
-    deprecated = True
-
-    def get_queryset(self):
-        sublist_qs = super(JobExtraCredentialsList, self).get_queryset()
-        sublist_qs = sublist_qs.filter(credential_type__kind__in=['cloud', 'net'])
-        return sublist_qs
-
-
 class JobLabelList(SubListAPIView):

    model = models.Label
@@ -4285,7 +4250,9 @@ class NotificationTemplateDetail(RetrieveUpdateDestroyAPIView):
        obj = self.get_object()
        if not request.user.can_access(self.model, 'delete', obj):
            return Response(status=status.HTTP_404_NOT_FOUND)
-        if obj.notifications.filter(status='pending').exists():
+
+        hours_old = now() - dateutil.relativedelta.relativedelta(hours=8)
+        if obj.notifications.filter(status='pending', created__gt=hours_old).exists():
            return Response({"error": _("Delete not allowed while there are pending notifications")},
                            status=status.HTTP_405_METHOD_NOT_ALLOWED)
        return super(NotificationTemplateDetail, self).delete(request, *args, **kwargs)
--- a/awx/api/views/inventory.py
+++ b/awx/api/views/inventory.py
@@ -134,7 +134,8 @@ class InventoryDetail(RelatedJobsPreventDeleteMixin, ControlledByScmMixin, Retri

        # Do not allow changes to an Inventory kind.
        if kind is not None and obj.kind != kind:
-            return self.http_method_not_allowed(request, *args, **kwargs)
+            return Response(dict(error=_('You cannot turn a regular inventory into a "smart" inventory.')),
+                            status=status.HTTP_405_METHOD_NOT_ALLOWED)
        return super(InventoryDetail, self).update(request, *args, **kwargs)

    def destroy(self, request, *args, **kwargs):
--- a/awx/api/views/metrics.py
+++ b/awx/api/views/metrics.py
@@ -22,7 +22,7 @@ from awx.api.generics import (
 )


-logger = logging.getLogger('awx.main.analytics')
+logger = logging.getLogger('awx.analytics')


 class MetricsView(APIView):
--- a/awx/api/views/organization.py
+++ b/awx/api/views/organization.py
@@ -7,11 +7,13 @@ import logging
 # Django
 from django.db.models import Count
 from django.contrib.contenttypes.models import ContentType
+from django.utils.translation import ugettext_lazy as _

 # AWX
 from awx.main.models import (
    ActivityStream,
    Inventory,
+    Host,
    Project,
    JobTemplate,
    WorkflowJobTemplate,
@@ -20,7 +22,8 @@ from awx.main.models import (
    Role,
    User,
    Team,
-    InstanceGroup
+    InstanceGroup,
+    Credential
 )
 from awx.api.generics import (
    ListCreateAPIView,
@@ -42,7 +45,8 @@ from awx.api.serializers import (
    RoleSerializer,
    NotificationTemplateSerializer,
    InstanceGroupSerializer,
-    ProjectSerializer, JobTemplateSerializer, WorkflowJobTemplateSerializer
+    ProjectSerializer, JobTemplateSerializer, WorkflowJobTemplateSerializer,
+    CredentialSerializer
 )
 from awx.api.views.mixin import (
    RelatedJobsPreventDeleteMixin,
@@ -95,6 +99,7 @@ class OrganizationDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPI
            organization__id=org_id).count()
        org_counts['job_templates'] = JobTemplate.accessible_objects(**access_kwargs).filter(
            organization__id=org_id).count()
+        org_counts['hosts'] = Host.objects.org_active_count(org_id)

        full_context['related_field_counts'] = {}
        full_context['related_field_counts'][org_id] = org_counts
@@ -214,6 +219,20 @@ class OrganizationInstanceGroupsList(SubListAttachDetachAPIView):
    relationship = 'instance_groups'


+class OrganizationGalaxyCredentialsList(SubListAttachDetachAPIView):
+
+    model = Credential
+    serializer_class = CredentialSerializer
+    parent_model = Organization
+    relationship = 'galaxy_credentials'
+
+    def is_valid_relation(self, parent, sub, created=False):
+        if sub.kind != 'galaxy_api_token':
+            return {'msg': _(
+                f"Credential must be a Galaxy credential, not {sub.credential_type.name}."
+            )}
+
+
 class OrganizationAccessList(ResourceAccessList):

    model = User # needs to be User for AccessLists's
--- a/awx/api/views/root.py
+++ b/awx/api/views/root.py
@@ -1,9 +1,10 @@
 # Copyright (c) 2018 Ansible, Inc.
 # All Rights Reserved.

+import base64
+import json
 import logging
 import operator
-import json
 from collections import OrderedDict

 from django.conf import settings
@@ -21,6 +22,7 @@ import requests

 from awx.api.generics import APIView
 from awx.conf.registry import settings_registry
+from awx.main.analytics import all_collectors
 from awx.main.ha import is_ha_environment
 from awx.main.utils import (
    get_awx_version,
@@ -28,8 +30,8 @@ from awx.main.utils import (
    get_custom_venv_choices,
    to_python_boolean,
 )
+from awx.main.utils.licensing import validate_entitlement_manifest
 from awx.api.versioning import reverse, drf_reverse
-from awx.conf.license import get_license
 from awx.main.constants import PRIVILEGE_ESCALATION_METHODS
 from awx.main.models import (
    Project,
@@ -177,7 +179,7 @@ class ApiV2PingView(APIView):
 class ApiV2SubscriptionView(APIView):

    permission_classes = (IsAuthenticated,)
-    name = _('Configuration')
+    name = _('Subscriptions')
    swagger_topic = 'System Configuration'

    def check_permissions(self, request):
@@ -188,18 +190,18 @@ class ApiV2SubscriptionView(APIView):
    def post(self, request):
        from awx.main.utils.common import get_licenser
        data = request.data.copy()
-        if data.get('rh_password') == '$encrypted$':
-            data['rh_password'] = settings.REDHAT_PASSWORD
+        if data.get('subscriptions_password') == '$encrypted$':
+            data['subscriptions_password'] = settings.SUBSCRIPTIONS_PASSWORD
        try:
-            user, pw = data.get('rh_username'), data.get('rh_password')
+            user, pw = data.get('subscriptions_username'), data.get('subscriptions_password')
            with set_environ(**settings.AWX_TASK_ENV):
                validated = get_licenser().validate_rh(user, pw)
            if user:
-                settings.REDHAT_USERNAME = data['rh_username']
+                settings.SUBSCRIPTIONS_USERNAME = data['subscriptions_username']
            if pw:
-                settings.REDHAT_PASSWORD = data['rh_password']
+                settings.SUBSCRIPTIONS_PASSWORD = data['subscriptions_password']
        except Exception as exc:
-            msg = _("Invalid License")
+            msg = _("Invalid Subscription")
            if (
                isinstance(exc, requests.exceptions.HTTPError) and
                getattr(getattr(exc, 'response', None), 'status_code', None) == 401
@@ -212,13 +214,63 @@ class ApiV2SubscriptionView(APIView):
            elif isinstance(exc, (ValueError, OSError)) and exc.args:
                msg = exc.args[0]
            else:
-                logger.exception(smart_text(u"Invalid license submitted."),
+                logger.exception(smart_text(u"Invalid subscription submitted."),
                                 extra=dict(actor=request.user.username))
            return Response({"error": msg}, status=status.HTTP_400_BAD_REQUEST)

        return Response(validated)


+class ApiV2AttachView(APIView):
+
+    permission_classes = (IsAuthenticated,)
+    name = _('Attach Subscription')
+    swagger_topic = 'System Configuration'
+
+    def check_permissions(self, request):
+        super(ApiV2AttachView, self).check_permissions(request)
+        if not request.user.is_superuser and request.method.lower() not in {'options', 'head'}:
+            self.permission_denied(request)  # Raises PermissionDenied exception.
+
+    def post(self, request):
+        data = request.data.copy()
+        pool_id = data.get('pool_id', None)
+        if not pool_id:
+            return Response({"error": _("No subscription pool ID provided.")}, status=status.HTTP_400_BAD_REQUEST)
+        user = getattr(settings, 'SUBSCRIPTIONS_USERNAME', None)
+        pw = getattr(settings, 'SUBSCRIPTIONS_PASSWORD', None)
+        if pool_id and user and pw:
+            from awx.main.utils.common import get_licenser
+            data = request.data.copy()
+            try:
+                with set_environ(**settings.AWX_TASK_ENV):
+                    validated = get_licenser().validate_rh(user, pw)
+            except Exception as exc:
+                msg = _("Invalid Subscription")
+                if (
+                    isinstance(exc, requests.exceptions.HTTPError) and
+                    getattr(getattr(exc, 'response', None), 'status_code', None) == 401
+                ):
+                    msg = _("The provided credentials are invalid (HTTP 401).")
+                elif isinstance(exc, requests.exceptions.ProxyError):
+                    msg = _("Unable to connect to proxy server.")
+                elif isinstance(exc, requests.exceptions.ConnectionError):
+                    msg = _("Could not connect to subscription service.")
+                elif isinstance(exc, (ValueError, OSError)) and exc.args:
+                    msg = exc.args[0]
+                else:
+                    logger.exception(smart_text(u"Invalid subscription submitted."),
+                                     extra=dict(actor=request.user.username))
+                return Response({"error": msg}, status=status.HTTP_400_BAD_REQUEST)
+        for sub in validated:
+            if sub['pool_id'] == pool_id:
+                sub['valid_key'] = True
+                settings.LICENSE = sub
+                return Response(sub)
+
+        return Response({"error": _("Error processing subscription metadata.")}, status=status.HTTP_400_BAD_REQUEST)
+
+
 class ApiV2ConfigView(APIView):

    permission_classes = (IsAuthenticated,)
@@ -233,15 +285,11 @@ class ApiV2ConfigView(APIView):
    def get(self, request, format=None):
        '''Return various sitewide configuration settings'''

-        if request.user.is_superuser or request.user.is_system_auditor:
-            license_data = get_license(show_key=True)
-        else:
-            license_data = get_license(show_key=False)
+        from awx.main.utils.common import get_licenser
+        license_data = get_licenser().validate()
+
        if not license_data.get('valid_key', False):
            license_data = {}
-        if license_data and 'features' in license_data and 'activity_streams' in license_data['features']:
-            # FIXME: Make the final setting value dependent on the feature?
-            license_data['features']['activity_streams'] &= settings.ACTIVITY_STREAM_ENABLED

        pendo_state = settings.PENDO_TRACKING_STATE if settings.PENDO_TRACKING_STATE in ('off', 'anonymous', 'detailed') else 'off'

@@ -252,6 +300,7 @@ class ApiV2ConfigView(APIView):
            ansible_version=get_ansible_version(),
            eula=render_to_string("eula.md") if license_data.get('license_type', 'UNLICENSED') != 'open' else '',
            analytics_status=pendo_state,
+            analytics_collectors=all_collectors(),
            become_methods=PRIVILEGE_ESCALATION_METHODS,
        )

@@ -279,9 +328,10 @@ class ApiV2ConfigView(APIView):

        return Response(data)

+
    def post(self, request):
        if not isinstance(request.data, dict):
-            return Response({"error": _("Invalid license data")}, status=status.HTTP_400_BAD_REQUEST)
+            return Response({"error": _("Invalid subscription data")}, status=status.HTTP_400_BAD_REQUEST)
        if "eula_accepted" not in request.data:
            return Response({"error": _("Missing 'eula_accepted' property")}, status=status.HTTP_400_BAD_REQUEST)
        try:
@@ -298,25 +348,47 @@ class ApiV2ConfigView(APIView):
            logger.info(smart_text(u"Invalid JSON submitted for license."),
                        extra=dict(actor=request.user.username))
            return Response({"error": _("Invalid JSON")}, status=status.HTTP_400_BAD_REQUEST)
-        try:
-            from awx.main.utils.common import get_licenser
-            license_data = json.loads(data_actual)
-            license_data_validated = get_licenser(**license_data).validate()
-        except Exception:
-            logger.warning(smart_text(u"Invalid license submitted."),
-                           extra=dict(actor=request.user.username))
-            return Response({"error": _("Invalid License")}, status=status.HTTP_400_BAD_REQUEST)
+
+        from awx.main.utils.common import get_licenser
+        license_data = json.loads(data_actual)
+        if 'license_key' in license_data:
+            return Response({"error": _('Legacy license submitted. A subscription manifest is now required.')}, status=status.HTTP_400_BAD_REQUEST)
+        if 'manifest' in license_data:
+            try:
+                json_actual = json.loads(base64.b64decode(license_data['manifest']))
+                if 'license_key' in json_actual:
+                    return Response(
+                        {"error": _('Legacy license submitted. A subscription manifest is now required.')},
+                        status=status.HTTP_400_BAD_REQUEST
+                    )
+            except Exception:
+                pass
+            try:
+                license_data = validate_entitlement_manifest(license_data['manifest'])
+            except ValueError as e:
+                return Response({"error": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+            except Exception:
+                logger.exception('Invalid manifest submitted. {}')
+                return Response({"error": _('Invalid manifest submitted.')}, status=status.HTTP_400_BAD_REQUEST)
+
+            try:
+                license_data_validated = get_licenser().license_from_manifest(license_data)
+            except Exception:
+                logger.warning(smart_text(u"Invalid subscription submitted."),
+                               extra=dict(actor=request.user.username))
+                return Response({"error": _("Invalid License")}, status=status.HTTP_400_BAD_REQUEST)
+        else:
+            license_data_validated = get_licenser().validate()

        # If the license is valid, write it to the database.
        if license_data_validated['valid_key']:
-            settings.LICENSE = license_data
            if not settings_registry.is_setting_read_only('TOWER_URL_BASE'):
                settings.TOWER_URL_BASE = "{}://{}".format(request.scheme, request.get_host())
            return Response(license_data_validated)

-        logger.warning(smart_text(u"Invalid license submitted."),
+        logger.warning(smart_text(u"Invalid subscription submitted."),
                       extra=dict(actor=request.user.username))
-        return Response({"error": _("Invalid license")}, status=status.HTTP_400_BAD_REQUEST)
+        return Response({"error": _("Invalid subscription")}, status=status.HTTP_400_BAD_REQUEST)

    def delete(self, request):
        try:
--- a/awx/asgi.py
+++ b/awx/asgi.py
@@ -4,12 +4,11 @@ import os
 import logging
 import django
 from awx import __version__ as tower_version
-
 # Prepare the AWX environment.
 from awx import prepare_env, MODE
+from channels.routing import get_default_application  # noqa
 prepare_env() # NOQA

-from channels.routing import get_default_application


 """
@@ -26,10 +25,12 @@ if MODE == 'production':
    try:
        fd = open("/var/lib/awx/.tower_version", "r")
        if fd.read().strip() != tower_version:
-            raise Exception()
-    except Exception:
+            raise ValueError()
+    except FileNotFoundError:
+        pass
+    except ValueError as e:
        logger.error("Missing or incorrect metadata for Tower version.  Ensure Tower was installed using the setup playbook.")
-        raise Exception("Missing or incorrect metadata for Tower version.  Ensure Tower was installed using the setup playbook.")
+        raise Exception("Missing or incorrect metadata for Tower version.  Ensure Tower was installed using the setup playbook.") from e


 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "awx.settings")
--- a/awx/conf/fields.py
+++ b/awx/conf/fields.py
@@ -172,9 +172,9 @@ class URLField(CharField):
                        netloc = '{}:{}'.format(netloc, url_parts.port)
                    if url_parts.username:
                        if url_parts.password:
-                            netloc = '{}:{}@{}' % (url_parts.username, url_parts.password, netloc)
+                            netloc = '{}:{}@{}'.format(url_parts.username, url_parts.password, netloc)
                        else:
-                            netloc = '{}@{}' % (url_parts.username, netloc)
+                            netloc = '{}@{}'.format(url_parts.username, netloc)
                    value = urlparse.urlunsplit([url_parts.scheme, netloc, url_parts.path, url_parts.query, url_parts.fragment])
            except Exception:
                raise  # If something fails here, just fall through and let the validators check it.
--- a/awx/conf/license.py
+++ b/awx/conf/license.py
@@ -1,18 +1,14 @@
 # Copyright (c) 2016 Ansible, Inc.
 # All Rights Reserved.

-
 __all__ = ['get_license']


 def _get_validated_license_data():
-    from awx.main.utils.common import get_licenser
+    from awx.main.utils import get_licenser
    return get_licenser().validate()


-def get_license(show_key=False):
+def get_license():
    """Return a dictionary representing the active license on this Tower instance."""
-    license_data = _get_validated_license_data()
-    if not show_key:
-        license_data.pop('license_key', None)
-    return license_data
+    return _get_validated_license_data()
--- a/awx/conf/migrations/0007_v380_rename_more_settings.py
+++ b/awx/conf/migrations/0007_v380_rename_more_settings.py
@@ -0,0 +1,19 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+from django.db import migrations
+from awx.conf.migrations import _rename_setting
+
+
+def copy_allowed_ips(apps, schema_editor):
+    _rename_setting.rename_setting(apps, schema_editor, old_key='PROXY_IP_WHITELIST', new_key='PROXY_IP_ALLOWED_LIST')
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('conf', '0006_v331_ldap_group_type'),
+    ]
+
+    operations = [
+        migrations.RunPython(copy_allowed_ips),
+    ]
--- a/awx/conf/migrations/0008_subscriptions.py
+++ b/awx/conf/migrations/0008_subscriptions.py
@@ -0,0 +1,26 @@
+# Generated by Django 2.2.11 on 2020-08-04 15:19
+
+import logging
+
+from django.db import migrations
+
+from awx.conf.migrations._subscriptions import clear_old_license, prefill_rh_credentials
+
+logger = logging.getLogger('awx.conf.migrations')
+
+
+def _noop(apps, schema_editor):
+    pass
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('conf', '0007_v380_rename_more_settings'),
+    ]
+
+
+    operations = [
+        migrations.RunPython(clear_old_license, _noop),
+        migrations.RunPython(prefill_rh_credentials, _noop)
+    ]
--- a/awx/conf/migrations/_subscriptions.py
+++ b/awx/conf/migrations/_subscriptions.py
@@ -0,0 +1,34 @@
+# -*- coding: utf-8 -*-
+import logging
+from django.utils.timezone import now
+from awx.main.utils.encryption import decrypt_field, encrypt_field
+
+logger = logging.getLogger('awx.conf.settings')
+
+__all__ = ['clear_old_license', 'prefill_rh_credentials']
+    
+
+def clear_old_license(apps, schema_editor):
+    Setting = apps.get_model('conf', 'Setting')
+    Setting.objects.filter(key='LICENSE').delete()
+
+
+def _migrate_setting(apps, old_key, new_key, encrypted=False):
+    Setting = apps.get_model('conf', 'Setting')
+    if not Setting.objects.filter(key=old_key).exists():
+        return
+    new_setting = Setting.objects.create(key=new_key,
+                                         created=now(),
+                                         modified=now()
+                                         )
+    if encrypted:
+        new_setting.value = decrypt_field(Setting.objects.filter(key=old_key).first(), 'value')
+        new_setting.value = encrypt_field(new_setting, 'value')
+    else:
+        new_setting.value = getattr(Setting.objects.filter(key=old_key).first(), 'value')
+    new_setting.save()
+
+
+def prefill_rh_credentials(apps, schema_editor):
+    _migrate_setting(apps, 'REDHAT_USERNAME', 'SUBSCRIPTIONS_USERNAME', encrypted=False)
+    _migrate_setting(apps, 'REDHAT_PASSWORD', 'SUBSCRIPTIONS_PASSWORD', encrypted=True)
--- a/awx/conf/models.py
+++ b/awx/conf/models.py
@@ -78,14 +78,6 @@ class Setting(CreatedModifiedModel):
    def get_cache_id_key(self, key):
        return '{}_ID'.format(key)

-    def display_value(self):
-        if self.key == 'LICENSE' and 'license_key' in self.value:
-            # don't log the license key in activity stream
-            value = self.value.copy()
-            value['license_key'] = '********'
-            return value
-        return self.value
-

 import awx.conf.signals  # noqa

--- a/awx/conf/registry.py
+++ b/awx/conf/registry.py
@@ -129,12 +129,14 @@ class SettingsRegistry(object):
        placeholder = field_kwargs.pop('placeholder', empty)
        encrypted = bool(field_kwargs.pop('encrypted', False))
        defined_in_file = bool(field_kwargs.pop('defined_in_file', False))
+        unit = field_kwargs.pop('unit', None)
        if getattr(field_kwargs.get('child', None), 'source', None) is not None:
            field_kwargs['child'].source = None
        field_instance = field_class(**field_kwargs)
        field_instance.category_slug = category_slug
        field_instance.category = category
        field_instance.depends_on = depends_on
+        field_instance.unit = unit
        if placeholder is not empty:
            field_instance.placeholder = placeholder
        field_instance.defined_in_file = defined_in_file
--- a/awx/conf/settings.py
+++ b/awx/conf/settings.py
@@ -4,6 +4,7 @@ import logging
 import sys
 import threading
 import time
+import os

 # Django
 from django.conf import LazySettings
@@ -11,12 +12,14 @@ from django.conf import settings, UserSettingsHolder
 from django.core.cache import cache as django_cache
 from django.core.exceptions import ImproperlyConfigured
 from django.db import transaction, connection
-from django.db.utils import Error as DBError
+from django.db.utils import Error as DBError, ProgrammingError
 from django.utils.functional import cached_property

 # Django REST Framework
 from rest_framework.fields import empty, SkipField

+import cachetools
+
 # Tower
 from awx.main.utils import encrypt_field, decrypt_field
 from awx.conf import settings_registry
@@ -28,21 +31,23 @@ from awx.conf.migrations._reencrypt import decrypt_field as old_decrypt_field

 logger = logging.getLogger('awx.conf.settings')

+SETTING_MEMORY_TTL = 5 if 'callback_receiver' in ' '.join(sys.argv) else 0
+
 # Store a special value to indicate when a setting is not set in the database.
 SETTING_CACHE_NOTSET = '___notset___'

-# Cannot store None in memcached; use a special value instead to indicate None.
+# Cannot store None in cache; use a special value instead to indicate None.
 # If the special value for None is the same as the "not set" value, then a value
 # of None will be equivalent to the setting not being set (and will raise an
 # AttributeError if there is no other default defined).
 # SETTING_CACHE_NONE = '___none___'
 SETTING_CACHE_NONE = SETTING_CACHE_NOTSET

-# Cannot store empty list/tuple in memcached; use a special value instead to
+# Cannot store empty list/tuple in cache; use a special value instead to
 # indicate an empty list.
 SETTING_CACHE_EMPTY_LIST = '___[]___'

-# Cannot store empty dict in memcached; use a special value instead to indicate
+# Cannot store empty dict in cache; use a special value instead to indicate
 # an empty dict.
 SETTING_CACHE_EMPTY_DICT = '___{}___'

@@ -74,10 +79,19 @@ def _ctit_db_wrapper(trans_safe=False):
                    logger.debug('Obtaining database settings in spite of broken transaction.')
                    transaction.set_rollback(False)
        yield
-    except DBError:
+    except DBError as exc:
        if trans_safe:
            if 'migrate' not in sys.argv and 'check_migrations' not in sys.argv:
-                logger.exception('Database settings are not available, using defaults.')
+                level = logger.exception
+                if isinstance(exc, ProgrammingError):
+                    if 'relation' in str(exc) and 'does not exist' in str(exc):
+                        # this generally means we can't fetch Tower configuration
+                        # because the database hasn't actually finished migrating yet;
+                        # this is usually a sign that a service in a container (such as ws_broadcast)
+                        # has come up *before* the database has finished migrating, and
+                        # especially that the conf.settings table doesn't exist yet
+                        level = logger.debug
+                level('Database settings are not available, using defaults.')
        else:
            logger.exception('Error modifying something related to database settings.')
    finally:
@@ -234,6 +248,7 @@ class SettingsWrapper(UserSettingsHolder):
        # These values have to be stored via self.__dict__ in this way to get
        # around the magic __setattr__ method on this class (which is used to
        # store API-assigned settings in the database).
+        self.__dict__['__forks__'] = {}
        self.__dict__['default_settings'] = default_settings
        self.__dict__['_awx_conf_settings'] = self
        self.__dict__['_awx_conf_preload_expires'] = None
@@ -242,6 +257,26 @@ class SettingsWrapper(UserSettingsHolder):
        self.__dict__['cache'] = EncryptedCacheProxy(cache, registry)
        self.__dict__['registry'] = registry

+        # record the current pid so we compare it post-fork for
+        # processes like the dispatcher and callback receiver
+        self.__dict__['pid'] = os.getpid()
+
+    def __clean_on_fork__(self):
+        pid = os.getpid()
+        # if the current pid does *not* match the value on self, it means
+        # that value was copied on fork, and we're now in a *forked* process;
+        # the *first* time we enter this code path (on setting access),
+        # forcibly close DB/cache sockets and set a marker so we don't run
+        # this code again _in this process_
+        #
+        if pid != self.__dict__['pid'] and pid not in self.__dict__['__forks__']:
+            self.__dict__['__forks__'][pid] = True
+            # It's important to close these post-fork, because we
+            # don't want the forked processes to inherit the open sockets
+            # for the DB and cache connections (that way lies race conditions)
+            connection.close()
+            django_cache.close()
+
    @cached_property
    def all_supported_settings(self):
        return self.registry.get_registered_settings()
@@ -317,6 +352,7 @@ class SettingsWrapper(UserSettingsHolder):
        self.cache.set_many(settings_to_cache, timeout=SETTING_CACHE_TIMEOUT)

    def _get_local(self, name, validate=True):
+        self.__clean_on_fork__()
        self._preload_cache()
        cache_key = Setting.get_cache_key(name)
        try:
@@ -397,6 +433,7 @@ class SettingsWrapper(UserSettingsHolder):
    def SETTINGS_MODULE(self):
        return self._get_default('SETTINGS_MODULE')

+    @cachetools.cached(cache=cachetools.TTLCache(maxsize=2048, ttl=SETTING_MEMORY_TTL))
    def __getattr__(self, name):
        value = empty
        if name in self.all_supported_settings:
@@ -410,7 +447,7 @@ class SettingsWrapper(UserSettingsHolder):
        field = self.registry.get_setting_field(name)
        if field.read_only:
            logger.warning('Attempt to set read only setting "%s".', name)
-            raise ImproperlyConfigured('Setting "%s" is read only.'.format(name))
+            raise ImproperlyConfigured('Setting "{}" is read only.'.format(name))

        try:
            data = field.to_representation(value)
@@ -441,7 +478,7 @@ class SettingsWrapper(UserSettingsHolder):
        field = self.registry.get_setting_field(name)
        if field.read_only:
            logger.warning('Attempt to delete read only setting "%s".', name)
-            raise ImproperlyConfigured('Setting "%s" is read only.'.format(name))
+            raise ImproperlyConfigured('Setting "{}" is read only.'.format(name))
        for setting in Setting.objects.filter(key=name, user__isnull=True):
            setting.delete()
            # pre_delete handler will delete from cache.
--- a/awx/conf/tests/unit/test_registry.py
+++ b/awx/conf/tests/unit/test_registry.py
@@ -29,9 +29,10 @@ def reg(request):
    # as "defined in a settings file".  This is analogous to manually
    # specifying a setting on the filesystem (e.g., in a local_settings.py in
    # development, or in /etc/tower/conf.d/<something>.py)
-    defaults = request.node.get_marker('defined_in_file')
-    if defaults:
-        settings.configure(**defaults.kwargs)
+    for marker in request.node.own_markers:
+        if marker.name == 'defined_in_file':
+            settings.configure(**marker.kwargs)
+
    settings._wrapped = SettingsWrapper(settings._wrapped,
                                        cache,
                                        registry)
--- a/awx/conf/tests/unit/test_settings.py
+++ b/awx/conf/tests/unit/test_settings.py
@@ -41,13 +41,16 @@ def settings(request):
    cache = LocMemCache(str(uuid4()), {})  # make a new random cache each time
    settings = LazySettings()
    registry = SettingsRegistry(settings)
+    defaults = {}

    # @pytest.mark.defined_in_file can be used to mark specific setting values
    # as "defined in a settings file".  This is analogous to manually
    # specifying a setting on the filesystem (e.g., in a local_settings.py in
    # development, or in /etc/tower/conf.d/<something>.py)
-    in_file_marker = request.node.get_marker('defined_in_file')
-    defaults = in_file_marker.kwargs if in_file_marker else {}
+    for marker in request.node.own_markers:
+        if marker.name == 'defined_in_file':
+            defaults = marker.kwargs
+
    defaults['DEFAULTS_SNAPSHOT'] = {}
    settings.configure(**defaults)
    settings._wrapped = SettingsWrapper(settings._wrapped,
@@ -63,15 +66,6 @@ def test_unregistered_setting(settings):
    assert settings.cache.get('DEBUG') is None


-def test_cached_settings_unicode_is_auto_decoded(settings):
-    # https://github.com/linsomniac/python-memcached/issues/79
-    # https://github.com/linsomniac/python-memcached/blob/288c159720eebcdf667727a859ef341f1e908308/memcache.py#L961
-
-    value = 'Iñtërnâtiônàlizætiøn'  # this simulates what python-memcached does on cache.set()
-    settings.cache.set('DEBUG', value)
-    assert settings.cache.get('DEBUG') == 'Iñtërnâtiônàlizætiøn'
-
-
 def test_read_only_setting(settings):
    settings.registry.register(
        'AWX_READ_ONLY',
@@ -251,31 +245,6 @@ def test_setting_from_db(settings, mocker):
        assert settings.cache.get('AWX_SOME_SETTING') == 'FROM_DB'


-@pytest.mark.parametrize('encrypted', (True, False))
-def test_setting_from_db_with_unicode(settings, mocker, encrypted):
-    settings.registry.register(
-        'AWX_SOME_SETTING',
-        field_class=fields.CharField,
-        category=_('System'),
-        category_slug='system',
-        default='DEFAULT',
-        encrypted=encrypted
-    )
-    # this simulates a bug in python-memcached; see https://github.com/linsomniac/python-memcached/issues/79
-    value = 'Iñtërnâtiônàlizætiøn'
-
-    setting_from_db = mocker.Mock(id=1, key='AWX_SOME_SETTING', value=value)
-    mocks = mocker.Mock(**{
-        'order_by.return_value': mocker.Mock(**{
-            '__iter__': lambda self: iter([setting_from_db]),
-            'first.return_value': setting_from_db
-        }),
-    })
-    with mocker.patch('awx.conf.models.Setting.objects.filter', return_value=mocks):
-        assert settings.AWX_SOME_SETTING == 'Iñtërnâtiônàlizætiøn'
-        assert settings.cache.get('AWX_SOME_SETTING') == 'Iñtërnâtiônàlizætiøn'
-
-
@pytest.mark.defined_in_file(AWX_SOME_SETTING='DEFAULT')
 def test_read_only_setting_assignment(settings):
    "read-only settings cannot be overwritten"
--- a/awx/conf/views.py
+++ b/awx/conf/views.py
@@ -10,6 +10,7 @@ import socket
 from socket import SHUT_RDWR

 # Django
+from django.db import connection
 from django.conf import settings
 from django.http import Http404
 from django.utils.translation import ugettext_lazy as _
@@ -130,7 +131,8 @@ class SettingSingletonDetail(RetrieveUpdateDestroyAPIView):
                setting.save(update_fields=['value'])
                settings_change_list.append(key)
        if settings_change_list:
-            handle_setting_changes.delay(settings_change_list)
+            connection.on_commit(lambda: handle_setting_changes.delay(settings_change_list))
+

    def destroy(self, request, *args, **kwargs):
        instance = self.get_object()
@@ -145,7 +147,7 @@ class SettingSingletonDetail(RetrieveUpdateDestroyAPIView):
            setting.delete()
            settings_change_list.append(setting.key)
        if settings_change_list:
-            handle_setting_changes.delay(settings_change_list)
+            connection.on_commit(lambda: handle_setting_changes.delay(settings_change_list))

        # When TOWER_URL_BASE is deleted from the API, reset it to the hostname
        # used to make the request as a default.
@@ -170,7 +172,13 @@ class SettingLoggingTest(GenericAPIView):
            return Response({'error': 'Logging not enabled'}, status=status.HTTP_409_CONFLICT)
        
        # Send test message to configured logger based on db settings
-        logging.getLogger('awx').error('AWX Connection Test Message')
+        try:
+            default_logger = settings.LOG_AGGREGATOR_LOGGERS[0]
+            if default_logger != 'awx':
+                default_logger = f'awx.analytics.{default_logger}'
+        except IndexError:
+            default_logger = 'awx'
+        logging.getLogger(default_logger).error('AWX Connection Test Message')
        
        hostname = getattr(settings, 'LOG_AGGREGATOR_HOST', None)
        protocol = getattr(settings, 'LOG_AGGREGATOR_PROTOCOL', None)
--- a/awx/locale/django.pot
+++ b/awx/locale/django.pot
--- a/awx/locale/en-us/LC_MESSAGES/django.po
+++ b/awx/locale/en-us/LC_MESSAGES/django.po
--- a/awx/locale/es/LC_MESSAGES/django.po
+++ b/awx/locale/es/LC_MESSAGES/django.po
--- a/awx/locale/fr/LC_MESSAGES/django.po
+++ b/awx/locale/fr/LC_MESSAGES/django.po
--- a/awx/locale/ja/LC_MESSAGES/django.po
+++ b/awx/locale/ja/LC_MESSAGES/django.po
--- a/awx/locale/nl/LC_MESSAGES/django.po
+++ b/awx/locale/nl/LC_MESSAGES/django.po
--- a/awx/locale/zh/LC_MESSAGES/django.po
+++ b/awx/locale/zh/LC_MESSAGES/django.po
--- a/awx/main/access.py
+++ b/awx/main/access.py
@@ -333,14 +333,14 @@ class BaseAccess(object):
            report_violation(_("License has expired."))

        free_instances = validation_info.get('free_instances', 0)
-        available_instances = validation_info.get('available_instances', 0)
+        instance_count = validation_info.get('instance_count', 0)

        if add_host_name:
            host_exists = Host.objects.filter(name=add_host_name).exists()
            if not host_exists and free_instances == 0:
-                report_violation(_("License count of %s instances has been reached.") % available_instances)
+                report_violation(_("License count of %s instances has been reached.") % instance_count)
            elif not host_exists and free_instances < 0:
-                report_violation(_("License count of %s instances has been exceeded.") % available_instances)
+                report_violation(_("License count of %s instances has been exceeded.") % instance_count)
        elif not add_host_name and free_instances < 0:
            report_violation(_("Host count exceeds available instances."))

@@ -495,7 +495,7 @@ class NotificationAttachMixin(BaseAccess):
            # due to this special case, we use symmetrical logic with attach permission
            return self._can_attach(notification_template=sub_obj, resource_obj=obj)
        return super(NotificationAttachMixin, self).can_unattach(
-            obj, sub_obj, relationship, relationship, data=data
+            obj, sub_obj, relationship, data=data
        )


@@ -1103,11 +1103,6 @@ class CredentialTypeAccess(BaseAccess):
    def can_use(self, obj):
        return True

-    def get_method_capability(self, method, obj, parent_obj):
-        if obj.managed_by_tower:
-            return False
-        return super(CredentialTypeAccess, self).get_method_capability(method, obj, parent_obj)
-
    def filtered_queryset(self):
        return self.model.objects.all()

@@ -1182,6 +1177,8 @@ class CredentialAccess(BaseAccess):
    def get_user_capabilities(self, obj, **kwargs):
        user_capabilities = super(CredentialAccess, self).get_user_capabilities(obj, **kwargs)
        user_capabilities['use'] = self.can_use(obj)
+        if getattr(obj, 'managed_by_tower', False) is True:
+            user_capabilities['edit'] = user_capabilities['delete'] = False
        return user_capabilities


@@ -1513,8 +1510,7 @@ class JobTemplateAccess(NotificationAttachMixin, BaseAccess):
        thus can be made by a job template administrator which may not have access
        to the any inventory, project, or credentials associated with the template.
        '''
-        # We are white listing fields that can
-        field_whitelist = [
+        allowed_fields = [
            'name', 'description', 'forks', 'limit', 'verbosity', 'extra_vars',
            'job_tags', 'force_handlers', 'skip_tags', 'ask_variables_on_launch',
            'ask_tags_on_launch', 'ask_job_type_on_launch', 'ask_skip_tags_on_launch',
@@ -1529,7 +1525,7 @@ class JobTemplateAccess(NotificationAttachMixin, BaseAccess):
            if k not in [x.name for x in obj._meta.concrete_fields]:
                continue
            if hasattr(obj, k) and getattr(obj, k) != v:
-                if k not in field_whitelist and v != getattr(obj, '%s_id' % k, None) \
+                if k not in allowed_fields and v != getattr(obj, '%s_id' % k, None) \
                        and not (hasattr(obj, '%s_id' % k) and getattr(obj, '%s_id' % k) is None and v == ''): # Equate '' to None in the case of foreign keys
                    return False
        return True
@@ -2480,13 +2476,16 @@ class NotificationAccess(BaseAccess):

 class LabelAccess(BaseAccess):
    '''
-    I can see/use a Label if I have permission to associated organization
+    I can see/use a Label if I have permission to associated organization, or to a JT that the label is on
    '''
    model = Label
    prefetch_related = ('modified_by', 'created_by', 'organization',)

    def filtered_queryset(self):
-        return self.model.objects.all()
+        return self.model.objects.filter(
+            Q(organization__in=Organization.accessible_pk_qs(self.user, 'read_role')) |
+            Q(unifiedjobtemplate_labels__in=UnifiedJobTemplate.accessible_pk_qs(self.user, 'read_role'))
+        )

    @check_superuser
    def can_add(self, data):
@@ -2751,6 +2750,9 @@ class WorkflowApprovalTemplateAccess(BaseAccess):
        else:
            return (self.check_related('workflow_approval_template', UnifiedJobTemplate, role_field='admin_role'))

+    def can_change(self, obj, data):
+        return self.user.can_access(WorkflowJobTemplate, 'change', obj.workflow_job_template, data={})
+
    def can_start(self, obj, validate_license=False):
        # for copying WFJTs that contain approval nodes
        if self.user.is_superuser:
--- a/awx/main/analytics/init.py
+++ b/awx/main/analytics/init.py
@@ -1 +1 @@
-from .core import register, gather, ship, table_version  # noqa
+from .core import all_collectors, expensive_collectors, register, gather, ship  # noqa
--- a/awx/main/analytics/broadcast_websocket.py
+++ b/awx/main/analytics/broadcast_websocket.py
@@ -20,7 +20,7 @@ from django.conf import settings
 BROADCAST_WEBSOCKET_REDIS_KEY_NAME = 'broadcast_websocket_stats'


-logger = logging.getLogger('awx.main.analytics.broadcast_websocket')
+logger = logging.getLogger('awx.analytics.broadcast_websocket')


 def dt_to_seconds(dt):
@@ -44,8 +44,8 @@ class FixedSlidingWindow():

    def cleanup(self, now_bucket=None):
        now_bucket = now_bucket or now_seconds()
-        if self.start_time + 60 <= now_bucket:
-            self.start_time = now_bucket + 60 + 1
+        if self.start_time + 60 < now_bucket:
+            self.start_time = now_bucket - 60

            # Delete old entries
            for k in list(self.buckets.keys()):
@@ -53,16 +53,15 @@ class FixedSlidingWindow():
                    del self.buckets[k]

    def record(self, ts=None):
-        ts = ts or datetime.datetime.now()
-        now_bucket = int((ts - datetime.datetime(1970,1,1)).total_seconds())
+        now_bucket = ts or dt_to_seconds(datetime.datetime.now())

        val = self.buckets.get(now_bucket, 0)
        self.buckets[now_bucket] = val + 1

        self.cleanup(now_bucket)

-    def render(self):
-        self.cleanup()
+    def render(self, ts=None):
+        self.cleanup(now_bucket=ts)
        return sum(self.buckets.values()) or 0


--- a/awx/main/analytics/collectors.py
+++ b/awx/main/analytics/collectors.py
@@ -1,3 +1,4 @@
+import io
 import os
 import os.path
 import platform
@@ -6,13 +7,14 @@ from django.db import connection
 from django.db.models import Count
 from django.conf import settings
 from django.utils.timezone import now
+from django.utils.translation import ugettext_lazy as _

 from awx.conf.license import get_license
 from awx.main.utils import (get_awx_version, get_ansible_version,
                            get_custom_venv_choices, camelcase_to_underscore)
 from awx.main import models
 from django.contrib.sessions.models import Session
-from awx.main.analytics import register, table_version
+from awx.main.analytics import register

 '''
 This module is used to define metrics collected by awx.main.analytics.gather()
@@ -31,9 +33,9 @@ data _since_ the last report date - i.e., new data in the last 24 hours)
 '''


-@register('config', '1.1')
-def config(since):
-    license_info = get_license(show_key=False)
+@register('config', '1.2', description=_('General platform configuration.'))
+def config(since, **kwargs):
+    license_info = get_license()
    install_type = 'traditional'
    if os.environ.get('container') == 'oci':
        install_type = 'openshift'
@@ -63,8 +65,8 @@ def config(since):
    }


-@register('counts', '1.0')
-def counts(since):
+@register('counts', '1.0', description=_('Counts of objects such as organizations, inventories, and projects'))
+def counts(since, **kwargs):
    counts = {}
    for cls in (models.Organization, models.Team, models.User,
                models.Inventory, models.Credential, models.Project,
@@ -98,8 +100,8 @@ def counts(since):
    return counts

    
-@register('org_counts', '1.0')
-def org_counts(since):
+@register('org_counts', '1.0', description=_('Counts of users and teams by organization'))
+def org_counts(since, **kwargs):
    counts = {}
    for org in models.Organization.objects.annotate(num_users=Count('member_role__members', distinct=True), 
                                                    num_teams=Count('teams', distinct=True)).values('name', 'id', 'num_users', 'num_teams'):
@@ -110,8 +112,8 @@ def org_counts(since):
    return counts
    
    
-@register('cred_type_counts', '1.0')
-def cred_type_counts(since):
+@register('cred_type_counts', '1.0', description=_('Counts of credentials by credential type'))
+def cred_type_counts(since, **kwargs):
    counts = {}
    for cred_type in models.CredentialType.objects.annotate(num_credentials=Count(
                                                            'credentials', distinct=True)).values('name', 'id', 'managed_by_tower', 'num_credentials'):  
@@ -122,8 +124,8 @@ def cred_type_counts(since):
    return counts
    
    
-@register('inventory_counts', '1.2')
-def inventory_counts(since):
+@register('inventory_counts', '1.2', description=_('Inventories, their inventory sources, and host counts'))
+def inventory_counts(since, **kwargs):
    counts = {}
    for inv in models.Inventory.objects.filter(kind='').annotate(num_sources=Count('inventory_sources', distinct=True), 
                                                                 num_hosts=Count('hosts', distinct=True)).only('id', 'name', 'kind'):
@@ -147,8 +149,8 @@ def inventory_counts(since):
    return counts


-@register('projects_by_scm_type', '1.0')
-def projects_by_scm_type(since):
+@register('projects_by_scm_type', '1.0', description=_('Counts of projects by source control type'))
+def projects_by_scm_type(since, **kwargs):
    counts = dict(
        (t[0] or 'manual', 0)
        for t in models.Project.SCM_TYPE_CHOICES
@@ -166,8 +168,8 @@ def _get_isolated_datetime(last_check):
    return last_check


-@register('instance_info', '1.0')
-def instance_info(since, include_hostnames=False):
+@register('instance_info', '1.0', description=_('Cluster topology and capacity'))
+def instance_info(since, include_hostnames=False, **kwargs):
    info = {}
    instances = models.Instance.objects.values_list('hostname').values(
        'uuid', 'version', 'capacity', 'cpu', 'memory', 'managed_by_policy', 'hostname', 'last_isolated_check', 'enabled')
@@ -192,8 +194,7 @@ def instance_info(since, include_hostnames=False):
    return info


-@register('job_counts', '1.0')
-def job_counts(since):
+def job_counts(since, **kwargs):
    counts = {}
    counts['total_jobs'] = models.UnifiedJob.objects.exclude(launch_type='sync').count()
    counts['status'] = dict(models.UnifiedJob.objects.exclude(launch_type='sync').values_list('status').annotate(Count('status')).order_by())
@@ -202,8 +203,7 @@ def job_counts(since):
    return counts
    
    
-@register('job_instance_counts', '1.0')
-def job_instance_counts(since):
+def job_instance_counts(since, **kwargs):
    counts = {}
    job_types = models.UnifiedJob.objects.exclude(launch_type='sync').values_list(
        'execution_node', 'launch_type').annotate(job_launch_type=Count('launch_type')).order_by()
@@ -217,34 +217,79 @@ def job_instance_counts(since):
    return counts


-@register('query_info', '1.0')
-def query_info(since, collection_type):
+@register('query_info', '1.0', description=_('Metadata about the analytics collected'))
+def query_info(since, collection_type, until, **kwargs):
    query_info = {}
    query_info['last_run'] = str(since)
-    query_info['current_time'] = str(now())
+    query_info['current_time'] = str(until)
    query_info['collection_type'] = collection_type
    return query_info


-# Copies Job Events from db to a .csv to be shipped
-@table_version('events_table.csv', '1.1')
-@table_version('unified_jobs_table.csv', '1.0')
-@table_version('unified_job_template_table.csv', '1.0')
-def copy_tables(since, full_path):
-    def _copy_table(table, query, path):
-        file_path = os.path.join(path, table + '_table.csv')
-        file = open(file_path, 'w', encoding='utf-8')
-        with connection.cursor() as cursor:
-            cursor.copy_expert(query, file)
-            file.close()
-        return file_path
+'''
+The event table can be *very* large, and we have a 100MB upload limit.

+Split large table dumps at dump time into a series of files.
+'''
+MAX_TABLE_SIZE = 200 * 1048576
+
+
+class FileSplitter(io.StringIO):
+    def __init__(self, filespec=None, *args, **kwargs):
+        self.filespec = filespec
+        self.files = []
+        self.currentfile = None
+        self.header = None
+        self.counter = 0
+        self.cycle_file()
+
+    def cycle_file(self):
+        if self.currentfile:
+            self.currentfile.close()
+        self.counter = 0
+        fname = '{}_split{}'.format(self.filespec, len(self.files))
+        self.currentfile = open(fname, 'w', encoding='utf-8')
+        self.files.append(fname)
+        if self.header:
+            self.currentfile.write('{}\n'.format(self.header))
+
+    def file_list(self):
+        self.currentfile.close()
+        # Check for an empty dump
+        if len(self.header) + 1 == self.counter:
+            os.remove(self.files[-1])
+            self.files = self.files[:-1]
+        # If we only have one file, remove the suffix
+        if len(self.files) == 1:
+            os.rename(self.files[0],self.files[0].replace('_split0',''))
+        return self.files
+
+    def write(self, s):
+        if not self.header:
+            self.header = s[0:s.index('\n')]
+        self.counter += self.currentfile.write(s)
+        if self.counter >= MAX_TABLE_SIZE:
+            self.cycle_file()
+
+
+def _copy_table(table, query, path):
+    file_path = os.path.join(path, table + '_table.csv')
+    file = FileSplitter(filespec=file_path)
+    with connection.cursor() as cursor:
+        cursor.copy_expert(query, file)
+    return file.file_list()
+
+
+@register('events_table', '1.2', format='csv', description=_('Automation task records'), expensive=True)
+def events_table(since, full_path, until, **kwargs):
    events_query = '''COPY (SELECT main_jobevent.id, 
                              main_jobevent.created,
+                              main_jobevent.modified,
                              main_jobevent.uuid,
                              main_jobevent.parent_uuid,
                              main_jobevent.event, 
                              main_jobevent.event_data::json->'task_action' AS task_action,
+                              (CASE WHEN event = 'playbook_on_stats' THEN event_data END) as playbook_on_stats,
                              main_jobevent.failed, 
                              main_jobevent.changed, 
                              main_jobevent.playbook, 
@@ -260,15 +305,21 @@ def copy_tables(since, full_path):
                              main_jobevent.event_data::json->'res'->'warnings' AS warnings,
                              main_jobevent.event_data::json->'res'->'deprecations' AS deprecations
                              FROM main_jobevent 
-                              WHERE main_jobevent.created > {}
-                              ORDER BY main_jobevent.id ASC) TO STDOUT WITH CSV HEADER'''.format(since.strftime("'%Y-%m-%d %H:%M:%S'"))
-    _copy_table(table='events', query=events_query, path=full_path)
+                              WHERE (main_jobevent.created > '{}' AND main_jobevent.created <= '{}')
+                              ORDER BY main_jobevent.id ASC) TO STDOUT WITH CSV HEADER
+                   '''.format(since.isoformat(),until.isoformat())
+    return _copy_table(table='events', query=events_query, path=full_path)

+
+@register('unified_jobs_table', '1.1', format='csv', description=_('Data on jobs run'), expensive=True)
+def unified_jobs_table(since, full_path, until, **kwargs):
    unified_job_query = '''COPY (SELECT main_unifiedjob.id,
                                 main_unifiedjob.polymorphic_ctype_id,
                                 django_content_type.model,
                                 main_unifiedjob.organization_id,
                                 main_organization.name as organization_name,
+                                 main_job.inventory_id, 
+                                 main_inventory.name as inventory_name,
                                 main_unifiedjob.created,  
                                 main_unifiedjob.name,  
                                 main_unifiedjob.unified_job_template_id, 
@@ -286,12 +337,19 @@ def copy_tables(since, full_path):
                                 main_unifiedjob.instance_group_id
                                 FROM main_unifiedjob
                                 JOIN django_content_type ON main_unifiedjob.polymorphic_ctype_id = django_content_type.id
+                                 LEFT JOIN main_job ON main_unifiedjob.id = main_job.unifiedjob_ptr_id
+                                 LEFT JOIN main_inventory ON main_job.inventory_id = main_inventory.id
                                 LEFT JOIN main_organization ON main_organization.id = main_unifiedjob.organization_id
-                                 WHERE (main_unifiedjob.created > {0} OR main_unifiedjob.finished > {0})
+                                 WHERE ((main_unifiedjob.created > '{0}' AND main_unifiedjob.created <= '{1}')
+                                       OR (main_unifiedjob.finished > '{0}' AND main_unifiedjob.finished <= '{1}'))
                                       AND main_unifiedjob.launch_type != 'sync'
-                                 ORDER BY main_unifiedjob.id ASC) TO STDOUT WITH CSV HEADER'''.format(since.strftime("'%Y-%m-%d %H:%M:%S'"))    
-    _copy_table(table='unified_jobs', query=unified_job_query, path=full_path)
+                                 ORDER BY main_unifiedjob.id ASC) TO STDOUT WITH CSV HEADER
+                        '''.format(since.isoformat(),until.isoformat())
+    return _copy_table(table='unified_jobs', query=unified_job_query, path=full_path)

+
+@register('unified_job_template_table', '1.0', format='csv', description=_('Data on job templates'))
+def unified_job_template_table(since, full_path, **kwargs):
    unified_job_template_query = '''COPY (SELECT main_unifiedjobtemplate.id, 
                                 main_unifiedjobtemplate.polymorphic_ctype_id,
                                 django_content_type.model,
@@ -309,6 +367,73 @@ def copy_tables(since, full_path):
                                 main_unifiedjobtemplate.status 
                                 FROM main_unifiedjobtemplate, django_content_type
                                 WHERE main_unifiedjobtemplate.polymorphic_ctype_id = django_content_type.id
-                                 ORDER BY main_unifiedjobtemplate.id ASC) TO STDOUT WITH CSV HEADER'''.format(since.strftime("'%Y-%m-%d %H:%M:%S'"))    
-    _copy_table(table='unified_job_template', query=unified_job_template_query, path=full_path)
-    return
+                                 ORDER BY main_unifiedjobtemplate.id ASC) TO STDOUT WITH CSV HEADER'''  
+    return _copy_table(table='unified_job_template', query=unified_job_template_query, path=full_path)
+
+
+@register('workflow_job_node_table', '1.0', format='csv', description=_('Data on workflow runs'), expensive=True)
+def workflow_job_node_table(since, full_path, until, **kwargs):
+    workflow_job_node_query = '''COPY (SELECT main_workflowjobnode.id,
+                                 main_workflowjobnode.created,
+                                 main_workflowjobnode.modified, 
+                                 main_workflowjobnode.job_id, 
+                                 main_workflowjobnode.unified_job_template_id, 
+                                 main_workflowjobnode.workflow_job_id, 
+                                 main_workflowjobnode.inventory_id, 
+                                 success_nodes.nodes AS success_nodes,
+                                 failure_nodes.nodes AS failure_nodes,
+                                 always_nodes.nodes AS always_nodes,
+                                 main_workflowjobnode.do_not_run, 
+                                 main_workflowjobnode.all_parents_must_converge
+                                 FROM main_workflowjobnode
+                                 LEFT JOIN (
+                                     SELECT from_workflowjobnode_id, ARRAY_AGG(to_workflowjobnode_id) AS nodes
+                                     FROM main_workflowjobnode_success_nodes
+                                     GROUP BY from_workflowjobnode_id
+                                 ) success_nodes ON main_workflowjobnode.id = success_nodes.from_workflowjobnode_id
+                                 LEFT JOIN (
+                                     SELECT from_workflowjobnode_id, ARRAY_AGG(to_workflowjobnode_id) AS nodes
+                                     FROM main_workflowjobnode_failure_nodes
+                                     GROUP BY from_workflowjobnode_id
+                                 ) failure_nodes ON main_workflowjobnode.id = failure_nodes.from_workflowjobnode_id
+                                 LEFT JOIN (
+                                     SELECT from_workflowjobnode_id, ARRAY_AGG(to_workflowjobnode_id) AS nodes
+                                     FROM main_workflowjobnode_always_nodes
+                                     GROUP BY from_workflowjobnode_id
+                                 ) always_nodes ON main_workflowjobnode.id = always_nodes.from_workflowjobnode_id
+                                 WHERE (main_workflowjobnode.modified > '{}' AND main_workflowjobnode.modified <= '{}')
+                                 ORDER BY main_workflowjobnode.id ASC) TO STDOUT WITH CSV HEADER
+                              '''.format(since.isoformat(),until.isoformat())
+    return _copy_table(table='workflow_job_node', query=workflow_job_node_query, path=full_path)
+
+
+@register('workflow_job_template_node_table', '1.0', format='csv', description=_('Data on workflows'))
+def workflow_job_template_node_table(since, full_path, **kwargs):
+    workflow_job_template_node_query = '''COPY (SELECT main_workflowjobtemplatenode.id, 
+                                 main_workflowjobtemplatenode.created,
+                                 main_workflowjobtemplatenode.modified, 
+                                 main_workflowjobtemplatenode.unified_job_template_id, 
+                                 main_workflowjobtemplatenode.workflow_job_template_id, 
+                                 main_workflowjobtemplatenode.inventory_id, 
+                                 success_nodes.nodes AS success_nodes,
+                                 failure_nodes.nodes AS failure_nodes,
+                                 always_nodes.nodes AS always_nodes,
+                                 main_workflowjobtemplatenode.all_parents_must_converge
+                                 FROM main_workflowjobtemplatenode
+                                 LEFT JOIN (
+                                     SELECT from_workflowjobtemplatenode_id, ARRAY_AGG(to_workflowjobtemplatenode_id) AS nodes
+                                     FROM main_workflowjobtemplatenode_success_nodes
+                                     GROUP BY from_workflowjobtemplatenode_id
+                                 ) success_nodes ON main_workflowjobtemplatenode.id = success_nodes.from_workflowjobtemplatenode_id
+                                 LEFT JOIN (
+                                     SELECT from_workflowjobtemplatenode_id, ARRAY_AGG(to_workflowjobtemplatenode_id) AS nodes
+                                     FROM main_workflowjobtemplatenode_failure_nodes
+                                     GROUP BY from_workflowjobtemplatenode_id
+                                 ) failure_nodes ON main_workflowjobtemplatenode.id = failure_nodes.from_workflowjobtemplatenode_id
+                                 LEFT JOIN (
+                                     SELECT from_workflowjobtemplatenode_id, ARRAY_AGG(to_workflowjobtemplatenode_id) AS nodes
+                                     FROM main_workflowjobtemplatenode_always_nodes
+                                     GROUP BY from_workflowjobtemplatenode_id
+                                 ) always_nodes ON main_workflowjobtemplatenode.id = always_nodes.from_workflowjobtemplatenode_id
+                                 ORDER BY main_workflowjobtemplatenode.id ASC) TO STDOUT WITH CSV HEADER'''   
+    return _copy_table(table='workflow_job_template_node', query=workflow_job_template_node_query, path=full_path)
--- a/awx/main/analytics/core.py
+++ b/awx/main/analytics/core.py
@@ -14,21 +14,17 @@ from rest_framework.exceptions import PermissionDenied
 from awx.conf.license import get_license
 from awx.main.models import Job
 from awx.main.access import access_registry
-from awx.main.models.ha import TowerAnalyticsState
 from awx.main.utils import get_awx_http_client_headers, set_environ

-
-__all__ = ['register', 'gather', 'ship', 'table_version']
+__all__ = ['register', 'gather', 'ship']


 logger = logging.getLogger('awx.main.analytics')

-manifest = dict()
-

 def _valid_license():
    try:
-        if get_license(show_key=False).get('license_type', 'UNLICENSED') == 'open':
+        if get_license().get('license_type', 'UNLICENSED') == 'open':
            return False
        access_registry[Job](None).check_license()
    except PermissionDenied:
@@ -37,114 +33,194 @@ def _valid_license():
    return True


-def register(key, version):
+def all_collectors():
+    from awx.main.analytics import collectors
+
+    collector_dict = {}
+    module = collectors
+    for name, func in inspect.getmembers(module):
+        if inspect.isfunction(func) and hasattr(func, '__awx_analytics_key__'):
+            key = func.__awx_analytics_key__
+            desc = func.__awx_analytics_description__ or ''
+            version = func.__awx_analytics_version__
+            collector_dict[key] = { 'name': key, 'version': version, 'description': desc}
+    return collector_dict
+
+
+def expensive_collectors():
+    from awx.main.analytics import collectors
+
+    ret = []
+    module = collectors
+    for name, func in inspect.getmembers(module):
+        if inspect.isfunction(func) and hasattr(func, '__awx_analytics_key__') and func.__awx_expensive__:
+            ret.append(func.__awx_analytics_key__)
+    return ret
+
+
+def register(key, version, description=None, format='json', expensive=False):
    """
    A decorator used to register a function as a metric collector.

-    Decorated functions should return JSON-serializable objects.
+    Decorated functions should do the following based on format:
+    - json: return JSON-serializable objects.
+    - csv: write CSV data to a filename named 'key'

    @register('projects_by_scm_type', 1)
    def projects_by_scm_type():
-        return {'git': 5, 'svn': 1, 'hg': 0}
+        return {'git': 5, 'svn': 1}
    """

    def decorate(f):
        f.__awx_analytics_key__ = key
        f.__awx_analytics_version__ = version
+        f.__awx_analytics_description__ = description
+        f.__awx_analytics_type__ = format
+        f.__awx_expensive__ = expensive
        return f

    return decorate


-def table_version(file_name, version):
-
-    global manifest
-    manifest[file_name] = version
-
-    def decorate(f):
-        return f
-
-    return decorate
-
-
-def gather(dest=None, module=None, collection_type='scheduled'):
+def gather(dest=None, module=None, subset = None, since = None, until = now(), collection_type='scheduled'):
    """
    Gather all defined metrics and write them as JSON files in a .tgz

    :param dest:    the (optional) absolute path to write a compressed tarball
-    :pararm module: the module to search for registered analytic collector
+    :param module: the module to search for registered analytic collector
                    functions; defaults to awx.main.analytics.collectors
    """
+    def _write_manifest(destdir, manifest):
+        path = os.path.join(destdir, 'manifest.json')
+        with open(path, 'w', encoding='utf-8') as f:
+            try:
+                json.dump(manifest, f)
+            except Exception:
+                f.close()
+                os.remove(f.name)
+                logger.exception("Could not generate manifest.json")

-    run_now = now()
-    state = TowerAnalyticsState.get_solo()
-    last_run = state.last_run
-    logger.debug("Last analytics run was: {}".format(last_run))
-    
-    max_interval = now() - timedelta(weeks=4)
-    if last_run < max_interval or not last_run:
-        last_run = max_interval
+    last_run = since or settings.AUTOMATION_ANALYTICS_LAST_GATHER or (now() - timedelta(weeks=4))
+    logger.debug("Last analytics run was: {}".format(settings.AUTOMATION_ANALYTICS_LAST_GATHER))

    if _valid_license() is False:
        logger.exception("Invalid License provided, or No License Provided")
-        return "Error: Invalid License provided, or No License Provided"
+        return None

    if collection_type != 'dry-run' and not settings.INSIGHTS_TRACKING_STATE:
        logger.error("Automation Analytics not enabled. Use --dry-run to gather locally without sending.")
-        return
+        return None

-    if module is None:
+    collector_list = []
+    if module:
+        collector_module = module
+    else:
        from awx.main.analytics import collectors
-        module = collectors
-
+        collector_module = collectors
+    for name, func in inspect.getmembers(collector_module):
+        if (
+            inspect.isfunction(func) and
+            hasattr(func, '__awx_analytics_key__') and
+            (not subset or name in subset)
+        ):
+            collector_list.append((name, func))

+    manifest = dict()
    dest = dest or tempfile.mkdtemp(prefix='awx_analytics')
-    for name, func in inspect.getmembers(module):
-        if inspect.isfunction(func) and hasattr(func, '__awx_analytics_key__'):
+    gather_dir = os.path.join(dest, 'stage')
+    os.mkdir(gather_dir, 0o700)
+    num_splits = 1
+    for name, func in collector_list:
+        if func.__awx_analytics_type__ == 'json':
            key = func.__awx_analytics_key__
-            manifest['{}.json'.format(key)] = func.__awx_analytics_version__
-            path = '{}.json'.format(os.path.join(dest, key))
+            path = '{}.json'.format(os.path.join(gather_dir, key))
            with open(path, 'w', encoding='utf-8') as f:
                try:
-                    if func.__name__ == 'query_info':
-                        json.dump(func(last_run, collection_type=collection_type), f)
-                    else:
-                        json.dump(func(last_run), f)
+                    json.dump(func(last_run, collection_type=collection_type, until=until), f)
+                    manifest['{}.json'.format(key)] = func.__awx_analytics_version__
                except Exception:
                    logger.exception("Could not generate metric {}.json".format(key))
                    f.close()
                    os.remove(f.name)
-    
-    path = os.path.join(dest, 'manifest.json')
-    with open(path, 'w', encoding='utf-8') as f:
-        try:
-            json.dump(manifest, f)
-        except Exception:
-            logger.exception("Could not generate manifest.json")
-            f.close()
-            os.remove(f.name)
+        elif func.__awx_analytics_type__ == 'csv':
+            key = func.__awx_analytics_key__
+            try:
+                files = func(last_run, full_path=gather_dir, until=until)
+                if files:
+                    manifest['{}.csv'.format(key)] = func.__awx_analytics_version__
+                if len(files) > num_splits:
+                    num_splits = len(files)
+            except Exception:
+                logger.exception("Could not generate metric {}.csv".format(key))

-    try:
-        collectors.copy_tables(since=last_run, full_path=dest)
-    except Exception:
-        logger.exception("Could not copy tables")
-        
-    # can't use isoformat() since it has colons, which GNU tar doesn't like
-    tarname = '_'.join([
-        settings.SYSTEM_UUID,
-        run_now.strftime('%Y-%m-%d-%H%M%S%z')
-    ])
-    try:
-        tgz = shutil.make_archive(
-            os.path.join(os.path.dirname(dest), tarname),
-            'gztar',
-            dest
-        )
-        return tgz
-    except Exception:
-        logger.exception("Failed to write analytics archive file")
-    finally: 
+    if not manifest:
+        # No data was collected
+        logger.warning("No data from {} to {}".format(last_run, until))
        shutil.rmtree(dest)
+        return None
+
+    # Always include config.json if we're using our collectors
+    if 'config.json' not in manifest.keys() and not module:
+        from awx.main.analytics import collectors
+        config = collectors.config
+        path = '{}.json'.format(os.path.join(gather_dir, config.__awx_analytics_key__))
+        with open(path, 'w', encoding='utf-8') as f:
+            try:
+                json.dump(collectors.config(last_run), f)
+                manifest['config.json'] = config.__awx_analytics_version__
+            except Exception:
+                logger.exception("Could not generate metric {}.json".format(key))
+                f.close()
+                os.remove(f.name)
+                shutil.rmtree(dest)
+                return None
+
+    stage_dirs = [gather_dir]
+    if num_splits > 1:
+        for i in range(0, num_splits):
+            split_path = os.path.join(dest, 'split{}'.format(i))
+            os.mkdir(split_path, 0o700)
+            filtered_manifest = {}
+            shutil.copy(os.path.join(gather_dir, 'config.json'), split_path)
+            filtered_manifest['config.json'] = manifest['config.json']
+            suffix = '_split{}'.format(i)
+            for file in os.listdir(gather_dir):
+                if file.endswith(suffix):
+                    old_file = os.path.join(gather_dir, file)
+                    new_filename = file.replace(suffix, '')
+                    new_file = os.path.join(split_path, new_filename)
+                    shutil.move(old_file, new_file)
+                    filtered_manifest[new_filename] = manifest[new_filename]
+            _write_manifest(split_path, filtered_manifest)
+            stage_dirs.append(split_path)
+
+    for item in list(manifest.keys()):
+        if not os.path.exists(os.path.join(gather_dir, item)):
+            manifest.pop(item)
+    _write_manifest(gather_dir, manifest)
+
+    tarfiles = []
+    try:
+        for i in range(0, len(stage_dirs)):
+            stage_dir = stage_dirs[i]
+            # can't use isoformat() since it has colons, which GNU tar doesn't like
+            tarname = '_'.join([
+                settings.SYSTEM_UUID,
+                until.strftime('%Y-%m-%d-%H%M%S%z'),
+                str(i)
+            ])
+            tgz = shutil.make_archive(
+                os.path.join(os.path.dirname(dest), tarname),
+                'gztar',
+                stage_dir
+            )
+            tarfiles.append(tgz)
+    except Exception:
+        shutil.rmtree(stage_dir, ignore_errors = True)
+        logger.exception("Failed to write analytics archive file")
+    finally:
+        shutil.rmtree(dest, ignore_errors = True)
+    return tarfiles


 def ship(path):
@@ -154,6 +230,9 @@ def ship(path):
    if not path:
        logger.error('Automation Analytics TAR not found')
        return
+    if not os.path.exists(path):
+        logger.error('Automation Analytics TAR {} not found'.format(path))
+        return
    if "Error:" in str(path):
        return
    try:
@@ -180,13 +259,11 @@ def ship(path):
                                  auth=(rh_user, rh_password),
                                  headers=s.headers,
                                  timeout=(31, 31))
-            if response.status_code != 202:
+            # Accept 2XX status_codes
+            if response.status_code >= 300:
                return logger.exception('Upload failed with status {}, {}'.format(response.status_code,
                                                                                  response.text))
-        run_now = now()
-        state = TowerAnalyticsState.get_solo()
-        state.last_run = run_now
-        state.save()
    finally:
        # cleanup tar.gz
-        os.remove(path)
+        if os.path.exists(path):
+            os.remove(path)
--- a/awx/main/analytics/metrics.py
+++ b/awx/main/analytics/metrics.py
@@ -12,7 +12,7 @@ from prometheus_client import (
 from awx.conf.license import get_license
 from awx.main.utils import (get_awx_version, get_ansible_version)
 from awx.main.analytics.collectors import (
-    counts, 
+    counts,
    instance_info,
    job_instance_counts,
    job_counts,
@@ -54,7 +54,7 @@ LICENSE_INSTANCE_FREE = Gauge('awx_license_instance_free', 'Number of remaining


 def metrics():
-    license_info = get_license(show_key=False)
+    license_info = get_license()
    SYSTEM_INFO.info({
        'install_uuid': settings.INSTALL_UUID,
        'insights_analytics': str(settings.INSIGHTS_TRACKING_STATE),
@@ -68,7 +68,7 @@ def metrics():
        'external_logger_type': getattr(settings, 'LOG_AGGREGATOR_TYPE', 'None')
    })

-    LICENSE_INSTANCE_TOTAL.set(str(license_info.get('available_instances', 0)))
+    LICENSE_INSTANCE_TOTAL.set(str(license_info.get('instance_count', 0)))
    LICENSE_INSTANCE_FREE.set(str(license_info.get('free_instances', 0)))

    current_counts = counts(None)
--- a/awx/main/conf.py
+++ b/awx/main/conf.py
@@ -1,8 +1,5 @@
 # Python
-import json
 import logging
-import os
-from distutils.version import LooseVersion as Version

 # Django
 from django.utils.translation import ugettext_lazy as _
@@ -14,6 +11,7 @@ from rest_framework.fields import FloatField
 # Tower
 from awx.conf import fields, register, register_validate

+
 logger = logging.getLogger('awx.main.conf')

 register(
@@ -80,11 +78,11 @@ register(
 )

 register(
-    'PROXY_IP_WHITELIST',
+    'PROXY_IP_ALLOWED_LIST',
    field_class=fields.StringListField,
-    label=_('Proxy IP Whitelist'),
+    label=_('Proxy IP Allowed List'),
    help_text=_("If Tower is behind a reverse proxy/load balancer, use this setting "
-                "to whitelist the proxy IP addresses from which Tower should trust "
+                "to configure the proxy IP addresses from which Tower should trust "
                "custom REMOTE_HOST_HEADERS header values. "
                "If this setting is an empty list (the default), the headers specified by "
                "REMOTE_HOST_HEADERS will be trusted unconditionally')"),
@@ -93,22 +91,10 @@ register(
 )


-def _load_default_license_from_file():
-    try:
-        license_file = os.environ.get('AWX_LICENSE_FILE', '/etc/tower/license')
-        if os.path.exists(license_file):
-            license_data = json.load(open(license_file))
-            logger.debug('Read license data from "%s".', license_file)
-            return license_data
-    except Exception:
-        logger.warning('Could not read license from "%s".', license_file, exc_info=True)
-    return {}
-
-
 register(
    'LICENSE',
    field_class=fields.DictField,
-    default=_load_default_license_from_file,
+    default=lambda: {},
    label=_('License'),
    help_text=_('The license controls which features and functionality are '
                'enabled. Use /api/v2/config/ to update or change '
@@ -125,7 +111,7 @@ register(
    encrypted=False,
    read_only=False,
    label=_('Red Hat customer username'),
-    help_text=_('This username is used to retrieve license information and to send Automation Analytics'),  # noqa
+    help_text=_('This username is used to send data to Automation Analytics'),
    category=_('System'),
    category_slug='system',
 )
@@ -138,7 +124,33 @@ register(
    encrypted=True,
    read_only=False,
    label=_('Red Hat customer password'),
-    help_text=_('This password is used to retrieve license information and to send Automation Analytics'),  # noqa
+    help_text=_('This password is used to send data to Automation Analytics'),
+    category=_('System'),
+    category_slug='system',
+)
+
+register(
+    'SUBSCRIPTIONS_USERNAME',
+    field_class=fields.CharField,
+    default='',
+    allow_blank=True,
+    encrypted=False,
+    read_only=False,
+    label=_('Red Hat or Satellite username'),
+    help_text=_('This username is used to retrieve subscription and content information'),  # noqa
+    category=_('System'),
+    category_slug='system',
+)
+
+register(
+    'SUBSCRIPTIONS_PASSWORD',
+    field_class=fields.CharField,
+    default='',
+    allow_blank=True,
+    encrypted=True,
+    read_only=False,
+    label=_('Red Hat or Satellite password'),
+    help_text=_('This password is used to retrieve subscription and content information'),  # noqa
    category=_('System'),
    category_slug='system',
 )
@@ -149,7 +161,7 @@ register(
    default='https://example.com',
    schemes=('http', 'https'),
    allow_plain_hostname=True,  # Allow hostname only without TLD.
-    label=_('Automation Analytics upload URL.'),
+    label=_('Automation Analytics upload URL'),
    help_text=_('This setting is used to to configure data collection for the Automation Analytics dashboard'),
    category=_('System'),
    category_slug='system',
@@ -241,23 +253,11 @@ register(
    field_class=fields.StringListField,
    required=False,
    label=_('Paths to expose to isolated jobs'),
-    help_text=_('Whitelist of paths that would otherwise be hidden to expose to isolated jobs. Enter one path per line.'),
+    help_text=_('List of paths that would otherwise be hidden to expose to isolated jobs. Enter one path per line.'),
    category=_('Jobs'),
    category_slug='jobs',
 )

-register(
-    'AWX_ISOLATED_VERBOSITY',
-    field_class=fields.IntegerField,
-    min_value=0,
-    max_value=5,
-    label=_('Verbosity level for isolated node management tasks'),
-    help_text=_('This can be raised to aid in debugging connection issues for isolated task execution'),
-    category=_('Jobs'),
-    category_slug='jobs',
-    default=0
-)
-
 register(
    'AWX_ISOLATED_CHECK_INTERVAL',
    field_class=fields.IntegerField,
@@ -266,6 +266,7 @@ register(
    help_text=_('The number of seconds to sleep between status checks for jobs running on isolated instances.'),
    category=_('Jobs'),
    category_slug='jobs',
+    unit=_('seconds'),
 )

 register(
@@ -277,6 +278,7 @@ register(
                'This includes the time needed to copy source control files (playbooks) to the isolated instance.'),
    category=_('Jobs'),
    category_slug='jobs',
+    unit=_('seconds'),
 )

 register(
@@ -289,6 +291,7 @@ register(
                'Value should be substantially greater than expected network latency.'),
    category=_('Jobs'),
    category_slug='jobs',
+    unit=_('seconds'),
 )

 register(
@@ -436,82 +439,16 @@ register(
 )

 register(
-    'PRIMARY_GALAXY_URL',
-    field_class=fields.URLField,
-    required=False,
-    allow_blank=True,
-    label=_('Primary Galaxy Server URL'),
+    'AWX_SHOW_PLAYBOOK_LINKS',
+    field_class=fields.BooleanField,
+    default=False,
+    label=_('Follow symlinks'),
    help_text=_(
-        'For organizations that run their own Galaxy service, this gives the option to specify a '
-        'host as the primary galaxy server. Requirements will be downloaded from the primary if the '
-        'specific role or collection is available there. If the content is not avilable in the primary, '
-        'or if this field is left blank, it will default to galaxy.ansible.com.'
+        'Follow symbolic links when scanning for playbooks. Be aware that setting this to True can lead '
+        'to infinite recursion if a link points to a parent directory of itself.'
    ),
    category=_('Jobs'),
-    category_slug='jobs'
-)
-
-register(
-    'PRIMARY_GALAXY_USERNAME',
-    field_class=fields.CharField,
-    required=False,
-    allow_blank=True,
-    label=_('Primary Galaxy Server Username'),
-    help_text=_('For using a galaxy server at higher precedence than the public Ansible Galaxy. '
-                'The username to use for basic authentication against the Galaxy instance, '
-                'this is mutually exclusive with PRIMARY_GALAXY_TOKEN.'),
-    category=_('Jobs'),
-    category_slug='jobs'
-)
-
-register(
-    'PRIMARY_GALAXY_PASSWORD',
-    field_class=fields.CharField,
-    encrypted=True,
-    required=False,
-    allow_blank=True,
-    label=_('Primary Galaxy Server Password'),
-    help_text=_('For using a galaxy server at higher precedence than the public Ansible Galaxy. '
-                'The password to use for basic authentication against the Galaxy instance, '
-                'this is mutually exclusive with PRIMARY_GALAXY_TOKEN.'),
-    category=_('Jobs'),
-    category_slug='jobs'
-)
-
-register(
-    'PRIMARY_GALAXY_TOKEN',
-    field_class=fields.CharField,
-    encrypted=True,
-    required=False,
-    allow_blank=True,
-    label=_('Primary Galaxy Server Token'),
-    help_text=_('For using a galaxy server at higher precedence than the public Ansible Galaxy. '
-                'The token to use for connecting with the Galaxy instance, '
-                'this is mutually exclusive with corresponding username and password settings.'),
-    category=_('Jobs'),
-    category_slug='jobs'
-)
-
-register(
-    'PRIMARY_GALAXY_AUTH_URL',
-    field_class=fields.CharField,
-    required=False,
-    allow_blank=True,
-    label=_('Primary Galaxy Authentication URL'),
-    help_text=_('For using a galaxy server at higher precedence than the public Ansible Galaxy. '
-                'The token_endpoint of a Keycloak server.'),
-    category=_('Jobs'),
-    category_slug='jobs'
-)
-
-register(
-    'PUBLIC_GALAXY_ENABLED',
-    field_class=fields.BooleanField,
-    default=True,
-    label=_('Allow Access to Public Galaxy'),
-    help_text=_('Allow or deny access to the public Ansible Galaxy during project updates.'),
-    category=_('Jobs'),
-    category_slug='jobs'
+    category_slug='jobs',
 )

 register(
@@ -519,7 +456,7 @@ register(
    field_class=fields.BooleanField,
    default=False,
    label=_('Ignore Ansible Galaxy SSL Certificate Verification'),
-    help_text=_('If set to true, certificate validation will not be done when'
+    help_text=_('If set to true, certificate validation will not be done when '
                'installing content from any Galaxy server.'),
    category=_('Jobs'),
    category_slug='jobs'
@@ -576,6 +513,7 @@ register(
                'timeout should be imposed. A timeout set on an individual job template will override this.'),
    category=_('Jobs'),
    category_slug='jobs',
+    unit=_('seconds'),
 )

 register(
@@ -588,6 +526,7 @@ register(
                'timeout should be imposed. A timeout set on an individual inventory source will override this.'),
    category=_('Jobs'),
    category_slug='jobs',
+    unit=_('seconds'),
 )

 register(
@@ -600,6 +539,7 @@ register(
                'timeout should be imposed. A timeout set on an individual project will override this.'),
    category=_('Jobs'),
    category_slug='jobs',
+    unit=_('seconds'),
 )

 register(
@@ -614,6 +554,7 @@ register(
                'Use a value of 0 to indicate that no timeout should be imposed.'),
    category=_('Jobs'),
    category_slug='jobs',
+    unit=_('seconds'),
 )

 register(
@@ -621,7 +562,7 @@ register(
    field_class=fields.IntegerField,
    allow_null=False,
    default=200,
-    label=_('Maximum number of forks per job.'),
+    label=_('Maximum number of forks per job'),
    help_text=_('Saving a Job Template with more than this number of forks will result in an error. '
                'When set to 0, no limit is applied.'),
    category=_('Jobs'),
@@ -751,6 +692,7 @@ register(
                'aggregator protocols.'),
    category=_('Logging'),
    category_slug='logging',
+    unit=_('seconds'),
 )
 register(
    'LOG_AGGREGATOR_VERIFY_CERT',
@@ -778,26 +720,40 @@ register(
    category_slug='logging',
 )
 register(
-    'LOG_AGGREGATOR_AUDIT',
+    'LOG_AGGREGATOR_MAX_DISK_USAGE_GB',
+    field_class=fields.IntegerField,
+    default=1,
+    min_value=1,
+    label=_('Maximum disk persistance for external log aggregation (in GB)'),
+    help_text=_('Amount of data to store (in gigabytes) during an outage of '
+                'the external log aggregator (defaults to 1). '
+                'Equivalent to the rsyslogd queue.maxdiskspace setting.'),
+    category=_('Logging'),
+    category_slug='logging',
+)
+register(
+    'LOG_AGGREGATOR_MAX_DISK_USAGE_PATH',
+    field_class=fields.CharField,
+    default='/var/lib/awx',
+    label=_('File system location for rsyslogd disk persistence'),
+    help_text=_('Location to persist logs that should be retried after an outage '
+                'of the external log aggregator (defaults to /var/lib/awx). '
+                'Equivalent to the rsyslogd queue.spoolDirectory setting.'),
+    category=_('Logging'),
+    category_slug='logging',
+)
+register(
+    'LOG_AGGREGATOR_RSYSLOGD_DEBUG',
    field_class=fields.BooleanField,
-    allow_null=True,
    default=False,
-    label=_('Enabled external log aggregation auditing'),
-    help_text=_('When enabled, all external logs emitted by Tower will also be written to /var/log/tower/external.log.  This is an experimental setting intended to be used for debugging external log aggregation issues (and may be subject to change in the future).'),  # noqa
+    label=_('Enable rsyslogd debugging'),
+    help_text=_('Enabled high verbosity debugging for rsyslogd.  '
+                'Useful for debugging connection issues for external log aggregation.'),
    category=_('Logging'),
    category_slug='logging',
 )


-register(
-    'BROKER_DURABILITY',
-    field_class=fields.BooleanField,
-    label=_('Message Durability'),
-    help_text=_('When set (the default), underlying queues will be persisted to disk.  Disable this to enable higher message bus throughput.'),
-    category=_('System'),
-    category_slug='system',
-)
-

 register(
    'AUTOMATION_ANALYTICS_LAST_GATHER',
@@ -817,7 +773,8 @@ register(
    default=14400,	# every 4 hours
    min_value=1800,	# every 30 minutes
    category=_('System'),
-    category_slug='system'
+    category_slug='system',
+    unit=_('seconds'),
 )


@@ -839,84 +796,4 @@ def logging_validate(serializer, attrs):
    return attrs


-def galaxy_validate(serializer, attrs):
-    """Ansible Galaxy config options have mutual exclusivity rules, these rules
-    are enforced here on serializer validation so that users will not be able
-    to save settings which obviously break all project updates.
-    """
-    prefix = 'PRIMARY_GALAXY_'
-    errors = {}
-
-    def _new_value(setting_name):
-        if setting_name in attrs:
-            return attrs[setting_name]
-        elif not serializer.instance:
-            return ''
-        return getattr(serializer.instance, setting_name, '')
-
-    if not _new_value('PRIMARY_GALAXY_URL'):
-        if _new_value('PUBLIC_GALAXY_ENABLED') is False:
-            msg = _('A URL for Primary Galaxy must be defined before disabling public Galaxy.')
-            # put error in both keys because UI has trouble with errors in toggles
-            for key in ('PRIMARY_GALAXY_URL', 'PUBLIC_GALAXY_ENABLED'):
-                errors.setdefault(key, [])
-                errors[key].append(msg)
-            raise serializers.ValidationError(errors)
-
-    from awx.main.constants import GALAXY_SERVER_FIELDS
-    if not any('{}{}'.format(prefix, subfield.upper()) in attrs for subfield in GALAXY_SERVER_FIELDS):
-        return attrs
-
-    galaxy_data = {}
-    for subfield in GALAXY_SERVER_FIELDS:
-        galaxy_data[subfield] = _new_value('{}{}'.format(prefix, subfield.upper()))
-    if not galaxy_data['url']:
-        for k, v in galaxy_data.items():
-            if v:
-                setting_name = '{}{}'.format(prefix, k.upper())
-                errors.setdefault(setting_name, [])
-                errors[setting_name].append(_(
-                    'Cannot provide field if PRIMARY_GALAXY_URL is not set.'
-                ))
-    for k in GALAXY_SERVER_FIELDS:
-        if galaxy_data[k]:
-            setting_name = '{}{}'.format(prefix, k.upper())
-            if (not serializer.instance) or (not getattr(serializer.instance, setting_name, '')):
-                # new auth is applied, so check if compatible with version
-                from awx.main.utils import get_ansible_version
-                current_version = get_ansible_version()
-                min_version = '2.9'
-                if Version(current_version) < Version(min_version):
-                    errors.setdefault(setting_name, [])
-                    errors[setting_name].append(_(
-                        'Galaxy server settings are not available until Ansible {min_version}, '
-                        'you are running {current_version}.'
-                    ).format(min_version=min_version, current_version=current_version))
-    if (galaxy_data['password'] or galaxy_data['username']) and (galaxy_data['token'] or galaxy_data['auth_url']):
-        for k in ('password', 'username', 'token', 'auth_url'):
-            setting_name = '{}{}'.format(prefix, k.upper())
-            if setting_name in attrs:
-                errors.setdefault(setting_name, [])
-                errors[setting_name].append(_(
-                    'Setting Galaxy token and authentication URL is mutually exclusive with username and password.'
-                ))
-    if bool(galaxy_data['username']) != bool(galaxy_data['password']):
-        msg = _('If authenticating via username and password, both must be provided.')
-        for k in ('username', 'password'):
-            setting_name = '{}{}'.format(prefix, k.upper())
-            errors.setdefault(setting_name, [])
-            errors[setting_name].append(msg)
-    if bool(galaxy_data['token']) != bool(galaxy_data['auth_url']):
-        msg = _('If authenticating via token, both token and authentication URL must be provided.')
-        for k in ('token', 'auth_url'):
-            setting_name = '{}{}'.format(prefix, k.upper())
-            errors.setdefault(setting_name, [])
-            errors[setting_name].append(msg)
-
-    if errors:
-        raise serializers.ValidationError(errors)
-    return attrs
-
-
 register_validate('logging', logging_validate)
-register_validate('jobs', galaxy_validate)
--- a/awx/main/constants.py
+++ b/awx/main/constants.py
@@ -10,8 +10,7 @@ __all__ = [
    'ANSI_SGR_PATTERN', 'CAN_CANCEL', 'ACTIVE_STATES', 'STANDARD_INVENTORY_UPDATE_ENV'
 ]

-
-CLOUD_PROVIDERS = ('azure_rm', 'ec2', 'gce', 'vmware', 'openstack', 'rhv', 'satellite6', 'cloudforms', 'tower')
+CLOUD_PROVIDERS = ('azure_rm', 'ec2', 'gce', 'vmware', 'openstack', 'rhv', 'satellite6', 'tower')
 SCHEDULEABLE_PROVIDERS = CLOUD_PROVIDERS + ('custom', 'scm',)
 PRIVILEGE_ESCALATION_METHODS = [
    ('sudo', _('Sudo')), ('su', _('Su')), ('pbrun', _('Pbrun')), ('pfexec', _('Pfexec')),
@@ -32,7 +31,7 @@ STANDARD_INVENTORY_UPDATE_ENV = {
 CAN_CANCEL = ('new', 'pending', 'waiting', 'running')
 ACTIVE_STATES = CAN_CANCEL
 CENSOR_VALUE = '************'
-ENV_BLACKLIST = frozenset((
+ENV_BLOCKLIST = frozenset((
    'VIRTUAL_ENV', 'PATH', 'PYTHONPATH', 'PROOT_TMP_DIR', 'JOB_ID',
    'INVENTORY_ID', 'INVENTORY_SOURCE_ID', 'INVENTORY_UPDATE_ID',
    'AD_HOC_COMMAND_ID', 'REST_API_URL', 'REST_API_TOKEN', 'MAX_EVENT_RES',
@@ -42,7 +41,7 @@ ENV_BLACKLIST = frozenset((
 ))

 # loggers that may be called in process of emitting a log
-LOGGER_BLACKLIST = (
+LOGGER_BLOCKLIST = (
    'awx.main.utils.handlers',
    'awx.main.utils.formatters',
    'awx.main.utils.filters',
@@ -51,7 +50,3 @@ LOGGER_BLACKLIST = (
    # loggers that may be called getting logging settings
    'awx.conf'
 )
-
-# these correspond to both AWX and Ansible settings to keep naming consistent
-# for instance, settings.PRIMARY_GALAXY_AUTH_URL vs env var ANSIBLE_GALAXY_SERVER_FOO_AUTH_URL
-GALAXY_SERVER_FIELDS = ('url', 'username', 'password', 'token', 'auth_url')
--- a/awx/main/consumers.py
+++ b/awx/main/consumers.py
@@ -75,7 +75,7 @@ class WebsocketSecretAuthHelper:
        nonce_diff = now - nonce_parsed
        if abs(nonce_diff) > nonce_tolerance:
            logger.warn(f"Potential replay attack or machine(s) time out of sync by {nonce_diff} seconds.")
-            raise ValueError("Potential replay attack or machine(s) time out of sync by {nonce_diff} seconds.")
+            raise ValueError(f"Potential replay attack or machine(s) time out of sync by {nonce_diff} seconds.")

        return True

@@ -95,19 +95,17 @@ class BroadcastConsumer(AsyncJsonWebsocketConsumer):
        try:
            WebsocketSecretAuthHelper.is_authorized(self.scope)
        except Exception:
-            # TODO: log ip of connected client
-            logger.warn("Broadcast client failed to authorize for reason.")
+            logger.warn(f"client '{self.channel_name}' failed to authorize against the broadcast endpoint.")
            await self.close()
            return

-        # TODO: log ip of connected client
-        logger.info(f"Broadcast client connected.")
        await self.accept()
        await self.channel_layer.group_add(settings.BROADCAST_WEBSOCKET_GROUP_NAME, self.channel_name)
+        logger.info(f"client '{self.channel_name}' joined the broadcast group.")

    async def disconnect(self, code):
-        # TODO: log ip of disconnected client
-        logger.info("Client disconnected")
+        logger.info(f"client '{self.channel_name}' disconnected from the broadcast group.")
+        await self.channel_layer.group_discard(settings.BROADCAST_WEBSOCKET_GROUP_NAME, self.channel_name)

    async def internal_message(self, event):
        await self.send(event['text'])
@@ -132,6 +130,14 @@ class EventConsumer(AsyncJsonWebsocketConsumer):
            await self.send_json({"close": True})
            await self.close()

+    async def disconnect(self, code):
+        current_groups = set(self.scope['session'].pop('groups') if 'groups' in self.scope['session'] else [])
+        for group_name in current_groups:
+            await self.channel_layer.group_discard(
+                group_name,
+                self.channel_name,
+            )
+
    @database_sync_to_async
    def user_can_see_object_id(self, user_access, oid):
        # At this point user is a channels.auth.UserLazyObject object
@@ -189,7 +195,6 @@ class EventConsumer(AsyncJsonWebsocketConsumer):
                    group_name,
                    self.channel_name
                )
-            logger.debug(f"Channel {self.channel_name} left groups {old_groups} and joined {new_groups_exclusive}")
            self.scope['session']['groups'] = new_groups
            await self.send_json({
                "groups_current": list(new_groups),
--- a/awx/main/credential_plugins/aim.py
+++ b/awx/main/credential_plugins/aim.py
@@ -1,15 +1,10 @@
-from .plugin import CredentialPlugin
+from .plugin import CredentialPlugin, CertFiles, raise_for_status

 from urllib.parse import quote, urlencode, urljoin

 from django.utils.translation import ugettext_lazy as _
 import requests

-# AWX
-from awx.main.utils import (
-    create_temporary_fifo,
-)
-
 aim_inputs = {
    'fields': [{
        'id': 'url',
@@ -81,22 +76,15 @@ def aim_backend(**kwargs):
    request_qs = '?' + urlencode(query_params, quote_via=quote)
    request_url = urljoin(url, '/'.join(['AIMWebService', 'api', 'Accounts']))

-    cert = None
-    if client_cert and client_key:
-        cert = (
-            create_temporary_fifo(client_cert.encode()),
-            create_temporary_fifo(client_key.encode())
+    with CertFiles(client_cert, client_key) as cert:
+        res = requests.get(
+            request_url + request_qs,
+            timeout=30,
+            cert=cert,
+            verify=verify,
+            allow_redirects=False,
        )
-    elif client_cert:
-        cert = create_temporary_fifo(client_cert.encode())
-
-    res = requests.get(
-        request_url + request_qs,
-        timeout=30,
-        cert=cert,
-        verify=verify,
-    )
-    res.raise_for_status()
+    raise_for_status(res)
    return res.json()['Content']


--- a/awx/main/credential_plugins/conjur.py
+++ b/awx/main/credential_plugins/conjur.py
@@ -1,16 +1,11 @@
-from .plugin import CredentialPlugin
+from .plugin import CredentialPlugin, CertFiles, raise_for_status

 import base64
-from urllib.parse import urljoin, quote_plus
+from urllib.parse import urljoin, quote

 from django.utils.translation import ugettext_lazy as _
 import requests

-# AWX
-from awx.main.utils import (
-    create_temporary_fifo,
-)
-

 conjur_inputs = {
    'fields': [{
@@ -55,32 +50,32 @@ conjur_inputs = {
 def conjur_backend(**kwargs):
    url = kwargs['url']
    api_key = kwargs['api_key']
-    account = quote_plus(kwargs['account'])
-    username = quote_plus(kwargs['username'])
-    secret_path = quote_plus(kwargs['secret_path'])
+    account = quote(kwargs['account'], safe='') 
+    username = quote(kwargs['username'], safe='')
+    secret_path = quote(kwargs['secret_path'], safe='')
    version = kwargs.get('secret_version')
    cacert = kwargs.get('cacert', None)

    auth_kwargs = {
        'headers': {'Content-Type': 'text/plain'},
-        'data': api_key
+        'data': api_key,
+        'allow_redirects': False,
    }
-    if cacert:
-        auth_kwargs['verify'] = create_temporary_fifo(cacert.encode())

-    # https://www.conjur.org/api.html#authentication-authenticate-post
-    resp = requests.post(
-        urljoin(url, '/'.join(['authn', account, username, 'authenticate'])),
-        **auth_kwargs
-    )
-    resp.raise_for_status()
+    with CertFiles(cacert) as cert:
+        # https://www.conjur.org/api.html#authentication-authenticate-post
+        auth_kwargs['verify'] = cert
+        resp = requests.post(
+            urljoin(url, '/'.join(['authn', account, username, 'authenticate'])),
+            **auth_kwargs
+        )
+    raise_for_status(resp)
    token = base64.b64encode(resp.content).decode('utf-8')

    lookup_kwargs = {
        'headers': {'Authorization': 'Token token="{}"'.format(token)},
+        'allow_redirects': False,
    }
-    if cacert:
-        lookup_kwargs['verify'] = create_temporary_fifo(cacert.encode())

    # https://www.conjur.org/api.html#secrets-retrieve-a-secret-get
    path = urljoin(url, '/'.join([
@@ -92,8 +87,10 @@ def conjur_backend(**kwargs):
    if version:
        path = '?'.join([path, version])

-    resp = requests.get(path, timeout=30, **lookup_kwargs)
-    resp.raise_for_status()
+    with CertFiles(cacert) as cert:
+        lookup_kwargs['verify'] = cert
+        resp = requests.get(path, timeout=30, **lookup_kwargs)
+    raise_for_status(resp)
    return resp.text


--- a/awx/main/credential_plugins/hashivault.py
+++ b/awx/main/credential_plugins/hashivault.py
@@ -3,16 +3,11 @@ import os
 import pathlib
 from urllib.parse import urljoin

-from .plugin import CredentialPlugin
+from .plugin import CredentialPlugin, CertFiles, raise_for_status

 import requests
 from django.utils.translation import ugettext_lazy as _

-# AWX
-from awx.main.utils import (
-    create_temporary_fifo,
-)
-
 base_inputs = {
    'fields': [{
        'id': 'url',
@@ -32,14 +27,41 @@ base_inputs = {
        'type': 'string',
        'multiline': True,
        'help_text': _('The CA certificate used to verify the SSL certificate of the Vault server')
-    }],
+    }, {
+        'id': 'role_id',
+        'label': _('AppRole role_id'),
+        'type': 'string',
+        'multiline': False,
+        'help_text': _('The Role ID for AppRole Authentication')
+    }, {
+        'id': 'secret_id',
+        'label': _('AppRole secret_id'),
+        'type': 'string',
+        'multiline': False,
+        'secret': True,
+        'help_text': _('The Secret ID for AppRole Authentication')
+    }, {
+        'id': 'default_auth_path',
+        'label': _('Path to Approle Auth'),
+        'type': 'string',
+        'multiline': False,
+        'default': 'approle',
+        'help_text': _('The AppRole Authentication path to use if one isn\'t provided in the metadata when linking to an input field. Defaults to \'approle\'')
+    }
+    ],
    'metadata': [{
        'id': 'secret_path',
        'label': _('Path to Secret'),
        'type': 'string',
        'help_text': _('The path to the secret stored in the secret backend e.g, /some/secret/')
+    }, {
+        'id': 'auth_path',
+        'label': _('Path to Auth'),
+        'type': 'string',
+        'multiline': False,
+        'help_text': _('The path where the Authentication method is mounted e.g, approle')
    }],
-    'required': ['url', 'token', 'secret_path'],
+    'required': ['url', 'secret_path'],
 }

 hashi_kv_inputs = copy.deepcopy(base_inputs)
@@ -88,8 +110,44 @@ hashi_ssh_inputs['metadata'] = [{
 hashi_ssh_inputs['required'].extend(['public_key', 'role'])


+def handle_auth(**kwargs):
+    token = None
+
+    if kwargs.get('token'):
+        token = kwargs['token']
+    elif kwargs.get('role_id') and kwargs.get('secret_id'):
+        token = approle_auth(**kwargs)
+    else:
+        raise Exception('Either token or AppRole parameters must be set')
+
+    return token
+
+
+def approle_auth(**kwargs):
+    role_id = kwargs['role_id']
+    secret_id = kwargs['secret_id']
+    # we first try to use the 'auth_path' from the metadata
+    # if not found we try to fetch the 'default_auth_path' from inputs
+    auth_path = kwargs.get('auth_path') or kwargs['default_auth_path']
+
+    url = urljoin(kwargs['url'], 'v1')
+    cacert = kwargs.get('cacert', None)
+
+    request_kwargs = {'timeout': 30}
+    # AppRole Login
+    request_kwargs['json'] = {'role_id': role_id, 'secret_id': secret_id}
+    sess = requests.Session()
+    request_url = '/'.join([url, 'auth', auth_path, 'login']).rstrip('/')
+    with CertFiles(cacert) as cert:
+        request_kwargs['verify'] = cert
+        resp = sess.post(request_url, **request_kwargs)
+    resp.raise_for_status()
+    token = resp.json()['auth']['client_token']
+    return token
+
+
 def kv_backend(**kwargs):
-    token = kwargs['token']
+    token = handle_auth(**kwargs)
    url = kwargs['url']
    secret_path = kwargs['secret_path']
    secret_backend = kwargs.get('secret_backend', None)
@@ -97,13 +155,14 @@ def kv_backend(**kwargs):
    cacert = kwargs.get('cacert', None)
    api_version = kwargs['api_version']

-    request_kwargs = {'timeout': 30}
-    if cacert:
-        request_kwargs['verify'] = create_temporary_fifo(cacert.encode())
+    request_kwargs = {
+        'timeout': 30,
+        'allow_redirects': False,
+    }

    sess = requests.Session()
    sess.headers['Authorization'] = 'Bearer {}'.format(token)
-    # Compatability header for older installs of Hashicorp Vault
+    # Compatibility header for older installs of Hashicorp Vault
    sess.headers['X-Vault-Token'] = token

    if api_version == 'v2':
@@ -126,8 +185,10 @@ def kv_backend(**kwargs):
            path_segments = [secret_path]

    request_url = urljoin(url, '/'.join(['v1'] + path_segments)).rstrip('/')
-    response = sess.get(request_url, **request_kwargs)
-    response.raise_for_status()
+    with CertFiles(cacert) as cert:
+        request_kwargs['verify'] = cert
+        response = sess.get(request_url, **request_kwargs)
+    raise_for_status(response)

    json = response.json()
    if api_version == 'v2':
@@ -144,15 +205,16 @@ def kv_backend(**kwargs):


 def ssh_backend(**kwargs):
-    token = kwargs['token']
+    token = handle_auth(**kwargs)
    url = urljoin(kwargs['url'], 'v1')
    secret_path = kwargs['secret_path']
    role = kwargs['role']
    cacert = kwargs.get('cacert', None)

-    request_kwargs = {'timeout': 30}
-    if cacert:
-        request_kwargs['verify'] = create_temporary_fifo(cacert.encode())
+    request_kwargs = {
+        'timeout': 30,
+        'allow_redirects': False,
+    }

    request_kwargs['json'] = {'public_key': kwargs['public_key']}
    if kwargs.get('valid_principals'):
@@ -164,9 +226,12 @@ def ssh_backend(**kwargs):
    sess.headers['X-Vault-Token'] = token
    # https://www.vaultproject.io/api/secret/ssh/index.html#sign-ssh-key
    request_url = '/'.join([url, secret_path, 'sign', role]).rstrip('/')
-    resp = sess.post(request_url, **request_kwargs)

-    resp.raise_for_status()
+    with CertFiles(cacert) as cert:
+        request_kwargs['verify'] = cert
+        resp = sess.post(request_url, **request_kwargs)
+
+    raise_for_status(resp)
    return resp.json()['data']['signed_key']


--- a/awx/main/credential_plugins/plugin.py
+++ b/awx/main/credential_plugins/plugin.py
@@ -1,3 +1,55 @@
+import os
+import tempfile
+
 from collections import namedtuple

+from requests.exceptions import HTTPError
+
 CredentialPlugin = namedtuple('CredentialPlugin', ['name', 'inputs', 'backend'])
+
+
+def raise_for_status(resp):
+    resp.raise_for_status()
+    if resp.status_code >= 300:
+        exc = HTTPError()
+        setattr(exc, 'response', resp)
+        raise exc
+
+
+class CertFiles():
+    """
+    A context manager used for writing a certificate and (optional) key
+    to $TMPDIR, and cleaning up afterwards.
+
+    This is particularly useful as a shared resource for credential plugins
+    that want to pull cert/key data out of the database and persist it
+    temporarily to the file system so that it can loaded into the openssl
+    certificate chain (generally, for HTTPS requests plugins make via the
+    Python requests library)
+
+    with CertFiles(cert_data, key_data) as cert:
+        # cert is string representing a path to the cert or pemfile
+        # temporarily written to disk
+        requests.post(..., cert=cert)
+    """
+
+    certfile = None
+
+    def __init__(self, cert, key=None):
+        self.cert = cert
+        self.key = key
+
+    def __enter__(self):
+        if not self.cert:
+            return None
+        self.certfile = tempfile.NamedTemporaryFile('wb', delete=False)
+        self.certfile.write(self.cert.encode())
+        if self.key:
+            self.certfile.write(b'\n')
+            self.certfile.write(self.key.encode())
+        self.certfile.flush()
+        return str(self.certfile.name)
+
+    def __exit__(self, *args):
+        if self.certfile and os.path.exists(self.certfile.name):
+            os.remove(self.certfile.name)
--- a/awx/main/db/profiled_pg/base.py
+++ b/awx/main/db/profiled_pg/base.py
@@ -24,7 +24,7 @@ class RecordedQueryLog(object):
        try:
            self.threshold = cache.get('awx-profile-sql-threshold')
        except Exception:
-            # if we can't reach memcached, just assume profiling's off
+            # if we can't reach the cache, just assume profiling's off
            self.threshold = None

    def append(self, query):
@@ -110,7 +110,7 @@ class RecordedQueryLog(object):
 class DatabaseWrapper(BaseDatabaseWrapper):
    """
    This is a special subclass of Django's postgres DB backend which - based on
-    the value of a special flag in memcached - captures slow queries and
+    the value of a special flag in cache - captures slow queries and
    writes profile and Python stack metadata to the disk.
    """

@@ -133,19 +133,19 @@ class DatabaseWrapper(BaseDatabaseWrapper):
        # is the same mechanism used by libraries like the django-debug-toolbar)
        #
        # in _this_ implementation, we represent it as a property which will
-        # check memcache for a special flag to be set (when the flag is set, it
+        # check the cache for a special flag to be set (when the flag is set, it
        # means we should start recording queries because somebody called
        # `awx-manage profile_sql`)
        #
        # it's worth noting that this property is wrapped w/ @memoize because
        # Django references this attribute _constantly_ (in particular, once
-        # per executed query); doing a memcached.get()  _at most_ once per
+        # per executed query); doing a cache.get()  _at most_ once per
        # second is a good enough window to detect when profiling is turned
        # on/off by a system administrator
        try:
            threshold = cache.get('awx-profile-sql-threshold')
        except Exception:
-            # if we can't reach memcached, just assume profiling's off
+            # if we can't reach the cache, just assume profiling's off
            threshold = None
        self.queries_log.threshold = threshold
        return threshold is not None
--- a/awx/main/dispatch/control.py
+++ b/awx/main/dispatch/control.py
@@ -2,6 +2,9 @@ import logging
 import uuid
 import json

+from django.conf import settings
+import redis
+
 from awx.main.dispatch import get_local_queuename

 from . import pg_bus_conn
@@ -21,7 +24,15 @@ class Control(object):
        self.queuename = host or get_local_queuename()

    def status(self, *args, **kwargs):
-        return self.control_with_reply('status', *args, **kwargs)
+        r = redis.Redis.from_url(settings.BROKER_URL)
+        if self.service == 'dispatcher':
+            stats = r.get(f'awx_{self.service}_statistics') or b''
+            return stats.decode('utf-8')
+        else:
+            workers = []
+            for key in r.keys('awx_callback_receiver_statistics_*'):
+                workers.append(r.get(key).decode('utf-8'))
+            return '\n'.join(workers)

    def running(self, *args, **kwargs):
        return self.control_with_reply('running', *args, **kwargs)
@@ -43,7 +54,7 @@ class Control(object):
            for reply in conn.events(select_timeout=timeout, yield_timeouts=True):
                if reply is None:
                    logger.error(f'{self.service} did not reply within {timeout}s')
-                    raise RuntimeError("{self.service} did not reply within {timeout}s")
+                    raise RuntimeError(f"{self.service} did not reply within {timeout}s")
                break

        return json.loads(reply.payload)
--- a/awx/main/dispatch/periodic.py
+++ b/awx/main/dispatch/periodic.py
@@ -22,7 +22,7 @@ class Scheduler(Scheduler):

        def run():
            ppid = os.getppid()
-            logger.warn(f'periodic beat started')
+            logger.warn('periodic beat started')
            while True:
                if os.getppid() != ppid:
                    # if the parent PID changes, this process has been orphaned
--- a/awx/main/dispatch/pool.py
+++ b/awx/main/dispatch/pool.py
@@ -5,6 +5,7 @@ import signal
 import sys
 import time
 import traceback
+from datetime import datetime
 from uuid import uuid4

 import collections
@@ -27,6 +28,12 @@ else:
    logger = logging.getLogger('awx.main.dispatch')


+class NoOpResultQueue(object):
+
+    def put(self, item):
+        pass
+
+
 class PoolWorker(object):
    '''
    Used to track a worker child process and its pending and finished messages.
@@ -56,11 +63,13 @@ class PoolWorker(object):
    It is "idle" when self.managed_tasks is empty.
    '''

-    def __init__(self, queue_size, target, args):
+    track_managed_tasks = False
+
+    def __init__(self, queue_size, target, args, **kwargs):
        self.messages_sent = 0
        self.messages_finished = 0
        self.managed_tasks = collections.OrderedDict()
-        self.finished = MPQueue(queue_size)
+        self.finished = MPQueue(queue_size) if self.track_managed_tasks else NoOpResultQueue()
        self.queue = MPQueue(queue_size)
        self.process = Process(target=target, args=(self.queue, self.finished) + args)
        self.process.daemon = True
@@ -74,7 +83,8 @@ class PoolWorker(object):
            if not body.get('uuid'):
                body['uuid'] = str(uuid4())
            uuid = body['uuid']
-        self.managed_tasks[uuid] = body
+        if self.track_managed_tasks:
+            self.managed_tasks[uuid] = body
        self.queue.put(body, block=True, timeout=5)
        self.messages_sent += 1
        self.calculate_managed_tasks()
@@ -111,6 +121,8 @@ class PoolWorker(object):
        return str(self.process.exitcode)

    def calculate_managed_tasks(self):
+        if not self.track_managed_tasks:
+            return
        # look to see if any tasks were finished
        finished = []
        for _ in range(self.finished.qsize()):
@@ -135,6 +147,8 @@ class PoolWorker(object):

    @property
    def current_task(self):
+        if not self.track_managed_tasks:
+            return None
        self.calculate_managed_tasks()
        # the task at [0] is the one that's running right now (or is about to
        # be running)
@@ -145,6 +159,8 @@ class PoolWorker(object):

    @property
    def orphaned_tasks(self):
+        if not self.track_managed_tasks:
+            return []
        orphaned = []
        if not self.alive:
            # if this process had a running task that never finished,
@@ -179,6 +195,11 @@ class PoolWorker(object):
        return not self.busy


+class StatefulPoolWorker(PoolWorker):
+
+    track_managed_tasks = True
+
+
 class WorkerPool(object):
    '''
    Creates a pool of forked PoolWorkers.
@@ -200,6 +221,7 @@ class WorkerPool(object):
    )
    '''

+    pool_cls = PoolWorker
    debug_meta = ''

    def __init__(self, min_workers=None, queue_size=None):
@@ -222,10 +244,10 @@ class WorkerPool(object):
        idx = len(self.workers)
        # It's important to close these because we're _about_ to fork, and we
        # don't want the forked processes to inherit the open sockets
-        # for the DB and memcached connections (that way lies race conditions)
+        # for the DB and cache connections (that way lies race conditions)
        django_connection.close()
        django_cache.close()
-        worker = PoolWorker(self.queue_size, self.target, (idx,) + self.target_args)
+        worker = self.pool_cls(self.queue_size, self.target, (idx,) + self.target_args)
        self.workers.append(worker)
        try:
            worker.start()
@@ -236,13 +258,13 @@ class WorkerPool(object):
        return idx, worker

    def debug(self, *args, **kwargs):
-        self.cleanup()
        tmpl = Template(
+            'Recorded at: {{ dt }} \n'
            '{{ pool.name }}[pid:{{ pool.pid }}] workers total={{ workers|length }} {{ meta }} \n'
            '{% for w in workers %}'
            '.  worker[pid:{{ w.pid }}]{% if not w.alive %} GONE exit={{ w.exitcode }}{% endif %}'
            ' sent={{ w.messages_sent }}'
-            ' finished={{ w.messages_finished }}'
+            '{% if w.messages_finished %} finished={{ w.messages_finished }}{% endif %}'
            ' qsize={{ w.managed_tasks|length }}'
            ' rss={{ w.mb }}MB'
            '{% for task in w.managed_tasks.values() %}'
@@ -260,7 +282,11 @@ class WorkerPool(object):
            '\n'
            '{% endfor %}'
        )
-        return tmpl.render(pool=self, workers=self.workers, meta=self.debug_meta)
+        now = datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S UTC')
+        return tmpl.render(
+            pool=self, workers=self.workers, meta=self.debug_meta,
+            dt=now
+        )

    def write(self, preferred_queue, body):
        queue_order = sorted(range(len(self.workers)), key=lambda x: -1 if x==preferred_queue else x)
@@ -293,6 +319,8 @@ class AutoscalePool(WorkerPool):
    down based on demand
    '''

+    pool_cls = StatefulPoolWorker
+
    def __init__(self, *args, **kwargs):
        self.max_workers = kwargs.pop('max_workers', None)
        super(AutoscalePool, self).__init__(*args, **kwargs)
@@ -309,6 +337,10 @@ class AutoscalePool(WorkerPool):
        # max workers can't be less than min_workers
        self.max_workers = max(self.min_workers, self.max_workers)

+    def debug(self, *args, **kwargs):
+        self.cleanup()
+        return super(AutoscalePool, self).debug(*args, **kwargs)
+
    @property
    def should_grow(self):
        if len(self.workers) < self.min_workers:
--- a/awx/main/dispatch/worker/base.py
+++ b/awx/main/dispatch/worker/base.py
@@ -8,6 +8,7 @@ import sys
 import redis
 import json
 import psycopg2
+import time
 from uuid import UUID
 from queue import Empty as QueueEmpty

@@ -34,6 +35,7 @@ class WorkerSignalHandler:

    def __init__(self):
        self.kill_now = False
+        signal.signal(signal.SIGTERM, signal.SIG_DFL)
        signal.signal(signal.SIGINT, self.exit_gracefully)

    def exit_gracefully(self, *args, **kwargs):
@@ -41,6 +43,9 @@ class WorkerSignalHandler:


 class AWXConsumerBase(object):
+
+    last_stats = time.time()
+
    def __init__(self, name, worker, queues=[], pool=None):
        self.should_stop = False

@@ -52,6 +57,7 @@ class AWXConsumerBase(object):
        if pool is None:
            self.pool = WorkerPool()
        self.pool.init_workers(self.worker.work_loop)
+        self.redis = redis.Redis.from_url(settings.BROKER_URL)

    @property
    def listening_on(self):
@@ -97,6 +103,16 @@ class AWXConsumerBase(object):
            queue = 0
        self.pool.write(queue, body)
        self.total_messages += 1
+        self.record_statistics()
+
+    def record_statistics(self):
+        if time.time() - self.last_stats > 1:  # buffer stat recording to once per second
+            try:
+                self.redis.set(f'awx_{self.name}_statistics', self.pool.debug())
+                self.last_stats = time.time()
+            except Exception:
+                logger.exception(f"encountered an error communicating with redis to store {self.name} statistics")
+                self.last_stats = time.time()

    def run(self, *args, **kwargs):
        signal.signal(signal.SIGINT, self.stop)
@@ -116,18 +132,9 @@ class AWXConsumerRedis(AWXConsumerBase):
        super(AWXConsumerRedis, self).run(*args, **kwargs)
        self.worker.on_start()

-        queue = redis.Redis.from_url(settings.BROKER_URL)
        while True:
-            try:
-                res = queue.blpop(self.queues)
-                res = json.loads(res[1])
-                self.process_task(res)
-            except redis.exceptions.RedisError:
-                logger.exception(f"encountered an error communicating with redis")
-            except (json.JSONDecodeError, KeyError):
-                logger.exception(f"failed to decode JSON message from redis")
-            if self.should_stop:
-                return
+            logger.debug(f'{os.getpid()} is alive')
+            time.sleep(60)


 class AWXConsumerPG(AWXConsumerBase):
--- a/awx/main/dispatch/worker/callback.py
+++ b/awx/main/dispatch/worker/callback.py
@@ -1,17 +1,18 @@
-import cProfile
+import json
 import logging
 import os
-import pstats
 import signal
-import tempfile
 import time
 import traceback
-from queue import Empty as QueueEmpty

 from django.conf import settings
 from django.utils.timezone import now as tz_now
 from django.db import DatabaseError, OperationalError, connection as django_connection
-from django.db.utils import InterfaceError, InternalError, IntegrityError
+from django.db.utils import InterfaceError, InternalError
+
+import psutil
+
+import redis

 from awx.main.consumers import emit_channel_notification
 from awx.main.models import (JobEvent, AdHocCommandEvent, ProjectUpdateEvent,
@@ -19,15 +20,12 @@ from awx.main.models import (JobEvent, AdHocCommandEvent, ProjectUpdateEvent,
                             Job)
 from awx.main.tasks import handle_success_and_failure_notifications
 from awx.main.models.events import emit_event_detail
+from awx.main.utils.profiling import AWXProfiler

 from .base import BaseWorker

 logger = logging.getLogger('awx.main.commands.run_callback_receiver')

-# the number of seconds to buffer events in memory before flushing
-# using JobEvent.objects.bulk_create()
-BUFFER_SECONDS = .1
-

 class CallbackBrokerWorker(BaseWorker):
    '''
@@ -39,31 +37,62 @@ class CallbackBrokerWorker(BaseWorker):
    '''

    MAX_RETRIES = 2
+    last_stats = time.time()
+    last_flush = time.time()
+    total = 0
+    last_event = ''
    prof = None

    def __init__(self):
        self.buff = {}
+        self.pid = os.getpid()
+        self.redis = redis.Redis.from_url(settings.BROKER_URL)
+        self.prof = AWXProfiler("CallbackBrokerWorker")
+        for key in self.redis.keys('awx_callback_receiver_statistics_*'):
+            self.redis.delete(key)

    def read(self, queue):
        try:
-            return queue.get(block=True, timeout=BUFFER_SECONDS)
-        except QueueEmpty:
-            return {'event': 'FLUSH'}
+            res = self.redis.blpop(settings.CALLBACK_QUEUE, timeout=1)
+            if res is None:
+                return {'event': 'FLUSH'}
+            self.total += 1
+            return json.loads(res[1])
+        except redis.exceptions.RedisError:
+            logger.exception("encountered an error communicating with redis")
+            time.sleep(1)
+        except (json.JSONDecodeError, KeyError):
+            logger.exception("failed to decode JSON message from redis")
+        finally:
+            self.record_statistics()
+        return {'event': 'FLUSH'}
+
+    def record_statistics(self):
+        # buffer stat recording to once per (by default) 5s
+        if time.time() - self.last_stats > settings.JOB_EVENT_STATISTICS_INTERVAL:
+            try:
+                self.redis.set(f'awx_callback_receiver_statistics_{self.pid}', self.debug())
+                self.last_stats = time.time()
+            except Exception:
+                logger.exception("encountered an error communicating with redis")
+                self.last_stats = time.time()
+
+    def debug(self):
+        return f'.  worker[pid:{self.pid}] sent={self.total} rss={self.mb}MB {self.last_event}'
+
+    @property
+    def mb(self):
+        return '{:0.3f}'.format(
+            psutil.Process(self.pid).memory_info().rss / 1024.0 / 1024.0
+        )

    def toggle_profiling(self, *args):
-        if self.prof:
-            self.prof.disable()
-            filename = f'callback-{os.getpid()}.pstats'
-            filepath = os.path.join(tempfile.gettempdir(), filename)
-            with open(filepath, 'w') as f:
-                pstats.Stats(self.prof, stream=f).sort_stats('cumulative').print_stats()
-            pstats.Stats(self.prof).dump_stats(filepath + '.raw')
-            self.prof = False
-            logger.error(f'profiling is disabled, wrote {filepath}')
-        else:
-            self.prof = cProfile.Profile()
-            self.prof.enable()
+        if not self.prof.is_started():
+            self.prof.start()
            logger.error('profiling is enabled')
+        else:
+            filepath = self.prof.stop()
+            logger.error(f'profiling is disabled, wrote {filepath}')

    def work_loop(self, *args, **kw):
        if settings.AWX_CALLBACK_PROFILE:
@@ -74,6 +103,7 @@ class CallbackBrokerWorker(BaseWorker):
        now = tz_now()
        if (
            force or
+            (time.time() - self.last_flush) > settings.JOB_EVENT_BUFFER_SECONDS or
            any([len(events) >= 1000 for events in self.buff.values()])
        ):
            for cls, events in self.buff.items():
@@ -84,30 +114,25 @@ class CallbackBrokerWorker(BaseWorker):
                    e.modified = now
                try:
                    cls.objects.bulk_create(events)
-                except Exception as exc:
+                except Exception:
                    # if an exception occurs, we should re-attempt to save the
                    # events one-by-one, because something in the list is
-                    # broken/stale (e.g., an IntegrityError on a specific event)
+                    # broken/stale
                    for e in events:
                        try:
-                            if (
-                                isinstance(exc, IntegrityError),
-                                getattr(e, 'host_id', '')
-                            ):
-                                # this is one potential IntegrityError we can
-                                # work around - if the host disappears before
-                                # the event can be processed
-                                e.host_id = None
                            e.save()
                        except Exception:
                            logger.exception('Database Error Saving Job Event')
                for e in events:
                    emit_event_detail(e)
            self.buff = {}
+            self.last_flush = time.time()

    def perform_work(self, body):
        try:
            flush = body.get('event') == 'FLUSH'
+            if flush:
+                self.last_event = ''
            if not flush:
                event_map = {
                    'job_id': JobEvent,
@@ -123,6 +148,8 @@ class CallbackBrokerWorker(BaseWorker):
                        job_identifier = body[key]
                        break

+                self.last_event = f'\n\t- {cls.__name__} for #{job_identifier} ({body.get("event", "")} {body.get("uuid", "")})'  # noqa
+
                if body.get('event') == 'EOF':
                    try:
                        final_counter = body.get('final_counter', 0)
--- a/awx/main/exceptions.py
+++ b/awx/main/exceptions.py
@@ -30,3 +30,10 @@ class _AwxTaskError():


 AwxTaskError = _AwxTaskError()
+
+
+class PostRunError(Exception):
+    def __init__(self, msg, status='failed', tb=''):
+        self.status = status
+        self.tb = tb
+        super(PostRunError, self).__init__(msg)
--- a/awx/main/fields.py
+++ b/awx/main/fields.py
@@ -7,8 +7,8 @@ import json
 import re
 import urllib.parse

-from jinja2 import Environment, StrictUndefined
-from jinja2.exceptions import UndefinedError, TemplateSyntaxError
+from jinja2 import sandbox, StrictUndefined
+from jinja2.exceptions import UndefinedError, TemplateSyntaxError, SecurityError

 # Django
 from django.contrib.postgres.fields import JSONField as upstream_JSONBField
@@ -50,7 +50,7 @@ from awx.main.models.rbac import (
    batch_role_ancestor_rebuilding, Role,
    ROLE_SINGLETON_SYSTEM_ADMINISTRATOR, ROLE_SINGLETON_SYSTEM_AUDITOR
 )
-from awx.main.constants import ENV_BLACKLIST
+from awx.main.constants import ENV_BLOCKLIST
 from awx.main import utils


@@ -637,6 +637,14 @@ class CredentialInputField(JSONSchemaField):
            else:
                decrypted_values[k] = v

+        # don't allow secrets with $encrypted$ on new object creation
+        if not model_instance.pk:
+            for field in model_instance.credential_type.secret_fields:
+                if value.get(field) == '$encrypted$':
+                    raise serializers.ValidationError({
+                        self.name: [f'$encrypted$ is a reserved keyword, and cannot be used for {field}.']
+                    })
+
        super(JSONSchemaField, self).validate(decrypted_values, model_instance)
        errors = {}
        for error in Draft4Validator(
@@ -870,9 +878,9 @@ class CredentialTypeInjectorField(JSONSchemaField):
                  'use is not allowed in credentials.').format(env_var),
                code='invalid', params={'value': env_var},
            )
-        if env_var in ENV_BLACKLIST:
+        if env_var in ENV_BLOCKLIST:
            raise django_exceptions.ValidationError(
-                _('Environment variable {} is blacklisted from use in credentials.').format(env_var),
+                _('Environment variable {} is not allowed to be used in credentials.').format(env_var),
                code='invalid', params={'value': env_var},
            )

@@ -932,7 +940,7 @@ class CredentialTypeInjectorField(JSONSchemaField):
                    self.validate_env_var_allowed(key)
            for key, tmpl in injector.items():
                try:
-                    Environment(
+                    sandbox.ImmutableSandboxedEnvironment(
                        undefined=StrictUndefined
                    ).from_string(tmpl).render(valid_namespace)
                except UndefinedError as e:
@@ -942,6 +950,10 @@ class CredentialTypeInjectorField(JSONSchemaField):
                        code='invalid',
                        params={'value': value},
                    )
+                except SecurityError as e:
+                    raise django_exceptions.ValidationError(
+                        _('Encountered unsafe code execution: {}').format(e)
+                    )
                except TemplateSyntaxError as e:
                    raise django_exceptions.ValidationError(
                        _('Syntax error rendering template for {sub_key} inside of {type} ({error_msg})').format(
--- a/awx/main/isolated/manager.py
+++ b/awx/main/isolated/manager.py
@@ -7,6 +7,7 @@ import tempfile
 import time
 import logging
 import yaml
+import datetime

 from django.conf import settings
 import ansible_runner
@@ -74,6 +75,7 @@ class IsolatedManager(object):
        env['ANSIBLE_RETRY_FILES_ENABLED'] = 'False'
        env['ANSIBLE_HOST_KEY_CHECKING'] = str(settings.AWX_ISOLATED_HOST_KEY_CHECKING)
        env['ANSIBLE_LIBRARY'] = os.path.join(os.path.dirname(awx.__file__), 'plugins', 'isolated')
+        env['ANSIBLE_COLLECTIONS_PATHS'] = settings.AWX_ANSIBLE_COLLECTIONS_PATHS
        set_pythonpath(os.path.join(settings.ANSIBLE_VENV_PATH, 'lib'), env)

        def finished_callback(runner_obj):
@@ -109,7 +111,6 @@ class IsolatedManager(object):
            'cancel_callback': self.canceled_callback,
            'settings': {
                'job_timeout': settings.AWX_ISOLATED_LAUNCH_TIMEOUT,
-                'pexpect_timeout': getattr(settings, 'PEXPECT_TIMEOUT', 5),
                'suppress_ansible_output': True,
            },
        }
@@ -123,6 +124,7 @@ class IsolatedManager(object):
            dir=private_data_dir
        )
        params = self.runner_params.copy()
+        params.get('envvars', dict())['ANSIBLE_CALLBACK_WHITELIST'] = 'profile_tasks'
        params['playbook'] = playbook
        params['private_data_dir'] = iso_dir
        if idle_timeout:
@@ -149,7 +151,6 @@ class IsolatedManager(object):
            # don't rsync source control metadata (it can be huge!)
            '- /project/.git',
            '- /project/.svn',
-            '- /project/.hg',
            # don't rsync job events that are in the process of being written
            '- /artifacts/job_events/*-partial.json.tmp',
            # don't rsync the ssh_key FIFO
@@ -169,7 +170,8 @@ class IsolatedManager(object):
        extravars = {
            'src': self.private_data_dir,
            'dest': settings.AWX_PROOT_BASE_PATH,
-            'ident': self.ident
+            'ident': self.ident,
+            'job_id': self.instance.id,
        }
        if playbook:
            extravars['playbook'] = playbook
@@ -205,7 +207,10 @@ class IsolatedManager(object):
        :param interval: an interval (in seconds) to wait between status polls
        """
        interval = interval if interval is not None else settings.AWX_ISOLATED_CHECK_INTERVAL
-        extravars = {'src': self.private_data_dir}
+        extravars = {
+            'src': self.private_data_dir,
+            'job_id': self.instance.id
+        }
        status = 'failed'
        rc = None
        last_check = time.time()
@@ -221,9 +226,13 @@ class IsolatedManager(object):
                logger.warning('Isolated job {} was manually canceled.'.format(self.instance.id))

            logger.debug('Checking on isolated job {} with `check_isolated.yml`.'.format(self.instance.id))
+            time_start = datetime.datetime.now()
            runner_obj = self.run_management_playbook('check_isolated.yml',
                                                      self.private_data_dir,
                                                      extravars=extravars)
+            time_end = datetime.datetime.now()
+            time_diff = time_end - time_start
+            logger.debug('Finished checking on isolated job {} with `check_isolated.yml` took {} seconds.'.format(self.instance.id, time_diff.total_seconds()))
            status, rc = runner_obj.status, runner_obj.rc

            if self.check_callback is not None and not self.captured_command_artifact:
--- a/awx/main/management/commands/bottleneck.py
+++ b/awx/main/management/commands/bottleneck.py
@@ -0,0 +1,96 @@
+from django.core.management.base import BaseCommand
+from django.db import connection
+
+from awx.main.models import JobTemplate
+
+
+class Command(BaseCommand):
+    help = "Find the slowest tasks and hosts for a Job Template's most recent runs."
+
+    def add_arguments(self, parser):
+        parser.add_argument('--template', dest='jt', type=int,
+                            help='ID of the Job Template to profile')
+        parser.add_argument('--threshold', dest='threshold', type=float, default=30,
+                            help='Only show tasks that took at least this many seconds (defaults to 30)')
+        parser.add_argument('--history', dest='history', type=float, default=25,
+                            help='The number of historic jobs to look at')
+        parser.add_argument('--ignore', action='append', help='ignore a specific action (e.g., --ignore git)')
+
+    def handle(self, *args, **options):
+        jt = options['jt']
+        threshold = options['threshold']
+        history = options['history']
+        ignore = options['ignore']
+
+        print('## ' + JobTemplate.objects.get(pk=jt).name + f' (last {history} runs)\n')
+        with connection.cursor() as cursor:
+            cursor.execute(
+                f'''
+                SELECT 
+                    b.id, b.job_id, b.host_name, b.created - a.created delta,
+                    b.task task,
+                    b.event_data::json->'task_action' task_action,
+                    b.event_data::json->'task_path' task_path
+                FROM main_jobevent a JOIN main_jobevent b
+                ON b.parent_uuid = a.parent_uuid  AND a.host_name = b.host_name
+                WHERE
+                    a.event = 'runner_on_start' AND
+                    b.event != 'runner_on_start' AND
+                    b.event != 'runner_on_skipped' AND
+                    b.failed = false AND
+                    a.job_id IN (
+                        SELECT unifiedjob_ptr_id FROM main_job
+                        WHERE job_template_id={jt}
+                        ORDER BY unifiedjob_ptr_id DESC
+                        LIMIT {history}
+                    )
+                ORDER BY delta DESC;
+                '''
+            )
+            slowest_events = cursor.fetchall()
+
+        def format_td(x):
+            return str(x).split('.')[0]
+
+        fastest = dict()
+        for event in slowest_events:
+            _id, job_id, host, duration, task, action, playbook = event
+            playbook = playbook.rsplit('/')[-1]
+            if ignore and action in ignore:
+                continue
+            if host:
+                fastest[(action, playbook)] = (_id, host, format_td(duration))
+
+        host_counts = dict()
+        warned = set()
+        print(f'slowest tasks (--threshold={threshold})\n---')
+
+        for event in slowest_events:
+            _id, job_id, host, duration, task, action, playbook = event
+            if ignore and action in ignore:
+                continue
+            if duration.total_seconds() < threshold:
+                break
+            playbook = playbook.rsplit('/')[-1]
+            human_duration = format_td(duration)
+
+            fastest_summary = ''
+            fastest_match = fastest.get((action, playbook))
+            if fastest_match[2] != human_duration and (host, action, playbook) not in warned:
+                warned.add((host, action, playbook))
+                fastest_summary = ' ' + self.style.WARNING(f'{fastest_match[1]} ran this in {fastest_match[2]}s at /api/v2/job_events/{fastest_match[0]}/')
+
+            url = f'/api/v2/jobs/{job_id}/'
+            print(' -- '.join([url, host, human_duration, action, task, playbook]) + fastest_summary)
+            host_counts.setdefault(host, [])
+            host_counts[host].append(duration)
+
+        host_counts = sorted(host_counts.items(), key=lambda item: [e.total_seconds() for e in item[1]], reverse=True)
+
+        print('\nslowest hosts\n---')
+        for h, matches in host_counts:
+            total = len(matches)
+            total_seconds = sum([e.total_seconds() for e in matches])
+            print(f'{h} had {total} tasks that ran longer than {threshold} second(s) for a total of {total_seconds}')
+
+        print('')
--- a/awx/main/management/commands/callback_stats.py
+++ b/awx/main/management/commands/callback_stats.py
@@ -34,7 +34,7 @@ class Command(BaseCommand):
                if clear:
                    for i in range(12):
                        sys.stdout.write('\x1b[1A\x1b[2K')
-                for l in lines:
-                    print(l)
+                for line in lines:
+                    print(line)
                clear = True
                time.sleep(.25)
--- a/awx/main/management/commands/check_license.py
+++ b/awx/main/management/commands/check_license.py
@@ -18,7 +18,5 @@ class Command(BaseCommand):
        super(Command, self).__init__()
        license = get_licenser().validate()
        if options.get('data'):
-            if license.get('license_key', '') != 'UNLICENSED':
-                license['license_key'] = '********'
            return json.dumps(license)
        return license.get('license_type', 'none')
--- a/awx/main/management/commands/check_migrations.py
+++ b/awx/main/management/commands/check_migrations.py
@@ -8,5 +8,7 @@ class Command(MakeMigrations):
    def execute(self, *args, **options):
        settings = connections['default'].settings_dict.copy()
        settings['ENGINE'] = 'sqlite3'
+        if 'application_name' in settings['OPTIONS']:
+            del settings['OPTIONS']['application_name']
        connections['default'] = DatabaseWrapper(settings)
        return MakeMigrations().execute(*args, **options)
--- a/awx/main/management/commands/cleanup_jobs.py
+++ b/awx/main/management/commands/cleanup_jobs.py
@@ -21,7 +21,7 @@ from awx.main.signals import (
    disable_computed_fields
 )

-from awx.main.management.commands.deletion import AWXCollector, pre_delete
+from awx.main.utils.deletion import AWXCollector, pre_delete


 class Command(BaseCommand):
--- a/awx/main/management/commands/create_preload_data.py
+++ b/awx/main/management/commands/create_preload_data.py
@@ -42,6 +42,16 @@ class Command(BaseCommand):
                                              },
                                              created_by=superuser)
                c.admin_role.members.add(superuser)
+                public_galaxy_credential = Credential(
+                    name='Ansible Galaxy',
+                    managed_by_tower=True,
+                    credential_type=CredentialType.objects.get(kind='galaxy'),
+                    inputs = {
+                        'url': 'https://galaxy.ansible.com/'
+                    }
+                )
+                public_galaxy_credential.save()
+                o.galaxy_credentials.add(public_galaxy_credential)
                i = Inventory.objects.create(name='Demo Inventory',
                                             organization=o,
                                             created_by=superuser)
--- a/awx/main/management/commands/gather_analytics.py
+++ b/awx/main/management/commands/gather_analytics.py
@@ -1,6 +1,9 @@
 import logging
+
 from awx.main.analytics import gather, ship
+from dateutil import parser
 from django.core.management.base import BaseCommand
+from django.utils.timezone import now


 class Command(BaseCommand):
@@ -15,6 +18,10 @@ class Command(BaseCommand):
                            help='Gather analytics without shipping. Works even if analytics are disabled in settings.')
        parser.add_argument('--ship', dest='ship', action='store_true',
                            help='Enable to ship metrics to the Red Hat Cloud')
+        parser.add_argument('--since', dest='since', action='store',
+                            help='Start date for collection')
+        parser.add_argument('--until', dest='until', action='store',
+                            help='End date for collection')

    def init_logging(self):
        self.logger = logging.getLogger('awx.main.analytics')
@@ -28,11 +35,28 @@ class Command(BaseCommand):
        self.init_logging()
        opt_ship = options.get('ship')
        opt_dry_run = options.get('dry-run')
+        opt_since = options.get('since') or None
+        opt_until = options.get('until') or None
+
+        if opt_since:
+            since = parser.parse(opt_since)
+        else:
+            since = None
+        if opt_until:
+            until = parser.parse(opt_until)
+        else:
+            until = now()
+
        if opt_ship and opt_dry_run:
            self.logger.error('Both --ship and --dry-run cannot be processed at the same time.')
            return
-        tgz = gather(collection_type='manual' if not opt_dry_run else 'dry-run')
-        if tgz:
-            self.logger.debug(tgz)
+        tgzfiles = gather(collection_type='manual' if not opt_dry_run else 'dry-run', since = since, until = until)
+        if tgzfiles:
+            for tgz in tgzfiles:
+                self.logger.info(tgz)
+        else:
+            self.logger.error('No analytics collected')
        if opt_ship:
-            ship(tgz)
+            if tgzfiles:
+                for tgz in tgzfiles:
+                    ship(tgz)
--- a/awx/main/management/commands/graph_jobs.py
+++ b/awx/main/management/commands/graph_jobs.py
@@ -0,0 +1,117 @@
+# Python
+import asciichartpy as chart
+import collections
+import time
+import sys
+
+# Django
+from django.db.models import Count
+from django.core.management.base import BaseCommand
+
+# AWX
+from awx.main.models import (
+    Job,
+    Instance
+)
+
+
+DEFAULT_WIDTH = 100
+DEFAULT_HEIGHT = 30
+
+
+def chart_color_lookup(color_str):
+    return getattr(chart, color_str)
+
+
+def clear_screen():
+    print(chr(27) + "[2J")
+
+
+class JobStatus():
+    def __init__(self, status, color, width):
+        self.status = status
+        self.color = color
+        self.color_code = chart_color_lookup(color)
+        self.x = collections.deque(maxlen=width)
+        self.y = collections.deque(maxlen=width)
+
+    def tick(self, x, y):
+        self.x.append(x)
+        self.y.append(y)
+
+
+class JobStatusController:
+    RESET = chart_color_lookup('reset')
+
+    def __init__(self, width):
+        self.plots = [
+            JobStatus('pending', 'red', width),
+            JobStatus('waiting', 'blue', width),
+            JobStatus('running', 'green', width)
+        ]
+        self.ts_start = int(time.time())
+
+    def tick(self):
+        ts = int(time.time()) - self.ts_start
+        q = Job.objects.filter(status__in=['pending','waiting','running']).values_list('status').order_by().annotate(Count('status'))
+        status_count = dict(pending=0, waiting=0, running=0)
+        for status, count in q:
+            status_count[status] = count
+
+        for p in self.plots:
+            p.tick(ts, status_count[p.status])
+
+    def series(self):
+        return [list(p.y) for p in self.plots]
+
+    def generate_status(self):
+        line = ""
+        lines = []
+        for p in self.plots:
+            lines.append(f'{p.color_code}{p.status} {p.y[-1]}{self.RESET}')
+
+        line += ", ".join(lines) + '\n'
+
+        width = 5
+        time_running = int(time.time()) - self.ts_start
+        instances = Instance.objects.all().order_by('hostname')
+        line += "Capacity:  " + ", ".join([f"{instance.capacity:{width}}" for instance in instances]) + '\n'
+        line += "Remaining: " + ", ".join([f"{instance.remaining_capacity:{width}}" for instance in instances]) + '\n'
+        line += f"Seconds running: {time_running}" + '\n'
+
+        return line
+
+
+class Command(BaseCommand):
+    help = "Plot pending, waiting, running jobs over time on the terminal"
+
+    def add_arguments(self, parser):
+        parser.add_argument('--refresh', dest='refresh', type=float, default=1.0,
+                            help='Time between refreshes of the graph and data in seconds (defaults to 1.0)')
+        parser.add_argument('--width', dest='width', type=int, default=DEFAULT_WIDTH,
+                            help=f'Width of the graph (defaults to {DEFAULT_WIDTH})')
+        parser.add_argument('--height', dest='height', type=int, default=DEFAULT_HEIGHT,
+                            help=f'Height of the graph (defaults to {DEFAULT_HEIGHT})')
+
+    def handle(self, *args, **options):
+        refresh_seconds = options['refresh']
+        width = options['width']
+        height = options['height']
+
+        jctl = JobStatusController(width)
+
+        conf = {
+            'colors': [chart_color_lookup(p.color) for p in jctl.plots],
+            'height': height,
+        }
+
+        while True:
+            jctl.tick()
+
+            draw = chart.plot(jctl.series(), conf)
+            status_line = jctl.generate_status()
+            clear_screen()
+            print(draw)
+            sys.stdout.write(status_line)
+            time.sleep(refresh_seconds)
+
--- a/awx/main/management/commands/inventory_import.py
+++ b/awx/main/management/commands/inventory_import.py
@@ -12,7 +12,6 @@ import sys
 import time
 import traceback
 import shutil
-from distutils.version import LooseVersion as Version

 # Django
 from django.conf import settings
@@ -20,6 +19,9 @@ from django.core.management.base import BaseCommand, CommandError
 from django.db import connection, transaction
 from django.utils.encoding import smart_text

+# DRF error class to distinguish license exceptions
+from rest_framework.exceptions import PermissionDenied
+
 # AWX inventory imports
 from awx.main.models.inventory import (
    Inventory,
@@ -32,14 +34,14 @@ from awx.main.utils.safe_yaml import sanitize_jinja

 # other AWX imports
 from awx.main.models.rbac import batch_role_ancestor_rebuilding
+# TODO: remove proot utils once we move to running inv. updates in containers
 from awx.main.utils import (
-    ignore_inventory_computed_fields,
    check_proot_installed,
    wrap_args_with_proot,
    build_proot_temp_dir,
+    ignore_inventory_computed_fields,
    get_licenser
 )
-from awx.main.utils.common import _get_ansible_version
 from awx.main.signals import disable_activity_stream
 from awx.main.constants import STANDARD_INVENTORY_UPDATE_ENV
 from awx.main.utils.pglock import advisory_lock
@@ -55,11 +57,11 @@ No license.
 See http://www.ansible.com/renew for license information.'''

 LICENSE_MESSAGE = '''\
-Number of licensed instances exceeded, would bring available instances to %(new_count)d, system is licensed for %(available_instances)d.
+Number of licensed instances exceeded, would bring available instances to %(new_count)d, system is licensed for %(instance_count)d.
 See http://www.ansible.com/renew for license extension information.'''

 DEMO_LICENSE_MESSAGE = '''\
-Demo mode free license count exceeded, would bring available instances to %(new_count)d, demo mode allows %(available_instances)d.
+Demo mode free license count exceeded, would bring available instances to %(new_count)d, demo mode allows %(instance_count)d.
 See http://www.ansible.com/renew for licensing information.'''


@@ -77,13 +79,11 @@ class AnsibleInventoryLoader(object):
        /usr/bin/ansible/ansible-inventory -i hosts --list
    '''

-    def __init__(self, source, is_custom=False, venv_path=None, verbosity=0):
+    def __init__(self, source, venv_path=None, verbosity=0):
        self.source = source
-        self.source_dir = functioning_dir(self.source)
-        self.is_custom = is_custom
-        self.tmp_private_dir = None
-        self.method = 'ansible-inventory'
        self.verbosity = verbosity
+        # TODO: remove once proot has been removed
+        self.tmp_private_dir = None
        if venv_path:
            self.venv_path = venv_path
        else:
@@ -133,43 +133,34 @@ class AnsibleInventoryLoader(object):
        # NOTE: why do we add "python" to the start of these args?
        # the script that runs ansible-inventory specifies a python interpreter
        # that makes no sense in light of the fact that we put all the dependencies
-        # inside of /venv/ansible, so we override the specified interpreter
+        # inside of /var/lib/awx/venv/ansible, so we override the specified interpreter
        # https://github.com/ansible/ansible/issues/50714
        bargs = ['python', ansible_inventory_path, '-i', self.source]
-        ansible_version = _get_ansible_version(ansible_inventory_path[:-len('-inventory')])
-        if ansible_version != 'unknown':
-            this_version = Version(ansible_version)
-            if this_version >= Version('2.5'):
-                bargs.extend(['--playbook-dir', self.source_dir])
-            if this_version >= Version('2.8'):
-                if self.verbosity:
-                    # INFO: -vvv, DEBUG: -vvvvv, for inventory, any more than 3 makes little difference
-                    bargs.append('-{}'.format('v' * min(5, self.verbosity * 2 + 1)))
+        bargs.extend(['--playbook-dir', functioning_dir(self.source)])
+        if self.verbosity:
+            # INFO: -vvv, DEBUG: -vvvvv, for inventory, any more than 3 makes little difference
+            bargs.append('-{}'.format('v' * min(5, self.verbosity * 2 + 1)))
        logger.debug('Using base command: {}'.format(' '.join(bargs)))
        return bargs

+    # TODO: Remove this once we move to running ansible-inventory in containers
+    # and don't need proot for process isolation anymore
    def get_proot_args(self, cmd, env):
        cwd = os.getcwd()
        if not check_proot_installed():
            raise RuntimeError("proot is not installed but is configured for use")

        kwargs = {}
-        if self.is_custom:
-            # use source's tmp dir for proot, task manager will delete folder
-            logger.debug("Using provided directory '{}' for isolation.".format(self.source_dir))
-            kwargs['proot_temp_dir'] = self.source_dir
-            cwd = self.source_dir
-        else:
-            # we cannot safely store tmp data in source dir or trust script contents
-            if env['AWX_PRIVATE_DATA_DIR']:
-                # If this is non-blank, file credentials are being used and we need access
-                private_data_dir = functioning_dir(env['AWX_PRIVATE_DATA_DIR'])
-                logger.debug("Using private credential data in '{}'.".format(private_data_dir))
-                kwargs['private_data_dir'] = private_data_dir
-            self.tmp_private_dir = build_proot_temp_dir()
-            logger.debug("Using fresh temporary directory '{}' for isolation.".format(self.tmp_private_dir))
-            kwargs['proot_temp_dir'] = self.tmp_private_dir
-            kwargs['proot_show_paths'] = [functioning_dir(self.source)]
+        # we cannot safely store tmp data in source dir or trust script contents
+        if env['AWX_PRIVATE_DATA_DIR']:
+            # If this is non-blank, file credentials are being used and we need access
+            private_data_dir = functioning_dir(env['AWX_PRIVATE_DATA_DIR'])
+            logger.debug("Using private credential data in '{}'.".format(private_data_dir))
+            kwargs['private_data_dir'] = private_data_dir
+        self.tmp_private_dir = build_proot_temp_dir()
+        logger.debug("Using fresh temporary directory '{}' for isolation.".format(self.tmp_private_dir))
+        kwargs['proot_temp_dir'] = self.tmp_private_dir
+        kwargs['proot_show_paths'] = [functioning_dir(self.source), settings.AWX_ANSIBLE_COLLECTIONS_PATHS]
        logger.debug("Running from `{}` working directory.".format(cwd))

        if self.venv_path != settings.ANSIBLE_VENV_PATH:
@@ -177,12 +168,14 @@ class AnsibleInventoryLoader(object):

        return wrap_args_with_proot(cmd, cwd, **kwargs)

+
    def command_to_json(self, cmd):
        data = {}
        stdout, stderr = '', ''
        env = self.build_env()

-        if ((self.is_custom or 'AWX_PRIVATE_DATA_DIR' in env) and
+        # TODO: remove proot args once inv. updates run in containers
+        if (('AWX_PRIVATE_DATA_DIR' in env) and
                getattr(settings, 'AWX_PROOT_ENABLED', False)):
            cmd = self.get_proot_args(cmd, env)

@@ -191,11 +184,13 @@ class AnsibleInventoryLoader(object):
        stdout = smart_text(stdout)
        stderr = smart_text(stderr)

+        # TODO: can be removed when proot is removed
        if self.tmp_private_dir:
            shutil.rmtree(self.tmp_private_dir, True)
+
        if proc.returncode != 0:
            raise RuntimeError('%s failed (rc=%d) with stdout:\n%s\nstderr:\n%s' % (
-                self.method, proc.returncode, stdout, stderr))
+                'ansible-inventory', proc.returncode, stdout, stderr))

        for line in stderr.splitlines():
            logger.error(line)
@@ -238,9 +233,9 @@ class Command(BaseCommand):
                            action='store_true', default=False,
                            help='overwrite (rather than merge) variables')
        parser.add_argument('--keep-vars', dest='keep_vars', action='store_true', default=False,
-                            help='use database variables if set')
+                            help='DEPRECATED legacy option, has no effect')
        parser.add_argument('--custom', dest='custom', action='store_true', default=False,
-                            help='this is a custom inventory script')
+                            help='DEPRECATED indicates a custom inventory script, no longer used')
        parser.add_argument('--source', dest='source', type=str, default=None,
                            metavar='s', help='inventory directory, file, or script to load')
        parser.add_argument('--enabled-var', dest='enabled_var', type=str,
@@ -266,12 +261,12 @@ class Command(BaseCommand):
                            'specifies the unique, immutable instance ID, may be '
                            'specified as "foo.bar" to traverse nested dicts.')

-    def set_logging_level(self):
+    def set_logging_level(self, verbosity):
        log_levels = dict(enumerate([logging.WARNING, logging.INFO,
                                     logging.DEBUG, 0]))
-        logger.setLevel(log_levels.get(self.verbosity, 0))
+        logger.setLevel(log_levels.get(verbosity, 0))

-    def _get_instance_id(self, from_dict, default=''):
+    def _get_instance_id(self, variables, default=''):
        '''
        Retrieve the instance ID from the given dict of host variables.

@@ -279,15 +274,23 @@ class Command(BaseCommand):
        the lookup will traverse into nested dicts, equivalent to:

        from_dict.get('foo', {}).get('bar', default)
+
+        Multiple ID variables may be specified as 'foo.bar,foobar', so that
+        it will first try to find 'bar' inside of 'foo', and if unable,
+        will try to find 'foobar' as a fallback
        '''
        instance_id = default
        if getattr(self, 'instance_id_var', None):
-            for key in self.instance_id_var.split('.'):
-                if not hasattr(from_dict, 'get'):
-                    instance_id = default
+            for single_instance_id in self.instance_id_var.split(','):
+                from_dict = variables
+                for key in single_instance_id.split('.'):
+                    if not hasattr(from_dict, 'get'):
+                        instance_id = default
+                        break
+                    instance_id = from_dict.get(key, default)
+                    from_dict = instance_id
+                if instance_id:
                    break
-                instance_id = from_dict.get(key, default)
-                from_dict = instance_id
        return smart_text(instance_id)

    def _get_enabled(self, from_dict, default=None):
@@ -321,7 +324,8 @@ class Command(BaseCommand):
        else:
            raise NotImplementedError('Value of enabled {} not understood.'.format(enabled))

-    def get_source_absolute_path(self, source):
+    @staticmethod
+    def get_source_absolute_path(source):
        if not os.path.exists(source):
            raise IOError('Source does not exist: %s' % source)
        source = os.path.join(os.getcwd(), os.path.dirname(source),
@@ -329,61 +333,6 @@ class Command(BaseCommand):
        source = os.path.normpath(os.path.abspath(source))
        return source

-    def load_inventory_from_database(self):
-        '''
-        Load inventory and related objects from the database.
-        '''
-        # Load inventory object based on name or ID.
-        if self.inventory_id:
-            q = dict(id=self.inventory_id)
-        else:
-            q = dict(name=self.inventory_name)
-        try:
-            self.inventory = Inventory.objects.get(**q)
-        except Inventory.DoesNotExist:
-            raise CommandError('Inventory with %s = %s cannot be found' % list(q.items())[0])
-        except Inventory.MultipleObjectsReturned:
-            raise CommandError('Inventory with %s = %s returned multiple results' % list(q.items())[0])
-        logger.info('Updating inventory %d: %s' % (self.inventory.pk,
-                                                   self.inventory.name))
-
-        # Load inventory source if specified via environment variable (when
-        # inventory_import is called from an InventoryUpdate task).
-        inventory_source_id = os.getenv('INVENTORY_SOURCE_ID', None)
-        inventory_update_id = os.getenv('INVENTORY_UPDATE_ID', None)
-        if inventory_source_id:
-            try:
-                self.inventory_source = InventorySource.objects.get(pk=inventory_source_id,
-                                                                    inventory=self.inventory)
-            except InventorySource.DoesNotExist:
-                raise CommandError('Inventory source with id=%s not found' %
-                                   inventory_source_id)
-            try:
-                self.inventory_update = InventoryUpdate.objects.get(pk=inventory_update_id)
-            except InventoryUpdate.DoesNotExist:
-                raise CommandError('Inventory update with id=%s not found' %
-                                   inventory_update_id)
-        # Otherwise, create a new inventory source to capture this invocation
-        # via command line.
-        else:
-            with ignore_inventory_computed_fields():
-                self.inventory_source, created = InventorySource.objects.get_or_create(
-                    inventory=self.inventory,
-                    source='file',
-                    source_path=os.path.abspath(self.source),
-                    overwrite=self.overwrite,
-                    overwrite_vars=self.overwrite_vars,
-                )
-                self.inventory_update = self.inventory_source.create_inventory_update(
-                    _eager_fields=dict(
-                        job_args=json.dumps(sys.argv),
-                        job_env=dict(os.environ.items()),
-                        job_cwd=os.getcwd())
-                )
-
-        # FIXME: Wait or raise error if inventory is being updated by another
-        # source.
-
    def _batch_add_m2m(self, related_manager, *objs, **kwargs):
        key = (related_manager.instance.pk, related_manager.through._meta.db_table)
        flush = bool(kwargs.get('flush', False))
@@ -422,7 +371,7 @@ class Command(BaseCommand):
            for mem_host in self.all_group.all_hosts.values():
                instance_id = self._get_instance_id(mem_host.variables)
                if not instance_id:
-                    logger.warning('Host "%s" has no "%s" variable',
+                    logger.warning('Host "%s" has no "%s" variable(s)',
                                   mem_host.name, self.instance_id_var)
                    continue
                mem_host.instance_id = instance_id
@@ -649,11 +598,12 @@ class Command(BaseCommand):
            if group_name in existing_group_names:
                continue
            mem_group = self.all_group.all_groups[group_name]
+            group_desc = mem_group.variables.pop('_awx_description', 'imported')
            group = self.inventory.groups.update_or_create(
                name=group_name,
                defaults={
                    'variables':json.dumps(mem_group.variables),
-                    'description':'imported'
+                    'description':group_desc
                }
            )[0]
            logger.debug('Group "%s" added', group.name)
@@ -776,8 +726,9 @@ class Command(BaseCommand):
        # Create any new hosts.
        for mem_host_name in sorted(mem_host_names_to_update):
            mem_host = self.all_group.all_hosts[mem_host_name]
-            host_attrs = dict(variables=json.dumps(mem_host.variables),
-                              description='imported')
+            import_vars = mem_host.variables
+            host_desc = import_vars.pop('_awx_description', 'imported')
+            host_attrs = dict(variables=json.dumps(import_vars), description=host_desc)
            enabled = self._get_enabled(mem_host.variables)
            if enabled is not None:
                host_attrs['enabled'] = enabled
@@ -871,42 +822,41 @@ class Command(BaseCommand):
        Load inventory from in-memory groups to the database, overwriting or
        merging as appropriate.
        '''
-        with advisory_lock('inventory_{}_update'.format(self.inventory.id)):
-            # FIXME: Attribute changes to superuser?
-            # Perform __in queries in batches (mainly for unit tests using SQLite).
-            self._batch_size = 500
-            self._build_db_instance_id_map()
-            self._build_mem_instance_id_map()
-            if self.overwrite:
-                self._delete_hosts()
-                self._delete_groups()
-                self._delete_group_children_and_hosts()
-            self._update_inventory()
-            self._create_update_groups()
-            self._create_update_hosts()
-            self._create_update_group_children()
-            self._create_update_group_hosts()
+        # FIXME: Attribute changes to superuser?
+        # Perform __in queries in batches (mainly for unit tests using SQLite).
+        self._batch_size = 500
+        self._build_db_instance_id_map()
+        self._build_mem_instance_id_map()
+        if self.overwrite:
+            self._delete_hosts()
+            self._delete_groups()
+            self._delete_group_children_and_hosts()
+        self._update_inventory()
+        self._create_update_groups()
+        self._create_update_hosts()
+        self._create_update_group_children()
+        self._create_update_group_hosts()

    def remote_tower_license_compare(self, local_license_type):
        # this requires https://github.com/ansible/ansible/pull/52747
        source_vars = self.all_group.variables
        remote_license_type = source_vars.get('tower_metadata', {}).get('license_type', None)
        if remote_license_type is None:
-            raise CommandError('Unexpected Error: Tower inventory plugin missing needed metadata!')
+            raise PermissionDenied('Unexpected Error: Tower inventory plugin missing needed metadata!')
        if local_license_type != remote_license_type:
-            raise CommandError('Tower server licenses must match: source: {} local: {}'.format(
+            raise PermissionDenied('Tower server licenses must match: source: {} local: {}'.format(
                remote_license_type, local_license_type
            ))

    def check_license(self):
        license_info = get_licenser().validate()
        local_license_type = license_info.get('license_type', 'UNLICENSED')
-        if license_info.get('license_key', 'UNLICENSED') == 'UNLICENSED':
+        if local_license_type == 'UNLICENSED':
            logger.error(LICENSE_NON_EXISTANT_MESSAGE)
-            raise CommandError('No license found!')
+            raise PermissionDenied('No license found!')
        elif local_license_type == 'open':
            return
-        available_instances = license_info.get('available_instances', 0)
+        instance_count = license_info.get('instance_count', 0)
        free_instances = license_info.get('free_instances', 0)
        time_remaining = license_info.get('time_remaining', 0)
        hard_error = license_info.get('trial', False) is True or license_info['instance_count'] == 10
@@ -914,24 +864,24 @@ class Command(BaseCommand):
        if time_remaining <= 0:
            if hard_error:
                logger.error(LICENSE_EXPIRED_MESSAGE)
-                raise CommandError("License has expired!")
+                raise PermissionDenied("License has expired!")
            else:
                logger.warning(LICENSE_EXPIRED_MESSAGE)
        # special check for tower-type inventory sources
        # but only if running the plugin
        TOWER_SOURCE_FILES = ['tower.yml', 'tower.yaml']
-        if self.inventory_source.source == 'tower' and any(f in self.source for f in TOWER_SOURCE_FILES):
+        if self.inventory_source.source == 'tower' and any(f in self.inventory_source.source_path for f in TOWER_SOURCE_FILES):
            # only if this is the 2nd call to license check, we cannot compare before running plugin
            if hasattr(self, 'all_group'):
                self.remote_tower_license_compare(local_license_type)
        if free_instances < 0:
            d = {
                'new_count': new_count,
-                'available_instances': available_instances,
+                'instance_count': instance_count,
            }
            if hard_error:
                logger.error(LICENSE_MESSAGE % d)
-                raise CommandError('License count exceeded!')
+                raise PermissionDenied('License count exceeded!')
            else:
                logger.warning(LICENSE_MESSAGE % d)

@@ -946,7 +896,7 @@ class Command(BaseCommand):

        active_count = Host.objects.org_active_count(org.id)
        if active_count > org.max_hosts:
-            raise CommandError('Host limit for organization exceeded!')
+            raise PermissionDenied('Host limit for organization exceeded!')

    def mark_license_failure(self, save=True):
        self.inventory_update.license_error = True
@@ -957,16 +907,103 @@ class Command(BaseCommand):
        self.inventory_update.save(update_fields=['org_host_limit_error'])

    def handle(self, *args, **options):
-        self.verbosity = int(options.get('verbosity', 1))
-        self.set_logging_level()
-        self.inventory_name = options.get('inventory_name', None)
-        self.inventory_id = options.get('inventory_id', None)
-        venv_path = options.get('venv', None)
+        # Load inventory and related objects from database.
+        inventory_name = options.get('inventory_name', None)
+        inventory_id = options.get('inventory_id', None)
+        if inventory_name and inventory_id:
+            raise CommandError('--inventory-name and --inventory-id are mutually exclusive')
+        elif not inventory_name and not inventory_id:
+            raise CommandError('--inventory-name or --inventory-id is required')
+
+        with advisory_lock('inventory_{}_import'.format(inventory_id)):
+            # Obtain rest of the options needed to run update
+            raw_source = options.get('source', None)
+            if not raw_source:
+                raise CommandError('--source is required')
+            verbosity = int(options.get('verbosity', 1))
+            self.set_logging_level(verbosity)
+            venv_path = options.get('venv', None)
+
+            # Load inventory object based on name or ID.
+            if inventory_id:
+                q = dict(id=inventory_id)
+            else:
+                q = dict(name=inventory_name)
+            try:
+                inventory = Inventory.objects.get(**q)
+            except Inventory.DoesNotExist:
+                raise CommandError('Inventory with %s = %s cannot be found' % list(q.items())[0])
+            except Inventory.MultipleObjectsReturned:
+                raise CommandError('Inventory with %s = %s returned multiple results' % list(q.items())[0])
+            logger.info('Updating inventory %d: %s' % (inventory.pk, inventory.name))
+
+
+            # Create ad-hoc inventory source and inventory update objects
+            with ignore_inventory_computed_fields():
+                source = Command.get_source_absolute_path(raw_source)
+
+                inventory_source, created = InventorySource.objects.get_or_create(
+                    inventory=inventory,
+                    source='file',
+                    source_path=os.path.abspath(source),
+                    overwrite=bool(options.get('overwrite', False)),
+                    overwrite_vars=bool(options.get('overwrite_vars', False)),
+                )
+                inventory_update = inventory_source.create_inventory_update(
+                    _eager_fields=dict(
+                        job_args=json.dumps(sys.argv),
+                        job_env=dict(os.environ.items()),
+                        job_cwd=os.getcwd())
+                )
+
+            data = AnsibleInventoryLoader(
+                source=source, venv_path=venv_path, verbosity=verbosity
+            ).load()
+
+            logger.debug('Finished loading from source: %s', source)
+
+            status, tb, exc = 'error', '', None
+            try:
+                self.perform_update(options, data, inventory_update)
+                status = 'successful'
+            except Exception as e:
+                exc = e
+                if isinstance(e, KeyboardInterrupt):
+                    status = 'canceled'
+                else:
+                    tb = traceback.format_exc()
+
+            with ignore_inventory_computed_fields():
+                inventory_update = InventoryUpdate.objects.get(pk=inventory_update.pk)
+                inventory_update.result_traceback = tb
+                inventory_update.status = status
+                inventory_update.save(update_fields=['status', 'result_traceback'])
+                inventory_source.status = status
+                inventory_source.save(update_fields=['status'])
+
+        if exc:
+            logger.error(str(exc))
+
+        if exc:
+            if isinstance(exc, CommandError):
+                sys.exit(1)
+            raise exc
+
+    def perform_update(self, options, data, inventory_update):
+        """Shared method for both awx-manage CLI updates and inventory updates
+        from the tasks system.
+
+        This saves the inventory data to the database, calling load_into_database
+        but also wraps that method in a host of options processing
+        """
+        # outside of normal options, these are needed as part of programatic interface
+        self.inventory = inventory_update.inventory
+        self.inventory_source = inventory_update.inventory_source
+        self.inventory_update = inventory_update
+
+        # the update options, could be parser object or dict
        self.overwrite = bool(options.get('overwrite', False))
        self.overwrite_vars = bool(options.get('overwrite_vars', False))
-        self.keep_vars = bool(options.get('keep_vars', False))
-        self.is_custom = bool(options.get('custom', False))
-        self.source = options.get('source', None)
        self.enabled_var = options.get('enabled_var', None)
        self.enabled_value = options.get('enabled_value', None)
        self.group_filter = options.get('group_filter', None) or r'^.+$'
@@ -974,17 +1011,6 @@ class Command(BaseCommand):
        self.exclude_empty_groups = bool(options.get('exclude_empty_groups', False))
        self.instance_id_var = options.get('instance_id_var', None)

-        self.invoked_from_dispatcher = False if os.getenv('INVENTORY_SOURCE_ID', None) is None else True
-
-        # Load inventory and related objects from database.
-        if self.inventory_name and self.inventory_id:
-            raise CommandError('--inventory-name and --inventory-id are mutually exclusive')
-        elif not self.inventory_name and not self.inventory_id:
-            raise CommandError('--inventory-name or --inventory-id is required')
-        if (self.overwrite or self.overwrite_vars) and self.keep_vars:
-            raise CommandError('--overwrite/--overwrite-vars and --keep-vars are mutually exclusive')
-        if not self.source:
-            raise CommandError('--source is required')
        try:
            self.group_filter_re = re.compile(self.group_filter)
        except re.error:
@@ -995,47 +1021,43 @@ class Command(BaseCommand):
            raise CommandError('invalid regular expression for --host-filter')

        begin = time.time()
-        self.load_inventory_from_database()

-        try:
-            self.check_license()
-        except CommandError as e:
-            self.mark_license_failure(save=True)
-            raise e
+        # Since perform_update can be invoked either through the awx-manage CLI
+        # or from the task system, we need to create a new lock at this level
+        # (even though inventory_import.Command.handle -- which calls
+        # perform_update -- has its own lock, inventory_ID_import)
+        with advisory_lock('inventory_{}_perform_update'.format(self.inventory.id)):

-        try:
-            # Check the per-org host limits
-            self.check_org_host_limit()
-        except CommandError as e:
-            self.mark_org_limits_failure(save=True)
-            raise e
+            try:
+                self.check_license()
+            except PermissionDenied as e:
+                self.mark_license_failure(save=True)
+                raise e
+
+            try:
+                # Check the per-org host limits
+                self.check_org_host_limit()
+            except PermissionDenied as e:
+                self.mark_org_limits_failure(save=True)
+                raise e

-        status, tb, exc = 'error', '', None
-        try:
            if settings.SQL_DEBUG:
                queries_before = len(connection.queries)

            # Update inventory update for this command line invocation.
            with ignore_inventory_computed_fields():
+                # TODO: move this to before perform_update
                iu = self.inventory_update
                if iu.status != 'running':
                    with transaction.atomic():
                        self.inventory_update.status = 'running'
                        self.inventory_update.save()

-            source = self.get_source_absolute_path(self.source)
-
-            data = AnsibleInventoryLoader(source=source, is_custom=self.is_custom,
-                                          venv_path=venv_path, verbosity=self.verbosity).load()
-
-            logger.debug('Finished loading from source: %s', source)
            logger.info('Processing JSON output...')
            inventory = MemInventory(
                group_filter_re=self.group_filter_re, host_filter_re=self.host_filter_re)
            inventory = dict_to_mem_data(data, inventory=inventory)

-            del data  # forget dict from import, could be large
-
            logger.info('Loaded %d groups, %d hosts', len(inventory.all_group.all_groups),
                        len(inventory.all_group.all_hosts))

@@ -1075,10 +1097,11 @@ class Command(BaseCommand):
                            if settings.SQL_DEBUG:
                                queries_before2 = len(connection.queries)
                            self.inventory.update_computed_fields()
-                            if settings.SQL_DEBUG:
-                                logger.warning('update computed fields took %d queries',
-                                               len(connection.queries) - queries_before2)
-                        # Check if the license is valid. 
+                        if settings.SQL_DEBUG:
+                            logger.warning('update computed fields took %d queries',
+                                           len(connection.queries) - queries_before2)
+
+                        # Check if the license is valid.
                        # If the license is not valid, a CommandError will be thrown,
                        # and inventory update will be marked as invalid.
                        # with transaction.atomic() will roll back the changes.
@@ -1088,11 +1111,11 @@ class Command(BaseCommand):
                        # Check the per-org host limits
                        license_fail = False
                        self.check_org_host_limit()
-                except CommandError as e:
+                except PermissionDenied as e:
                    if license_fail:
-                        self.mark_license_failure()
+                        self.mark_license_failure(save=True)
                    else:
-                        self.mark_org_limits_failure()
+                        self.mark_org_limits_failure(save=True)
                    raise e

                if settings.SQL_DEBUG:
@@ -1101,7 +1124,6 @@ class Command(BaseCommand):
                else:
                    logger.info('Inventory import completed for %s in %0.1fs',
                                self.inventory_source.name, time.time() - begin)
-                status = 'successful'

            # If we're in debug mode, then log the queries and time
            # used to do the operation.
@@ -1111,29 +1133,3 @@ class Command(BaseCommand):
                logger.warning('Inventory import required %d queries '
                               'taking %0.3fs', len(queries_this_import),
                               sqltime)
-        except Exception as e:
-            if isinstance(e, KeyboardInterrupt):
-                status = 'canceled'
-                exc = e
-            elif isinstance(e, CommandError):
-                exc = e
-            else:
-                tb = traceback.format_exc()
-                exc = e
-
-        if not self.invoked_from_dispatcher:
-            with ignore_inventory_computed_fields():
-                self.inventory_update = InventoryUpdate.objects.get(pk=self.inventory_update.pk)
-                self.inventory_update.result_traceback = tb
-                self.inventory_update.status = status
-                self.inventory_update.save(update_fields=['status', 'result_traceback'])
-                self.inventory_source.status = status
-                self.inventory_source.save(update_fields=['status'])
-
-            if exc:
-                logger.error(str(exc))
-
-        if exc:
-            if isinstance(exc, CommandError):
-                sys.exit(1)
-            raise exc
--- a/awx/main/management/commands/profile_sql.py
+++ b/awx/main/management/commands/profile_sql.py
@@ -19,3 +19,9 @@ class Command(BaseCommand):
        profile_sql.delay(
            threshold=options['threshold'], minutes=options['minutes']
        )
+        if options['threshold'] > 0:
+            print(f"SQL profiling initiated with a threshold of {options['threshold']} second(s) and a"
+                  f" duration of {options['minutes']} minute(s), any queries that meet criteria can"
+                  f" be found in /var/log/tower/profile/.")
+        else:
+            print("SQL profiling disabled.")
--- a/awx/main/management/commands/provision_instance.py
+++ b/awx/main/management/commands/provision_instance.py
@@ -13,7 +13,7 @@ from django.core.management.base import BaseCommand, CommandError
 class Command(BaseCommand):
    """
    Internal tower command.
-    Regsiter this instance with the database for HA tracking.
+    Register this instance with the database for HA tracking.
    """

    help = (
--- a/awx/main/management/commands/regenerate_secret_key.py
+++ b/awx/main/management/commands/regenerate_secret_key.py
@@ -82,7 +82,7 @@ class Command(BaseCommand):
            OAuth2Application.objects.filter(pk=app.pk).update(client_secret=encrypted)

    def _settings(self):
-        # don't update memcached, the *actual* value isn't changing
+        # don't update the cache, the *actual* value isn't changing
        post_save.disconnect(on_post_save_setting, sender=Setting)
        for setting in Setting.objects.filter().order_by('pk'):
            if settings_registry.is_setting_encrypted(setting.key):
--- a/awx/main/management/commands/register_queue.py
+++ b/awx/main/management/commands/register_queue.py
@@ -16,31 +16,24 @@ class InstanceNotFound(Exception):
        super(InstanceNotFound, self).__init__(*args, **kwargs)


-class Command(BaseCommand):
+class RegisterQueue:
+    def __init__(self, queuename, controller, instance_percent, inst_min, hostname_list):
+        self.instance_not_found_err = None
+        self.queuename = queuename
+        self.controller = controller
+        self.instance_percent = instance_percent
+        self.instance_min = inst_min
+        self.hostname_list = hostname_list

-    def add_arguments(self, parser):
-        parser.add_argument('--queuename', dest='queuename', type=str,
-                            help='Queue to create/update')
-        parser.add_argument('--hostnames', dest='hostnames', type=str,
-                            help='Comma-Delimited Hosts to add to the Queue (will not remove already assigned instances)')
-        parser.add_argument('--controller', dest='controller', type=str,
-                            default='', help='The controlling group (makes this an isolated group)')
-        parser.add_argument('--instance_percent', dest='instance_percent', type=int, default=0,
-                            help='The percentage of active instances that will be assigned to this group'),
-        parser.add_argument('--instance_minimum', dest='instance_minimum', type=int, default=0,
-                            help='The minimum number of instance that will be retained for this group from available instances')
-
-
-    def get_create_update_instance_group(self, queuename, instance_percent, instance_min):
+    def get_create_update_instance_group(self):
        created = False
        changed = False
-
-        (ig, created) = InstanceGroup.objects.get_or_create(name=queuename)
-        if ig.policy_instance_percentage != instance_percent:
-            ig.policy_instance_percentage = instance_percent
+        (ig, created) = InstanceGroup.objects.get_or_create(name=self.queuename)
+        if ig.policy_instance_percentage != self.instance_percent:
+            ig.policy_instance_percentage = self.instance_percent
            changed = True
-        if ig.policy_instance_minimum != instance_min:
-            ig.policy_instance_minimum = instance_min
+        if ig.policy_instance_minimum != self.instance_min:
+            ig.policy_instance_minimum = self.instance_min
            changed = True

        if changed:
@@ -48,12 +41,12 @@ class Command(BaseCommand):

        return (ig, created, changed)

-    def update_instance_group_controller(self, ig, controller):
+    def update_instance_group_controller(self, ig):
        changed = False
        control_ig = None

-        if controller:
-            control_ig = InstanceGroup.objects.filter(name=controller).first()
+        if self.controller:
+            control_ig = InstanceGroup.objects.filter(name=self.controller).first()

        if control_ig and ig.controller_id != control_ig.pk:
            ig.controller = control_ig
@@ -62,10 +55,10 @@ class Command(BaseCommand):

        return (control_ig, changed)

-    def add_instances_to_group(self, ig, hostname_list):
+    def add_instances_to_group(self, ig):
        changed = False

-        instance_list_unique = set([x.strip() for x in hostname_list if x])
+        instance_list_unique = set([x.strip() for x in self.hostname_list if x])
        instances = []
        for inst_name in instance_list_unique:
            instance = Instance.objects.filter(hostname=inst_name)
@@ -86,43 +79,61 @@ class Command(BaseCommand):

        return (instances, changed)

-    def handle(self, **options):
-        instance_not_found_err = None
-        queuename = options.get('queuename')
-        if not queuename:
-            raise CommandError("Specify `--queuename` to use this command.")
-        ctrl = options.get('controller')
-        inst_per = options.get('instance_percent')
-        inst_min = options.get('instance_minimum')
-        hostname_list = []
-        if options.get('hostnames'):
-            hostname_list = options.get('hostnames').split(",")
-
+    def register(self):
        with advisory_lock('cluster_policy_lock'):
            with transaction.atomic():
                changed2 = False
                changed3 = False
-                (ig, created, changed1) = self.get_create_update_instance_group(queuename, inst_per, inst_min)
+                (ig, created, changed1) = self.get_create_update_instance_group()
                if created:
                    print("Creating instance group {}".format(ig.name))
                elif not created:
                    print("Instance Group already registered {}".format(ig.name))

-                if ctrl:
-                    (ig_ctrl, changed2) = self.update_instance_group_controller(ig, ctrl)
+                if self.controller:
+                    (ig_ctrl, changed2) = self.update_instance_group_controller(ig)
                    if changed2:
-                        print("Set controller group {} on {}.".format(ctrl, queuename))
+                        print("Set controller group {} on {}.".format(self.controller, self.queuename))

                try:
-                    (instances, changed3) = self.add_instances_to_group(ig, hostname_list)
+                    (instances, changed3) = self.add_instances_to_group(ig)
                    for i in instances:
                        print("Added instance {} to {}".format(i.hostname, ig.name))
                except InstanceNotFound as e:
-                    instance_not_found_err = e
+                    self.instance_not_found_err = e

        if any([changed1, changed2, changed3]):
            print('(changed: True)')

-        if instance_not_found_err:
-            print(instance_not_found_err.message)
+
+class Command(BaseCommand):
+
+    def add_arguments(self, parser):
+        parser.add_argument('--queuename', dest='queuename', type=str,
+                            help='Queue to create/update')
+        parser.add_argument('--hostnames', dest='hostnames', type=str,
+                            help='Comma-Delimited Hosts to add to the Queue (will not remove already assigned instances)')
+        parser.add_argument('--controller', dest='controller', type=str,
+                            default='', help='The controlling group (makes this an isolated group)')
+        parser.add_argument('--instance_percent', dest='instance_percent', type=int, default=0,
+                            help='The percentage of active instances that will be assigned to this group'),
+        parser.add_argument('--instance_minimum', dest='instance_minimum', type=int, default=0,
+                            help='The minimum number of instance that will be retained for this group from available instances')
+
+
+    def handle(self, **options):
+        queuename = options.get('queuename')
+        if not queuename:
+            raise CommandError("Specify `--queuename` to use this command.")
+        ctrl = options.get('controller')
+        inst_per = options.get('instance_percent')
+        instance_min = options.get('instance_minimum')
+        hostname_list = []
+        if options.get('hostnames'):
+            hostname_list = options.get('hostnames').split(",")
+
+        rq = RegisterQueue(queuename, ctrl, inst_per, instance_min, hostname_list)
+        rq.register()
+        if rq.instance_not_found_err:
+            print(rq.instance_not_found_err.message)
            sys.exit(1)
--- a/awx/main/management/commands/remove_from_queue.py
+++ b/awx/main/management/commands/remove_from_queue.py
@@ -32,4 +32,7 @@ class Command(BaseCommand):
            sys.exit(1)
        i = i.first()
        ig.instances.remove(i)
+        if i.hostname in ig.policy_instance_list:
+            ig.policy_instance_list.remove(i.hostname)
+            ig.save()
        print("Instance removed from instance group")
--- a/awx/main/management/commands/run_callback_receiver.py
+++ b/awx/main/management/commands/run_callback_receiver.py
@@ -4,6 +4,7 @@
 from django.conf import settings
 from django.core.management.base import BaseCommand

+from awx.main.dispatch.control import Control
 from awx.main.dispatch.worker import AWXConsumerRedis, CallbackBrokerWorker


@@ -15,7 +16,14 @@ class Command(BaseCommand):
    '''
    help = 'Launch the job callback receiver'

+    def add_arguments(self, parser):
+        parser.add_argument('--status', dest='status', action='store_true',
+                            help='print the internal state of any running dispatchers')
+
    def handle(self, *arg, **options):
+        if options.get('status'):
+            print(Control('callback_receiver').status())
+            return
        consumer = None
        try:
            consumer = AWXConsumerRedis(
--- a/awx/main/management/commands/run_dispatcher.py
+++ b/awx/main/management/commands/run_dispatcher.py
@@ -44,7 +44,7 @@ class Command(BaseCommand):

        # It's important to close these because we're _about_ to fork, and we
        # don't want the forked processes to inherit the open sockets
-        # for the DB and memcached connections (that way lies race conditions)
+        # for the DB and cache connections (that way lies race conditions)
        django_connection.close()
        django_cache.close()

--- a/awx/main/management/commands/run_wsbroadcast.py
+++ b/awx/main/management/commands/run_wsbroadcast.py
@@ -5,17 +5,19 @@ import asyncio
 import datetime
 import re
 import redis
+import time
 from datetime import datetime as dt

 from django.core.management.base import BaseCommand
+from django.db import connection
 from django.db.models import Q
+from django.db.migrations.executor import MigrationExecutor

 from awx.main.analytics.broadcast_websocket import (
    BroadcastWebsocketStatsManager,
    safe_name,
 )
 from awx.main.wsbroadcast import BroadcastWebsocketManager
-from awx.main.models.ha import Instance


 logger = logging.getLogger('awx.main.wsbroadcast')
@@ -91,6 +93,36 @@ class Command(BaseCommand):
        return host_stats

    def handle(self, *arg, **options):
+        # it's necessary to delay this import in case
+        # database migrations are still running
+        from awx.main.models.ha import Instance
+
+        executor = MigrationExecutor(connection)
+        migrating = bool(executor.migration_plan(executor.loader.graph.leaf_nodes()))
+        registered = False
+
+        if not migrating:
+            try:
+                Instance.objects.me()
+                registered = True
+            except RuntimeError:
+                pass
+
+        if migrating or not registered:
+            # In containerized deployments, migrations happen in the task container,
+            # and the services running there don't start until migrations are
+            # finished.
+            # *This* service runs in the web container, and it's possible that it can
+            # start _before_ migrations are finished, thus causing issues with the ORM
+            # queries it makes (specifically, conf.settings queries).
+            # This block is meant to serve as a sort of bail-out for the situation
+            # where migrations aren't yet finished (similar to the migration
+            # detection middleware that the uwsgi processes have) or when instance
+            # registration isn't done yet
+            logger.error('AWX is currently installing/upgrading.  Trying again in 5s...')
+            time.sleep(5)
+            return
+
        if options.get('status'):
            try:
                stats_all = BroadcastWebsocketStatsManager.get_stats_sync()
@@ -107,6 +139,7 @@ class Command(BaseCommand):
                            break
                else:
                    data[family.name] = family.samples[0].value
+
            me = Instance.objects.me()
            hostnames = [i.hostname for i in Instance.objects.exclude(Q(hostname=me.hostname) | Q(rampart_groups__controller__isnull=False))]

--- a/awx/main/managers.py
+++ b/awx/main/managers.py
@@ -44,25 +44,17 @@ class HostManager(models.Manager):
            inventory_sources__source='tower'
        ).filter(inventory__organization=org_id).values('name').distinct().count()

-    def active_counts_by_org(self):
-        """Return the counts of active, unique hosts for each organization.
-        Construction of query involves:
-         - remove any ordering specified in model's Meta
-         - Exclude hosts sourced from another Tower
-         - Consider only hosts where the canonical inventory is owned by each organization
-         - Restrict the query to only count distinct names
-         - Return the counts
-        """
-        return self.order_by().exclude(
-            inventory_sources__source='tower'
-        ).values('inventory__organization').annotate(
-            inventory__organization__count=models.Count('name', distinct=True))
-
    def get_queryset(self):
        """When the parent instance of the host query set has a `kind=smart` and a `host_filter`
        set. Use the `host_filter` to generate the queryset for the hosts.
        """
-        qs = super(HostManager, self).get_queryset()
+        qs = super(HostManager, self).get_queryset().defer(
+            'last_job__extra_vars',
+            'last_job_host_summary__job__extra_vars',
+            'last_job__artifacts',
+            'last_job_host_summary__job__artifacts',
+        )
+
        if (hasattr(self, 'instance') and
           hasattr(self.instance, 'host_filter') and
           hasattr(self.instance, 'kind')):
@@ -121,6 +113,17 @@ class InstanceManager(models.Manager):
        if not hostname:
            hostname = settings.CLUSTER_HOST_ID
        with advisory_lock('instance_registration_%s' % hostname):
+            if settings.AWX_AUTO_DEPROVISION_INSTANCES:
+                # detect any instances with the same IP address.
+                # if one exists, set it to None
+                inst_conflicting_ip = self.filter(ip_address=ip_address).exclude(hostname=hostname)
+                if inst_conflicting_ip.exists():
+                    for other_inst in inst_conflicting_ip:
+                        other_hostname = other_inst.hostname
+                        other_inst.ip_address = None
+                        other_inst.save(update_fields=['ip_address'])
+                        logger.warning("IP address {0} conflict detected, ip address unset for host {1}.".format(ip_address, other_hostname))
+
            instance = self.filter(hostname=hostname)
            if instance.exists():
                instance = instance.get()
@@ -138,8 +141,11 @@ class InstanceManager(models.Manager):

    def get_or_register(self):
        if settings.AWX_AUTO_DEPROVISION_INSTANCES:
+            from awx.main.management.commands.register_queue import RegisterQueue
            pod_ip = os.environ.get('MY_POD_IP')
-            return self.register(ip_address=pod_ip)
+            registered = self.register(ip_address=pod_ip)
+            RegisterQueue('tower', None, 100, 0, []).register()
+            return registered
        else:
            return (False, self.me())

--- a/awx/main/middleware.py
+++ b/awx/main/middleware.py
@@ -1,20 +1,16 @@
 # Copyright (c) 2015 Ansible, Inc.
 # All Rights Reserved.

-import uuid
 import logging
 import threading
 import time
-import cProfile
-import pstats
-import os
 import urllib.parse

 from django.conf import settings
 from django.contrib.auth.models import User
 from django.db.migrations.executor import MigrationExecutor
 from django.db import connection
-from django.shortcuts import get_object_or_404, redirect
+from django.shortcuts import redirect
 from django.apps import apps
 from django.utils.deprecation import MiddlewareMixin
 from django.utils.translation import ugettext_lazy as _
@@ -22,6 +18,7 @@ from django.urls import reverse, resolve

 from awx.main.utils.named_url_graph import generate_graph, GraphNode
 from awx.conf import fields, register
+from awx.main.utils.profiling import AWXProfiler


 logger = logging.getLogger('awx.main.middleware')
@@ -32,11 +29,14 @@ class TimingMiddleware(threading.local, MiddlewareMixin):

    dest = '/var/log/tower/profile'

+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.prof = AWXProfiler("TimingMiddleware")
+
    def process_request(self, request):
        self.start_time = time.time()
        if settings.AWX_REQUEST_PROFILE:
-            self.prof = cProfile.Profile()
-            self.prof.enable()
+            self.prof.start()

    def process_response(self, request, response):
        if not hasattr(self, 'start_time'):  # some tools may not invoke process_request
@@ -44,33 +44,10 @@ class TimingMiddleware(threading.local, MiddlewareMixin):
        total_time = time.time() - self.start_time
        response['X-API-Total-Time'] = '%0.3fs' % total_time
        if settings.AWX_REQUEST_PROFILE:
-            self.prof.disable()
-            cprofile_file = self.save_profile_file(request)
-            response['cprofile_file'] = cprofile_file
+            response['X-API-Profile-File'] = self.prof.stop()
        perf_logger.info('api response times', extra=dict(python_objects=dict(request=request, response=response)))
        return response

-    def save_profile_file(self, request):
-        if not os.path.isdir(self.dest):
-            os.makedirs(self.dest)
-        filename = '%.3fs-%s.pstats' % (pstats.Stats(self.prof).total_tt, uuid.uuid4())
-        filepath = os.path.join(self.dest, filename)
-        with open(filepath, 'w') as f:
-            f.write('%s %s\n' % (request.method, request.get_full_path()))
-            pstats.Stats(self.prof, stream=f).sort_stats('cumulative').print_stats()
-
-        if settings.AWX_REQUEST_PROFILE_WITH_DOT:
-            from gprof2dot import main as generate_dot
-            raw = os.path.join(self.dest, filename) + '.raw'
-            pstats.Stats(self.prof).dump_stats(raw)
-            generate_dot([
-                '-n', '2.5', '-f', 'pstats', '-o',
-                os.path.join( self.dest, filename).replace('.pstats', '.dot'),
-                raw
-            ])
-            os.remove(raw)
-        return filepath
-

 class SessionTimeoutMiddleware(MiddlewareMixin):
    """
@@ -148,7 +125,21 @@ class URLModificationMiddleware(MiddlewareMixin):
    def _named_url_to_pk(cls, node, resource, named_url):
        kwargs = {}
        if node.populate_named_url_query_kwargs(kwargs, named_url):
-            return str(get_object_or_404(node.model, **kwargs).pk)
+            match = node.model.objects.filter(**kwargs).first()
+            if match:
+                return str(match.pk)
+            else:
+                # if the name does *not* resolve to any actual resource,
+                # we should still attempt to route it through so that 401s are
+                # respected
+                # using "zero" here will cause the URL regex to match e.g.,
+                # /api/v2/users/<integer>/, but it also means that anonymous
+                # users will go down the path of having their credentials
+                # verified; in this way, *anonymous* users will that visit
+                # /api/v2/users/invalid-username/ *won't* see a 404, they'll
+                # see a 401 as if they'd gone to /api/v2/users/0/
+                #
+                return '0'
        if resource == 'job_templates' and '++' not in named_url:
            # special case for deprecated job template case
            # will not raise a 404 on its own
@@ -178,6 +169,7 @@ class URLModificationMiddleware(MiddlewareMixin):
            old_path = request.path_info
        new_path = self._convert_named_url(old_path)
        if request.path_info != new_path:
+            request.environ['awx.named_url_rewritten'] = request.path
            request.path = request.path.replace(request.path_info, new_path)
            request.path_info = new_path

@@ -189,4 +181,4 @@ class MigrationRanCheckMiddleware(MiddlewareMixin):
        plan = executor.migration_plan(executor.loader.graph.leaf_nodes())
        if bool(plan) and \
                getattr(resolve(request.path), 'url_name', '') != 'migrations_notran':
-            return redirect(reverse("ui:migrations_notran"))
+            return redirect(reverse("ui_next:migrations_notran"))
--- a/awx/main/migrations/0115_v370_schedule_set_null.py
+++ b/awx/main/migrations/0115_v370_schedule_set_null.py
@@ -0,0 +1,24 @@
+# Generated by Django 2.2.11 on 2020-05-04 02:26
+
+import awx.main.utils.polymorphic
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0114_v370_remove_deprecated_manual_inventory_sources'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='unifiedjob',
+            name='schedule',
+            field=models.ForeignKey(default=None, editable=False, null=True, on_delete=awx.main.utils.polymorphic.SET_NULL, to='main.Schedule'),
+        ),
+        migrations.AlterField(
+            model_name='unifiedjobtemplate',
+            name='next_schedule',
+            field=models.ForeignKey(default=None, editable=False, null=True, on_delete=awx.main.utils.polymorphic.SET_NULL, related_name='unifiedjobtemplate_as_next_schedule+', to='main.Schedule'),
+        ),
+    ]
--- a/awx/main/migrations/0116_v400_remove_hipchat_notifications.py
+++ b/awx/main/migrations/0116_v400_remove_hipchat_notifications.py
@@ -0,0 +1,34 @@
+# Generated by Django 2.2.11 on 2020-05-19 02:27
+
+from django.db import migrations, models
+
+
+def remove_hipchat_notifications(apps, schema_editor):
+    '''
+    HipChat notifications are no longer in service, remove any that are found.
+    '''
+    Notification = apps.get_model('main', 'Notification')
+    Notification.objects.filter(notification_type='hipchat').delete()
+    NotificationTemplate = apps.get_model('main', 'NotificationTemplate')
+    NotificationTemplate.objects.filter(notification_type='hipchat').delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0115_v370_schedule_set_null'),
+    ]
+
+    operations = [
+        migrations.RunPython(remove_hipchat_notifications),
+        migrations.AlterField(
+            model_name='notification',
+            name='notification_type',
+            field=models.CharField(choices=[('email', 'Email'), ('grafana', 'Grafana'), ('irc', 'IRC'), ('mattermost', 'Mattermost'), ('pagerduty', 'Pagerduty'), ('rocketchat', 'Rocket.Chat'), ('slack', 'Slack'), ('twilio', 'Twilio'), ('webhook', 'Webhook')], max_length=32),
+        ),
+        migrations.AlterField(
+            model_name='notificationtemplate',
+            name='notification_type',
+            field=models.CharField(choices=[('email', 'Email'), ('grafana', 'Grafana'), ('irc', 'IRC'), ('mattermost', 'Mattermost'), ('pagerduty', 'Pagerduty'), ('rocketchat', 'Rocket.Chat'), ('slack', 'Slack'), ('twilio', 'Twilio'), ('webhook', 'Webhook')], max_length=32),
+        ),
+    ]
--- a/awx/main/migrations/0117_v400_remove_cloudforms_inventory.py
+++ b/awx/main/migrations/0117_v400_remove_cloudforms_inventory.py
@@ -0,0 +1,25 @@
+# Generated by Django 2.2.11 on 2020-05-01 13:25
+
+from django.db import migrations, models
+from awx.main.migrations._inventory_source import delete_cloudforms_inv_source
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0116_v400_remove_hipchat_notifications'),
+    ]
+
+    operations = [
+        migrations.RunPython(delete_cloudforms_inv_source),
+        migrations.AlterField(
+            model_name='inventorysource',
+            name='source',
+            field=models.CharField(choices=[('file', 'File, Directory or Script'), ('scm', 'Sourced from a Project'), ('ec2', 'Amazon EC2'), ('gce', 'Google Compute Engine'), ('azure_rm', 'Microsoft Azure Resource Manager'), ('vmware', 'VMware vCenter'), ('satellite6', 'Red Hat Satellite 6'), ('openstack', 'OpenStack'), ('rhv', 'Red Hat Virtualization'), ('tower', 'Ansible Tower'), ('custom', 'Custom Script')], default=None, max_length=32),
+        ),
+        migrations.AlterField(
+            model_name='inventoryupdate',
+            name='source',
+            field=models.CharField(choices=[('file', 'File, Directory or Script'), ('scm', 'Sourced from a Project'), ('ec2', 'Amazon EC2'), ('gce', 'Google Compute Engine'), ('azure_rm', 'Microsoft Azure Resource Manager'), ('vmware', 'VMware vCenter'), ('satellite6', 'Red Hat Satellite 6'), ('openstack', 'OpenStack'), ('rhv', 'Red Hat Virtualization'), ('tower', 'Ansible Tower'), ('custom', 'Custom Script')], default=None, max_length=32),
+        ),
+    ]
--- a/awx/main/migrations/0118_add_remote_archive_scm_type.py
+++ b/awx/main/migrations/0118_add_remote_archive_scm_type.py
@@ -0,0 +1,23 @@
+# Generated by Django 2.2.11 on 2020-08-18 22:35
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0117_v400_remove_cloudforms_inventory'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='project',
+            name='scm_type',
+            field=models.CharField(blank=True, choices=[('', 'Manual'), ('git', 'Git'), ('hg', 'Mercurial'), ('svn', 'Subversion'), ('insights', 'Red Hat Insights'), ('archive', 'Remote Archive')], default='', help_text='Specifies the source control system used to store the project.', max_length=8, verbose_name='SCM Type'),
+        ),
+        migrations.AlterField(
+            model_name='projectupdate',
+            name='scm_type',
+            field=models.CharField(blank=True, choices=[('', 'Manual'), ('git', 'Git'), ('hg', 'Mercurial'), ('svn', 'Subversion'), ('insights', 'Red Hat Insights'), ('archive', 'Remote Archive')], default='', help_text='Specifies the source control system used to store the project.', max_length=8, verbose_name='SCM Type'),
+        ),
+    ]
--- a/awx/main/migrations/0119_inventory_plugins.py
+++ b/awx/main/migrations/0119_inventory_plugins.py
@@ -0,0 +1,104 @@
+# Generated by Django 2.2.11 on 2020-07-20 19:56
+
+import logging
+import yaml
+
+from django.db import migrations, models
+
+from awx.main.models.base import VarsDictProperty
+
+from ._inventory_source_vars import FrozenInjectors
+
+
+logger = logging.getLogger('awx.main.migrations')
+
+
+def _get_inventory_sources(InventorySource):
+    return InventorySource.objects.filter(source__in=['ec2', 'gce', 'azure_rm', 'vmware', 'satellite6', 'openstack', 'rhv', 'tower'])
+
+
+def inventory_source_vars_forward(apps, schema_editor):
+    InventorySource = apps.get_model("main", "InventorySource")
+    '''
+    The Django app registry does not keep track of model inheritance. The
+    source_vars_dict property comes from InventorySourceOptions via inheritance.
+    This adds that property. Luckily, other properteries and functionality from
+    InventorySourceOptions is not needed by the injector logic.
+    '''
+    setattr(InventorySource, 'source_vars_dict', VarsDictProperty('source_vars'))
+    source_vars_backup = dict()
+
+    for inv_source_obj in _get_inventory_sources(InventorySource):
+
+        if inv_source_obj.source in FrozenInjectors:
+            source_vars_backup[inv_source_obj.id] = dict(inv_source_obj.source_vars_dict)
+
+            injector = FrozenInjectors[inv_source_obj.source]()
+            new_inv_source_vars = injector.inventory_as_dict(inv_source_obj, None)
+            inv_source_obj.source_vars = yaml.dump(new_inv_source_vars)
+            inv_source_obj.save()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0118_add_remote_archive_scm_type'),
+    ]
+
+    operations = [
+        migrations.RunPython(inventory_source_vars_forward),
+        migrations.RemoveField(
+            model_name='inventorysource',
+            name='group_by',
+        ),
+        migrations.RemoveField(
+            model_name='inventoryupdate',
+            name='group_by',
+        ),
+        migrations.RemoveField(
+            model_name='inventorysource',
+            name='instance_filters',
+        ),
+        migrations.RemoveField(
+            model_name='inventoryupdate',
+            name='instance_filters',
+        ),
+        migrations.RemoveField(
+            model_name='inventorysource',
+            name='source_regions',
+        ),
+        migrations.RemoveField(
+            model_name='inventoryupdate',
+            name='source_regions',
+        ),
+        migrations.AddField(
+            model_name='inventorysource',
+            name='enabled_value',
+            field=models.TextField(blank=True, default='', help_text='Only used when enabled_var is set. Value when the host is considered enabled. For example if enabled_var="status.power_state"and enabled_value="powered_on" with host variables:{   "status": {     "power_state": "powered_on",     "created": "2020-08-04T18:13:04+00:00",     "healthy": true    },    "name": "foobar",    "ip_address": "192.168.2.1"}The host would be marked enabled. If power_state where any value other than powered_on then the host would be disabled when imported into Tower. If the key is not found then the host will be enabled'),
+        ),
+        migrations.AddField(
+            model_name='inventorysource',
+            name='enabled_var',
+            field=models.TextField(blank=True, default='', help_text='Retrieve the enabled state from the given dict of host variables. The enabled variable may be specified as "foo.bar", in which case the lookup will traverse into nested dicts, equivalent to: from_dict.get("foo", {}).get("bar", default)'),
+        ),
+        migrations.AddField(
+            model_name='inventorysource',
+            name='host_filter',
+            field=models.TextField(blank=True, default='', help_text='Regex where only matching hosts will be imported into Tower.'),
+        ),
+        migrations.AddField(
+            model_name='inventoryupdate',
+            name='enabled_value',
+            field=models.TextField(blank=True, default='', help_text='Only used when enabled_var is set. Value when the host is considered enabled. For example if enabled_var="status.power_state"and enabled_value="powered_on" with host variables:{   "status": {     "power_state": "powered_on",     "created": "2020-08-04T18:13:04+00:00",     "healthy": true    },    "name": "foobar",    "ip_address": "192.168.2.1"}The host would be marked enabled. If power_state where any value other than powered_on then the host would be disabled when imported into Tower. If the key is not found then the host will be enabled'),
+        ),
+        migrations.AddField(
+            model_name='inventoryupdate',
+            name='enabled_var',
+            field=models.TextField(blank=True, default='', help_text='Retrieve the enabled state from the given dict of host variables. The enabled variable may be specified as "foo.bar", in which case the lookup will traverse into nested dicts, equivalent to: from_dict.get("foo", {}).get("bar", default)'),
+        ),
+        migrations.AddField(
+            model_name='inventoryupdate',
+            name='host_filter',
+            field=models.TextField(blank=True, default='', help_text='Regex where only matching hosts will be imported into Tower.'),
+        ),
+    ]
--- a/awx/main/migrations/0120_galaxy_credentials.py
+++ b/awx/main/migrations/0120_galaxy_credentials.py
@@ -0,0 +1,51 @@
+# Generated by Django 2.2.11 on 2020-08-04 15:19
+
+import logging
+
+import awx.main.fields
+from awx.main.utils.encryption import encrypt_field, decrypt_field
+
+from django.db import migrations, models
+from django.utils.timezone import now
+import django.db.models.deletion
+
+from awx.main.migrations import _galaxy as galaxy
+from awx.main.models import CredentialType as ModernCredentialType
+from awx.main.utils.common import set_current_apps
+
+logger = logging.getLogger('awx.main.migrations')
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0119_inventory_plugins'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='credentialtype',
+            name='kind',
+            field=models.CharField(choices=[('ssh', 'Machine'), ('vault', 'Vault'), ('net', 'Network'), ('scm', 'Source Control'), ('cloud', 'Cloud'), ('token', 'Personal Access Token'), ('insights', 'Insights'), ('external', 'External'), ('kubernetes', 'Kubernetes'), ('galaxy', 'Galaxy/Automation Hub')], max_length=32),
+        ),
+        migrations.CreateModel(
+            name='OrganizationGalaxyCredentialMembership',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('position', models.PositiveIntegerField(db_index=True, default=None, null=True)),
+                ('credential', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='main.Credential')),
+                ('organization', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='main.Organization')),
+            ],
+        ),
+        migrations.AddField(
+            model_name='organization',
+            name='galaxy_credentials',
+            field=awx.main.fields.OrderedManyToManyField(blank=True, related_name='organization_galaxy_credentials', through='main.OrganizationGalaxyCredentialMembership', to='main.Credential'),
+        ),
+        migrations.AddField(
+            model_name='credential',
+            name='managed_by_tower',
+            field=models.BooleanField(default=False, editable=False),
+        ),
+        migrations.RunPython(galaxy.migrate_galaxy_settings)
+    ]
--- a/awx/main/migrations/0121_delete_toweranalyticsstate.py
+++ b/awx/main/migrations/0121_delete_toweranalyticsstate.py
@@ -0,0 +1,16 @@
+# Generated by Django 2.2.11 on 2020-07-24 17:41
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0120_galaxy_credentials'),
+    ]
+
+    operations = [
+        migrations.DeleteModel(
+            name='TowerAnalyticsState',
+        ),
+    ]
--- a/awx/main/migrations/0122_really_remove_cloudforms_inventory.py
+++ b/awx/main/migrations/0122_really_remove_cloudforms_inventory.py
@@ -0,0 +1,13 @@
+from django.db import migrations
+from awx.main.migrations._inventory_source import delete_cloudforms_inv_source
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0121_delete_toweranalyticsstate'),
+    ]
+
+    operations = [
+        migrations.RunPython(delete_cloudforms_inv_source),
+    ]
--- a/awx/main/migrations/0123_drop_hg_support.py
+++ b/awx/main/migrations/0123_drop_hg_support.py
@@ -0,0 +1,23 @@
+from django.db import migrations, models
+from awx.main.migrations._hg_removal import delete_hg_scm
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0122_really_remove_cloudforms_inventory'),
+    ]
+
+    operations = [
+        migrations.RunPython(delete_hg_scm),
+        migrations.AlterField(
+            model_name='project',
+            name='scm_type',
+            field=models.CharField(blank=True, choices=[('', 'Manual'), ('git', 'Git'), ('svn', 'Subversion'), ('insights', 'Red Hat Insights'), ('archive', 'Remote Archive')], default='', help_text='Specifies the source control system used to store the project.', max_length=8, verbose_name='SCM Type'),
+        ),
+        migrations.AlterField(
+            model_name='projectupdate',
+            name='scm_type',
+            field=models.CharField(blank=True, choices=[('', 'Manual'), ('git', 'Git'), ('svn', 'Subversion'), ('insights', 'Red Hat Insights'), ('archive', 'Remote Archive')], default='', help_text='Specifies the source control system used to store the project.', max_length=8, verbose_name='SCM Type'),
+        ),
+    ]
--- a/awx/main/migrations/_galaxy.py
+++ b/awx/main/migrations/_galaxy.py
@@ -0,0 +1,125 @@
+# Generated by Django 2.2.11 on 2020-08-04 15:19
+
+import logging
+
+from awx.main.utils.encryption import encrypt_field, decrypt_field
+
+from django.conf import settings
+from django.utils.timezone import now
+
+from awx.main.models import CredentialType as ModernCredentialType
+from awx.main.utils.common import set_current_apps
+
+logger = logging.getLogger('awx.main.migrations')
+
+
+def migrate_galaxy_settings(apps, schema_editor):
+    Organization = apps.get_model('main', 'Organization')
+    if Organization.objects.count() == 0:
+        # nothing to migrate
+        return
+    set_current_apps(apps)
+    ModernCredentialType.setup_tower_managed_defaults()
+    CredentialType = apps.get_model('main', 'CredentialType')
+    Credential = apps.get_model('main', 'Credential')
+    Setting = apps.get_model('conf', 'Setting')
+
+    galaxy_type = CredentialType.objects.get(kind='galaxy')
+    private_galaxy_url = Setting.objects.filter(key='PRIMARY_GALAXY_URL').first()
+
+    # by default, prior versions of AWX/Tower automatically pulled content
+    # from galaxy.ansible.com
+    public_galaxy_enabled = True
+    public_galaxy_setting = Setting.objects.filter(key='PUBLIC_GALAXY_ENABLED').first()
+    if public_galaxy_setting and public_galaxy_setting.value is False:
+        # ...UNLESS this behavior was explicitly disabled via this setting
+        public_galaxy_enabled = False
+
+    public_galaxy_credential = Credential(
+        created=now(),
+        modified=now(),
+        name='Ansible Galaxy',
+        managed_by_tower=True,
+        credential_type=galaxy_type,
+        inputs = {
+            'url': 'https://galaxy.ansible.com/'
+        }
+    )
+    public_galaxy_credential.save()
+
+    for org in Organization.objects.all():
+        if private_galaxy_url and private_galaxy_url.value:
+            # If a setting exists for a private Galaxy URL, make a credential for it
+            username = Setting.objects.filter(key='PRIMARY_GALAXY_USERNAME').first()
+            password = Setting.objects.filter(key='PRIMARY_GALAXY_PASSWORD').first()
+            if (username and username.value) or (password and password.value):
+                logger.error(
+                    f'Specifying HTTP basic auth for the Ansible Galaxy API '
+                    f'({private_galaxy_url.value}) is no longer supported. '
+                    'Please provide an API token instead after your upgrade '
+                    'has completed',
+                )
+            inputs = {
+                'url': private_galaxy_url.value
+            }
+            token = Setting.objects.filter(key='PRIMARY_GALAXY_TOKEN').first()
+            if token and token.value:
+                inputs['token'] = decrypt_field(token, 'value')
+            auth_url = Setting.objects.filter(key='PRIMARY_GALAXY_AUTH_URL').first()
+            if auth_url and auth_url.value:
+                inputs['auth_url'] = auth_url.value
+            name = f'Private Galaxy ({private_galaxy_url.value})'
+            if 'cloud.redhat.com' in inputs['url']:
+                name = f'Ansible Automation Hub ({private_galaxy_url.value})'
+            cred = Credential(
+                created=now(),
+                modified=now(),
+                name=name,
+                organization=org,
+                credential_type=galaxy_type,
+                inputs=inputs
+            )
+            cred.save()
+            if token and token.value:
+                # encrypt based on the primary key from the prior save
+                cred.inputs['token'] = encrypt_field(cred, 'token')
+                cred.save()
+            org.galaxy_credentials.add(cred)
+
+        fallback_servers = getattr(settings, 'FALLBACK_GALAXY_SERVERS', [])
+        for fallback in fallback_servers:
+            url = fallback.get('url', None)
+            auth_url = fallback.get('auth_url', None)
+            username = fallback.get('username', None)
+            password = fallback.get('password', None)
+            token = fallback.get('token', None)
+            if username or password:
+                logger.error(
+                    f'Specifying HTTP basic auth for the Ansible Galaxy API '
+                    f'({url}) is no longer supported. '
+                    'Please provide an API token instead after your upgrade '
+                    'has completed',
+                )
+            inputs = {'url': url}
+            if token:
+                inputs['token'] = token
+            if auth_url:
+                inputs['auth_url'] = auth_url
+            cred = Credential(
+                created=now(),
+                modified=now(),
+                name=f'Ansible Galaxy ({url})',
+                organization=org,
+                credential_type=galaxy_type,
+                inputs=inputs
+            )
+            cred.save()
+            if token:
+                # encrypt based on the primary key from the prior save
+                cred.inputs['token'] = encrypt_field(cred, 'token')
+                cred.save()
+            org.galaxy_credentials.add(cred)
+
+        if public_galaxy_enabled:
+            # If public Galaxy was enabled, associate it to the org
+            org.galaxy_credentials.add(public_galaxy_credential)
--- a/awx/main/migrations/_hg_removal.py
+++ b/awx/main/migrations/_hg_removal.py
@@ -0,0 +1,19 @@
+import logging
+
+from awx.main.utils.common import set_current_apps
+
+logger = logging.getLogger('awx.main.migrations')
+
+
+def delete_hg_scm(apps, schema_editor):
+    set_current_apps(apps)
+    Project = apps.get_model('main', 'Project')
+    ProjectUpdate = apps.get_model('main', 'ProjectUpdate')
+
+    ProjectUpdate.objects.filter(project__scm_type='hg').update(scm_type='')
+    update_ct = Project.objects.filter(scm_type='hg').update(scm_type='')
+
+    if update_ct:
+        logger.warn('Changed {} mercurial projects to manual, deprecation period ended'.format(
+            update_ct
+        ))
--- a/awx/main/migrations/_inventory_source.py
+++ b/awx/main/migrations/_inventory_source.py
@@ -1,7 +1,11 @@
 import logging

-from django.utils.encoding import smart_text
+from uuid import uuid4

+from django.utils.encoding import smart_text
+from django.utils.timezone import now
+
+from awx.main.utils.common import set_current_apps
 from awx.main.utils.common import parse_yaml_or_json

 logger = logging.getLogger('awx.main.migrations')
@@ -87,3 +91,15 @@ def back_out_new_instance_id(apps, source, new_id):
            modified_ct, source
        ))

+
+def delete_cloudforms_inv_source(apps, schema_editor):
+    set_current_apps(apps)
+    InventorySource = apps.get_model('main', 'InventorySource')
+    InventoryUpdate = apps.get_model('main', 'InventoryUpdate')
+    CredentialType = apps.get_model('main', 'CredentialType')
+    InventoryUpdate.objects.filter(inventory_source__source='cloudforms').delete()
+    InventorySource.objects.filter(source='cloudforms').delete()
+    ct = CredentialType.objects.filter(namespace='cloudforms').first()
+    if ct:
+        ct.credentials.all().delete()
+        ct.delete()
--- a/Show More
+++ b/Show More
@@ -1 +1 @@
 .0.0
 .0.1