add command to introspect job performance

This command outputs json with information about past job event processing delays, how long jobs spent in pending, and what times of day saw more pending jobs. To aid in investigations, the raw queries are supplied alongside data.
Re-do PR #14685 for alt-text inventories. (#15394 )
2026-02-08 21:14:47 -03:30 · 2024-07-31 13:18:31 -04:00 · 2024-07-31 09:40:28 +01:00 · 2024-07-29 14:36:22 -06:00 · 2024-07-29 15:13:09 -04:00 · 2024-07-26 09:11:12 -04:00
120 changed files with 1540 additions and 341 deletions
--- a/.github/actions/awx_devel_image/action.yml
+++ b/.github/actions/awx_devel_image/action.yml
@@ -24,7 +24,7 @@ runs:

    - name: Pre-pull latest devel image to warm cache
      shell: bash
-      run: docker pull ghcr.io/${OWNER_LC}/awx_devel:${{ github.base_ref }}
+      run: docker pull -q ghcr.io/${OWNER_LC}/awx_devel:${{ github.base_ref }}

    - name: Build image for current source checkout
      shell: bash
--- a/.github/actions/run_awx_devel/action.yml
+++ b/.github/actions/run_awx_devel/action.yml
@@ -57,16 +57,6 @@ runs:
          awx-manage update_password --username=admin --password=password
        EOSH

-    - name: Build UI
-      # This must be a string comparison in composite actions:
-      # https://github.com/actions/runner/issues/2238
-      if: ${{ inputs.build-ui == 'true' }}
-      shell: bash
-      run: |
-        docker exec -i tools_awx_1 sh <<-EOSH
-          make ui-devel
-        EOSH
-
    - name: Get instance data
      id: data
      shell: bash
--- a/.github/triage_replies.md
+++ b/.github/triage_replies.md
@@ -1,7 +1,7 @@
 ## General
 - For the roundup of all the different mailing lists available from AWX, Ansible, and beyond visit: https://docs.ansible.com/ansible/latest/community/communication.html
 - Hello, we think your question is answered in our FAQ. Does this: https://www.ansible.com/products/awx-project/faq cover your question?
- You can find the latest documentation here: https://docs.ansible.com/automation-controller/latest/html/userguide/index.html
+- You can find the latest documentation here: https://ansible.readthedocs.io/projects/awx/en/latest/userguide/index.html



--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -38,7 +38,9 @@ jobs:
          - name: ui-test-general
            command: make ui-test-general
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      - name: Build awx_devel image for running checks
        uses: ./.github/actions/awx_devel_image
@@ -52,7 +54,9 @@ jobs:
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      - uses: ./.github/actions/run_awx_devel
        id: awx
@@ -70,13 +74,15 @@ jobs:
      DEBUG_OUTPUT_DIR: /tmp/awx_operator_molecule_test
    steps:
      - name: Checkout awx
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
        with:
+          show-progress: false
          path: awx

      - name: Checkout awx-operator
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
        with:
+          show-progress: false\
          repository: ansible/awx-operator
          path: awx-operator

@@ -130,7 +136,9 @@ jobs:
    strategy:
      fail-fast: false
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      # The containers that GitHub Actions use have Ansible installed, so upgrade to make sure we have the latest version.
      - name: Upgrade ansible-core
@@ -154,7 +162,9 @@ jobs:
          - name: r-z0-9
            regex: ^[r-z0-9]
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      - uses: ./.github/actions/run_awx_devel
        id: awx
@@ -200,7 +210,9 @@ jobs:
    strategy:
      fail-fast: false
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      - name: Upgrade ansible-core
        run: python3 -m pip install --upgrade ansible-core
--- a/.github/workflows/dab-release.yml
+++ b/.github/workflows/dab-release.yml
@@ -0,0 +1,57 @@
+---
+name: django-ansible-base requirements update
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: '0 6 * * *' # once an day @ 6 AM
+permissions:
+  pull-requests: write
+  contents: write
+jobs:
+  dab-pin-newest:
+    if: (github.repository_owner == 'ansible' && endsWith(github.repository, 'awx')) || github.event_name != 'schedule'
+    runs-on: ubuntu-latest
+    steps:
+      - id: dab-release
+        name: Get current django-ansible-base release version
+        uses: pozetroninc/github-action-get-latest-release@2a61c339ea7ef0a336d1daa35ef0cb1418e7676c # v0.8.0
+        with:
+          owner: ansible
+          repo: django-ansible-base
+          excludes: prerelease, draft
+
+      - name: Check out respository code
+        uses: actions/checkout@v4
+
+      - id: dab-pinned
+        name: Get current django-ansible-base pinned version
+        run:
+          echo "version=$(requirements/django-ansible-base-pinned-version.sh)" >> "$GITHUB_OUTPUT"
+
+      - name: Update django-ansible-base pinned version to upstream release
+        run:
+          requirements/django-ansible-base-pinned-version.sh -s ${{ steps.dab-release.outputs.release }}
+
+      - name: Create Pull Request
+        uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6
+        with:
+          base: devel
+          branch: bump-django-ansible-base
+          title: Bump django-ansible-base to ${{ steps.dab-release.outputs.release }}
+          body: |
+            ##### SUMMARY
+            Automated .github/workflows/dab-release.yml
+
+            django-ansible-base upstream released version == ${{ steps.dab-release.outputs.release }}
+            requirements_git.txt django-ansible-base pinned version ==  ${{ steps.dab-pinned.outputs.version }}
+
+            ##### ISSUE TYPE
+            - Bug, Docs Fix or other nominal change
+
+            ##### COMPONENT NAME
+            - API
+
+          commit-message: |
+            Update django-ansible-base version to ${{ steps.dab-pinned.outputs.version }}
+          add-paths:
+            requirements/requirements_git.txt
--- a/.github/workflows/devel_images.yml
+++ b/.github/workflows/devel_images.yml
@@ -2,6 +2,7 @@
 name: Build/Push Development Images
 env:
  LC_ALL: "C.UTF-8" # prevent ERROR: Ansible could not initialize the preferred locale: unsupported locale setting
+  DOCKER_CACHE: "--no-cache" # using the cache will not rebuild git requirements and other things
 on:
  workflow_dispatch:
  push:
@@ -34,7 +35,9 @@ jobs:
          exit 0
        if: matrix.build-targets.image-name == 'awx' && !endsWith(github.repository, '/awx')

-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
@@ -59,16 +62,14 @@ jobs:
        run: |
          echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin

-      - name: Setup node and npm
+      - name: Setup node and npm for the new UI build
        uses: actions/setup-node@v2
        with:
-          node-version: '16.13.1'
+          node-version: '18'
        if: matrix.build-targets.image-name == 'awx'

-      - name: Prebuild UI for awx image (to speed up build process)
+      - name: Prebuild new UI for awx image (to speed up build process)
        run: |
-          sudo apt-get install gettext
-          make ui-release
          make ui-next
        if: matrix.build-targets.image-name == 'awx'

--- a/.github/workflows/docs.yml
+++ b/.github/workflows/docs.yml
@@ -8,7 +8,9 @@ jobs:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      - name: install tox
        run: pip install tox
--- a/.github/workflows/label_issue.yml
+++ b/.github/workflows/label_issue.yml
@@ -30,7 +30,10 @@ jobs:
    timeout-minutes: 20
    name: Label Issue - Community
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false
+
      - uses: actions/setup-python@v4
      - name: Install python requests
        run: pip install requests
--- a/.github/workflows/label_pr.yml
+++ b/.github/workflows/label_pr.yml
@@ -29,7 +29,10 @@ jobs:
    timeout-minutes: 20
    name: Label PR - Community
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false
+
      - uses: actions/setup-python@v4
      - name: Install python requests
        run: pip install requests
--- a/.github/workflows/promote.yml
+++ b/.github/workflows/promote.yml
@@ -32,7 +32,9 @@ jobs:
          echo "TAG_NAME=${{ github.event.release.tag_name }}" >> $GITHUB_ENV

      - name: Checkout awx
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
+        with:
+          show-progress: false

      - name: Get python version from Makefile
        run: echo py_version=`make PYTHON_VERSION` >> $GITHUB_ENV
--- a/.github/workflows/stage.yml
+++ b/.github/workflows/stage.yml
@@ -45,19 +45,22 @@ jobs:
          exit 0

      - name: Checkout awx
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
        with:
+          show-progress: false
          path: awx

      - name: Checkout awx-operator
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
        with:
+          show-progress: false
          repository: ${{ github.repository_owner }}/awx-operator
          path: awx-operator

      - name: Checkout awx-logos
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
        with:
+          show-progress: false
          repository: ansible/awx-logos
          path: awx-logos

@@ -86,17 +89,14 @@ jobs:
        run: |
          cp ../awx-logos/awx/ui/client/assets/* awx/ui/public/static/media/

-      - name: Setup node and npm
+      - name: Setup node and npm for new UI build
        uses: actions/setup-node@v2
        with:
-          node-version: '16.13.1'
+          node-version: '18'

-      - name: Prebuild UI for awx image (to speed up build process)
+      - name: Prebuild new UI for awx image (to speed up build process)
        working-directory: awx
-        run: |
-          sudo apt-get install gettext
-          make ui-release
-          make ui-next
+        run: make ui-next

      - name: Set build env variables
        run: |
@@ -136,9 +136,9 @@ jobs:
      - name: Pulling images for test deployment with awx-operator
        # awx operator molecue test expect to kind load image and buildx exports image to registry and not local
        run: |
-          docker pull ${AWX_OPERATOR_TEST_IMAGE}
-          docker pull ${AWX_EE_TEST_IMAGE}
-          docker pull ${AWX_TEST_IMAGE}:${AWX_TEST_VERSION}
+          docker pull -q ${AWX_OPERATOR_TEST_IMAGE}
+          docker pull -q ${AWX_EE_TEST_IMAGE}
+          docker pull -q ${AWX_TEST_IMAGE}:${AWX_TEST_VERSION}

      - name: Run test deployment with awx-operator
        working-directory: awx-operator
--- a/.github/workflows/update_dependabot_prs.yml
+++ b/.github/workflows/update_dependabot_prs.yml
@@ -13,7 +13,9 @@ jobs:

    steps:
      - name: Checkout branch
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
+        with:
+          show-progress: false

      - name: Update PR Body
        env:
--- a/.github/workflows/upload_schema.yml
+++ b/.github/workflows/upload_schema.yml
@@ -18,7 +18,9 @@ jobs:
      packages: write
      contents: read
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+        with:
+          show-progress: false

      - name: Get python version from Makefile
        run: echo py_version=`make PYTHON_VERSION` >> $GITHUB_ENV
@@ -34,7 +36,7 @@ jobs:

      - name: Pre-pull image to warm build cache
        run: |
-          docker pull ghcr.io/${{ github.repository_owner }}/awx_devel:${GITHUB_REF##*/} || :
+          docker pull -q ghcr.io/${{ github.repository_owner }}/awx_devel:${GITHUB_REF##*/} || :

      - name: Build image
        run: |
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -67,7 +67,7 @@ If you're not using Docker for Mac, or Docker for Windows, you may need, or choo

 #### Frontend Development

-See [the ui development documentation](awx/ui/CONTRIBUTING.md).
+See [the ansible-ui development documentation](https://github.com/ansible/ansible-ui/blob/main/CONTRIBUTING.md).

 #### Fork and clone the AWX repo

@@ -121,7 +121,7 @@ If it has someone assigned to it then that person is the person responsible for

 **NOTES**

-> Issue assignment will only be done for maintainers of the project. If you decide to work on an issue, please feel free to add a comment in the issue to let others know that you are working on it; but know that we will accept the first pull request from whomever is able to fix an issue. Once your PR is accepted we can add you as an assignee to an issue upon request. 
+> Issue assignment will only be done for maintainers of the project. If you decide to work on an issue, please feel free to add a comment in the issue to let others know that you are working on it; but know that we will accept the first pull request from whomever is able to fix an issue. Once your PR is accepted we can add you as an assignee to an issue upon request.


 > If you work in a part of the codebase that is going through active development, your changes may be rejected, or you may be asked to `rebase`. A good idea before starting work is to have a discussion with us in the `#ansible-awx` channel on irc.libera.chat, or on the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
@@ -132,7 +132,7 @@ If it has someone assigned to it then that person is the person responsible for

 At this time we do not accept PRs for adding additional language translations as we have an automated process for generating our translations. This is because translations require constant care as new strings are added and changed in the code base. Because of this the .po files are overwritten during every translation release cycle. We also can't support a lot of translations on AWX as its an open source project and each language adds time and cost to maintain. If you would like to see AWX translated into a new language please create an issue and ask others you know to upvote the issue. Our translation team will review the needs of the community and see what they can do around supporting additional language.

-If you find an issue with an existing translation, please see the [Reporting Issues](#reporting-issues) section to open an issue and our translation team will work with you on a resolution. 
+If you find an issue with an existing translation, please see the [Reporting Issues](#reporting-issues) section to open an issue and our translation team will work with you on a resolution.


 ## Submitting Pull Requests
@@ -161,7 +161,7 @@ Sometimes it might take us a while to fully review your PR. We try to keep the `
 When your PR is initially submitted the checks will not be run until a maintainer allows them to be. Once a maintainer has done a quick review of your work the PR will have the linter and unit tests run against them via GitHub Actions, and the status reported in the PR.

 ## Reporting Issues
- 
+
 We welcome your feedback, and encourage you to file an issue when you run into a problem. But before opening a new issues, we ask that you please view our [Issues guide](./ISSUES.md).

 ## Getting Help
--- a/41
+++ b/41
@@ -63,6 +63,8 @@ DEV_DOCKER_OWNER ?= ansible
 DEV_DOCKER_OWNER_LOWER = $(shell echo $(DEV_DOCKER_OWNER) | tr A-Z a-z)
 DEV_DOCKER_TAG_BASE ?= ghcr.io/$(DEV_DOCKER_OWNER_LOWER)
 DEVEL_IMAGE_NAME ?= $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
+IMAGE_KUBE_DEV=$(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG)
+IMAGE_KUBE=$(DEV_DOCKER_TAG_BASE)/awx:$(COMPOSE_TAG)

 # Common command to use for running ansible-playbook
 ANSIBLE_PLAYBOOK ?= ansible-playbook -e ansible_python_interpreter=$(PYTHON)
@@ -89,6 +91,18 @@ I18N_FLAG_FILE = .i18n_built
 ## PLATFORMS defines the target platforms for  the manager image be build to provide support to multiple
 PLATFORMS ?= linux/amd64,linux/arm64  # linux/ppc64le,linux/s390x

+# Set up cache variables for image builds, allowing to control whether cache is used or not, ex:
+# DOCKER_CACHE=--no-cache make docker-compose-build
+ifeq ($(DOCKER_CACHE),)
+ DOCKER_DEVEL_CACHE_FLAG=--cache-from=$(DEVEL_IMAGE_NAME)
+ DOCKER_KUBE_DEV_CACHE_FLAG=--cache-from=$(IMAGE_KUBE_DEV)
+ DOCKER_KUBE_CACHE_FLAG=--cache-from=$(IMAGE_KUBE)
+else
+ DOCKER_DEVEL_CACHE_FLAG=$(DOCKER_CACHE)
+ DOCKER_KUBE_DEV_CACHE_FLAG=$(DOCKER_CACHE)
+ DOCKER_KUBE_CACHE_FLAG=$(DOCKER_CACHE)
+endif
+
 .PHONY: awx-link clean clean-tmp clean-venv requirements requirements_dev \
 	develop refresh adduser migrate dbchange \
 	receiver test test_unit test_coverage coverage_html \
@@ -488,13 +502,7 @@ ui-test-general:
 	$(NPM_BIN) run --prefix awx/ui pretest
 	$(NPM_BIN) run --prefix awx/ui/ test-general --runInBand

-# NOTE: The make target ui-next is imported from awx/ui_next/Makefile
-HEADLESS ?= no
-ifeq ($(HEADLESS), yes)
 dist/$(SDIST_TAR_FILE):
-else
-dist/$(SDIST_TAR_FILE): $(UI_BUILD_FLAG_FILE) ui-next
-endif
 	$(PYTHON) -m build -s
 	ln -sf $(SDIST_TAR_FILE) dist/awx.tar.gz

@@ -606,8 +614,7 @@ docker-compose-build: Dockerfile.dev
 		-f Dockerfile.dev \
 		-t $(DEVEL_IMAGE_NAME) \
 		--build-arg BUILDKIT_INLINE_CACHE=1 \
-		--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) .
-
+		$(DOCKER_DEVEL_CACHE_FLAG) .

 .PHONY: docker-compose-buildx
 ## Build awx_devel image for docker compose development environment for multiple architectures
@@ -617,7 +624,7 @@ docker-compose-buildx: Dockerfile.dev
 	- docker buildx build \
 		--push \
 		--build-arg BUILDKIT_INLINE_CACHE=1 \
-		--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) \
+		$(DOCKER_DEVEL_CACHE_FLAG) \
 		--platform=$(PLATFORMS) \
 		--tag $(DEVEL_IMAGE_NAME) \
 		-f Dockerfile.dev .
@@ -680,7 +687,8 @@ awx-kube-build: Dockerfile
 		--build-arg VERSION=$(VERSION) \
 		--build-arg SETUPTOOLS_SCM_PRETEND_VERSION=$(VERSION) \
 		--build-arg HEADLESS=$(HEADLESS) \
-		-t $(DEV_DOCKER_TAG_BASE)/awx:$(COMPOSE_TAG) .
+		$(DOCKER_KUBE_CACHE_FLAG) \
+		-t $(IMAGE_KUBE) .

 ## Build multi-arch awx image for deployment on Kubernetes environment.
 awx-kube-buildx: Dockerfile
@@ -692,7 +700,8 @@ awx-kube-buildx: Dockerfile
 		--build-arg SETUPTOOLS_SCM_PRETEND_VERSION=$(VERSION) \
 		--build-arg HEADLESS=$(HEADLESS) \
 		--platform=$(PLATFORMS) \
-		--tag $(DEV_DOCKER_TAG_BASE)/awx:$(COMPOSE_TAG) \
+		$(DOCKER_KUBE_CACHE_FLAG) \
+		--tag $(IMAGE_KUBE) \
 		-f Dockerfile .
 	- docker buildx rm awx-kube-buildx

@@ -710,8 +719,8 @@ Dockerfile.kube-dev: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
 awx-kube-dev-build: Dockerfile.kube-dev
 	DOCKER_BUILDKIT=1 docker build -f Dockerfile.kube-dev \
 	    --build-arg BUILDKIT_INLINE_CACHE=1 \
-	    --cache-from=$(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) \
-	    -t $(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) .
+	     $(DOCKER_KUBE_DEV_CACHE_FLAG) \
+	    -t $(IMAGE_KUBE_DEV) .

 ## Build and push multi-arch awx_kube_devel image for development on local Kubernetes environment.
 awx-kube-dev-buildx: Dockerfile.kube-dev
@@ -720,14 +729,14 @@ awx-kube-dev-buildx: Dockerfile.kube-dev
 	- docker buildx build \
 		--push \
 		--build-arg BUILDKIT_INLINE_CACHE=1 \
-		--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) \
+		$(DOCKER_KUBE_DEV_CACHE_FLAG) \
 		--platform=$(PLATFORMS) \
-		--tag $(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) \
+		--tag $(IMAGE_KUBE_DEV) \
 		-f Dockerfile.kube-dev .
 	- docker buildx rm awx-kube-dev-buildx

 kind-dev-load: awx-kube-dev-build
-	$(KIND_BIN) load docker-image $(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG)
+	$(KIND_BIN) load docker-image $(IMAGE_KUBE_DEV)

 # Translation TASKS
 # --------------------------------------
--- a/awx/api/generics.py
+++ b/awx/api/generics.py
@@ -227,7 +227,10 @@ class APIView(views.APIView):
            if type(response.data) is dict:
                msg_data['error'] = response.data.get('error', response.status_text)
            elif type(response.data) is list:
-                msg_data['error'] = ", ".join(list(map(lambda x: x.get('error', response.status_text), response.data)))
+                if len(response.data) > 0 and isinstance(response.data[0], str):
+                    msg_data['error'] = str(response.data[0])
+                else:
+                    msg_data['error'] = ", ".join(list(map(lambda x: x.get('error', response.status_text), response.data)))
            else:
                msg_data['error'] = response.status_text

--- a/awx/api/metadata.py
+++ b/awx/api/metadata.py
@@ -103,7 +103,7 @@ class Metadata(metadata.SimpleMetadata):
            default = field.get_default()
            if type(default) is UUID:
                default = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
-            if field.field_name == 'TOWER_URL_BASE' and default == 'https://towerhost':
+            if field.field_name == 'TOWER_URL_BASE' and default == 'https://platformhost':
                default = '{}://{}'.format(self.request.scheme, self.request.get_host())
            field_info['default'] = default
        except serializers.SkipField:
--- a/awx/api/templates/instance_install_bundle/install_receptor.yml
+++ b/awx/api/templates/instance_install_bundle/install_receptor.yml
@@ -2,6 +2,12 @@
 - hosts: all
  become: yes
  tasks:
+    - name: Create the receptor group
+      group:
+{% verbatim %}
+        name: "{{ receptor_group }}"
+{% endverbatim %}
+        state: present
    - name: Create the receptor user
      user:
 {% verbatim %}
--- a/awx/api/views/init.py
+++ b/awx/api/views/init.py
@@ -2392,6 +2392,14 @@ class JobTemplateList(ListCreateAPIView):
    serializer_class = serializers.JobTemplateSerializer
    always_allow_superuser = False

+    def check_permissions(self, request):
+        if request.method == 'POST':
+            can_access, messages = request.user.can_access_with_errors(self.model, 'add', request.data)
+            if not can_access:
+                self.permission_denied(request, message=messages)
+
+        super(JobTemplateList, self).check_permissions(request)
+

 class JobTemplateDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView):
    model = models.JobTemplate
@@ -3111,6 +3119,14 @@ class WorkflowJobTemplateList(ListCreateAPIView):
    serializer_class = serializers.WorkflowJobTemplateSerializer
    always_allow_superuser = False

+    def check_permissions(self, request):
+        if request.method == 'POST':
+            can_access, messages = request.user.can_access_with_errors(self.model, 'add', request.data)
+            if not can_access:
+                self.permission_denied(request, message=messages)
+
+        super(WorkflowJobTemplateList, self).check_permissions(request)
+

 class WorkflowJobTemplateDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView):
    model = models.WorkflowJobTemplate
--- a/awx/main/access.py
+++ b/awx/main/access.py
@@ -1387,12 +1387,11 @@ class TeamAccess(BaseAccess):
 class ExecutionEnvironmentAccess(BaseAccess):
    """
    I can see an execution environment when:
-     - I'm a superuser
-     - I'm a member of the same organization
-     - it is a global ExecutionEnvironment
+     - I can see its organization
+     - It is a global ExecutionEnvironment
    I can create/change an execution environment when:
     - I'm a superuser
-     - I'm an admin for the organization(s)
+     - I have an organization or object role that gives access
    """

    model = ExecutionEnvironment
@@ -1401,7 +1400,9 @@ class ExecutionEnvironmentAccess(BaseAccess):

    def filtered_queryset(self):
        return ExecutionEnvironment.objects.filter(
-            Q(organization__in=Organization.accessible_pk_qs(self.user, 'read_role')) | Q(organization__isnull=True)
+            Q(organization__in=Organization.accessible_pk_qs(self.user, 'read_role'))
+            | Q(organization__isnull=True)
+            | Q(id__in=ExecutionEnvironment.access_ids_qs(self.user, 'change'))
        ).distinct()

    @check_superuser
@@ -1416,15 +1417,17 @@ class ExecutionEnvironmentAccess(BaseAccess):
            raise PermissionDenied
        if settings.ANSIBLE_BASE_ROLE_SYSTEM_ACTIVATED:
            if not self.user.has_obj_perm(obj, 'change'):
-                raise PermissionDenied
+                return False
        else:
            if self.user not in obj.organization.execution_environment_admin_role:
                raise PermissionDenied
-        if data and 'organization' in data:
-            new_org = get_object_from_data('organization', Organization, data, obj=obj)
-            if not new_org or self.user not in new_org.execution_environment_admin_role:
+        if not self.check_related('organization', Organization, data, obj=obj, role_field='execution_environment_admin_role'):
+            return False
+        # Special case that check_related does not catch, org users can not remove the organization from the EE
+        if data and ('organization' in data or 'organization_id' in data):
+            if (not data.get('organization')) and (not data.get('organization_id')):
                return False
-        return self.check_related('organization', Organization, data, obj=obj, mandatory=True, role_field='execution_environment_admin_role')
+        return True

    def can_delete(self, obj):
        if obj.managed:
@@ -1596,6 +1599,8 @@ class JobTemplateAccess(NotificationAttachMixin, UnifiedCredentialsMixin, BaseAc
        inventory = get_value(Inventory, 'inventory')
        if inventory:
            if self.user not in inventory.use_role:
+                if self.save_messages:
+                    self.messages['inventory'] = [_('You do not have use permission on Inventory')]
                return False

        if not self.check_related('execution_environment', ExecutionEnvironment, data, role_field='read_role'):
@@ -1604,11 +1609,16 @@ class JobTemplateAccess(NotificationAttachMixin, UnifiedCredentialsMixin, BaseAc
        project = get_value(Project, 'project')
        # If the user has admin access to the project (as an org admin), should
        # be able to proceed without additional checks.
-        if project:
-            return self.user in project.use_role
-        else:
+        if not project:
            return False

+        if self.user not in project.use_role:
+            if self.save_messages:
+                self.messages['project'] = [_('You do not have use permission on Project')]
+            return False
+
+        return True
+
    @check_superuser
    def can_copy_related(self, obj):
        """
@@ -2092,11 +2102,23 @@ class WorkflowJobTemplateAccess(NotificationAttachMixin, BaseAccess):
        if not data:  # So the browseable API will work
            return Organization.accessible_objects(self.user, 'workflow_admin_role').exists()

-        return bool(
-            self.check_related('organization', Organization, data, role_field='workflow_admin_role', mandatory=True)
-            and self.check_related('inventory', Inventory, data, role_field='use_role')
-            and self.check_related('execution_environment', ExecutionEnvironment, data, role_field='read_role')
-        )
+        if not self.check_related('organization', Organization, data, role_field='workflow_admin_role', mandatory=True):
+            if data.get('organization', None) is None:
+                if self.save_messages:
+                    self.messages['organization'] = [_('An organization is required to create a workflow job template for normal user')]
+            return False
+
+        if not self.check_related('inventory', Inventory, data, role_field='use_role'):
+            if self.save_messages:
+                self.messages['inventory'] = [_('You do not have use_role to the inventory')]
+            return False
+
+        if not self.check_related('execution_environment', ExecutionEnvironment, data, role_field='read_role'):
+            if self.save_messages:
+                self.messages['execution_environment'] = [_('You do not have read_role to the execution environment')]
+            return False
+
+        return True

    def can_copy(self, obj):
        if self.save_messages:
--- a/awx/main/analytics/broadcast_websocket.py
+++ b/awx/main/analytics/broadcast_websocket.py
@@ -66,10 +66,8 @@ class FixedSlidingWindow:


 class RelayWebsocketStatsManager:
-    def __init__(self, event_loop, local_hostname):
+    def __init__(self, local_hostname):
        self._local_hostname = local_hostname
-
-        self._event_loop = event_loop
        self._stats = dict()
        self._redis_key = BROADCAST_WEBSOCKET_REDIS_KEY_NAME

@@ -94,7 +92,10 @@ class RelayWebsocketStatsManager:
            self.start()

    def start(self):
-        self.async_task = self._event_loop.create_task(self.run_loop())
+        self.async_task = asyncio.get_running_loop().create_task(
+            self.run_loop(),
+            name='RelayWebsocketStatsManager.run_loop',
+        )
        return self.async_task

    @classmethod
--- a/awx/main/conf.py
+++ b/awx/main/conf.py
@@ -929,6 +929,16 @@ register(
    category_slug='debug',
 )

+register(
+    'RECEPTOR_KEEP_WORK_ON_ERROR',
+    field_class=fields.BooleanField,
+    label=_('Keep receptor work on error'),
+    default=False,
+    help_text=_('Prevent receptor work from being released on when error is detected'),
+    category=('Debug'),
+    category_slug='debug',
+)
+

 def logging_validate(serializer, attrs):
    if not serializer.instance or not hasattr(serializer.instance, 'LOG_AGGREGATOR_HOST') or not hasattr(serializer.instance, 'LOG_AGGREGATOR_TYPE'):
--- a/awx/main/constants.py
+++ b/awx/main/constants.py
@@ -43,6 +43,7 @@ STANDARD_INVENTORY_UPDATE_ENV = {
 }
 CAN_CANCEL = ('new', 'pending', 'waiting', 'running')
 ACTIVE_STATES = CAN_CANCEL
+ERROR_STATES = ('error',)
 MINIMAL_EVENTS = set(['playbook_on_play_start', 'playbook_on_task_start', 'playbook_on_stats', 'EOF'])
 CENSOR_VALUE = '************'
 ENV_BLOCKLIST = frozenset(
--- a/awx/main/dispatch/init.py
+++ b/awx/main/dispatch/init.py
@@ -102,7 +102,8 @@ def create_listener_connection():

    # Apply overrides specifically for the listener connection
    for k, v in settings.LISTENER_DATABASES.get('default', {}).items():
-        conf[k] = v
+        if k != 'OPTIONS':
+            conf[k] = v
    for k, v in settings.LISTENER_DATABASES.get('default', {}).get('OPTIONS', {}).items():
        conf['OPTIONS'][k] = v

--- a/awx/main/management/commands/create_preload_data.py
+++ b/awx/main/management/commands/create_preload_data.py
@@ -2,6 +2,7 @@
 # All Rights Reserved

 from django.core.management.base import BaseCommand
+from django.db import transaction
 from crum import impersonate
 from awx.main.models import User, Organization, Project, Inventory, CredentialType, Credential, Host, JobTemplate
 from awx.main.signals import disable_computed_fields
@@ -13,6 +14,12 @@ class Command(BaseCommand):
    help = 'Creates a preload tower data if there is none.'

    def handle(self, *args, **kwargs):
+        # Wrap the operation in an atomic block, so we do not on accident
+        # create the organization but not create the project, etc.
+        with transaction.atomic():
+            self._handle()
+
+    def _handle(self):
        changed = False

        # Create a default organization as the first superuser found.
@@ -43,10 +50,11 @@ class Command(BaseCommand):

                    ssh_type = CredentialType.objects.filter(namespace='ssh').first()
                    c, _ = Credential.objects.get_or_create(
-                        credential_type=ssh_type, name='Demo Credential', inputs={'username': superuser.username}, created_by=superuser
+                        credential_type=ssh_type, name='Demo Credential', inputs={'username': getattr(superuser, 'username', 'null')}, created_by=superuser
                    )

-                    c.admin_role.members.add(superuser)
+                    if superuser:
+                        c.admin_role.members.add(superuser)

                    public_galaxy_credential, _ = Credential.objects.get_or_create(
                        name='Ansible Galaxy',
--- a/awx/main/management/commands/job_performance_rollup.py
+++ b/awx/main/management/commands/job_performance_rollup.py
@@ -0,0 +1,151 @@
+# Copyright (c) 2015 Ansible, Inc.
+# All Rights Reserved
+
+# Django
+from django.core.management.base import BaseCommand
+from django.db import connection
+
+import json
+import re
+
+
+class Command(BaseCommand):
+    """
+    Emits some simple statistics suitable for external monitoring
+    """
+
+    help = 'Run queries that provide an overview of the performance of the system over a given period of time'
+
+    def add_arguments(self, parser):
+        parser.add_argument('--since', action='store', dest='days', type=str, default="1", help='Max days to look back to for data')
+        parser.add_argument('--limit', action='store', dest='limit', type=str, default="10", help='Max number of records for database queries (LIMIT)')
+
+    def execute_query(self, query):
+        with connection.cursor() as cursor:
+            cursor.execute(query)
+            rows = cursor.fetchall()
+        return rows
+
+    def jsonify(self, title, keys, values, query):
+        result = []
+        query = re.sub('\n', ' ', query)
+        query = re.sub('\s{2,}', ' ', query)
+        for value in values:
+            result.append(dict(zip(keys, value)))
+        return {title: result, 'count': len(values), 'query': query}
+
+    def jobs_pending_duration(self, days, limit):
+        """Return list of jobs sorted by time in pending within configured number of days (within limit)"""
+        query = f"""
+            SELECT name, id AS job_id, unified_job_template_id, created, started - created AS pending_duration
+            FROM main_unifiedjob 
+            WHERE finished IS NOT null 
+            AND started IS NOT null 
+            AND cancel_flag IS NOT true 
+            AND created > NOW() - INTERVAL '{days} days' 
+            AND started - created > INTERVAL '0 seconds' 
+            ORDER BY pending_duration DESC
+            LIMIT {limit};"""
+        values = self.execute_query(query)
+        return self.jsonify(
+            title='completed_or_started_jobs_by_pending_duration',
+            keys=('job_name', 'job_id', 'unified_job_template_id', 'job_created', 'pending_duration'),
+            values=values,
+            query=query,
+        )
+
+    def times_of_day_pending_more_than_X_min(self, days, limit, minutes_pending):
+        """Return list of jobs sorted by time in pending within configured number of days (within limit)"""
+        query = f"""
+            SELECT
+                date_trunc('hour', created) as day_and_hour,
+                COUNT(created) as count_jobs_pending_greater_than_{minutes_pending}_min
+            FROM main_unifiedjob
+            WHERE started IS NOT NULL
+            AND started - created > INTERVAL '{minutes_pending} minutes'
+            AND created > NOW() - INTERVAL '{days} days' 
+            GROUP BY date_trunc('hour', created)
+            ORDER BY count_jobs_pending_greater_than_{minutes_pending}_min DESC
+            LIMIT {limit};"""
+        values = self.execute_query(query)
+        return self.jsonify(
+            title=f'times_of_day_pending_more_than_{minutes_pending}',
+            keys=('day_and_hour', f'count_jobs_pending_more_than_{minutes_pending}_min'),
+            values=values,
+            query=query,
+        )
+
+    def pending_jobs_details(self, days, limit):
+        """Return list of jobs that are in pending and list details such as reasons they may be blocked, within configured number of days and limit."""
+        query = f"""
+            SELECT DISTINCT ON(A.id) A.name, A.id, A.unified_job_template_id, A.created, NOW() - A.created as pending_duration, F.allow_simultaneous, B.current_job_id as current_ujt_job, I.to_unifiedjob_id as dependency_job_id, A.dependencies_processed 
+            FROM main_unifiedjob A
+            LEFT JOIN (
+                SELECT C.id, C.current_job_id FROM main_unifiedjobtemplate as C
+                ) B
+                ON A.unified_job_template_id = B.id
+                LEFT JOIN main_job F ON A.id = F.unifiedjob_ptr_id
+                LEFT JOIN (
+                    SELECT * FROM main_unifiedjob_dependent_jobs as G
+                    RIGHT JOIN main_unifiedjob H ON G.to_unifiedjob_id = H.id
+                    ) I 
+                ON A.id = I.from_unifiedjob_id
+            WHERE A.status = 'pending'
+            AND A.created > NOW() - INTERVAL '{days} days'
+            ORDER BY id DESC
+            LIMIT {limit};"""
+        values = self.execute_query(query)
+        return self.jsonify(
+            title='pending_jobs_details',
+            keys=(
+                'job_name',
+                'job_id',
+                'unified_job_template_id',
+                'job_created',
+                'pending_duration',
+                'allow_simultaneous',
+                'current_ujt_job',
+                'dependency_job_id',
+                'dependencies_processed',
+            ),
+            values=values,
+            query=query,
+        )
+
+    def jobs_by_FUNC_event_processing_time(self, func, days, limit):
+        """Return list of jobs sorted by MAX job event procesing time within configured number of days (within limit)"""
+        if func not in ('MAX', 'MIN', 'AVG', 'SUM'):
+            raise RuntimeError('Only able to asses job events grouped by job with MAX, MIN, AVG, SUM functions')
+
+        query = f"""SELECT job_id, {func}(A.modified - A.created) as job_event_processing_delay_{func}, B.name, B.created, B.finished, B.controller_node, B.execution_node
+            FROM main_jobevent A
+                RIGHT JOIN (
+                SELECT id, created, name, finished, controller_node, execution_node FROM
+                main_unifiedjob
+                WHERE created > NOW() - INTERVAL '{days} days'
+                AND created IS NOT null
+                AND finished IS NOT null
+                AND id IS NOT null
+                AND name IS NOT null
+                ) B
+                ON A.job_id=B.id
+            WHERE A.job_id is not null
+            GROUP BY job_id, B.name, B.created, B.finished, B.controller_node, B.execution_node
+            ORDER BY job_event_processing_delay_{func} DESC
+            LIMIT {limit};"""
+        values = self.execute_query(query)
+        return self.jsonify(
+            title=f'jobs_by_{func}_event_processing',
+            keys=('job_id', f'{func}_job_event_processing_delay', 'job_name', 'job_created_time', 'job_finished_time', 'controller_node', 'execution_node'),
+            values=values,
+            query=query,
+        )
+
+    def handle(self, *args, **options):
+        items = []
+        for func in ('MAX', 'MIN', 'AVG'):
+            items.append(self.jobs_by_FUNC_event_processing_time(func, options['days'], options['limit']))
+        items.append(self.jobs_pending_duration(options['days'], options['limit']))
+        items.append(self.pending_jobs_details(options['days'], options['limit']))
+        items.append(self.times_of_day_pending_more_than_X_min(options['days'], options['limit'], minutes_pending=10))
+        self.stdout.write(json.dumps(items, indent=4, sort_keys=True, default=str))
--- a/awx/main/migrations/0195_EE_permissions.py
+++ b/awx/main/migrations/0195_EE_permissions.py
@@ -0,0 +1,26 @@
+# Generated by Django 4.2.6 on 2024-06-20 15:55
+
+from django.db import migrations
+
+
+def delete_execution_environment_read_role(apps, schema_editor):
+    permission_classes = [apps.get_model('auth', 'Permission'), apps.get_model('dab_rbac', 'DABPermission')]
+    for permission_cls in permission_classes:
+        ee_read_perm = permission_cls.objects.filter(codename='view_executionenvironment').first()
+        if ee_read_perm:
+            ee_read_perm.delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('main', '0194_alter_inventorysource_source_and_more'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='executionenvironment',
+            options={'default_permissions': ('add', 'change', 'delete'), 'ordering': ('-created',)},
+        ),
+        migrations.RunPython(delete_execution_environment_read_role, migrations.RunPython.noop),
+    ]
--- a/awx/main/migrations/_dab_rbac.py
+++ b/awx/main/migrations/_dab_rbac.py
@@ -134,8 +134,7 @@ def get_permissions_for_role(role_field, children_map, apps):

    # more special cases for those same above special org-level roles
    if role_field.name == 'auditor_role':
-        for codename in ('view_notificationtemplate', 'view_executionenvironment'):
-            perm_list.append(Permission.objects.get(codename=codename))
+        perm_list.append(Permission.objects.get(codename='view_notificationtemplate'))

    return perm_list

@@ -292,12 +291,13 @@ def setup_managed_role_definitions(apps, schema_editor):
    org_perms = set()
    for cls in permission_registry.all_registered_models:
        ct = ContentType.objects.get_for_model(cls)
+        cls_name = cls._meta.model_name
        object_perms = set(Permission.objects.filter(content_type=ct))
        # Special case for InstanceGroup which has an organiation field, but is not an organization child object
-        if cls._meta.model_name != 'instancegroup':
+        if cls_name != 'instancegroup':
            org_perms.update(object_perms)

-        if 'object_admin' in to_create and cls != Organization:
+        if 'object_admin' in to_create and cls_name != 'organization':
            indiv_perms = object_perms.copy()
            add_perms = [perm for perm in indiv_perms if perm.codename.startswith('add_')]
            if add_perms:
@@ -310,7 +310,7 @@ def setup_managed_role_definitions(apps, schema_editor):
                )
            )

-        if 'org_children' in to_create and cls != Organization:
+        if 'org_children' in to_create and (cls_name not in ('organization', 'instancegroup', 'team')):
            org_child_perms = object_perms.copy()
            org_child_perms.add(Permission.objects.get(codename='view_organization'))

@@ -327,17 +327,25 @@ def setup_managed_role_definitions(apps, schema_editor):
        if 'special' in to_create:
            special_perms = []
            for perm in object_perms:
-                if perm.codename.split('_')[0] not in ('add', 'change', 'update', 'delete', 'view'):
+                # Organization auditor is handled separately
+                if perm.codename.split('_')[0] not in ('add', 'change', 'delete', 'view', 'audit'):
                    special_perms.append(perm)
            for perm in special_perms:
                action = perm.codename.split('_')[0]
                view_perm = Permission.objects.get(content_type=ct, codename__startswith='view_')
+                perm_list = [perm, view_perm]
+                # Handle special-case where adhoc role also listed use permission
+                if action == 'adhoc':
+                    for other_perm in object_perms:
+                        if other_perm.codename == 'use_inventory':
+                            perm_list.append(other_perm)
+                            break
                managed_role_definitions.append(
                    get_or_create_managed(
                        to_create['special'].format(cls=cls, action=action.title()),
                        f'Has {action} permissions to a single {cls._meta.verbose_name}',
                        ct,
-                        [perm, view_perm],
+                        perm_list,
                        RoleDefinition,
                    )
                )
@@ -353,6 +361,41 @@ def setup_managed_role_definitions(apps, schema_editor):
            )
        )

+    # Special "organization action" roles
+    audit_permissions = [perm for perm in org_perms if perm.codename.startswith('view_')]
+    audit_permissions.append(Permission.objects.get(codename='audit_organization'))
+    managed_role_definitions.append(
+        get_or_create_managed(
+            'Organization Audit',
+            'Has permission to view all objects inside of a single organization',
+            org_ct,
+            audit_permissions,
+            RoleDefinition,
+        )
+    )
+
+    org_execute_permissions = {'view_jobtemplate', 'execute_jobtemplate', 'view_workflowjobtemplate', 'execute_workflowjobtemplate', 'view_organization'}
+    managed_role_definitions.append(
+        get_or_create_managed(
+            'Organization Execute',
+            'Has permission to execute all runnable objects in the organization',
+            org_ct,
+            [perm for perm in org_perms if perm.codename in org_execute_permissions],
+            RoleDefinition,
+        )
+    )
+
+    org_approval_permissions = {'view_organization', 'view_workflowjobtemplate', 'approve_workflowjobtemplate'}
+    managed_role_definitions.append(
+        get_or_create_managed(
+            'Organization Approval',
+            'Has permission to approve any workflow steps within a single organization',
+            org_ct,
+            [perm for perm in org_perms if perm.codename in org_approval_permissions],
+            RoleDefinition,
+        )
+    )
+
    unexpected_role_definitions = RoleDefinition.objects.filter(managed=True).exclude(pk__in=[rd.pk for rd in managed_role_definitions])
    for role_definition in unexpected_role_definitions:
        logger.info(f'Deleting old managed role definition {role_definition.name}, pk={role_definition.pk}')
--- a/awx/main/models/init.py
+++ b/awx/main/models/init.py
@@ -176,17 +176,17 @@ pre_delete.connect(cleanup_created_modified_by, sender=User)

@property
 def user_get_organizations(user):
-    return Organization.objects.filter(member_role__members=user)
+    return Organization.access_qs(user, 'member')


@property
 def user_get_admin_of_organizations(user):
-    return Organization.objects.filter(admin_role__members=user)
+    return Organization.access_qs(user, 'change')


@property
 def user_get_auditor_of_organizations(user):
-    return Organization.objects.filter(auditor_role__members=user)
+    return Organization.access_qs(user, 'audit')


@property
--- a/awx/main/models/credential/init.py
+++ b/awx/main/models/credential/init.py
@@ -21,6 +21,10 @@ from django.conf import settings
 from django.utils.encoding import force_str
 from django.utils.functional import cached_property
 from django.utils.timezone import now
+from django.contrib.auth.models import User
+
+# DRF
+from rest_framework.serializers import ValidationError as DRFValidationError

 # AWX
 from awx.api.versioning import reverse
@@ -41,6 +45,7 @@ from awx.main.models.rbac import (
    ROLE_SINGLETON_SYSTEM_ADMINISTRATOR,
    ROLE_SINGLETON_SYSTEM_AUDITOR,
 )
+from awx.main.models import Team, Organization
 from awx.main.utils import encrypt_field
 from . import injectors as builtin_injectors

@@ -315,6 +320,16 @@ class Credential(PasswordFieldsModel, CommonModelNameNotUnique, ResourceMixin):
        else:
            raise ValueError('{} is not a dynamic input field'.format(field_name))

+    def validate_role_assignment(self, actor, role_definition):
+        if self.organization:
+            if isinstance(actor, User):
+                if actor.is_superuser or Organization.access_qs(actor, 'member').filter(id=self.organization.id).exists():
+                    return
+            if isinstance(actor, Team):
+                if actor.organization == self.organization:
+                    return
+            raise DRFValidationError({'detail': _(f"You cannot grant credential access to a {actor._meta.object_name} not in the credentials' organization")})
+

 class CredentialType(CommonModelNameNotUnique):
    """
--- a/awx/main/models/execution_environments.py
+++ b/awx/main/models/execution_environments.py
@@ -1,6 +1,8 @@
 from django.db import models
 from django.utils.translation import gettext_lazy as _

+from rest_framework.exceptions import ValidationError
+
 from awx.api.versioning import reverse
 from awx.main.models.base import CommonModel
 from awx.main.validators import validate_container_image_name
@@ -12,6 +14,8 @@ __all__ = ['ExecutionEnvironment']
 class ExecutionEnvironment(CommonModel):
    class Meta:
        ordering = ('-created',)
+        # Remove view permission, as a temporary solution, defer to organization read permission
+        default_permissions = ('add', 'change', 'delete')

    PULL_CHOICES = [
        ('always', _("Always pull container before running.")),
@@ -53,3 +57,12 @@ class ExecutionEnvironment(CommonModel):

    def get_absolute_url(self, request=None):
        return reverse('api:execution_environment_detail', kwargs={'pk': self.pk}, request=request)
+
+    def validate_role_assignment(self, actor, role_definition):
+        if self.managed:
+            raise ValidationError({'object_id': _('Can not assign object roles to managed Execution Environments')})
+        if self.organization_id is None:
+            raise ValidationError({'object_id': _('Can not assign object roles to global Execution Environments')})
+
+        if actor._meta.model_name == 'user' and (not actor.has_obj_perm(self.organization, 'view')):
+            raise ValidationError({'user': _('User must have view permission to Execution Environment organization')})
--- a/awx/main/models/notifications.py
+++ b/awx/main/models/notifications.py
@@ -396,11 +396,11 @@ class JobNotificationMixin(object):
                'verbosity': 0,
            },
            'job_friendly_name': 'Job',
-            'url': 'https://towerhost/#/jobs/playbook/1010',
+            'url': 'https://platformhost/#/jobs/playbook/1010',
            'approval_status': 'approved',
            'approval_node_name': 'Approve Me',
-            'workflow_url': 'https://towerhost/#/jobs/workflow/1010',
-            'job_metadata': """{'url': 'https://towerhost/$/jobs/playbook/13',
+            'workflow_url': 'https://platformhost/#/jobs/workflow/1010',
+            'job_metadata': """{'url': 'https://platformhost/$/jobs/playbook/13',
 'traceback': '',
 'status': 'running',
 'started': '2019-08-07T21:46:38.362630+00:00',
--- a/awx/main/models/rbac.py
+++ b/awx/main/models/rbac.py
@@ -591,14 +591,20 @@ def get_role_from_object_role(object_role):
        role_name = role_name.lower()
        model_cls = apps.get_model('main', target_model_name)
        target_model_name = get_type_for_model(model_cls)
+
+        # exception cases completely specific to one model naming convention
        if target_model_name == 'notification_template':
-            target_model_name = 'notification'  # total exception
+            target_model_name = 'notification'
+        elif target_model_name == 'workflow_job_template':
+            target_model_name = 'workflow'
+
        role_name = f'{target_model_name}_admin_role'
    elif rd.name.endswith(' Admin'):
        # cases like "project-admin"
        role_name = 'admin_role'
+    elif rd.name == 'Organization Audit':
+        role_name = 'auditor_role'
    else:
-        print(rd.name)
        model_name, role_name = rd.name.split()
        role_name = role_name.lower()
        role_name += '_role'
@@ -683,9 +689,15 @@ def sync_parents_to_new_rbac(instance, action, model, pk_set, reverse, **kwargs)

    for role_id in pk_set:
        if reverse:
-            child_role = Role.objects.get(id=role_id)
+            try:
+                child_role = Role.objects.get(id=role_id)
+            except Role.DoesNotExist:
+                continue
        else:
-            parent_role = Role.objects.get(id=role_id)
+            try:
+                parent_role = Role.objects.get(id=role_id)
+            except Role.DoesNotExist:
+                continue

        # To a fault, we want to avoid running this if triggered from implicit_parents management
        # we only want to do anything if we know for sure this is a non-implicit team role
--- a/awx/main/models/unified_jobs.py
+++ b/awx/main/models/unified_jobs.py
@@ -31,6 +31,7 @@ from rest_framework.exceptions import ParseError
 from polymorphic.models import PolymorphicModel

 from ansible_base.lib.utils.models import prevent_search, get_type_for_model
+from ansible_base.rbac import permission_registry

 # AWX
 from awx.main.models.base import CommonModelNameNotUnique, PasswordFieldsModel, NotificationFieldsModel
@@ -197,9 +198,7 @@ class UnifiedJobTemplate(PolymorphicModel, CommonModelNameNotUnique, ExecutionEn

    @classmethod
    def _submodels_with_roles(cls):
-        ujt_classes = [c for c in cls.__subclasses__() if c._meta.model_name not in ['inventorysource', 'systemjobtemplate']]
-        ct_dict = ContentType.objects.get_for_models(*ujt_classes)
-        return [ct.id for ct in ct_dict.values()]
+        return [c for c in cls.__subclasses__() if permission_registry.is_registered(c)]

    @classmethod
    def accessible_pk_qs(cls, accessor, role_field):
@@ -215,8 +214,16 @@ class UnifiedJobTemplate(PolymorphicModel, CommonModelNameNotUnique, ExecutionEn

        action = to_permissions[role_field]

+        # Special condition for super auditor
+        role_subclasses = cls._submodels_with_roles()
+        role_cts = ContentType.objects.get_for_models(*role_subclasses).values()
+        all_codenames = {f'{action}_{cls._meta.model_name}' for cls in role_subclasses}
+        if not (all_codenames - accessor.singleton_permissions()):
+            qs = cls.objects.filter(polymorphic_ctype__in=role_cts)
+            return qs.values_list('id', flat=True)
+
        return (
-            RoleEvaluation.objects.filter(role__in=accessor.has_roles.all(), codename__startswith=action, content_type_id__in=cls._submodels_with_roles())
+            RoleEvaluation.objects.filter(role__in=accessor.has_roles.all(), codename__in=all_codenames, content_type_id__in=[ct.id for ct in role_cts])
            .values_list('object_id')
            .distinct()
        )
--- a/awx/main/scheduler/task_manager.py
+++ b/awx/main/scheduler/task_manager.py
@@ -138,7 +138,8 @@ class TaskBase:

        # Lock
        with task_manager_bulk_reschedule():
-            with advisory_lock(f"{self.prefix}_lock", wait=False) as acquired:
+            lock_session_timeout_milliseconds = settings.TASK_MANAGER_LOCK_TIMEOUT * 1000  # convert to milliseconds
+            with advisory_lock(f"{self.prefix}_lock", lock_session_timeout_milliseconds=lock_session_timeout_milliseconds, wait=False) as acquired:
                with transaction.atomic():
                    if acquired is False:
                        logger.debug(f"Not running {self.prefix} scheduler, another task holds lock")
--- a/awx/main/tasks/receptor.py
+++ b/awx/main/tasks/receptor.py
@@ -405,10 +405,11 @@ class AWXReceptorJob:
        finally:
            # Make sure to always release the work unit if we established it
            if self.unit_id is not None and settings.RECEPTOR_RELEASE_WORK:
-                try:
-                    receptor_ctl.simple_command(f"work release {self.unit_id}")
-                except Exception:
-                    logger.exception(f"Error releasing work unit {self.unit_id}.")
+                if settings.RECPETOR_KEEP_WORK_ON_ERROR and getattr(res, 'status', 'error') == 'error':
+                    try:
+                        receptor_ctl.simple_command(f"work release {self.unit_id}")
+                    except Exception:
+                        logger.exception(f"Error releasing work unit {self.unit_id}.")

    def _run_internal(self, receptor_ctl):
        # Create a socketpair. Where the left side will be used for writing our payload
--- a/awx/main/tasks/system.py
+++ b/awx/main/tasks/system.py
@@ -54,7 +54,7 @@ from awx.main.models import (
    Job,
    convert_jsonfields,
 )
-from awx.main.constants import ACTIVE_STATES
+from awx.main.constants import ACTIVE_STATES, ERROR_STATES
 from awx.main.dispatch.publish import task
 from awx.main.dispatch import get_task_queuename, reaper
 from awx.main.utils.common import ignore_inventory_computed_fields, ignore_inventory_group_removal
@@ -685,6 +685,8 @@ def awx_receptor_workunit_reaper():

    unit_ids = [id for id in receptor_work_list]
    jobs_with_unreleased_receptor_units = UnifiedJob.objects.filter(work_unit_id__in=unit_ids).exclude(status__in=ACTIVE_STATES)
+    if settings.RECEPTOR_KEEP_WORK_ON_ERROR:
+        jobs_with_unreleased_receptor_units = jobs_with_unreleased_receptor_units.exclude(status__in=ERROR_STATES)
    for job in jobs_with_unreleased_receptor_units:
        logger.debug(f"{job.log_format} is not active, reaping receptor work unit {job.work_unit_id}")
        receptor_ctl.simple_command(f"work cancel {job.work_unit_id}")
@@ -704,7 +706,10 @@ def awx_k8s_reaper():
        logger.debug("Checking for orphaned k8s pods for {}.".format(group))
        pods = PodManager.list_active_jobs(group)
        time_cutoff = now() - timedelta(seconds=settings.K8S_POD_REAPER_GRACE_PERIOD)
-        for job in UnifiedJob.objects.filter(pk__in=pods.keys(), finished__lte=time_cutoff).exclude(status__in=ACTIVE_STATES):
+        reap_job_candidates = UnifiedJob.objects.filter(pk__in=pods.keys(), finished__lte=time_cutoff).exclude(status__in=ACTIVE_STATES)
+        if settings.RECEPTOR_KEEP_WORK_ON_ERROR:
+            reap_job_candidates = reap_job_candidates.exclude(status__in=ERROR_STATES)
+        for job in reap_job_candidates:
            logger.debug('{} is no longer active, reaping orphaned k8s pod'.format(job.log_format))
            try:
                pm = PodManager(job)
@@ -715,7 +720,8 @@ def awx_k8s_reaper():

@task(queue=get_task_queuename)
 def awx_periodic_scheduler():
-    with advisory_lock('awx_periodic_scheduler_lock', wait=False) as acquired:
+    lock_session_timeout_milliseconds = settings.TASK_MANAGER_LOCK_TIMEOUT * 1000
+    with advisory_lock('awx_periodic_scheduler_lock', lock_session_timeout_milliseconds=lock_session_timeout_milliseconds, wait=False) as acquired:
        if acquired is False:
            logger.debug("Not running periodic scheduler, another task holds lock")
            return
@@ -979,5 +985,15 @@ def periodic_resource_sync():
        if acquired is False:
            logger.debug("Not running periodic_resource_sync, another task holds lock")
            return
+        logger.debug("Running periodic resource sync")

-        SyncExecutor().run()
+        executor = SyncExecutor()
+        executor.run()
+        for key, item_list in executor.results.items():
+            if not item_list or key == 'noop':
+                continue
+            # Log creations and conflicts
+            if len(item_list) > 10 and settings.LOG_AGGREGATOR_LEVEL != 'DEBUG':
+                logger.info(f'Periodic resource sync {key}, first 10 items:\n{item_list[:10]}')
+            else:
+                logger.info(f'Periodic resource sync {key}:\n{item_list}')
--- a/awx/main/tests/functional/api/test_settings.py
+++ b/awx/main/tests/functional/api/test_settings.py
--- a/awx/main/tests/functional/conftest.py
+++ b/awx/main/tests/functional/conftest.py
@@ -92,6 +92,11 @@ def deploy_jobtemplate(project, inventory, credential):
    return jt


+@pytest.fixture()
+def execution_environment():
+    return ExecutionEnvironment.objects.create(name="test-ee", description="test-ee", managed=True)
+
+
@pytest.fixture
 def setup_managed_roles():
    "Run the migration script to pre-create managed role definitions"
--- a/awx/main/tests/functional/dab_rbac/test_access_list.py
+++ b/awx/main/tests/functional/dab_rbac/test_access_list.py
@@ -109,3 +109,17 @@ def test_team_indirect_access(get, team, admin_user, inventory):
    assert len(by_username['u1']['summary_fields']['indirect_access']) == 0
    access_entry = by_username['u1']['summary_fields']['direct_access'][0]
    assert sorted(access_entry['descendant_roles']) == sorted(['adhoc_role', 'use_role', 'update_role', 'read_role', 'admin_role'])
+
+
+@pytest.mark.django_db
+def test_workflow_access_list(workflow_job_template, alice, bob, setup_managed_roles, get, admin_user):
+    """Basic verification that WFJT access_list is functional"""
+    workflow_job_template.admin_role.members.add(alice)
+    workflow_job_template.organization.workflow_admin_role.members.add(bob)
+
+    url = reverse('api:workflow_job_template_access_list', kwargs={'pk': workflow_job_template.pk})
+    for u in (alice, bob, admin_user):
+        response = get(url, user=u, expect=200)
+        user_ids = [item['id'] for item in response.data['results']]
+        assert alice.pk in user_ids
+        assert bob.pk in user_ids
--- a/awx/main/tests/functional/dab_rbac/test_access_regressions.py
+++ b/awx/main/tests/functional/dab_rbac/test_access_regressions.py
@@ -21,3 +21,21 @@ def test_notification_template_object_role_change(rando, notification_template,
    rd.give_permission(rando, notification_template)
    access = NotificationTemplateAccess(rando)
    assert access.can_change(notification_template, {'name': 'new name'})
+
+
+@pytest.mark.django_db
+def test_organization_auditor_role(rando, setup_managed_roles, organization, inventory, project, jt_linked):
+    obj_list = (inventory, project, jt_linked)
+    for obj in obj_list:
+        assert obj.organization == organization, obj  # sanity
+
+    assert [rando.has_obj_perm(obj, 'view') for obj in obj_list] == [False for i in range(3)], obj_list
+
+    rd = RoleDefinition.objects.get(name='Organization Audit')
+    rd.give_permission(rando, organization)
+
+    codename_set = set(rd.permissions.values_list('codename', flat=True))
+    assert not ({'view_inventory', 'view_jobtemplate', 'audit_organization'} - codename_set)  # sanity
+
+    assert [obj in type(obj).access_qs(rando) for obj in obj_list] == [True for i in range(3)], obj_list
+    assert [rando.has_obj_perm(obj, 'view') for obj in obj_list] == [True for i in range(3)], obj_list
--- a/awx/main/tests/functional/dab_rbac/test_dab_rbac_api.py
+++ b/awx/main/tests/functional/dab_rbac/test_dab_rbac_api.py
@@ -2,9 +2,11 @@ import pytest

 from django.contrib.contenttypes.models import ContentType
 from django.urls import reverse as django_reverse
+from django.test.utils import override_settings

 from awx.api.versioning import reverse
 from awx.main.models import JobTemplate, Inventory, Organization
+from awx.main.access import JobTemplateAccess, WorkflowJobTemplateAccess

 from ansible_base.rbac.models import RoleDefinition

@@ -66,13 +68,17 @@ def test_assign_managed_role(admin_user, alice, rando, inventory, post, setup_ma

@pytest.mark.django_db
 def test_assign_custom_delete_role(admin_user, rando, inventory, delete, patch):
+    # TODO: just a delete_inventory, without change_inventory
    rd, _ = RoleDefinition.objects.get_or_create(
-        name='inventory-delete', permissions=['delete_inventory', 'view_inventory'], content_type=ContentType.objects.get_for_model(Inventory)
+        name='inventory-delete',
+        permissions=['delete_inventory', 'view_inventory', 'change_inventory'],
+        content_type=ContentType.objects.get_for_model(Inventory),
    )
    rd.give_permission(rando, inventory)
    inv_id = inventory.pk
    inv_url = reverse('api:inventory_detail', kwargs={'pk': inv_id})
-    patch(url=inv_url, data={"description": "new"}, user=rando, expect=403)
+    # TODO: eventually this will be valid test, for now ignore
+    # patch(url=inv_url, data={"description": "new"}, user=rando, expect=403)
    delete(url=inv_url, user=rando, expect=202)
    assert Inventory.objects.get(id=inv_id).pending_deletion

@@ -88,3 +94,63 @@ def test_assign_custom_add_role(admin_user, rando, organization, post, setup_man
    inv_id = r.data['id']
    inventory = Inventory.objects.get(id=inv_id)
    assert rando.has_obj_perm(inventory, 'change')
+
+
+@pytest.mark.django_db
+def test_jt_creation_permissions(setup_managed_roles, inventory, project, rando):
+    """This tests that if you assign someone required permissions in the new API
+    using the managed roles, then that works to give permissions to create a job template"""
+    inv_rd = RoleDefinition.objects.get(name='Inventory Admin')
+    proj_rd = RoleDefinition.objects.get(name='Project Admin')
+    # establish prior state
+    access = JobTemplateAccess(rando)
+    assert not access.can_add({'inventory': inventory.pk, 'project': project.pk, 'name': 'foo-jt'})
+
+    inv_rd.give_permission(rando, inventory)
+    proj_rd.give_permission(rando, project)
+
+    assert access.can_add({'inventory': inventory.pk, 'project': project.pk, 'name': 'foo-jt'})
+
+
+@pytest.mark.django_db
+def test_workflow_creation_permissions(setup_managed_roles, organization, workflow_job_template, rando):
+    """Similar to JT, assigning new roles gives creator permissions"""
+    org_wf_rd = RoleDefinition.objects.get(name='Organization WorkflowJobTemplate Admin')
+    assert workflow_job_template.organization == organization  # sanity
+    # establish prior state
+    access = WorkflowJobTemplateAccess(rando)
+    assert not access.can_add({'name': 'foo-flow', 'organization': organization.pk})
+    org_wf_rd.give_permission(rando, organization)
+
+    assert access.can_add({'name': 'foo-flow', 'organization': organization.pk})
+
+
+@pytest.mark.django_db
+def test_assign_credential_to_user_of_another_org(setup_managed_roles, credential, admin_user, rando, org_admin, organization, post):
+    '''Test that a credential can only be assigned to a user in the same organization'''
+    # cannot assign credential to rando, as rando is not in the same org as the credential
+    rd = RoleDefinition.objects.get(name="Credential Admin")
+    credential.organization = organization
+    credential.save(update_fields=['organization'])
+    assert credential.organization not in Organization.access_qs(rando, 'member')
+    url = django_reverse('roleuserassignment-list')
+    resp = post(url=url, data={"user": rando.id, "role_definition": rd.id, "object_id": credential.id}, user=admin_user, expect=400)
+    assert "You cannot grant credential access to a User not in the credentials' organization" in str(resp.data)
+
+    # can assign credential to superuser
+    rando.is_superuser = True
+    rando.save()
+    post(url=url, data={"user": rando.id, "role_definition": rd.id, "object_id": credential.id}, user=admin_user, expect=201)
+
+    # can assign credential to org_admin
+    assert credential.organization in Organization.access_qs(org_admin, 'member')
+    post(url=url, data={"user": org_admin.id, "role_definition": rd.id, "object_id": credential.id}, user=admin_user, expect=201)
+
+
+@pytest.mark.django_db
+@override_settings(ALLOW_LOCAL_RESOURCE_MANAGEMENT=False)
+def test_team_member_role_not_assignable(team, rando, post, admin_user, setup_managed_roles):
+    member_rd = RoleDefinition.objects.get(name='Organization Member')
+    url = django_reverse('roleuserassignment-list')
+    r = post(url, data={'object_id': team.id, 'role_definition': member_rd.id, 'user': rando.id}, user=admin_user, expect=400)
+    assert 'Not managed locally' in str(r.data)
--- a/awx/main/tests/functional/dab_rbac/test_external_auditor.py
+++ b/awx/main/tests/functional/dab_rbac/test_external_auditor.py
@@ -0,0 +1,120 @@
+import pytest
+
+from django.apps import apps
+
+from ansible_base.rbac.managed import SystemAuditor
+from ansible_base.rbac import permission_registry
+
+from awx.main.access import check_user_access, get_user_queryset
+from awx.main.models import User, AdHocCommandEvent
+from awx.api.versioning import reverse
+
+
+@pytest.fixture
+def ext_auditor_rd():
+    info = SystemAuditor(overrides={'name': 'Alien Auditor', 'shortname': 'ext_auditor'})
+    rd, _ = info.get_or_create(apps)
+    return rd
+
+
+@pytest.fixture
+def ext_auditor(ext_auditor_rd):
+    u = User.objects.create(username='external-auditor-user')
+    ext_auditor_rd.give_global_permission(u)
+    return u
+
+
+@pytest.fixture
+def obj_factory(request):
+    def _rf(fixture_name):
+        obj = request.getfixturevalue(fixture_name)
+
+        # special case to make obj organization-scoped
+        if obj._meta.model_name == 'executionenvironment':
+            obj.organization = request.getfixturevalue('organization')
+            obj.save(update_fields=['organization'])
+
+        return obj
+
+    return _rf
+
+
+@pytest.mark.django_db
+def test_access_qs_external_auditor(ext_auditor_rd, rando, job_template):
+    ext_auditor_rd.give_global_permission(rando)
+    jt_cls = apps.get_model('main', 'JobTemplate')
+    ujt_cls = apps.get_model('main', 'UnifiedJobTemplate')
+    assert job_template in jt_cls.access_qs(rando)
+    assert job_template.id in jt_cls.access_ids_qs(rando)
+    assert job_template.id in ujt_cls.accessible_pk_qs(rando, 'read_role')
+
+
+@pytest.mark.django_db
+@pytest.mark.parametrize('model', sorted(permission_registry.all_registered_models, key=lambda cls: cls._meta.model_name))
+class TestExternalAuditorRoleAllModels:
+    def test_access_can_read_method(self, obj_factory, model, ext_auditor, rando):
+        fixture_name = model._meta.verbose_name.replace(' ', '_')
+        obj = obj_factory(fixture_name)
+
+        assert check_user_access(rando, model, 'read', obj) is False
+        assert check_user_access(ext_auditor, model, 'read', obj) is True
+
+    def test_access_get_queryset(self, obj_factory, model, ext_auditor, rando):
+        fixture_name = model._meta.verbose_name.replace(' ', '_')
+        obj = obj_factory(fixture_name)
+
+        assert obj not in get_user_queryset(rando, model)
+        assert obj in get_user_queryset(ext_auditor, model)
+
+    def test_global_list(self, obj_factory, model, ext_auditor, rando, get):
+        fixture_name = model._meta.verbose_name.replace(' ', '_')
+        obj_factory(fixture_name)
+
+        url = reverse(f'api:{fixture_name}_list')
+        r = get(url, user=rando, expect=200)
+        initial_ct = r.data['count']
+
+        r = get(url, user=ext_auditor, expect=200)
+        assert r.data['count'] == initial_ct + 1
+
+        if fixture_name in ('job_template', 'workflow_job_template'):
+            url = reverse('api:unified_job_template_list')
+            r = get(url, user=rando, expect=200)
+            initial_ct = r.data['count']
+
+            r = get(url, user=ext_auditor, expect=200)
+            assert r.data['count'] == initial_ct + 1
+
+    def test_detail_view(self, obj_factory, model, ext_auditor, rando, get):
+        fixture_name = model._meta.verbose_name.replace(' ', '_')
+        obj = obj_factory(fixture_name)
+
+        url = reverse(f'api:{fixture_name}_detail', kwargs={'pk': obj.pk})
+        get(url, user=rando, expect=403)  # NOTE: should be 401
+        get(url, user=ext_auditor, expect=200)
+
+
+@pytest.mark.django_db
+class TestExternalAuditorNonRoleModels:
+    def test_ad_hoc_command_view(self, ad_hoc_command_factory, rando, ext_auditor, get):
+        """The AdHocCommandAccess class references is_system_auditor
+
+        this is to prove it works with other system-level view roles"""
+        ad_hoc_command = ad_hoc_command_factory()
+        url = reverse('api:ad_hoc_command_list')
+        r = get(url, user=rando, expect=200)
+        assert r.data['count'] == 0
+        r = get(url, user=ext_auditor, expect=200)
+        assert r.data['count'] == 1
+        assert r.data['results'][0]['id'] == ad_hoc_command.id
+
+        event = AdHocCommandEvent.objects.create(ad_hoc_command=ad_hoc_command)
+        url = reverse('api:ad_hoc_command_ad_hoc_command_events_list', kwargs={'pk': ad_hoc_command.id})
+        r = get(url, user=rando, expect=403)
+        r = get(url, user=ext_auditor, expect=200)
+        assert r.data['count'] == 1
+
+        url = reverse('api:ad_hoc_command_event_detail', kwargs={'pk': event.id})
+        r = get(url, user=rando, expect=403)
+        r = get(url, user=ext_auditor, expect=200)
+        assert r.data['id'] == event.id
--- a/awx/main/tests/functional/dab_rbac/test_managed_roles.py
+++ b/awx/main/tests/functional/dab_rbac/test_managed_roles.py
@@ -0,0 +1,31 @@
+import pytest
+
+from ansible_base.rbac.models import RoleDefinition, DABPermission
+
+
+@pytest.mark.django_db
+def test_roles_to_not_create(setup_managed_roles):
+    assert RoleDefinition.objects.filter(name='Organization Admin').count() == 1
+
+    SHOULD_NOT_EXIST = ('Organization Organization Admin', 'Organization Team Admin', 'Organization InstanceGroup Admin')
+
+    bad_rds = RoleDefinition.objects.filter(name__in=SHOULD_NOT_EXIST)
+    if bad_rds.exists():
+        bad_names = list(bad_rds.values_list('name', flat=True))
+        raise Exception(f'Found RoleDefinitions that should not exist: {bad_names}')
+
+
+@pytest.mark.django_db
+def test_project_update_role(setup_managed_roles):
+    """Role to allow updating a project on the object-level should exist"""
+    assert RoleDefinition.objects.filter(name='Project Update').count() == 1
+
+
+@pytest.mark.django_db
+def test_org_child_add_permission(setup_managed_roles):
+    for model_name in ('Project', 'NotificationTemplate', 'WorkflowJobTemplate', 'Inventory'):
+        rd = RoleDefinition.objects.get(name=f'Organization {model_name} Admin')
+        assert 'add_' in str(rd.permissions.values_list('codename', flat=True)), f'The {rd.name} role definition expected to contain add_ permissions'
+
+    # special case for JobTemplate, anyone can create one with use permission to project/inventory
+    assert not DABPermission.objects.filter(codename='add_jobtemplate').exists()
--- a/awx/main/tests/functional/dab_rbac/test_translation_layer.py
+++ b/awx/main/tests/functional/dab_rbac/test_translation_layer.py
@@ -1,4 +1,5 @@
 from unittest import mock
+import json

 import pytest

@@ -6,17 +7,29 @@ from django.contrib.contenttypes.models import ContentType

 from crum import impersonate

-from awx.main.models.rbac import get_role_from_object_role, give_creator_permissions
+from awx.main.fields import ImplicitRoleField
+from awx.main.models.rbac import get_role_from_object_role, give_creator_permissions, get_role_codenames, get_role_definition
 from awx.main.models import User, Organization, WorkflowJobTemplate, WorkflowJobTemplateNode, Team
 from awx.api.versioning import reverse

 from ansible_base.rbac.models import RoleUserAssignment, RoleDefinition
+from ansible_base.rbac import permission_registry


@pytest.mark.django_db
@pytest.mark.parametrize(
    'role_name',
-    ['execution_environment_admin_role', 'project_admin_role', 'admin_role', 'auditor_role', 'read_role', 'execute_role', 'notification_admin_role'],
+    [
+        'execution_environment_admin_role',
+        'workflow_admin_role',
+        'project_admin_role',
+        'admin_role',
+        'auditor_role',
+        'read_role',
+        'execute_role',
+        'approval_role',
+        'notification_admin_role',
+    ],
 )
 def test_round_trip_roles(organization, rando, role_name, setup_managed_roles):
    """
@@ -26,11 +39,41 @@ def test_round_trip_roles(organization, rando, role_name, setup_managed_roles):
    """
    getattr(organization, role_name).members.add(rando)
    assignment = RoleUserAssignment.objects.get(user=rando)
-    print(assignment.role_definition.name)
    old_role = get_role_from_object_role(assignment.object_role)
    assert old_role.id == getattr(organization, role_name).id


+@pytest.mark.django_db
+@pytest.mark.parametrize('model', sorted(permission_registry.all_registered_models, key=lambda cls: cls._meta.model_name))
+def test_role_migration_matches(request, model, setup_managed_roles):
+    fixture_name = model._meta.verbose_name.replace(' ', '_')
+    obj = request.getfixturevalue(fixture_name)
+    role_ct = 0
+    for field in obj._meta.get_fields():
+        if isinstance(field, ImplicitRoleField):
+            if field.name == 'read_role':
+                continue  # intentionally left as "Compat" roles
+            role_ct += 1
+            old_role = getattr(obj, field.name)
+            old_codenames = set(get_role_codenames(old_role))
+            rd = get_role_definition(old_role)
+            new_codenames = set(rd.permissions.values_list('codename', flat=True))
+            # all the old roles should map to a non-Compat role definition
+            if 'Compat' not in rd.name:
+                model_rds = RoleDefinition.objects.filter(content_type=ContentType.objects.get_for_model(obj))
+                rd_data = {}
+                for rd in model_rds:
+                    rd_data[rd.name] = list(rd.permissions.values_list('codename', flat=True))
+            assert (
+                'Compat' not in rd.name
+            ), f'Permissions for old vs new roles did not match.\nold {field.name}: {old_codenames}\nnew:\n{json.dumps(rd_data, indent=2)}'
+            assert new_codenames == set(old_codenames)
+
+    # In the old system these models did not have object-level roles, all others expect some model roles
+    if model._meta.model_name not in ('notificationtemplate', 'executionenvironment'):
+        assert role_ct > 0
+
+
@pytest.mark.django_db
 def test_role_naming(setup_managed_roles):
    qs = RoleDefinition.objects.filter(content_type=ContentType.objects.get(model='jobtemplate'), name__endswith='dmin')
@@ -141,3 +184,11 @@ def test_implicit_parents_no_assignments(organization):
    with mock.patch('awx.main.models.rbac.give_or_remove_permission') as mck:
        Team.objects.create(name='random team', organization=organization)
    mck.assert_not_called()
+
+
+@pytest.mark.django_db
+def test_user_auditor_rel(organization, rando, setup_managed_roles):
+    assert rando not in organization.auditor_role
+    audit_rd = RoleDefinition.objects.get(name='Organization Audit')
+    audit_rd.give_permission(rando, organization)
+    assert list(rando.auditor_of_organizations) == [organization]
--- a/awx/main/tests/functional/models/test_unified_job.py
+++ b/awx/main/tests/functional/models/test_unified_job.py
@@ -4,25 +4,19 @@ import pytest
 # CRUM
 from crum import impersonate

-# Django
-from django.contrib.contenttypes.models import ContentType
-
 # AWX
-from awx.main.models import UnifiedJobTemplate, Job, JobTemplate, WorkflowJobTemplate, WorkflowApprovalTemplate, Project, WorkflowJob, Schedule, Credential
+from awx.main.models import UnifiedJobTemplate, Job, JobTemplate, WorkflowJobTemplate, Project, WorkflowJob, Schedule, Credential
 from awx.api.versioning import reverse
 from awx.main.constants import JOB_VARIABLE_PREFIXES


@pytest.mark.django_db
 def test_subclass_types():
-    assert set(UnifiedJobTemplate._submodels_with_roles()) == set(
-        [
-            ContentType.objects.get_for_model(JobTemplate).id,
-            ContentType.objects.get_for_model(Project).id,
-            ContentType.objects.get_for_model(WorkflowJobTemplate).id,
-            ContentType.objects.get_for_model(WorkflowApprovalTemplate).id,
-        ]
-    )
+    assert set(UnifiedJobTemplate._submodels_with_roles()) == {
+        JobTemplate,
+        Project,
+        WorkflowJobTemplate,
+    }


@pytest.mark.django_db
--- a/awx/main/tests/functional/test_migrations.py
+++ b/awx/main/tests/functional/test_migrations.py
@@ -85,3 +85,17 @@ class TestMigrationSmoke:

        RoleUserAssignment = new_state.apps.get_model('dab_rbac', 'RoleUserAssignment')
        assert RoleUserAssignment.objects.filter(user=user.id, object_id=org.id).exists()
+
+        # Regression testing for bug that comes from current vs past models mismatch
+        RoleDefinition = new_state.apps.get_model('dab_rbac', 'RoleDefinition')
+        assert not RoleDefinition.objects.filter(name='Organization Organization Admin').exists()
+        # Test special cases in managed role creation
+        assert not RoleDefinition.objects.filter(name='Organization Team Admin').exists()
+        assert not RoleDefinition.objects.filter(name='Organization InstanceGroup Admin').exists()
+
+        # Test that a removed EE model permission has been deleted
+        new_state = migrator.apply_tested_migration(
+            ('main', '0195_EE_permissions'),
+        )
+        DABPermission = new_state.apps.get_model('dab_rbac', 'DABPermission')
+        assert not DABPermission.objects.filter(codename='view_executionenvironment').exists()
--- a/awx/main/tests/functional/test_rbac_execution_environment.py
+++ b/awx/main/tests/functional/test_rbac_execution_environment.py
@@ -0,0 +1,148 @@
+import pytest
+
+from django.contrib.contenttypes.models import ContentType
+
+from awx.main.access import ExecutionEnvironmentAccess
+from awx.main.models import ExecutionEnvironment, Organization, Team
+from awx.main.models.rbac import get_role_codenames
+
+from awx.api.versioning import reverse
+from django.urls import reverse as django_reverse
+
+from ansible_base.rbac.models import RoleDefinition
+
+
+@pytest.fixture
+def ee_rd():
+    return RoleDefinition.objects.create_from_permissions(
+        name='EE object admin',
+        permissions=['change_executionenvironment', 'delete_executionenvironment'],
+        content_type=ContentType.objects.get_for_model(ExecutionEnvironment),
+    )
+
+
+@pytest.fixture
+def org_ee_rd():
+    return RoleDefinition.objects.create_from_permissions(
+        name='EE org admin',
+        permissions=['add_executionenvironment', 'change_executionenvironment', 'delete_executionenvironment', 'view_organization'],
+        content_type=ContentType.objects.get_for_model(Organization),
+    )
+
+
+@pytest.mark.django_db
+def test_old_ee_role_maps_to_correct_permissions(organization):
+    assert set(get_role_codenames(organization.execution_environment_admin_role)) == {
+        'view_organization',
+        'add_executionenvironment',
+        'change_executionenvironment',
+        'delete_executionenvironment',
+    }
+
+
+@pytest.fixture
+def org_ee(organization):
+    return ExecutionEnvironment.objects.create(name='some user ee', organization=organization)
+
+
+@pytest.fixture
+def check_user_capabilities(get, setup_managed_roles):
+    def _rf(user, obj, expected):
+        url = reverse('api:execution_environment_list')
+        r = get(url, user=user, expect=200)
+        for item in r.data['results']:
+            if item['id'] == obj.pk:
+                assert expected == item['summary_fields']['user_capabilities']
+                break
+        else:
+            raise RuntimeError(f'Could not find expected object ({obj}) in EE list result: {r.data}')
+
+    return _rf
+
+
+# ___ begin tests ___
+
+
+@pytest.mark.django_db
+def test_any_user_can_view_global_ee(control_plane_execution_environment, rando):
+    assert ExecutionEnvironmentAccess(rando).can_read(control_plane_execution_environment)
+
+
+@pytest.mark.django_db
+def test_managed_ee_not_assignable(control_plane_execution_environment, ee_rd, rando, admin_user, post):
+    url = django_reverse('roleuserassignment-list')
+    r = post(url, {'role_definition': ee_rd.pk, 'user': rando.id, 'object_id': control_plane_execution_environment.pk}, user=admin_user, expect=400)
+    assert 'Can not assign object roles to managed Execution Environment' in str(r.data)
+
+
+@pytest.mark.django_db
+def test_org_member_required_for_assignment(org_ee, ee_rd, rando, admin_user, post):
+    url = django_reverse('roleuserassignment-list')
+    r = post(url, {'role_definition': ee_rd.pk, 'user': rando.id, 'object_id': org_ee.pk}, user=admin_user, expect=400)
+    assert 'User must have view permission to Execution Environment organization' in str(r.data)
+
+
+@pytest.mark.django_db
+def test_team_can_have_permission(org_ee, ee_rd, rando, admin_user, post):
+    org2 = Organization.objects.create(name='a different team')
+    team = Team.objects.create(name='a team', organization=org2)
+    team.member_role.members.add(rando)
+    assert org_ee not in ExecutionEnvironmentAccess(rando).get_queryset()  # user can not view the EE
+
+    url = django_reverse('roleteamassignment-list')
+
+    # can give object roles to the team now
+    post(url, {'role_definition': ee_rd.pk, 'team': team.id, 'object_id': org_ee.pk}, user=admin_user, expect=201)
+    assert rando.has_obj_perm(org_ee, 'change')
+    assert org_ee in ExecutionEnvironmentAccess(rando).get_queryset()  # user can view the EE now
+
+
+@pytest.mark.django_db
+def test_give_object_permission_to_ee(org_ee, ee_rd, org_member, check_user_capabilities):
+    access = ExecutionEnvironmentAccess(org_member)
+    assert access.can_read(org_ee)  # by virtue of being an org member
+    assert not access.can_change(org_ee, {'name': 'new'})
+    check_user_capabilities(org_member, org_ee, {'edit': False, 'delete': False, 'copy': False})
+
+    ee_rd.give_permission(org_member, org_ee)
+    assert access.can_change(org_ee, {'name': 'new', 'organization': org_ee.organization.id})
+
+    check_user_capabilities(org_member, org_ee, {'edit': True, 'delete': True, 'copy': False})
+
+
+@pytest.mark.django_db
+def test_need_related_organization_access(org_ee, ee_rd, org_member):
+    org2 = Organization.objects.create(name='another organization')
+    ee_rd.give_permission(org_member, org_ee)
+    org2.member_role.members.add(org_member)
+    access = ExecutionEnvironmentAccess(org_member)
+    assert access.can_change(org_ee, {'name': 'new', 'organization': org_ee.organization})
+    assert access.can_change(org_ee, {'name': 'new', 'organization': org_ee.organization.id})
+    assert not access.can_change(org_ee, {'name': 'new', 'organization': org2.id})
+    assert not access.can_change(org_ee, {'name': 'new', 'organization': org2})
+
+    # User can make the change if they have relevant permission to the new organization
+    org_ee.organization.execution_environment_admin_role.members.add(org_member)
+    org2.execution_environment_admin_role.members.add(org_member)
+    assert access.can_change(org_ee, {'name': 'new', 'organization': org2.id})
+    assert access.can_change(org_ee, {'name': 'new', 'organization': org2})
+
+
+@pytest.mark.django_db
+@pytest.mark.parametrize('style', ['new', 'old'])
+def test_give_org_permission_to_ee(org_ee, organization, org_member, check_user_capabilities, style, org_ee_rd):
+    access = ExecutionEnvironmentAccess(org_member)
+    assert not access.can_change(org_ee, {'name': 'new'})
+    check_user_capabilities(org_member, org_ee, {'edit': False, 'delete': False, 'copy': False})
+
+    if style == 'new':
+        org_ee_rd.give_permission(org_member, organization)
+        assert org_member.has_obj_perm(org_ee.organization, 'add_executionenvironment')  # sanity
+    else:
+        organization.execution_environment_admin_role.members.add(org_member)
+
+    assert access.can_change(org_ee, {'name': 'new', 'organization': organization.id})
+    check_user_capabilities(org_member, org_ee, {'edit': True, 'delete': True, 'copy': True})
+
+    # Extra check, user can not remove the EE from the organization
+    assert not access.can_change(org_ee, {'name': 'new', 'organization': None})
--- a/awx/main/tests/functional/test_rbac_job_templates.py
+++ b/awx/main/tests/functional/test_rbac_job_templates.py
@@ -182,8 +182,14 @@ def test_job_template_creator_access(project, organization, rando, post, setup_m

@pytest.mark.django_db
@pytest.mark.job_permissions
-@pytest.mark.parametrize('lacking', ['project', 'inventory'])
-def test_job_template_insufficient_creator_permissions(lacking, project, inventory, organization, rando, post):
+@pytest.mark.parametrize(
+    'lacking,reason',
+    [
+        ('project', 'You do not have use permission on Project'),
+        ('inventory', 'You do not have use permission on Inventory'),
+    ],
+)
+def test_job_template_insufficient_creator_permissions(lacking, reason, project, inventory, organization, rando, post):
    if lacking != 'project':
        project.use_role.members.add(rando)
    else:
@@ -192,12 +198,13 @@ def test_job_template_insufficient_creator_permissions(lacking, project, invento
        inventory.use_role.members.add(rando)
    else:
        inventory.read_role.members.add(rando)
-    post(
+    response = post(
        url=reverse('api:job_template_list'),
        data=dict(name='newly-created-jt', inventory=inventory.id, project=project.pk, playbook='helloworld.yml'),
        user=rando,
        expect=403,
    )
+    assert reason in response.data[lacking]


@pytest.mark.django_db
--- a/awx/main/tests/functional/test_rbac_organization.py
+++ b/awx/main/tests/functional/test_rbac_organization.py
@@ -48,3 +48,17 @@ def test_org_resource_role(ext_auth, organization, rando, org_admin):
        assert access.can_attach(organization, rando, 'member_role.members') == ext_auth
        organization.member_role.members.add(rando)
        assert access.can_unattach(organization, rando, 'member_role.members') == ext_auth
+
+
+@pytest.mark.django_db
+def test_delete_org_while_workflow_active(workflow_job_template):
+    '''
+    Delete org while workflow job is active (i.e. changing status)
+    '''
+    assert workflow_job_template.organization  # sanity check
+    wj = workflow_job_template.create_unified_job()  # status should be new
+    workflow_job_template.organization.delete()
+    wj.refresh_from_db()
+    assert wj.status != 'pending'  # sanity check
+    wj.status = 'pending'  # status needs to change in order to trigger workflow_job_template.save()
+    wj.save(update_fields=['status'])
--- a/awx/main/tests/functional/test_rbac_workflow.py
+++ b/awx/main/tests/functional/test_rbac_workflow.py
@@ -35,6 +35,13 @@ class TestWorkflowJobTemplateAccess:
        assert org_member in wfjt.execute_role
        assert org_member in wfjt.read_role

+    def test_non_super_admin_no_add_without_org(self, wfjt, organization, rando):
+        organization.member_role.members.add(rando)
+        wfjt.admin_role.members.add(rando)
+        access = WorkflowJobTemplateAccess(rando, save_messages=True)
+        assert not access.can_add({'name': 'without org'})
+        assert 'An organization is required to create a workflow job template for normal user' in access.messages['organization']
+

@pytest.mark.django_db
 class TestWorkflowJobTemplateNodeAccess:
--- a/awx/main/utils/pglock.py
+++ b/awx/main/utils/pglock.py
@@ -8,9 +8,22 @@ from django.db import connection


@contextmanager
-def advisory_lock(*args, **kwargs):
+def advisory_lock(*args, lock_session_timeout_milliseconds=0, **kwargs):
    if connection.vendor == 'postgresql':
+        cur = None
+        idle_in_transaction_session_timeout = None
+        idle_session_timeout = None
+        if lock_session_timeout_milliseconds > 0:
+            with connection.cursor() as cur:
+                idle_in_transaction_session_timeout = cur.execute('SHOW idle_in_transaction_session_timeout').fetchone()[0]
+                idle_session_timeout = cur.execute('SHOW idle_session_timeout').fetchone()[0]
+                cur.execute(f"SET idle_in_transaction_session_timeout = '{lock_session_timeout_milliseconds}'")
+                cur.execute(f"SET idle_session_timeout = '{lock_session_timeout_milliseconds}'")
        with django_pglocks_advisory_lock(*args, **kwargs) as internal_lock:
            yield internal_lock
+            if lock_session_timeout_milliseconds > 0:
+                with connection.cursor() as cur:
+                    cur.execute(f"SET idle_in_transaction_session_timeout = '{idle_in_transaction_session_timeout}'")
+                    cur.execute(f"SET idle_session_timeout = '{idle_session_timeout}'")
    else:
        yield True
--- a/awx/main/wsrelay.py
+++ b/awx/main/wsrelay.py
@@ -47,7 +47,6 @@ class WebsocketRelayConnection:
        verify_ssl: bool = settings.BROADCAST_WEBSOCKET_VERIFY_CERT,
    ):
        self.name = name
-        self.event_loop = asyncio.get_event_loop()
        self.stats = stats
        self.remote_host = remote_host
        self.remote_port = remote_port
@@ -110,7 +109,10 @@ class WebsocketRelayConnection:
            self.stats.record_connection_lost()

    def start(self):
-        self.async_task = self.event_loop.create_task(self.connect())
+        self.async_task = asyncio.get_running_loop().create_task(
+            self.connect(),
+            name=f"WebsocketRelayConnection.connect.{self.name}",
+        )
        return self.async_task

    def cancel(self):
@@ -121,7 +123,10 @@ class WebsocketRelayConnection:
        # metrics messages
        # the "metrics" group is not subscribed to in the typical fashion, so we
        # just explicitly create it
-        producer = self.event_loop.create_task(self.run_producer("metrics", websocket, "metrics"))
+        producer = asyncio.get_running_loop().create_task(
+            self.run_producer("metrics", websocket, "metrics"),
+            name="WebsocketRelayConnection.run_producer.metrics",
+        )
        self.producers["metrics"] = {"task": producer, "subscriptions": {"metrics"}}
        async for msg in websocket:
            self.stats.record_message_received()
@@ -143,7 +148,10 @@ class WebsocketRelayConnection:
                    name = f"{self.remote_host}-{group}"
                    origin_channel = payload['origin_channel']
                    if not self.producers.get(name):
-                        producer = self.event_loop.create_task(self.run_producer(name, websocket, group))
+                        producer = asyncio.get_running_loop().create_task(
+                            self.run_producer(name, websocket, group),
+                            name=f"WebsocketRelayConnection.run_producer.{name}",
+                        )
                        self.producers[name] = {"task": producer, "subscriptions": {origin_channel}}
                        logger.debug(f"Producer {name} started.")
                    else:
@@ -297,16 +305,15 @@ class WebSocketRelayManager(object):
            pass

    async def run(self):
-        event_loop = asyncio.get_running_loop()
-
-        self.stats_mgr = RelayWebsocketStatsManager(event_loop, self.local_hostname)
+        self.stats_mgr = RelayWebsocketStatsManager(self.local_hostname)
        self.stats_mgr.start()

        database_conf = deepcopy(settings.DATABASES['default'])
        database_conf['OPTIONS'] = deepcopy(database_conf.get('OPTIONS', {}))

        for k, v in settings.LISTENER_DATABASES.get('default', {}).items():
-            database_conf[k] = v
+            if k != 'OPTIONS':
+                database_conf[k] = v
        for k, v in settings.LISTENER_DATABASES.get('default', {}).get('OPTIONS', {}).items():
            database_conf['OPTIONS'][k] = v

@@ -322,7 +329,10 @@ class WebSocketRelayManager(object):
        )

        await async_conn.set_autocommit(True)
-        on_ws_heartbeat_task = event_loop.create_task(self.on_ws_heartbeat(async_conn))
+        on_ws_heartbeat_task = asyncio.get_running_loop().create_task(
+            self.on_ws_heartbeat(async_conn),
+            name="WebSocketRelayManager.on_ws_heartbeat",
+        )

        # Establishes a websocket connection to /websocket/relay on all API servers
        while True:
--- a/awx/settings/defaults.py
+++ b/awx/settings/defaults.py
@@ -262,6 +262,7 @@ START_TASK_LIMIT = 100
 # We have the grace period so the task manager can bail out before the timeout.
 TASK_MANAGER_TIMEOUT = 300
 TASK_MANAGER_TIMEOUT_GRACE_PERIOD = 60
+TASK_MANAGER_LOCK_TIMEOUT = TASK_MANAGER_TIMEOUT + TASK_MANAGER_TIMEOUT_GRACE_PERIOD

 # Number of seconds _in addition to_ the task manager timeout a job can stay
 # in waiting without being reaped
@@ -827,7 +828,7 @@ MANAGE_ORGANIZATION_AUTH = True
 DISABLE_LOCAL_AUTH = False

 # Note: This setting may be overridden by database settings.
-TOWER_URL_BASE = "https://towerhost"
+TOWER_URL_BASE = "https://platformhost"

 INSIGHTS_URL_BASE = "https://example.org"
 INSIGHTS_AGENT_MIME = 'application/example'
@@ -1008,6 +1009,7 @@ AWX_RUNNER_KEEPALIVE_SECONDS = 0

 # Delete completed work units in receptor
 RECEPTOR_RELEASE_WORK = True
+RECPETOR_KEEP_WORK_ON_ERROR = False

 # K8S only. Use receptor_log_level on AWX spec to set this properly
 RECEPTOR_LOG_LEVEL = 'info'
--- a/awx/templates/rest_framework/api.html
+++ b/awx/templates/rest_framework/api.html
@@ -64,7 +64,7 @@
      <div class="col-sm-6">
      </div>
      <div class="col-sm-6 footer-copyright">
-        Copyright &copy; 2021 <a href="http://www.redhat.com" target="_blank">Red Hat</a>, Inc. All Rights Reserved.
+        Copyright &copy; 2024 <a href="http://www.redhat.com" target="_blank">Red Hat</a>, Inc. All Rights Reserved.
      </div>
    </div>
  </div>
--- a/awx/ui/src/screens/ActivityStream/ActivityStream.js
+++ b/awx/ui/src/screens/ActivityStream/ActivityStream.js
@@ -59,7 +59,7 @@ function ActivityStream() {
    {
      page: 1,
      page_size: 20,
-      order_by: '-timestamp',
+      order_by: '-id',
    },
    ['id', 'page', 'page_size']
  );
--- a/awx/ui/src/screens/Host/data.hostFacts.json
+++ b/awx/ui/src/screens/Host/data.hostFacts.json
@@ -89,7 +89,7 @@
      "LC_ALL": "en_US.UTF-8",
      "MFLAGS": "-w",
      "OLDPWD": "/awx_devel",
-      "AWX_HOST": "https://towerhost",
+      "AWX_HOST": "https://platformhost",
      "HOSTNAME": "awx",
      "LANGUAGE": "en_US:en",
      "SDB_HOST": "0.0.0.0",
--- a/awx/ui/src/screens/Inventory/shared/data.hostFacts.json
+++ b/awx/ui/src/screens/Inventory/shared/data.hostFacts.json
@@ -89,7 +89,7 @@
      "LC_ALL": "en_US.UTF-8",
      "MFLAGS": "-w",
      "OLDPWD": "/awx_devel",
-      "AWX_HOST": "https://towerhost",
+      "AWX_HOST": "https://platformhost",
      "HOSTNAME": "awx",
      "LANGUAGE": "en_US:en",
      "SDB_HOST": "0.0.0.0",
--- a/awx/ui/src/screens/Job/shared/data.job.json
+++ b/awx/ui/src/screens/Job/shared/data.job.json
@@ -164,7 +164,7 @@
      "ANSIBLE_RETRY_FILES_ENABLED": "False",
      "MAX_EVENT_RES": "700000",
      "ANSIBLE_CALLBACK_PLUGINS": "/awx_devel/awx/plugins/callback",
-      "AWX_HOST": "https://towerhost",
+      "AWX_HOST": "https://platformhost",
      "ANSIBLE_SSH_CONTROL_PATH_DIR": "/tmp/awx_2_a4b1afiw/cp",
      "ANSIBLE_STDOUT_CALLBACK": "awx_display"
  },
--- a/awx/ui/src/screens/Setting/AzureAD/AzureAD.test.js
+++ b/awx/ui/src/screens/Setting/AzureAD/AzureAD.test.js
@@ -16,7 +16,7 @@ describe('<AzureAD />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_AZUREAD_OAUTH2_CALLBACK_URL:
-          'https://towerhost/sso/complete/azuread-oauth2/',
+          'https://platformhost/sso/complete/azuread-oauth2/',
        SOCIAL_AUTH_AZUREAD_OAUTH2_KEY: 'mock key',
        SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET: '$encrypted$',
        SOCIAL_AUTH_AZUREAD_OAUTH2_ORGANIZATION_MAP: {},
--- a/awx/ui/src/screens/Setting/AzureAD/AzureADDetail/AzureADDetail.test.js
+++ b/awx/ui/src/screens/Setting/AzureAD/AzureADDetail/AzureADDetail.test.js
@@ -22,7 +22,7 @@ describe('<AzureADDetail />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_AZUREAD_OAUTH2_CALLBACK_URL:
-          'https://towerhost/sso/complete/azuread-oauth2/',
+          'https://platformhost/sso/complete/azuread-oauth2/',
        SOCIAL_AUTH_AZUREAD_OAUTH2_KEY: 'mock key',
        SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET: '$encrypted$',
        SOCIAL_AUTH_AZUREAD_OAUTH2_ORGANIZATION_MAP: {},
@@ -62,7 +62,7 @@ describe('<AzureADDetail />', () => {
    assertDetail(
      wrapper,
      'Azure AD OAuth2 Callback URL',
-      'https://towerhost/sso/complete/azuread-oauth2/'
+      'https://platformhost/sso/complete/azuread-oauth2/'
    );
    assertDetail(wrapper, 'Azure AD OAuth2 Key', 'mock key');
    assertDetail(wrapper, 'Azure AD OAuth2 Secret', 'Encrypted');
--- a/awx/ui/src/screens/Setting/AzureAD/AzureADEdit/AzureADEdit.test.js
+++ b/awx/ui/src/screens/Setting/AzureAD/AzureADEdit/AzureADEdit.test.js
@@ -22,7 +22,7 @@ describe('<AzureADEdit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_AZUREAD_OAUTH2_CALLBACK_URL:
-          'https://towerhost/sso/complete/azuread-oauth2/',
+          'https://platformhost/sso/complete/azuread-oauth2/',
        SOCIAL_AUTH_AZUREAD_OAUTH2_KEY: 'mock key',
        SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET: '$encrypted$',
        SOCIAL_AUTH_AZUREAD_OAUTH2_ORGANIZATION_MAP: {},
--- a/awx/ui/src/screens/Setting/GitHub/GitHub.test.js
+++ b/awx/ui/src/screens/Setting/GitHub/GitHub.test.js
@@ -19,7 +19,7 @@ describe('<GitHub />', () => {
    SettingsAPI.readCategory.mockResolvedValueOnce({
      data: {
        SOCIAL_AUTH_GITHUB_CALLBACK_URL:
-          'https://towerhost/sso/complete/github/',
+          'https://platformhost/sso/complete/github/',
        SOCIAL_AUTH_GITHUB_KEY: 'mock github key',
        SOCIAL_AUTH_GITHUB_SECRET: '$encrypted$',
        SOCIAL_AUTH_GITHUB_ORGANIZATION_MAP: null,
@@ -29,7 +29,7 @@ describe('<GitHub />', () => {
    SettingsAPI.readCategory.mockResolvedValueOnce({
      data: {
        SOCIAL_AUTH_GITHUB_ORG_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-org/',
+          'https://platformhost/sso/complete/github-org/',
        SOCIAL_AUTH_GITHUB_ORG_KEY: '',
        SOCIAL_AUTH_GITHUB_ORG_SECRET: '$encrypted$',
        SOCIAL_AUTH_GITHUB_ORG_NAME: '',
@@ -40,7 +40,7 @@ describe('<GitHub />', () => {
    SettingsAPI.readCategory.mockResolvedValueOnce({
      data: {
        SOCIAL_AUTH_GITHUB_TEAM_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-team/',
+          'https://platformhost/sso/complete/github-team/',
        SOCIAL_AUTH_GITHUB_TEAM_KEY: 'OAuth2 key (Client ID)',
        SOCIAL_AUTH_GITHUB_TEAM_SECRET: '$encrypted$',
        SOCIAL_AUTH_GITHUB_TEAM_ID: 'team_id',
@@ -51,7 +51,7 @@ describe('<GitHub />', () => {
    SettingsAPI.readCategory.mockResolvedValueOnce({
      data: {
        SOCIAL_AUTH_GITHUB_ENTERPRISE_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-enterprise/',
+          'https://platformhost/sso/complete/github-enterprise/',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_URL: 'https://localhost/url',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_API_URL: 'https://localhost/apiurl',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_KEY: 'ent_key',
@@ -63,7 +63,7 @@ describe('<GitHub />', () => {
    SettingsAPI.readCategory.mockResolvedValueOnce({
      data: {
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-enterprise-org/',
+          'https://platformhost/sso/complete/github-enterprise-org/',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_URL: 'https://localhost/url',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_API_URL: 'https://localhost/apiurl',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_KEY: 'ent_org_key',
@@ -76,7 +76,7 @@ describe('<GitHub />', () => {
    SettingsAPI.readCategory.mockResolvedValueOnce({
      data: {
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-enterprise-team/',
+          'https://platformhost/sso/complete/github-enterprise-team/',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_URL: 'https://localhost/url',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_API_URL: 'https://localhost/apiurl',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_KEY: 'ent_team_key',
--- a/awx/ui/src/screens/Setting/GitHub/GitHubDetail/GitHubDetail.test.js
+++ b/awx/ui/src/screens/Setting/GitHub/GitHubDetail/GitHubDetail.test.js
@@ -22,7 +22,8 @@ jest.mock('../../../../api');

 const mockDefault = {
  data: {
-    SOCIAL_AUTH_GITHUB_CALLBACK_URL: 'https://towerhost/sso/complete/github/',
+    SOCIAL_AUTH_GITHUB_CALLBACK_URL:
+      'https://platformhost/sso/complete/github/',
    SOCIAL_AUTH_GITHUB_KEY: 'mock github key',
    SOCIAL_AUTH_GITHUB_SECRET: '$encrypted$',
    SOCIAL_AUTH_GITHUB_ORGANIZATION_MAP: null,
@@ -32,7 +33,7 @@ const mockDefault = {
 const mockOrg = {
  data: {
    SOCIAL_AUTH_GITHUB_ORG_CALLBACK_URL:
-      'https://towerhost/sso/complete/github-org/',
+      'https://platformhost/sso/complete/github-org/',
    SOCIAL_AUTH_GITHUB_ORG_KEY: '',
    SOCIAL_AUTH_GITHUB_ORG_SECRET: '$encrypted$',
    SOCIAL_AUTH_GITHUB_ORG_NAME: '',
@@ -43,7 +44,7 @@ const mockOrg = {
 const mockTeam = {
  data: {
    SOCIAL_AUTH_GITHUB_TEAM_CALLBACK_URL:
-      'https://towerhost/sso/complete/github-team/',
+      'https://platformhost/sso/complete/github-team/',
    SOCIAL_AUTH_GITHUB_TEAM_KEY: 'OAuth2 key (Client ID)',
    SOCIAL_AUTH_GITHUB_TEAM_SECRET: '$encrypted$',
    SOCIAL_AUTH_GITHUB_TEAM_ID: 'team_id',
@@ -54,7 +55,7 @@ const mockTeam = {
 const mockEnterprise = {
  data: {
    SOCIAL_AUTH_GITHUB_ENTERPRISE_CALLBACK_URL:
-      'https://towerhost/sso/complete/github-enterprise/',
+      'https://platformhost/sso/complete/github-enterprise/',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_URL: 'https://localhost/enterpriseurl',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_API_URL: 'https://localhost/enterpriseapi',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_KEY: 'foobar',
@@ -66,7 +67,7 @@ const mockEnterprise = {
 const mockEnterpriseOrg = {
  data: {
    SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_CALLBACK_URL:
-      'https://towerhost/sso/complete/github-enterprise-org/',
+      'https://platformhost/sso/complete/github-enterprise-org/',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_URL: 'https://localhost/orgurl',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_API_URL: 'https://localhost/orgapi',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_KEY: 'foobar',
@@ -79,7 +80,7 @@ const mockEnterpriseOrg = {
 const mockEnterpriseTeam = {
  data: {
    SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_CALLBACK_URL:
-      'https://towerhost/sso/complete/github-enterprise-team/',
+      'https://platformhost/sso/complete/github-enterprise-team/',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_URL: 'https://localhost/teamurl',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_API_URL: 'https://localhost/teamapi',
    SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_KEY: 'foobar',
@@ -143,7 +144,7 @@ describe('<GitHubDetail />', () => {
      assertDetail(
        wrapper,
        'GitHub OAuth2 Callback URL',
-        'https://towerhost/sso/complete/github/'
+        'https://platformhost/sso/complete/github/'
      );
      assertDetail(wrapper, 'GitHub OAuth2 Key', 'mock github key');
      assertDetail(wrapper, 'GitHub OAuth2 Secret', 'Encrypted');
@@ -218,7 +219,7 @@ describe('<GitHubDetail />', () => {
      assertDetail(
        wrapper,
        'GitHub Organization OAuth2 Callback URL',
-        'https://towerhost/sso/complete/github-org/'
+        'https://platformhost/sso/complete/github-org/'
      );
      assertDetail(wrapper, 'GitHub Organization OAuth2 Key', 'Not configured');
      assertDetail(wrapper, 'GitHub Organization OAuth2 Secret', 'Encrypted');
@@ -269,7 +270,7 @@ describe('<GitHubDetail />', () => {
      assertDetail(
        wrapper,
        'GitHub Team OAuth2 Callback URL',
-        'https://towerhost/sso/complete/github-team/'
+        'https://platformhost/sso/complete/github-team/'
      );
      assertDetail(wrapper, 'GitHub Team OAuth2 Key', 'OAuth2 key (Client ID)');
      assertDetail(wrapper, 'GitHub Team OAuth2 Secret', 'Encrypted');
@@ -316,7 +317,7 @@ describe('<GitHubDetail />', () => {
      assertDetail(
        wrapper,
        'GitHub Enterprise OAuth2 Callback URL',
-        'https://towerhost/sso/complete/github-enterprise/'
+        'https://platformhost/sso/complete/github-enterprise/'
      );
      assertDetail(
        wrapper,
@@ -343,7 +344,7 @@ describe('<GitHubDetail />', () => {
    });
  });

-  describe('Enterprise Org', () => {
+  describe('Enterprise Organization', () => {
    let wrapper;

    beforeAll(async () => {
@@ -376,7 +377,7 @@ describe('<GitHubDetail />', () => {
      assertDetail(
        wrapper,
        'GitHub Enterprise Organization OAuth2 Callback URL',
-        'https://towerhost/sso/complete/github-enterprise-org/'
+        'https://platformhost/sso/complete/github-enterprise-org/'
      );
      assertDetail(
        wrapper,
@@ -445,7 +446,7 @@ describe('<GitHubDetail />', () => {
      assertDetail(
        wrapper,
        'GitHub Enterprise Team OAuth2 Callback URL',
-        'https://towerhost/sso/complete/github-enterprise-team/'
+        'https://platformhost/sso/complete/github-enterprise-team/'
      );
      assertDetail(
        wrapper,
@@ -476,23 +477,4 @@ describe('<GitHubDetail />', () => {
      );
    });
  });
-
-  describe('Redirect', () => {
-    test('should render redirect when user navigates to erroneous category', async () => {
-      let wrapper;
-      useRouteMatch.mockImplementation(() => ({
-        url: '/settings/github/foo/details',
-        path: '/settings/github/:category/details',
-        params: { category: 'foo' },
-      }));
-      await act(async () => {
-        wrapper = mountWithContexts(
-          <SettingsProvider value={mockAllOptions.actions}>
-            <GitHubDetail />
-          </SettingsProvider>
-        );
-      });
-      await waitForElement(wrapper, 'Redirect');
-    });
-  });
 });
--- a/awx/ui/src/screens/Setting/GitHub/GitHubEnterpriseEdit/GitHubEnterpriseEdit.test.js
+++ b/awx/ui/src/screens/Setting/GitHub/GitHubEnterpriseEdit/GitHubEnterpriseEdit.test.js
@@ -22,7 +22,7 @@ describe('<GitHubEnterpriseEdit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GITHUB_ENTERPRISE_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-enterprise/',
+          'https://platformhost/sso/complete/github-enterprise/',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_URL: '',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_API_URL: '',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_KEY: '',
--- a/awx/ui/src/screens/Setting/GitHub/GitHubEnterpriseOrgEdit/GitHubEnterpriseOrgEdit.test.js
+++ b/awx/ui/src/screens/Setting/GitHub/GitHubEnterpriseOrgEdit/GitHubEnterpriseOrgEdit.test.js
@@ -22,7 +22,7 @@ describe('<GitHubEnterpriseOrgEdit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-enterprise-org/',
+          'https://platformhost/sso/complete/github-enterprise-org/',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_URL: '',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_API_URL: '',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_ORG_KEY: '',
--- a/awx/ui/src/screens/Setting/GitHub/GitHubEnterpriseTeamEdit/GitHubEnterpriseTeamEdit.test.js
+++ b/awx/ui/src/screens/Setting/GitHub/GitHubEnterpriseTeamEdit/GitHubEnterpriseTeamEdit.test.js
@@ -22,7 +22,7 @@ describe('<GitHubEnterpriseTeamEdit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-enterprise-team/',
+          'https://platformhost/sso/complete/github-enterprise-team/',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_URL: '',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_API_URL: '',
        SOCIAL_AUTH_GITHUB_ENTERPRISE_TEAM_KEY: '',
--- a/awx/ui/src/screens/Setting/GitHub/GitHubOrgEdit/GitHubOrgEdit.test.js
+++ b/awx/ui/src/screens/Setting/GitHub/GitHubOrgEdit/GitHubOrgEdit.test.js
@@ -22,7 +22,7 @@ describe('<GitHubOrgEdit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GITHUB_ORG_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-org/',
+          'https://platformhost/sso/complete/github-org/',
        SOCIAL_AUTH_GITHUB_ORG_KEY: '',
        SOCIAL_AUTH_GITHUB_ORG_SECRET: '$encrypted$',
        SOCIAL_AUTH_GITHUB_ORG_NAME: '',
--- a/awx/ui/src/screens/Setting/GitHub/GitHubTeamEdit/GitHubTeamEdit.test.js
+++ b/awx/ui/src/screens/Setting/GitHub/GitHubTeamEdit/GitHubTeamEdit.test.js
@@ -22,7 +22,7 @@ describe('<GitHubTeamEdit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GITHUB_TEAM_CALLBACK_URL:
-          'https://towerhost/sso/complete/github-team/',
+          'https://platformhost/sso/complete/github-team/',
        SOCIAL_AUTH_GITHUB_TEAM_KEY: 'OAuth2 key (Client ID)',
        SOCIAL_AUTH_GITHUB_TEAM_SECRET: '$encrypted$',
        SOCIAL_AUTH_GITHUB_TEAM_ID: 'team_id',
--- a/awx/ui/src/screens/Setting/GoogleOAuth2/GoogleOAuth2.test.js
+++ b/awx/ui/src/screens/Setting/GoogleOAuth2/GoogleOAuth2.test.js
@@ -16,7 +16,7 @@ describe('<GoogleOAuth2 />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GOOGLE_OAUTH2_CALLBACK_URL:
-          'https://towerhost/sso/complete/google-oauth2/',
+          'https://platformhost/sso/complete/google-oauth2/',
        SOCIAL_AUTH_GOOGLE_OAUTH2_KEY: 'mock key',
        SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET: '$encrypted$',
        SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS: [
--- a/awx/ui/src/screens/Setting/GoogleOAuth2/GoogleOAuth2Detail/GoogleOAuth2Detail.test.js
+++ b/awx/ui/src/screens/Setting/GoogleOAuth2/GoogleOAuth2Detail/GoogleOAuth2Detail.test.js
@@ -22,7 +22,7 @@ describe('<GoogleOAuth2Detail />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GOOGLE_OAUTH2_CALLBACK_URL:
-          'https://towerhost/sso/complete/google-oauth2/',
+          'https://platformhost/sso/complete/google-oauth2/',
        SOCIAL_AUTH_GOOGLE_OAUTH2_KEY: 'mock key',
        SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET: '$encrypted$',
        SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS: [
@@ -68,7 +68,7 @@ describe('<GoogleOAuth2Detail />', () => {
    assertDetail(
      wrapper,
      'Google OAuth2 Callback URL',
-      'https://towerhost/sso/complete/google-oauth2/'
+      'https://platformhost/sso/complete/google-oauth2/'
    );
    assertDetail(wrapper, 'Google OAuth2 Key', 'mock key');
    assertDetail(wrapper, 'Google OAuth2 Secret', 'Encrypted');
--- a/awx/ui/src/screens/Setting/GoogleOAuth2/GoogleOAuth2Edit/GoogleOAuth2Edit.test.js
+++ b/awx/ui/src/screens/Setting/GoogleOAuth2/GoogleOAuth2Edit/GoogleOAuth2Edit.test.js
@@ -22,7 +22,7 @@ describe('<GoogleOAuth2Edit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SOCIAL_AUTH_GOOGLE_OAUTH2_CALLBACK_URL:
-          'https://towerhost/sso/complete/google-oauth2/',
+          'https://platformhost/sso/complete/google-oauth2/',
        SOCIAL_AUTH_GOOGLE_OAUTH2_KEY: 'mock key',
        SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET: '$encrypted$',
        SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS: [
--- a/awx/ui/src/screens/Setting/MiscSystem/MiscSystemDetail/MiscSystemDetail.test.js
+++ b/awx/ui/src/screens/Setting/MiscSystem/MiscSystemDetail/MiscSystemDetail.test.js
@@ -26,7 +26,7 @@ describe('<MiscSystemDetail />', () => {
        ACTIVITY_STREAM_ENABLED_FOR_INVENTORY_SYNC: false,
        ORG_ADMINS_CAN_SEE_ALL_USERS: true,
        MANAGE_ORGANIZATION_AUTH: true,
-        TOWER_URL_BASE: 'https://towerhost',
+        TOWER_URL_BASE: 'https://platformhost',
        REMOTE_HOST_HEADERS: [],
        PROXY_IP_ALLOWED_LIST: [],
        CSRF_TRUSTED_ORIGINS: [],
@@ -94,7 +94,7 @@ describe('<MiscSystemDetail />', () => {
      'Automation Analytics upload URL',
      'https://example.com'
    );
-    assertDetail(wrapper, 'Base URL of the service', 'https://towerhost');
+    assertDetail(wrapper, 'Base URL of the service', 'https://platformhost');
    assertDetail(wrapper, 'Gather data for Automation Analytics', 'Off');
    assertDetail(
      wrapper,
--- a/awx/ui/src/screens/Setting/SAML/SAML.test.js
+++ b/awx/ui/src/screens/Setting/SAML/SAML.test.js
@@ -15,8 +15,10 @@ describe('<SAML />', () => {
  beforeEach(() => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
-        SOCIAL_AUTH_SAML_CALLBACK_URL: 'https://towerhost/sso/complete/saml/',
-        SOCIAL_AUTH_SAML_METADATA_URL: 'https://towerhost/sso/metadata/saml/',
+        SOCIAL_AUTH_SAML_CALLBACK_URL:
+          'https://platformhost/sso/complete/saml/',
+        SOCIAL_AUTH_SAML_METADATA_URL:
+          'https://platformhost/sso/metadata/saml/',
        SOCIAL_AUTH_SAML_SP_ENTITY_ID: '',
        SOCIAL_AUTH_SAML_SP_PUBLIC_CERT: '',
        SOCIAL_AUTH_SAML_SP_PRIVATE_KEY: '',
--- a/awx/ui/src/screens/Setting/SAML/SAMLDetail/SAMLDetail.test.js
+++ b/awx/ui/src/screens/Setting/SAML/SAMLDetail/SAMLDetail.test.js
@@ -21,8 +21,10 @@ describe('<SAMLDetail />', () => {
  beforeEach(() => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
-        SOCIAL_AUTH_SAML_CALLBACK_URL: 'https://towerhost/sso/complete/saml/',
-        SOCIAL_AUTH_SAML_METADATA_URL: 'https://towerhost/sso/metadata/saml/',
+        SOCIAL_AUTH_SAML_CALLBACK_URL:
+          'https://platformhost/sso/complete/saml/',
+        SOCIAL_AUTH_SAML_METADATA_URL:
+          'https://platformhost/sso/metadata/saml/',
        SOCIAL_AUTH_SAML_SP_ENTITY_ID: 'mock_id',
        SOCIAL_AUTH_SAML_SP_PUBLIC_CERT: 'mock_cert',
        SOCIAL_AUTH_SAML_SP_PRIVATE_KEY: '',
@@ -71,12 +73,12 @@ describe('<SAMLDetail />', () => {
    assertDetail(
      wrapper,
      'SAML Assertion Consumer Service (ACS) URL',
-      'https://towerhost/sso/complete/saml/'
+      'https://platformhost/sso/complete/saml/'
    );
    assertDetail(
      wrapper,
      'SAML Service Provider Metadata URL',
-      'https://towerhost/sso/metadata/saml/'
+      'https://platformhost/sso/metadata/saml/'
    );
    assertDetail(wrapper, 'SAML Service Provider Entity ID', 'mock_id');
    assertVariableDetail(
--- a/awx/ui/src/screens/Setting/SAML/SAMLEdit/SAMLEdit.test.js
+++ b/awx/ui/src/screens/Setting/SAML/SAMLEdit/SAMLEdit.test.js
@@ -22,8 +22,10 @@ describe('<SAMLEdit />', () => {
    SettingsAPI.readCategory.mockResolvedValue({
      data: {
        SAML_AUTO_CREATE_OBJECTS: true,
-        SOCIAL_AUTH_SAML_CALLBACK_URL: 'https://towerhost/sso/complete/saml/',
-        SOCIAL_AUTH_SAML_METADATA_URL: 'https://towerhost/sso/metadata/saml/',
+        SOCIAL_AUTH_SAML_CALLBACK_URL:
+          'https://platformhost/sso/complete/saml/',
+        SOCIAL_AUTH_SAML_METADATA_URL:
+          'https://platformhost/sso/metadata/saml/',
        SOCIAL_AUTH_SAML_SP_ENTITY_ID: 'mock_id',
        SOCIAL_AUTH_SAML_SP_PUBLIC_CERT: 'mock_cert',
        SOCIAL_AUTH_SAML_SP_PRIVATE_KEY: '$encrypted$',
--- a/awx/ui/src/screens/Setting/Troubleshooting/TroubleshootingEdit/TroubleshootingEdit.js
+++ b/awx/ui/src/screens/Setting/Troubleshooting/TroubleshootingEdit/TroubleshootingEdit.js
@@ -117,6 +117,10 @@ function TroubleshootingEdit() {
                  name="RECEPTOR_RELEASE_WORK"
                  config={debug.RECEPTOR_RELEASE_WORK}
                />
+                <BooleanField
+                  name="RECEPTOR_KEEP_WORK_ON_ERROR"
+                  config={debug.RECEPTOR_KEEP_WORK_ON_ERROR}
+                />
                {submitError && <FormSubmitError error={submitError} />}
                {revertError && <FormSubmitError error={revertError} />}
              </FormColumnLayout>
--- a/awx/ui/src/screens/Setting/Troubleshooting/TroubleshootingEdit/data.defaultTroubleshootingSettings.json
+++ b/awx/ui/src/screens/Setting/Troubleshooting/TroubleshootingEdit/data.defaultTroubleshootingSettings.json
@@ -1,5 +1,6 @@
 {
  "AWX_CLEANUP_PATHS": false,
  "AWX_REQUEST_PROFILE": false,
-  "RECEPTOR_RELEASE_WORK": false
-}
+  "RECEPTOR_RELEASE_WORK": false,
+  "RECEPTOR_KEEP_WORK_ON_ERROR": false
+}
--- a/awx/ui/src/screens/Setting/shared/data.allSettingOptions.json
+++ b/awx/ui/src/screens/Setting/shared/data.allSettingOptions.json
@@ -830,6 +830,15 @@
              "category_slug": "debug",
              "default": true
          },
+          "RECEPTOR_KEEP_WORK_ON_ERROR": {
+            "type": "boolean",
+            "required": false,
+            "label": "Keep receptor work on error",
+            "help_text": "Prevent receptor work from being released on when error is detected",
+            "category": "Debug",
+            "category_slug": "debug",
+            "default": false
+          },
          "SESSION_COOKIE_AGE": {
              "type": "integer",
              "required": true,
@@ -5173,6 +5182,14 @@
              "category_slug": "debug",
              "defined_in_file": false
          },
+          "RECEPTOR_KEEP_WORK_ON_ERROR": {
+              "type": "boolean",
+              "label": "Keep receptor work on error",
+              "help_text": "Prevent receptor work from being released on when error is detected",
+              "category": "Debug",
+              "category_slug": "debug",
+              "defined_in_file": false
+          },
          "SESSION_COOKIE_AGE": {
              "type": "integer",
              "label": "Idle Time Force Log Out",
--- a/awx/ui/src/screens/Setting/shared/data.allSettings.json
+++ b/awx/ui/src/screens/Setting/shared/data.allSettings.json
@@ -91,6 +91,7 @@
      "slirp4netns:enable_ipv6=true"
  ],
  "RECEPTOR_RELEASE_WORK": true,
+  "RECEPTOR_KEEP_WORK_ON_ERROR": false,
  "SESSION_COOKIE_AGE": 1800,
  "SESSIONS_PER_USER": -1,
  "DISABLE_LOCAL_AUTH": false,
--- a/awx/ui_next/Makefile
+++ b/awx/ui_next/Makefile
@@ -35,7 +35,7 @@ ui-next/src/build: $(UI_NEXT_DIR)/src/build/awx
 ## True target for ui-next/src/build. Build ui_next from source.
 $(UI_NEXT_DIR)/src/build/awx: $(UI_NEXT_DIR)/src $(UI_NEXT_DIR)/src/node_modules/webpack
 	@echo "=== Building ui_next ==="
-	@cd $(UI_NEXT_DIR)/src && PRODUCT="$(PRODUCT)" PUBLIC_PATH=/static/awx/ ROUTE_PREFIX=/ui_next npm run build:awx
+	@cd $(UI_NEXT_DIR)/src && PRODUCT="$(PRODUCT)" PUBLIC_PATH=/static/awx/ ROUTE_PREFIX=/ npm run build:awx
 	@mv $(UI_NEXT_DIR)/src/build/awx/index.html $(UI_NEXT_DIR)/src/build/awx/index_awx.html

 .PHONY: ui-next/src
--- a/awx/ui_next/urls.py
+++ b/awx/ui_next/urls.py
@@ -1,5 +1,3 @@
-from django.conf import settings
-from django.http import Http404
 from django.urls import re_path
 from django.views.generic.base import TemplateView

@@ -7,12 +5,6 @@ from django.views.generic.base import TemplateView
 class IndexView(TemplateView):
    template_name = 'index_awx.html'

-    def get_context_data(self, **kwargs):
-        if settings.UI_NEXT is False:
-            raise Http404()
-
-        return super().get_context_data(**kwargs)
-

 app_name = 'ui_next'

--- a/awx/urls.py
+++ b/awx/urls.py
@@ -18,8 +18,6 @@ def get_urlpatterns(prefix=None):
        prefix = f'/{prefix}/'

    urlpatterns = [
-        re_path(r'', include('awx.ui.urls', namespace='ui')),
-        re_path(r'^ui_next/.*', include('awx.ui_next.urls', namespace='ui_next')),
        path(f'api{prefix}', include('awx.api.urls', namespace='api')),
    ]

@@ -36,6 +34,9 @@ def get_urlpatterns(prefix=None):
        re_path(r'^(?:api/)?500.html$', handle_500),
        re_path(r'^csp-violation/', handle_csp_violation),
        re_path(r'^login/', handle_login_redirect),
+        # want api/v2/doesnotexist to return a 404, not match the ui_next urls,
+        # so use a negative lookahead assertion here
+        re_path(r'^(?!api/|sso/).*', include('awx.ui_next.urls', namespace='ui_next')),
    ]

    if settings.SETTINGS_MODULE == 'awx.settings.development':
--- a/awx_collection/README.md
+++ b/awx_collection/README.md
@@ -32,7 +32,7 @@ Installing the `tar.gz` involves no special instructions.
 ## Running

 Non-deprecated modules in this collection have no Python requirements, but
-may require the official [AWX CLI](https://docs.ansible.com/ansible-tower/latest/html/towercli/index.html)
+may require the official [AWX CLI](https://pypi.org/project/awxkit/)
 in the future. The `DOCUMENTATION` for each module will report this.

 You can specify authentication by a combination of either:
@@ -41,8 +41,7 @@ You can specify authentication by a combination of either:
 - host, OAuth2 token

 The OAuth2 token is the preferred method. You can obtain a token via the
-AWX CLI [login](https://docs.ansible.com/ansible-tower/latest/html/towercli/reference.html#awx-login)
-command.
+``login`` command with the AWX CLI.

 These can be specified via (from highest to lowest precedence):

--- a/awx_collection/plugins/lookup/controller_api.py
+++ b/awx_collection/plugins/lookup/controller_api.py
@@ -12,7 +12,7 @@ requirements:
  - None
 description:
  - Returns GET requests from the Automation Platform Controller API. See
-    U(https://docs.ansible.com/ansible-tower/latest/html/towerapi/index.html) for API usage.
+    U(https://docs.ansible.com/automation-controller/latest/html/towerapi/) for API usage.
  - For use that is cross-compatible between the awx.awx and ansible.controller collection
    see the controller_meta module
 options:
--- a/awx_collection/plugins/modules/job_template.py
+++ b/awx_collection/plugins/modules/job_template.py
@@ -16,9 +16,9 @@ DOCUMENTATION = '''
 ---
 module: job_template
 author: "Wayne Witzel III (@wwitzel3)"
-short_description: create, update, or destroy Automation Platform Controller job templates.
+short_description: create, update, or destroy job templates.
 description:
-    - Create, update, or destroy Automation Platform Controller job templates. See
+    - Create, update, or destroy job templates. See
      U(https://www.ansible.com/tower) for an overview.
 options:
    name:
@@ -320,8 +320,8 @@ extends_documentation_fragment: awx.awx.auth

 notes:
  - JSON for survey_spec can be found in the API Documentation. See
-    U(https://docs.ansible.com/ansible-tower/latest/html/towerapi/api_ref.html#/Job_Templates/Job_Templates_job_templates_survey_spec_create)
-    for POST operation payload example.
+    U(https://docs.ansible.com/automation-controller/latest/html/towerapi)
+    for job template survey creation and POST operation payload example.
 '''


--- a/awx_collection/tools/roles/template_galaxy/templates/README.md.j2
+++ b/awx_collection/tools/roles/template_galaxy/templates/README.md.j2
@@ -37,7 +37,7 @@ This collection should be installed from [Content Hub](https://cloud.redhat.com/
 ## Running

 Non-deprecated modules in this collection have no Python requirements, but
-may require the official [AWX CLI](https://docs.ansible.com/ansible-tower/latest/html/towercli/index.html)
+may require the AWX CLI
 in the future. The `DOCUMENTATION` for each module will report this.

 You can specify authentication by a combination of either:
@@ -46,8 +46,7 @@ You can specify authentication by a combination of either:
 - host, OAuth2 token

 The OAuth2 token is the preferred method. You can obtain a token via the
-AWX CLI [login](https://docs.ansible.com/ansible-tower/latest/html/towercli/reference.html#awx-login)
-command.
+``login`` command with the AWX CLI.

 These can be specified via (from highest to lowest precedence):

--- a/docs/auth/ldap.md
+++ b/docs/auth/ldap.md
@@ -4,7 +4,7 @@ The Lightweight Directory Access Protocol (LDAP) is an open, vendor-neutral, ind

 # Configure LDAP Authentication

-Please see the [Tower documentation](https://docs.ansible.com/ansible-tower/latest/html/administration/ldap_auth.html) as well as [Ansible blog post](https://www.ansible.com/blog/getting-started-ldap-authentication-in-ansible-tower) for basic LDAP configuration.
+Please see the [AWX documentation](https://ansible.readthedocs.io/projects/awx/en/latest/administration/ldap_auth.html) for basic LDAP configuration.

 LDAP Authentication provides duplicate sets of configuration fields for authentication with up to six different LDAP servers.
 The default set of configuration fields take the form `AUTH_LDAP_<field name>`. Configuration fields for additional LDAP servers are numbered `AUTH_LDAP_<n>_<field name>`.
--- a/docs/auth/saml.md
+++ b/docs/auth/saml.md
@@ -3,7 +3,7 @@ Security Assertion Markup Language, or SAML, is an open standard for exchanging


 # Configure SAML Authentication
-Please see the [Tower documentation](https://docs.ansible.com/ansible-tower/latest/html/administration/ent_auth.html#saml-authentication-settings) as well as the [Ansible blog post](https://www.ansible.com/blog/using-saml-with-red-hat-ansible-tower) for basic SAML configuration. Note that AWX's SAML implementation relies on `python-social-auth` which uses `python-saml`. AWX exposes three fields which are directly passed to the lower libraries:
+Please see the [AWX documentation](https://ansible.readthedocs.io/projects/awx/en/latest/administration/ent_auth.html#saml-settings) for basic SAML configuration. Note that AWX's SAML implementation relies on `python-social-auth` which uses `python-saml`. AWX exposes three fields which are directly passed to the lower libraries:
 * `SOCIAL_AUTH_SAML_SP_EXTRA` is passed to the `python-saml` library configuration's `sp` setting.  
 * `SOCIAL_AUTH_SAML_SECURITY_CONFIG` is passed to the `python-saml` library configuration's `security` setting.
 * `SOCIAL_AUTH_SAML_EXTRA_DATA`
--- a/docs/docsite/conf.py
+++ b/docs/docsite/conf.py
@@ -71,8 +71,8 @@ rst_epilog = """
 .. |aap| replace:: Ansible Automation Platform
 .. |ab| replace:: ansible-builder
 .. |ap| replace:: Automation Platform
-.. |at| replace:: automation controller
-.. |At| replace:: Automation controller
+.. |at| replace:: AWX
+.. |At| replace:: AWX
 .. |ah| replace:: Automation Hub
 .. |EE| replace:: Execution Environment
 .. |EEs| replace:: Execution Environments
--- a/docs/docsite/rst/administration/configure_awx.rst
+++ b/docs/docsite/rst/administration/configure_awx.rst
@@ -23,7 +23,6 @@ Authentication
 .. index::
    single: social authentication
    single: authentication
-    single: enterprise authentication
    pair: configuration; authentication

 .. include:: ./configure_awx_authentication.rst
--- a/docs/docsite/rst/administration/containers_instance_groups.rst
+++ b/docs/docsite/rst/administration/containers_instance_groups.rst
@@ -300,13 +300,10 @@ Container Groups
   single: container groups
   pair: containers; instance groups

-AWX supports :term:`Container Groups`, which allow you to execute jobs in AWX regardless of whether AWX is installed as a standalone, in  a virtual environment, or in a container. Container groups act as a pool of resources within a virtual environment. You can create instance groups to point to an OpenShift container, which are job environments that are provisioned on-demand as a Pod that exists only for the duration of the playbook run. This is known as the ephemeral execution model and ensures a clean environment for every job run.
+AWX supports :term:`Container Groups`, which allow you to execute jobs in pods on Kubernetes (k8s) or OpenShift clusters. Container groups act as a pool of resources within a virtual environment. These pods are created on-demand and only exist for the duration of the playbook run. This is known as the ephemeral execution model and ensures a clean environment for every job run.

 In some cases, it is desirable to have container groups be "always-on", which is configured through the creation of an instance.

-.. note::
-
-	Container Groups upgraded from versions prior to |at| 4.0 will revert back to default and completely remove the old pod definition, clearing out all custom pod definitions in the migration.


 Container groups are different from |ees| in that |ees| are container images and do not use a virtual environment. See :ref:`ug_execution_environments` in the |atu| for further detail.
@@ -335,19 +332,19 @@ To create a container group:

 .. _ag_customize_pod_spec:

-Customize the Pod spec
+Customize the pod spec
 ^^^^^^^^^^^^^^^^^^^^^^^^

-AWX provides a simple default Pod specification, however, you can provide a custom YAML (or JSON) document that overrides the default Pod spec. This field uses any custom fields (i.e. ``ImagePullSecrets``) that can be "serialized" as valid Pod JSON or YAML. A full list of options can be found in the `OpenShift documentation <https://docs.openshift.com/online/pro/architecture/core_concepts/pods_and_services.html>`_.
+AWX provides a simple default pod specification, however, you can provide a custom YAML (or JSON) document that overrides the default pod spec. This field uses any custom fields (for example, ``ImagePullSecrets``) that can be "serialized" as valid pod JSON or YAML. A full list of options can be found in the `OpenShift documentation <https://docs.openshift.com/online/pro/architecture/core_concepts/pods_and_services.html>`_.

-To customize the Pod spec, specify the namespace in the **Pod Spec Override** field by using the toggle to enable and expand the **Pod Spec Override** field and click **Save** when done.
+To customize the pod spec, check the **Customize pod specification** option to enable and expand the **Custom pod spec** field where you specify the namespace and provide additional customizations as needed.

 |IG - CG customize pod|

 .. |IG - CG customize pod| image:: ../common/images/instance-group-customize-cg-pod.png
 		:alt: Create new container group form with the option to custom the pod spec.

-You may provide additional customizations, if needed. Click **Expand** to view the entire customization window.
+Click **Expand** to view the entire customization window.

 .. image:: ../common/images/instance-group-customize-cg-pod-expanded.png
 	:alt: The expanded view for customizing the pod spec.
@@ -356,6 +353,21 @@ You may provide additional customizations, if needed. Click **Expand** to view t

 	The image used at job launch time is determined by which |ee| is associated with the job. If a Container Registry credential is associated with the |ee|, then AWX will attempt to make a ``ImagePullSecret`` to pull the image. If you prefer not to give the service account permission to manage secrets, you must pre-create the ``ImagePullSecret`` and specify it on the pod spec, and omit any credential from the |ee| used.

+.. tip::
+
+	In order to override DNS/host entries, use the ``hostAliases`` attribute on the pod spec. When the pod is created, these entries will be added to ``/etc/hosts`` in the container running the job.
+
+	::
+
+		spec:
+		  hostAliases:
+		  - ip: "127.0.0.1"
+		    hostnames:
+		    - "foo.local"
+
+	For more information, refer to Kubernetes' documentation on `Adding additional entries with hostAliases <https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/#adding-additional-entries-with-hostaliases>`_.
+
+
 Once the container group is successfully created, the **Details** tab of the newly created container group remains, which allows you to review and edit your container group information. This is the same menu that is opened if the Edit (|edit-button|) button is clicked from the **Instance Group** link. You can also edit **Instances** and review **Jobs** associated with this instance group.

 .. |edit-button| image:: ../common/images/edit-button.png
@@ -370,7 +382,7 @@ Container groups and instance groups are labeled accordingly.

 .. note::

-	Despite the fact that customers have custom Pod specs, upgrades may be difficult if the default ``pod_spec`` changes. Most any manifest can be applied to any namespace, with the namespace specified separately, most likely you will only need to override the namespace. Similarly, pinning a default image for different releases of the platform to different versions of the default job runner container is tricky. If the default image is specified in the Pod spec, then upgrades do not pick up the new default changes are made to the default Pod spec.
+	Using a custom pod spec may cause issues on upgrades if the default ``pod_spec`` changes. Since any manifest can be applied to any namespace, with the namespace specified separately, most likely you will only need to override the namespace. Similarly, pinning a default image for different releases of the platform to different versions of the default job runner container is tricky. If the default image is specified in the pod spec, then upgrades do not pick up the new default changes that are made to the default pod spec.


 Verify container group functions
@@ -411,7 +423,7 @@ You can see in the jobs detail view the container was reached successfully using
 .. |Inventory with localhost ping success| image:: ../common/images/inventories-launch-adhoc-cg-test-localhost-success.png
 	:alt: Jobs output view showing a successfully ran adhoc job.

-If you have an OpenShift UI, you can see Pods appear and disappear as they deploy and terminate. Alternatively, you can use the CLI to perform a ``get pod`` operation on your namespace to watch these same events occurring in real-time.
+If you have an OpenShift UI, you can see pods appear and disappear as they deploy and terminate. Alternatively, you can use the CLI to perform a ``get pod`` operation on your namespace to watch these same events occurring in real-time.


 View container group jobs
--- a/docs/docsite/rst/administration/logging.rst
+++ b/docs/docsite/rst/administration/logging.rst
@@ -320,6 +320,43 @@ Items surrounded by ``{}`` will be substituted when the log error is generated.
 - **error**: The error message returned by the API or, if no error is specified, the HTTP status as text


+.. _logging-api-otel:
+
+OTel configuration with AWX
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can integrate OTel with AWX by configuring logging manually to point to your OTel collector. To do this, add the following codeblock in your `settings file <https://github.com/ansible/awx/blob/devel/tools/docker-compose/ansible/roles/sources/templates/local_settings.py.j2#L50>`_ (``local_settings.py.j2``):
+
+.. code-block:: python
+
+    LOGGING['handlers']['otel'] |= {
+      'class': 'awx.main.utils.handlers.OTLPHandler',
+      'endpoint': 'http://otel:4317',
+    }
+    # Add otel log handler to all log handlers where propagate is False
+    for name in LOGGING['loggers'].keys():
+      if not LOGGING['loggers'][name].get('propagate', True):
+        handler = LOGGING['loggers'][name].get('handlers', [])
+        if 'otel' not in handler:
+            LOGGING['loggers'][name].get('handlers', []).append('otel')
+
+    # Everything without explicit propagate=False ends up logging to 'awx' so add it
+    handler = LOGGING['loggers']['awx'].get('handlers', [])
+    if 'otel' not in handler:
+      LOGGING['loggers']['awx'].get('handlers', []).append('otel')
+
+Edit ``'endpoint': 'http://otel:4317',`` to point to your OTel collector. 
+
+To see it working in the dev environment, set the following:
+
+::
+
+  OTEL=true GRAFANA=true LOKI=true PROMETHEUS=true make docker-compose
+
+Then go to `http://localhost:3001 <http://localhost:3001>`_ to access Grafana and see the logs.
+
+
+
 Troubleshoot Logging
 ---------------------

--- a/docs/docsite/rst/administration/troubleshooting.rst
+++ b/docs/docsite/rst/administration/troubleshooting.rst
@@ -8,6 +8,21 @@ Troubleshooting AWX
   single: troubleshooting
   single: help
  
+
+Some troubleshooting tools are built in the AWX user interface that may help you address some issues you might encounter. To access these tools, navigate to **Settings** and select **Troubleshooting**.
+
+.. image:: ../common/images/settings_troubleshooting_highlighted.png
+
+The options available are:
+
+- **Enable or Disable tmp dir cleanup**: choose whether you want to clean up the ``tmp`` directory.
+- **Debug Web Requests**: choose whether you want web requests to log messages for debugging purposes.
+- **Release Receptor Work**: disables cleaning up job pods. If you disable this, the jobs pods will remain in your cluster indefinitely, allowing you to examine them post-run. If you are missing data there, run ``kubectl logs <job-pod-name>`` and provide the logs in a issue report.
+
+.. image:: ../common/images/troubleshooting_options.png
+
+Click **Edit** to modify the settings. Use the toggle to enable and disable the appropriate settings.
+
 .. _admin_troubleshooting_extra_settings:

 Error logging and extra settings
@@ -220,3 +235,4 @@ If you receive the message "Skipping: No Hosts Matched" when you are trying to r
 - Make sure that if you have specified a Limit in the Job Template that it is a valid limit value and still matches something in your inventory. The Limit field takes a pattern argument, described here: http://docs.ansible.com/intro_patterns.html

 Please file a support ticket if you still run into issues after checking these options.
+
--- a/docs/docsite/rst/common/images/inventories-create-source-ocpvirt-example.png
+++ b/docs/docsite/rst/common/images/inventories-create-source-ocpvirt-example.png
--- a/docs/docsite/rst/common/images/settings_troubleshooting_highlighted.png
+++ b/docs/docsite/rst/common/images/settings_troubleshooting_highlighted.png
--- a/docs/docsite/rst/common/images/troubleshooting_options.png
+++ b/docs/docsite/rst/common/images/troubleshooting_options.png
--- a/docs/docsite/rst/common/logos_branding.rst
+++ b/docs/docsite/rst/common/logos_branding.rst
@@ -18,7 +18,7 @@ For example, if you uploaded a specific logo, and added the following text:
 	:alt: Edit User Interface Settings form populated with custom text and logo.


-The Tower login dialog would look like this:
+The AWX login dialog would look like this:

 .. image:: ../common/images/configure-awx-ui-angry-spud-login.png
 	:alt: AWX login screen with custom text and logo.
--- a/docs/docsite/rst/contributor/work_items.rst
+++ b/docs/docsite/rst/contributor/work_items.rst
@@ -40,6 +40,4 @@ Things to know prior to submitting revisions
 Translations
 -------------

-At this time we do not accept PRs for adding additional language translations as we have an automated process for generating our translations. This is because translations require constant care as new strings are added and changed in the code base. Because of this the .po files are overwritten during every translation release cycle. We also can't support a lot of translations on AWX as its an open source project and each language adds time and cost to maintain. If you would like to see AWX translated into a new language please create an issue and ask others you know to upvote the issue. Our translation team will review the needs of the community and see what they can do around supporting additional language.
-
-If you find an issue with an existing translation, please see the `Reporting Issues <https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md#reporting-issues>`_ section to open an issue and our translation team will work with you on a resolution. 
+At this time we do not accept PRs for language translations.
--- a/docs/docsite/rst/quickstart/create_organization.rst
+++ b/docs/docsite/rst/quickstart/create_organization.rst
@@ -9,7 +9,7 @@ An organization is a logical collection of users, teams, projects, and inventori
 From the left navigation bar, click **Organizations**. 

 .. note::
-    AWX creates a default organization automatically. Users of Tower with a Self-support level license only have the 
+    AWX creates a default organization automatically.

 |Organizations - default view|

--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Elijah DeLee	c5bea2b557	add command to introspect job performance This command outputs json with information about past job event processing delays, how long jobs spent in pending, and what times of day saw more pending jobs. To aid in investigations, the raw queries are supplied alongside data.	2024-07-31 13:18:31 -04:00
TVo	6d0c47fdd0	Re-do PR #14685 for alt-text inventories. (#15394 )	2024-07-31 09:40:28 +01:00
TVo	54b4acbdfc	Added docs for OTel - awx integration (#15408 )	2024-07-29 14:36:22 -06:00
Seth Foster	a41766090e	Make ui_next the default UI (#15405 ) Change django url dispatcher to serve up ui_next files instead of old ui files Old UI will not be served with this change Github CI still runs old ui tests (to be removed in another PR) Remove the Github workflows that build old UI --------- Signed-off-by: Seth Foster <fosterbseth@gmail.com>	2024-07-29 15:13:09 -04:00
github-actions[bot]	34fa897dda	Bump django-ansible-base to 2024.7.17 (#15373 ) Update django-ansible-base version to devel Co-authored-by: chrismeyersfsu <722880+chrismeyersfsu@users.noreply.github.com>	2024-07-26 09:11:12 -04:00
Hao Liu	32df114e41	Improve asyncio debugging (#15398 ) - use asyncio.get_running_loop() instead of passing around event_loops - add name to all of the asyncio tasks for easier debugging we are trying to figure out which task is ``` Task was destroyed but it is pending! task: <Task pending name='Task-<id>' coro=<RedisConnection._read_data() done, defined at /var/lib/awx/venv/awx/lib64/python3.9/site-packages/aioredis/connection.py:180> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fba77bf1700>()]> cb=[RedisConnection.__init__.<locals>.<lambda>() at /var/lib/awx/venv/awx/lib64/python3.9/site-packages/aioredis/connection.py:168]> ``` is referring to	2024-07-24 12:05:02 -04:00
TVo	018f235a64	Replaced all references of downstream docs to upstream docs (#15388 ) * Replaced all references of downstream docs to upstream docs. * Update README.md Co-authored-by: Don Naro <dnaro@redhat.com> * Update README.md.j2 Co-authored-by: Don Naro <dnaro@redhat.com> * Update README.md.j2 Co-authored-by: Don Naro <dnaro@redhat.com> * Incorpor'd review feedback from @oraNod and @samccann * Updated with agreed link (for now) until further change is needed. --------- Co-authored-by: Don Naro <dnaro@redhat.com>	2024-07-24 07:54:43 -06:00
Hao Liu	7e77235d5e	Add UI for RECEPTOR_KEEP_WORK_ON_ERROR In Troubleshooting settings	2024-07-22 17:02:37 -04:00
Hao Liu	139d8f0ae2	Add RECEPTOR_KEEP_WORK_ON_ERROR setting If RECEPTOR_KEEP_WORK_ON_ERROR is set to true receptor work unit will not be automatically released Co-Authored-By: Chris Meyers <chrismeyersfsu@users.noreply.github.com>	2024-07-22 17:02:37 -04:00
Hao Liu	7691365aea	Fix depends_on for awx devel when editable dependencies is enabled (#15393 ) Fix depends_on for awx devel... when editable dependencies is on bug introduced by https://github.com/ansible/awx/pull/15386	2024-07-22 16:48:53 -04:00
Alan Rominger	59f61517d4	Loosen up team EE restrictions (#15384 ) * Try to loosen up team EE restrictions * Fix missed permission case of nulling EE org	2024-07-22 14:51:32 -04:00
Alan Rominger	fa670e2d7f	Upgrade to v4 checkout, hide output (#15322 )	2024-07-19 15:33:11 -04:00
Jake Jackson	a87a044d64	Update test to conform with new DAB change (#15385 ) * update tests to not fail with new version of DAB * comment out conditional for now and add TODOs to fix it	2024-07-19 13:58:09 -04:00
Alan Rominger	381ade1148	Fix create_preload_data to allow running without an admin user created (#15356 ) * Allow create_preload_data without having superuser created first * Temporarily change the DAB requirement * Put DAB branch back to devel	2024-07-19 10:35:33 -04:00
Sandra McCann	864a30e3d4	Remove remnants of controller terms from quickstart docs (#15350 ) Remove remnants of controller terms from quickstart Signed-off-by: Sandra McCann <samccann@redhat.com>	2024-07-18 22:42:35 -06:00
Sandra McCann	5f42db67e6	Remove references to translated versions of the docs (#15354 ) remove references to translated versions of the docs Signed-off-by: Sandra McCann <samccann@redhat.com> Co-authored-by: TVo <thavo@redhat.com>	2024-07-19 01:53:06 +00:00
Hao Liu	ddf4f288d4	Remove links from docker-compose template (#15386 ) Links are use to indicate network connectivity and optionally provide alias it is not needed for communication since all the container are on the awx network in the prometheus container case since awx_ container now have valid hostname it's no longer required (also i think the link is missing a `-` anyway...) links also implicitly imply dependency between services in this i see awx container depends on redis and postgres so i switch to depends_on to retain that Making this change to be podman compatible because i get ``` Error response from daemon: bad parameter: link is not supported ```	2024-07-18 21:19:50 -04:00
Lila Yasin	e75bc8bc1e	Fix test_url_base_defaults_to_request to reference local host instead… (#15367 ) * Update all references to towerhost to platformhost * Run prettier on failing ui files	2024-07-18 15:28:54 -04:00
Roberto Chaud	bb533287b8	Create receptor group if missing (#15276 )	2024-07-18 17:59:23 +00:00
Sandra McCann	9979fc659e	Update docs replacements to AWX (#15349 ) Update replacements to AWX Signed-off-by: Sandra McCann <samccann@redhat.com>	2024-07-18 13:36:00 -04:00
Hao Liu	9e5babc093	Fix ui-next build for release staging GHA (#15383 )	2024-07-18 13:26:07 -04:00
Hao Liu	c71e2524ed	Disable dab-release GHA on fork unless explicitly triggered (#15382 ) Disable dab-release GHA on fork Unless explicitly triggered	2024-07-18 14:55:52 +00:00
Hao Liu	48b4c62186	Update DAB update automation PR template (#15376 ) To pass PR Check github action	2024-07-18 10:24:47 -04:00
Seth Foster	853730acb9	Allow deleting org of a running workflow job (#15374 ) Old RBAC system hits DOESNOTEXIST query errors if a user deletes an org while a workflow job is active. The error is triggered by 1. starting workflow job 2. delete the org that the workflow job is a part of 3. The workflow changes status (e.g. pending to waiting) This error message would surface awx.main.models.rbac.Role.DoesNotExist: Role matching query does not exist. The fix is wrap the query in a try catch, and skip over some logic if the roles don't exist. --------- Signed-off-by: Seth Foster <fosterbseth@gmail.com>	2024-07-18 09:40:58 -04:00
Don Naro	f1448fced1	update terminology (#15357 ) * update terminology Replace some instances of Tower with AWX and remove some references to enterprise left over from the migration of RST content from the Automation Controller docs. * Update docs/docsite/rst/userguide/overview.rst Co-authored-by: TVo <thavo@redhat.com> --------- Co-authored-by: TVo <thavo@redhat.com>	2024-07-18 10:29:21 +01:00
Hao Liu	7697b6a69b	Pin 3rd party action at SHA For safety	2024-07-17 22:58:15 +02:00
Chris Meyers	22a491c32c	Put DAB version in the PR title	2024-07-17 15:10:25 -04:00
Chris Meyers	cbd9dce940	Run at 6 am every day	2024-07-17 15:10:25 -04:00
Chris Meyers	a4fdcc1cca	Update .github/workflows/dab-release.yml Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>	2024-07-17 15:10:25 -04:00
Chris Meyers	df95439008	Update .github/workflows/dab-release.yml Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>	2024-07-17 15:10:25 -04:00
Chris Meyers	acd834df8b	Check and update django-ansible-base * Check upstream django-ansible-base releases. If the version upstream does not match the version we are pinned to then submit a PR with the upstream version.	2024-07-17 15:10:25 -04:00
TVo	587f0ecf98	Updated the api file to reflect 2024 date (#15369 )	2024-07-16 19:58:55 +00:00
Hao Liu	5a2091f7bf	Build new/old UI with different nodejs version (#15368 )	2024-07-16 13:18:47 -04:00
Hao Liu	fa7423819a	Fix minor docker build warning (#15362 ) Fix docker build warning Fix ``` WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 8) ```	2024-07-15 13:34:46 +00:00
Alan Rominger	fde8af9f11	Fix task ending in error due to bad iterator (#15355 )	2024-07-12 13:20:39 -04:00
Seth Foster	209e7e27b1	Check member of org when granting cred (#15353 ) A user needs to be a member of the org in order to use a credential in that org. We were incorrectly checking for "change" permission of the org, instead of "member". Signed-off-by: Seth Foster <fosterbseth@gmail.com>	2024-07-10 21:46:26 -04:00
Hao Liu	6c7d29a982	Fix command to set db session timeout for locks (#15352 ) Fix command to set db session timeout Add quote around the value of the setting Example failures ``` 2024-07-10 13:33:29,237 ERROR [a7e55a64e6744a0e920bb1fd78615e5f] awx.main.dispatch Worker failed to run task awx.main.tasks.system.awx_periodic_scheduler([], *{} Traceback (most recent call last): File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/backends/utils.py", line 87, in _execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/psycopg/cursor.py", line 732, in execute raise ex.with_traceback(None) psycopg.errors.SyntaxError: trailing junk after numeric literal at or near "1d" LINE 1: SET idle_in_transaction_session_timeout = 1d ^ ```	2024-07-10 11:11:12 -04:00
Alan Rominger	282ba36839	Fix EE admin not being able to PATCH/PUT object while providing `organization` (#15348 ) * Fix bug where EE object-level admin could not set organization * Finish polishing up test	2024-07-09 16:55:09 -04:00
Alan Rominger	b727d2c3b3	Log conflicts and created items by the periodic resource sync (#15337 ) * Initial lazy logging of periodic sync results * Add desired polish to log * Add debug log	2024-07-09 15:13:08 -04:00
Hao Liu	7fc3d5c7c7	Update ActivityStream UI query to order by id (#15346 ) Timestamp for activity stream is not indexed result in slow query. switching to ID (which effectively will is order by created time) to improve performance	2024-07-09 14:38:07 -04:00
TVo	4e055f46c4	Added note to API guide for filtering exact matches (#15332 )	2024-07-09 10:55:19 -06:00
Seth Foster	f595985b7c	Callback for role assignment (#15339 ) Validate role assignment if org defined Check that organization is defined on credential before running queries. Fixes a "None type does not have attribute id" error. Signed-off-by: Seth Foster <fosterbseth@gmail.com>	2024-07-09 09:44:27 -04:00
Alan Rominger	ea232315bf	Do not reference self.messages when it does not exist (#15331 )	2024-07-03 20:07:46 +00:00
Alan Rominger	ee251812b5	Add complete test that we have analogs to old versions of roles, fix some mismatches (#15321 ) * Add test that we got all permissions right for every role * Fix missing Org execute role and missing adhoc role permission * Add in missing Organization Approval Role as well * Remove Role from role names	2024-07-03 15:40:55 -04:00
Alan Rominger	00ba1ea569	Suppress docker pull output in checks (#15323 ) Supress docker pull output in checks	2024-07-03 15:04:59 -04:00
Alan Rominger	d91af132c1	Fix server error assigning teams EE object roles (#15320 )	2024-07-03 14:07:03 -04:00
Seth Foster	94e5795dfc	Prevent assigning credential to user of other org (#15296 ) Utilizes the `validate_role_assignment` callback from dab (see dab PR #490) to prevent granting credential access to a user of another organization. This logic will work for role_user_assignments and role_team_assignments endpoints. Signed-off-by: Seth Foster <fosterbseth@gmail.com>	2024-07-02 21:05:22 +00:00
Alan Rominger	c4688d6298	Add in missing read permissions for organization audit role (#15318 ) * Add in missing read permissions for organization audit role * Add missing audit permission, special case name handling	2024-07-02 15:20:40 -04:00
TVo	6763badea3	Added new OpenShift Virtualization inventory source to docs. (#15299 ) * Added new OpenShift Virtualization inventory source to docs. * Incorporated review feedback from @fosterseth and @TheRealHaoLiu. * Fixed link to correct kubevirt.core.kubevirt documentation.	2024-07-01 11:47:39 -06:00
Hao Liu	2c4ad6ef0f	Add better 403 error message for Job template create (#15307 ) * Add better 403 error message for Job template create To create Job template u need access to projects and inventory --------- Co-authored-by: Chris Meyers <chris.meyers.fsu@gmail.com>	2024-07-01 15:02:07 +00:00
Hao Liu	37f44d7214	Add better error message for wfjt create 403 (#15309 )	2024-07-01 10:50:49 -04:00
Alan Rominger	98bbc836a6	Fix server error from DAB ValidationError with strings (#15312 )	2024-07-01 10:11:22 -04:00
Alan Rominger	b59aff50dc	Update ExecutionEnvironment model so object-level roles work with DAB RBAC system (#15289 ) * Add initial test for deletion of stale permission * Delete existing EE view permission * Hypothetically complete update of EE model permissions setup * Tests passing locally * Issue with user_capabilities was a test bug, fixed	2024-06-28 16:09:42 -04:00
Alan Rominger	a70b0c1ddc	Do not use cache in github image build action (#15308 ) * Do not use cache in actual image build action * Add cache args to kube prod builds	2024-06-28 09:52:59 -04:00
Alan Rominger	db72c9d5b8	Fix permissions that come from an external auditor role (#15291 ) * Add tests for external auditor * Add assertion for unified JTs which fails * Fix UJT listing bug * Add test for ad hoc commands just to be sure	2024-06-27 15:57:39 -04:00
jamesmarshall24	4e0d19914f	LISTENER_DATABASES clobbers DATABASES OPTIONS (#15306 ) Do not overwrite DATABASES OPTIONS with LISTENER_DATABASES	2024-06-27 13:26:30 -04:00
Hao Liu	6f2307f50e	Add TASK_MANAGER_LOCK_TIMEOUT (#15300 ) * Add TASK_MANAGER_LOCK_TIMEOUT `TASK_MANAGER_LOCK_TIMEOUT` controls the `idle_in_transaction_session_timeout` and `idle_session_timeout` configuration for task manager connections and lock in database hope to prevent the situation that the task instance that holds the lock becomes unresponsive and preventing other instance to be able to run task manager * Add session timeout to periodic scheduler and all sub task manager locks	2024-06-27 09:42:41 -04:00
Alan Rominger	dbc2215bb6	Make attached user models adhere to new API assignments (#15298 )	2024-06-26 23:00:25 -04:00
Hao Liu	7c08b29827	Temporary workaround for CI failure (#15305 ) Workaround ``` ERROR awx/main/tests/functional/test_licenses.py - pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib' ```	2024-06-26 15:29:22 -04:00
TVo	407194d320	Added troubleshooting and tips tricks content (#15212 ) * Added troubleshooting and tips tricks content * Added troubleshooting and tips tricks content * Moved DNS host entry override info to customize pod spec section of CG chapter. * Added troubleshooting and tips tricks content * Moved DNS host entry override info to customize pod spec section of CG chapter. * Update docs/docsite/rst/administration/containers_instance_groups.rst Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com> * Update docs/docsite/rst/administration/containers_instance_groups.rst Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com> * Update docs/docsite/rst/administration/containers_instance_groups.rst Co-authored-by: Sandra McCann <samccann@redhat.com> * Incorp'd review feedback from @fosterseth and @samccann * Update docs/docsite/rst/administration/containers_instance_groups.rst Co-authored-by: Sandra McCann <samccann@redhat.com> * Final revisions based on @fosterseth's inputs. --------- Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com> Co-authored-by: Sandra McCann <samccann@redhat.com>	2024-06-24 12:17:31 -06:00
Alan Rominger	853af295d9	Various RBAC fixes related to managed RoleDefinitions (#15287 ) * Add migration testing for certain managed roles * Fix managed role bugs * Add more tests * Fix another bug with org workflow admin role reference * Add test because another issue is fixed * Mark reason for test * Remove internal markers * Reword failure message Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com> --------- Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>	2024-06-21 09:29:34 -04:00