Merge pull request #1810 from ryanpetrello/devel

restore the celery hostname

.github/BOTMETA.yml

@@ -12,5 +12,5 @@ files:
         labels: component:installer
 
 macros:
-    team_api: wwitzel3 matburt chrismeyersfsu cchurch AlanCoding ryanpetrello jangstur
-    team_ui: jlmitch5 jaredevantabor mabashian gconsidine marshmalien benthomasson
+    team_api: wwitzel3 matburt chrismeyersfsu cchurch AlanCoding ryanpetrello rooftopcellist
+    team_ui: jlmitch5 jaredevantabor mabashian marshmalien benthomasson jakemcdermott

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,3 @@
+# Community Code of Conduct
+
+Please see the official [Ansible Community Code of Conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html).

.github/ISSUE_TEMPLATE.md

@@ -1,3 +1,11 @@
+<!---
+The Ansible community is highly committed to the security of our open source
+projects.  Security concerns should be reported directly by email to
+security@ansible.com.  For more information on the Ansible community's
+practices regarding responsible disclosure, see
+https://www.ansible.com/security
+-->
+
 ##### ISSUE TYPE
 <!--- Pick one below and delete the rest: -->
  - Bug Report

.gitignore

@@ -22,6 +22,8 @@ awx/ui/build_test
 awx/ui/client/languages
 awx/ui/templates/ui/index.html
 awx/ui/templates/ui/installing.html
+/tower-license
+/tower-license/**
 
 # Tower setup playbook testing
 setup/test/roles/postgresql

.travis.yml

@@ -1,31 +0,0 @@
-sudo: false
-language: python
-python:
-    - '2.7'
-env:
-    - TOXENV=api-lint
-    - TOXENV=api
-    - TOXENV=ui-lint
-    - TOXENV=ui
-install:
-    - pip install tox
-script:
-    - tox
-# after_success:
-#     - TOXENV=coveralls tox
-addons:
-    apt:
-        packages:
-            - swig
-            - libxmlsec1-dev
-            - postgresql-9.5
-            - libssl-dev
-cache:
-    pip: true
-    directories:
-        - node_modules
-        - .tox
-services:
-    - mongodb
-    # Enable when we stop using sqlite for API tests
-    # - postgresql

CONTRIBUTING.md

@@ -24,6 +24,7 @@ Have questions about this document or anything not covered here? Come chat with
      * [Start a shell](#start-the-shell)
      * [Create a superuser](#create-a-superuser)
      * [Load the data](#load-the-data)
+     * [Building API Documentation](#build-documentation)
   * [Accessing the AWX web interface](#accessing-the-awx-web-interface)
   * [Purging containers and images](#purging-containers-and-images)
 * [What should I work on?](#what-should-i-work-on)
@@ -261,6 +262,20 @@ You can optionally load some demo data. This will create a demo project, invento
 > This information will persist in the database running in the `tools_postgres_1` container, until the container is removed. You may periodically need to recreate
 this container, and thus the database, if the database schema changes in an upstream commit.
 
+##### Building API Documentation
+
+AWX includes support for building [Swagger/OpenAPI
+documentation](https://swagger.io).  To build the documentation locally, run:
+
+```bash
+(container)/awx_devel$ make swagger
+```
+
+This will write a file named `swagger.json` that contains the API specification
+in OpenAPI format.  A variety of online tools are available for translating
+this data into more consumable formats (such as HTML). http://editor.swagger.io
+is an example of one such service.
+
 ### Accessing the AWX web interface 
 
 You can now log into the AWX web interface at [https://localhost:8043](https://localhost:8043), and access the API directly at [https://localhost:8043/api/](https://localhost:8043/api/).

INSTALL.md

@@ -23,6 +23,7 @@ This document provides a guide for installing AWX.
 - [Kubernetes](#kubernetes)
   - [Prerequisites](#prerequisites-2)
   - [Pre-build steps](#pre-build-steps-1)
+  - [Configuring Helm](#configuring-helm)
   - [Start the build](#start-the-build-1)
   - [Accessing AWX](#accessing-awx-1)
   - [SSL Termination](#ssl-termination)
@@ -61,6 +62,8 @@ Before you can run a deployment, you'll need the following installed in your loc
 - [docker-py](https://github.com/docker/docker-py) Python module
 - [GNU Make](https://www.gnu.org/software/make/)
 - [Git](https://git-scm.com/) Requires Version 1.8.4+
+- [Node 6.x LTS version](https://nodejs.org/en/download/)
+- [NPM 3.x LTS](https://docs.npmjs.com/)
 
 ### System Requirements
 
@@ -70,6 +73,7 @@ The system that runs the AWX service will need to satisfy the following requirem
 - At least 2 cpu cores
 - At least 20GB of space
 - Running Docker, Openshift, or Kubernetes
+- If you choose to use an external PostgreSQL database, please note that the minimum version is 9.4.
 
 ### AWX Tunables
 
@@ -82,9 +86,9 @@ We currently support running AWX as a containerized application using Docker ima
 The [installer](./installer) directory contains an [inventory](./installer/inventory) file, and a playbook, [install.yml](./installer/install.yml). You'll begin by setting variables in the inventory file according to the platform you wish to use, and then you'll start the image build and deployment process by running the playbook.
 
 In the sections below, you'll find deployment details and instructions for each platform:
-- [Docker and Docker Compose](#docker-and-docker-compose)
 - [OpenShift](#openshift)
-- [Kubernetes](#kubernetes).
+- [Kubernetes](#kubernetes)
+- [Docker or Docker Compose](#docker-or-docker-compose).
 
 ### Official vs Building Images
 
@@ -115,6 +119,15 @@ To complete a deployment to OpenShift, you will obviously need access to an Open
 
 You will also need to have the `oc` command in your PATH. The `install.yml` playbook will call out to `oc` when logging into, and creating objects on the cluster.
 
+The default resource requests per-pod requires:
+
+> Memory: 6GB
+> CPU: 3 cores
+
+This can be tuned by overriding the variables found in [/installer/openshift/defaults/main.yml](/installer/openshift/defaults/main.yml). Special care should be taken when doing this as undersized instances will experience crashes and resource exhaustion.
+
+For more detail on how resource requests are formed see: [https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-compute-resources](https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-compute-resources)
+
 #### Deploying to Minishift
 
 Install Minishift by following the [installation guide](https://docs.openshift.org/latest/minishift/getting-started/installing.html).
@@ -287,6 +300,15 @@ A Kubernetes deployment will require you to have access to a Kubernetes cluster
 
 The installation program will reference `kubectl` directly. `helm` is only necessary if you are letting the installer configure PostgreSQL for you.
 
+The default resource requests per-pod requires:
+
+> Memory: 6GB
+> CPU: 3 cores
+
+This can be tuned by overriding the variables found in [/installer/kubernetes/defaults/main.yml](/installer/kubernetes[/defaults/main.yml). Special care should be taken when doing this as undersized instances will experience crashes and resource exhaustion.
+
+For more detail on how resource requests are formed see: [https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
+
 ### Pre-build steps
 
 Before starting the build process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section uncommenting when necessary. Make sure the openshift and standalone docker sections are commented out:
@@ -303,6 +325,12 @@ Before starting the build process, review the [inventory](./installer/inventory)
 
 > These settings should be used if building your own base images. You'll need access to an external registry and are responsible for making sure your kube cluster can talk to it and use it. If these are undefined and the dockerhub_ configuration settings are uncommented then the images will be pulled from dockerhub instead
 
+### Configuring Helm
+
+If you want the AWX installer to manage creating the database pod (rather than installing and configuring postgres on your own). Then you will need to have a working `helm` installation, you can find details here: [https://docs.helm.sh/using_helm/#quickstart-guide](https://docs.helm.sh/using_helm/#quickstart-guide).
+
+Newer Kubernetes clusters with RBAC enabled will need to make sure a service account is created, make sure to follow the instructions here [https://docs.helm.sh/using_helm/#role-based-access-control](https://docs.helm.sh/using_helm/#role-based-access-control)
+
 ### Start the build
 
 After making changes to the `inventory` file use `ansible-playbook` to begin the install

Makefile

@@ -23,7 +23,7 @@ COMPOSE_HOST ?= $(shell hostname)
 
 VENV_BASE ?= /venv
 SCL_PREFIX ?=
-CELERY_SCHEDULE_FILE ?= /celerybeat-schedule
+CELERY_SCHEDULE_FILE ?= /var/lib/awx/beat.db
 
 DEV_DOCKER_TAG_BASE ?= gcr.io/ansible-tower-engineering
 # Python packages to install only from source (not from binary wheels)
@@ -72,7 +72,7 @@ UI_RELEASE_FLAG_FILE = awx/ui/.release_built
 
 I18N_FLAG_FILE = .i18n_built
 
-.PHONY: clean clean-tmp clean-venv requirements requirements_dev \
+.PHONY: awx-link clean clean-tmp clean-venv requirements requirements_dev \
 	develop refresh adduser migrate dbchange dbshell runserver celeryd \
 	receiver test test_unit test_ansible test_coverage coverage_html \
 	dev_build release_build release_clean sdist \
@@ -184,7 +184,6 @@ requirements_awx: virtualenv_awx
 
 requirements_awx_dev:
 	$(VENV_BASE)/awx/bin/pip install -r requirements/requirements_dev.txt
-	$(VENV_BASE)/awx/bin/pip uninstall --yes -r requirements/requirements_dev_uninstall.txt
 
 requirements: requirements_ansible requirements_awx
 
@@ -216,13 +215,11 @@ init:
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
 	$(MANAGEMENT_COMMAND) provision_instance --hostname=$(COMPOSE_HOST); \
-	$(MANAGEMENT_COMMAND) register_queue --queuename=tower --hostnames=$(COMPOSE_HOST);\
+	$(MANAGEMENT_COMMAND) register_queue --queuename=tower --instance_percent=100;\
 	if [ "$(AWX_GROUP_QUEUES)" == "tower,thepentagon" ]; then \
 		$(MANAGEMENT_COMMAND) provision_instance --hostname=isolated; \
 		$(MANAGEMENT_COMMAND) register_queue --queuename='thepentagon' --hostnames=isolated --controller=tower; \
 		$(MANAGEMENT_COMMAND) generate_isolated_key | ssh -o "StrictHostKeyChecking no" root@isolated 'cat > /root/.ssh/authorized_keys'; \
-	elif [ "$(AWX_GROUP_QUEUES)" != "tower" ]; then \
-		$(MANAGEMENT_COMMAND) register_queue --queuename=$(firstword $(subst $(comma), ,$(AWX_GROUP_QUEUES))) --hostnames=$(COMPOSE_HOST); \
 	fi;
 
 # Refresh development environment after pulling new code.
@@ -237,7 +234,7 @@ migrate:
 	if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
-	$(MANAGEMENT_COMMAND) migrate --noinput --fake-initial
+	$(MANAGEMENT_COMMAND) migrate --noinput
 
 # Run after making changes to the models to create a new migration.
 dbchange:
@@ -299,7 +296,7 @@ uwsgi: collectstatic
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
-    uwsgi -b 32768 --socket 127.0.0.1:8050 --module=awx.wsgi:application --home=/venv/awx --chdir=/awx_devel/ --vacuum --processes=5 --harakiri=120 --master --no-orphans --py-autoreload 1 --max-requests=1000 --stats /tmp/stats.socket --master-fifo=/awxfifo --lazy-apps --logformat "%(addr) %(method) %(uri) - %(proto) %(status)" --hook-accepting1-once="exec:/bin/sh -c '[ -f /tmp/celery_pid ] && kill -1 `cat /tmp/celery_pid`'"
+    uwsgi -b 32768 --socket 127.0.0.1:8050 --module=awx.wsgi:application --home=/venv/awx --chdir=/awx_devel/ --vacuum --processes=5 --harakiri=120 --master --no-orphans --py-autoreload 1 --max-requests=1000 --stats /tmp/stats.socket --master-fifo=/awxfifo --lazy-apps --logformat "%(addr) %(method) %(uri) - %(proto) %(status)" --hook-accepting1-once="exec:/bin/sh -c '[ -f /tmp/celery_pid ] && kill -1 `cat /tmp/celery_pid` || true'"
 
 daphne:
 	@if [ "$(VENV_BASE)" ]; then \
@@ -326,7 +323,7 @@ celeryd:
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
-	celery worker -A awx -l DEBUG -B -Ofair --autoscale=100,4 --schedule=$(CELERY_SCHEDULE_FILE) -Q tower_scheduler,tower_broadcast_all,$(COMPOSE_HOST),$(AWX_GROUP_QUEUES) -n celery@$(COMPOSE_HOST) --pidfile /tmp/celery_pid
+	celery worker -A awx -l DEBUG -B -Ofair --autoscale=100,4 --schedule=$(CELERY_SCHEDULE_FILE) -n celery@$(COMPOSE_HOST) --pidfile /tmp/celery_pid
 
 # Run to start the zeromq callback receiver
 receiver:
@@ -338,9 +335,6 @@ receiver:
 nginx:
 	nginx -g "daemon off;"
 
-rdb:
-	$(PYTHON) tools/rdb.py
-
 jupyter:
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
@@ -365,16 +359,29 @@ pyflakes: reports
 pylint: reports
 	@(set -o pipefail && $@ | reports/$@.report)
 
+swagger: reports
+	@if [ "$(VENV_BASE)" ]; then \
+		. $(VENV_BASE)/awx/bin/activate; \
+	fi; \
+	(set -o pipefail && py.test awx/conf/tests/functional awx/main/tests/functional/api awx/main/tests/docs --release=$(VERSION_TARGET) | tee reports/$@.report)
+
 check: flake8 pep8 # pyflakes pylint
 
+awx-link:
+	cp -R /tmp/awx.egg-info /awx_devel/ || true
+	sed -i "s/placeholder/$(shell git describe --long | sed 's/\./\\./g')/" /awx_devel/awx.egg-info/PKG-INFO
+	cp /tmp/awx.egg-link /venv/awx/lib/python2.7/site-packages/awx.egg-link
+
 TEST_DIRS ?= awx/main/tests/unit awx/main/tests/functional awx/conf/tests awx/sso/tests
 # Run all API unit tests.
-test: test_ansible
+test:
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \
 	fi; \
 	py.test $(TEST_DIRS)
 
+test_combined: test_ansible test
+
 test_unit:
 	@if [ "$(VENV_BASE)" ]; then \
 		. $(VENV_BASE)/awx/bin/activate; \

awx/__init__.py

@@ -7,7 +7,7 @@ import sys
 import warnings
 
 from pkg_resources import get_distribution
-from .celery import app as celery_app
+from .celery import app as celery_app # noqa
 
 __version__ = get_distribution('awx').version
 

awx/api/authentication.py

@@ -2,126 +2,21 @@
 # All Rights Reserved.
 
 # Python
-import urllib
 import logging
 
 # Django
 from django.conf import settings
-from django.utils.timezone import now as tz_now
 from django.utils.encoding import smart_text
-from django.utils.translation import ugettext_lazy as _
 
 # Django REST Framework
 from rest_framework import authentication
-from rest_framework import exceptions
-from rest_framework import HTTP_HEADER_ENCODING
 
-# AWX
-from awx.main.models import AuthToken
+# Django OAuth Toolkit
+from oauth2_provider.contrib.rest_framework import OAuth2Authentication
 
 logger = logging.getLogger('awx.api.authentication')
 
 
-class TokenAuthentication(authentication.TokenAuthentication):
-    '''
-    Custom token authentication using tokens that expire and are associated
-    with parameters specific to the request.
-    '''
-
-    model = AuthToken
-
-    @staticmethod
-    def _get_x_auth_token_header(request):
-        auth = request.META.get('HTTP_X_AUTH_TOKEN', '')
-        if isinstance(auth, type('')):
-            # Work around django test client oddness
-            auth = auth.encode(HTTP_HEADER_ENCODING)
-        return auth
-
-    @staticmethod
-    def _get_auth_token_cookie(request):
-        token = request.COOKIES.get('token', '')
-        if token:
-            token = urllib.unquote(token).strip('"')
-            return 'token %s' % token
-        return ''
-
-    def authenticate(self, request):
-        self.request = request
-
-        # Prefer the custom X-Auth-Token header over the Authorization header,
-        # to handle cases where the browser submits saved Basic auth and
-        # overrides the UI's normal use of the Authorization header.
-        auth = TokenAuthentication._get_x_auth_token_header(request).split()
-        if not auth or auth[0].lower() != 'token':
-            auth = authentication.get_authorization_header(request).split()
-            # Prefer basic auth over cookie token
-            if auth and auth[0].lower() == 'basic':
-                return None
-            elif not auth or auth[0].lower() != 'token':
-                auth = TokenAuthentication._get_auth_token_cookie(request).split()
-                if not auth or auth[0].lower() != 'token':
-                    return None
-
-        if len(auth) == 1:
-            msg = _('Invalid token header. No credentials provided.')
-            raise exceptions.AuthenticationFailed(msg)
-        elif len(auth) > 2:
-            msg = _('Invalid token header. Token string should not contain spaces.')
-            raise exceptions.AuthenticationFailed(msg)
-
-        return self.authenticate_credentials(auth[1])
-
-    def authenticate_credentials(self, key):
-        now = tz_now()
-        # Retrieve the request hash and token.
-        try:
-            request_hash = self.model.get_request_hash(self.request)
-            token = self.model.objects.select_related('user').get(
-                key=key,
-                request_hash=request_hash,
-            )
-        except self.model.DoesNotExist:
-            raise exceptions.AuthenticationFailed(AuthToken.reason_long('invalid_token'))
-
-        # Tell the user why their token was previously invalidated.
-        if token.invalidated:
-            raise exceptions.AuthenticationFailed(AuthToken.reason_long(token.reason))
-
-        # Explicitly handle expired tokens
-        if token.is_expired(now=now):
-            token.invalidate(reason='timeout_reached')
-            raise exceptions.AuthenticationFailed(AuthToken.reason_long('timeout_reached'))
-
-        # Token invalidated due to session limit config being reduced
-        # Session limit reached invalidation will also take place on authentication
-        if settings.AUTH_TOKEN_PER_USER != -1:
-            if not token.in_valid_tokens(now=now):
-                token.invalidate(reason='limit_reached')
-                raise exceptions.AuthenticationFailed(AuthToken.reason_long('limit_reached'))
-
-        # If the user is inactive, then return an error.
-        if not token.user.is_active:
-            raise exceptions.AuthenticationFailed(_('User inactive or deleted'))
-
-        # Refresh the token.
-        # The token is extended from "right now" + configurable setting amount.
-        token.refresh(now=now)
-
-        # Return the user object and the token.
-        return (token.user, token)
-
-
-class TokenGetAuthentication(TokenAuthentication):
-
-    def authenticate(self, request):
-        if request.method.lower() == 'get':
-            token = request.GET.get('token', None)
-            if token:
-                request.META['HTTP_X_AUTH_TOKEN'] = 'Token %s' % token
-        return super(TokenGetAuthentication, self).authenticate(request)
-
-
 class LoggedBasicAuthentication(authentication.BasicAuthentication):
 
     def authenticate(self, request):
@@ -137,3 +32,28 @@ class LoggedBasicAuthentication(authentication.BasicAuthentication):
         if not settings.AUTH_BASIC_ENABLED:
             return
         return super(LoggedBasicAuthentication, self).authenticate_header(request)
+
+
+class SessionAuthentication(authentication.SessionAuthentication):
+    
+    def authenticate_header(self, request):
+        return 'Session'
+
+    def enforce_csrf(self, request):
+        return None
+
+
+class LoggedOAuth2Authentication(OAuth2Authentication):
+
+    def authenticate(self, request):
+        ret = super(LoggedOAuth2Authentication, self).authenticate(request)
+        if ret:
+            user, token = ret
+            username = user.username if user else '<none>'
+            logger.debug(smart_text(
+                u"User {} performed a {} to {} through the API using OAuth token {}.".format(
+                    username, request.method, request.path, token.pk
+                )
+            ))
+            setattr(user, 'oauth_scopes', [x for x in token.scope.split() if x])
+        return ret

awx/api/conf.py

@@ -1,30 +1,30 @@
 # Django
 from django.utils.translation import ugettext_lazy as _
 
-# Tower
+# AWX
 from awx.conf import fields, register
+from awx.api.fields import OAuth2ProviderField
 
 
 register(
-    'AUTH_TOKEN_EXPIRATION',
+    'SESSION_COOKIE_AGE',
     field_class=fields.IntegerField,
     min_value=60,
+    max_value=30000000000,  # approx 1,000 years, higher values give OverflowError
     label=_('Idle Time Force Log Out'),
     help_text=_('Number of seconds that a user is inactive before they will need to login again.'),
     category=_('Authentication'),
     category_slug='authentication',
 )
-
 register(
-    'AUTH_TOKEN_PER_USER',
+    'SESSIONS_PER_USER',
     field_class=fields.IntegerField,
     min_value=-1,
-    label=_('Maximum number of simultaneous logins'),
-    help_text=_('Maximum number of simultaneous logins a user may have. To disable enter -1.'),
+    label=_('Maximum number of simultaneous logged in sessions'),
+    help_text=_('Maximum number of simultaneous logged in sessions a user may have. To disable enter -1.'),
     category=_('Authentication'),
     category_slug='authentication',
 )
-
 register(
     'AUTH_BASIC_ENABLED',
     field_class=fields.BooleanField,
@@ -33,3 +33,16 @@ register(
     category=_('Authentication'),
     category_slug='authentication',
 )
+register(
+    'OAUTH2_PROVIDER',
+    field_class=OAuth2ProviderField,
+    default={'ACCESS_TOKEN_EXPIRE_SECONDS': 315360000000,
+             'AUTHORIZATION_CODE_EXPIRE_SECONDS': 600},
+    label=_('OAuth 2 Timeout Settings'),
+    help_text=_('Dictionary for customizing OAuth 2 timeouts, available items are '
+                '`ACCESS_TOKEN_EXPIRE_SECONDS`, the duration of access tokens in the number '
+                'of seconds, and `AUTHORIZATION_CODE_EXPIRE_SECONDS`, the duration of '
+                'authorization grants in the number of seconds.'),
+    category=_('Authentication'),
+    category_slug='authentication',
+)

awx/api/exceptions.py

@@ -0,0 +1,18 @@
+# Copyright (c) 2018 Ansible by Red Hat
+# All Rights Reserved.
+
+# Django
+from django.utils.translation import ugettext_lazy as _
+
+# Django REST Framework
+from rest_framework.exceptions import ValidationError
+
+
+class ActiveJobConflict(ValidationError):
+    status_code = 409
+
+    def __init__(self, active_jobs):
+        super(ActiveJobConflict, self).__init__({
+            "error": _("Resource is being used by running jobs."),
+            "active_jobs": active_jobs
+        })

awx/api/fields.py

@@ -1,10 +1,17 @@
 # Copyright (c) 2016 Ansible, Inc.
 # All Rights Reserved.
 
+# Django
+from django.utils.translation import ugettext_lazy as _
+from django.core.exceptions import ObjectDoesNotExist
 
 # Django REST Framework
 from rest_framework import serializers
 
+# AWX
+from awx.conf import fields
+from awx.main.models import Credential
+
 __all__ = ['BooleanNullField', 'CharNullField', 'ChoiceNullField', 'VerbatimField']
 
 
@@ -66,3 +73,36 @@ class VerbatimField(serializers.Field):
 
     def to_representation(self, value):
         return value
+
+
+class OAuth2ProviderField(fields.DictField):
+
+    default_error_messages = {
+        'invalid_key_names': _('Invalid key names: {invalid_key_names}'),
+    }
+    valid_key_names = {'ACCESS_TOKEN_EXPIRE_SECONDS', 'AUTHORIZATION_CODE_EXPIRE_SECONDS'}
+    child = fields.IntegerField(min_value=1)
+
+    def to_internal_value(self, data):
+        data = super(OAuth2ProviderField, self).to_internal_value(data)
+        invalid_flags = (set(data.keys()) - self.valid_key_names)
+        if invalid_flags:
+            self.fail('invalid_key_names', invalid_key_names=', '.join(list(invalid_flags)))
+        return data
+
+
+class DeprecatedCredentialField(serializers.IntegerField):
+
+    def __init__(self, **kwargs):
+        kwargs['allow_null'] = True
+        kwargs['default'] = None
+        kwargs['min_value'] = 1
+        kwargs['help_text'] = 'This resource has been deprecated and will be removed in a future release'
+        super(DeprecatedCredentialField, self).__init__(**kwargs)
+
+    def to_internal_value(self, pk):
+        try:
+            Credential.objects.get(pk=pk)
+        except ObjectDoesNotExist:
+            raise serializers.ValidationError(_('Credential {} does not exist').format(pk))
+        return pk

awx/api/filters.py

@@ -27,13 +27,6 @@ from awx.main.models.credential import CredentialType
 from awx.main.models.rbac import RoleAncestorEntry
 
 
-class MongoFilterBackend(BaseFilterBackend):
-
-    # FIX: Note that MongoEngine can't use the filter backends from DRF
-    def filter_queryset(self, request, queryset, view):
-        return queryset
-
-
 class V1CredentialFilterBackend(BaseFilterBackend):
     '''
     For /api/v1/ requests, filter out v2 (custom) credentials
@@ -138,6 +131,8 @@ class FieldLookupBackend(BaseFilterBackend):
                     new_parts.append(name_alt)
                 else:
                     field = model._meta.get_field(name)
+                if 'auth' in name or 'token' in name:
+                    raise PermissionDenied(_('Filtering on %s is not allowed.' % name))
                 if isinstance(field, ForeignObjectRel) and getattr(field.field, '__prevent_search__', False):
                     raise PermissionDenied(_('Filtering on %s is not allowed.' % name))
                 elif getattr(field, '__prevent_search__', False):
@@ -276,7 +271,7 @@ class FieldLookupBackend(BaseFilterBackend):
                 # TODO: remove after API v1 deprecation period
                 if queryset.model._meta.object_name in ('JobTemplate', 'Job') and key in (
                         'credential', 'vault_credential', 'cloud_credential', 'network_credential'
-                ):
+                ) or queryset.model._meta.object_name in ('InventorySource', 'InventoryUpdate') and key == 'credential':
                     key = 'credentials'
 
                 # Make legacy v1 Credential fields work for backwards compatability

awx/api/generics.py

@@ -5,6 +5,7 @@
 import inspect
 import logging
 import time
+import six
 
 # Django
 from django.conf import settings
@@ -18,14 +19,19 @@ from django.utils.encoding import smart_text
 from django.utils.safestring import mark_safe
 from django.contrib.contenttypes.models import ContentType
 from django.utils.translation import ugettext_lazy as _
+from django.contrib.auth import views as auth_views
 
 # Django REST Framework
 from rest_framework.authentication import get_authorization_header
-from rest_framework.exceptions import PermissionDenied, AuthenticationFailed
+from rest_framework.exceptions import PermissionDenied, AuthenticationFailed, ParseError
 from rest_framework import generics
 from rest_framework.response import Response
 from rest_framework import status
 from rest_framework import views
+from rest_framework.permissions import AllowAny
+
+# cryptography
+from cryptography.fernet import InvalidToken
 
 # AWX
 from awx.api.filters import FieldLookupBackend
@@ -33,9 +39,9 @@ from awx.main.models import *  # noqa
 from awx.main.access import access_registry
 from awx.main.utils import * # noqa
 from awx.main.utils.db import get_all_field_names
-from awx.api.serializers import ResourceAccessListElementSerializer
+from awx.api.serializers import ResourceAccessListElementSerializer, CopySerializer
 from awx.api.versioning import URLPathVersioning, get_request_version
-from awx.api.metadata import SublistAttachDetatchMetadata
+from awx.api.metadata import SublistAttachDetatchMetadata, Metadata
 
 __all__ = ['APIView', 'GenericAPIView', 'ListAPIView', 'SimpleListAPIView',
            'ListCreateAPIView', 'SubListAPIView', 'SubListCreateAPIView',
@@ -47,12 +53,41 @@ __all__ = ['APIView', 'GenericAPIView', 'ListAPIView', 'SimpleListAPIView',
            'ResourceAccessList',
            'ParentMixin',
            'DeleteLastUnattachLabelMixin',
-           'SubListAttachDetachAPIView',]
+           'SubListAttachDetachAPIView',
+           'CopyAPIView']
 
 logger = logging.getLogger('awx.api.generics')
 analytics_logger = logging.getLogger('awx.analytics.performance')
 
 
+class LoggedLoginView(auth_views.LoginView):
+
+    def post(self, request, *args, **kwargs):
+        original_user = getattr(request, 'user', None)
+        ret = super(LoggedLoginView, self).post(request, *args, **kwargs)
+        current_user = getattr(request, 'user', None)
+
+        if current_user and getattr(current_user, 'pk', None) and current_user != original_user:
+            logger.info("User {} logged in.".format(current_user.username))
+        if request.user.is_authenticated:
+            return ret
+        else:
+            ret.status_code = 401
+            return ret
+
+
+class LoggedLogoutView(auth_views.LogoutView):
+
+    def dispatch(self, request, *args, **kwargs):
+        original_user = getattr(request, 'user', None)
+        ret = super(LoggedLogoutView, self).dispatch(request, *args, **kwargs)
+        current_user = getattr(request, 'user', None)
+        if (not current_user or not getattr(current_user, 'pk', True)) \
+                and current_user != original_user:
+            logger.info("User {} logged out.".format(original_user.username))
+        return ret
+
+
 def get_view_name(cls, suffix=None):
     '''
     Wrapper around REST framework get_view_name() to support get_name() method
@@ -91,8 +126,17 @@ def get_view_description(cls, request, html=False):
     return mark_safe(desc)
 
 
+def get_default_schema():
+    if settings.SETTINGS_MODULE == 'awx.settings.development':
+        from awx.api.swagger import AutoSchema
+        return AutoSchema()
+    else:
+        return views.APIView.schema
+
+
 class APIView(views.APIView):
 
+    schema = get_default_schema()
     versioning_class = URLPathVersioning
 
     def initialize_request(self, request, *args, **kwargs):
@@ -121,6 +165,9 @@ class APIView(views.APIView):
             request.drf_request_user = getattr(drf_request, 'user', False)
         except AuthenticationFailed:
             request.drf_request_user = None
+        except ParseError as exc:
+            request.drf_request_user = None
+            self.__init_request_error__ = exc
         return drf_request
 
     def finalize_response(self, request, response, *args, **kwargs):
@@ -130,6 +177,8 @@ class APIView(views.APIView):
         if response.status_code >= 400:
             status_msg = "status %s received by user %s attempting to access %s from %s" % \
                          (response.status_code, request.user, request.path, request.META.get('REMOTE_ADDR', None))
+            if hasattr(self, '__init_request_error__'):
+                response = self.handle_exception(self.__init_request_error__)
             if response.status_code == 401:
                 logger.info(status_msg)
             else:
@@ -176,27 +225,14 @@ class APIView(views.APIView):
         and in the browsable API.
         """
         func = self.settings.VIEW_DESCRIPTION_FUNCTION
-        return func(self.__class__, self._request, html)
+        return func(self.__class__, getattr(self, '_request', None), html)
 
     def get_description_context(self):
         return {
             'view': self,
             'docstring': type(self).__doc__ or '',
-            'new_in_13': getattr(self, 'new_in_13', False),
-            'new_in_14': getattr(self, 'new_in_14', False),
-            'new_in_145': getattr(self, 'new_in_145', False),
-            'new_in_148': getattr(self, 'new_in_148', False),
-            'new_in_200': getattr(self, 'new_in_200', False),
-            'new_in_210': getattr(self, 'new_in_210', False),
-            'new_in_220': getattr(self, 'new_in_220', False),
-            'new_in_230': getattr(self, 'new_in_230', False),
-            'new_in_240': getattr(self, 'new_in_240', False),
-            'new_in_300': getattr(self, 'new_in_300', False),
-            'new_in_310': getattr(self, 'new_in_310', False),
-            'new_in_320': getattr(self, 'new_in_320', False),
-            'new_in_330': getattr(self, 'new_in_330', False),
-            'new_in_api_v2': getattr(self, 'new_in_api_v2', False),
             'deprecated': getattr(self, 'deprecated', False),
+            'swagger_method': getattr(self.request, 'swagger_method', None),
         }
 
     def get_description(self, request, html=False):
@@ -214,7 +250,7 @@ class APIView(views.APIView):
             context['deprecated'] = True
 
         description = render_to_string(template_list, context)
-        if context.get('deprecated'):
+        if context.get('deprecated') and context.get('swagger_method') is None:
             # render deprecation messages at the very top
             description = '\n'.join([render_to_string('api/_deprecated.md', context), description])
         return description
@@ -325,13 +361,6 @@ class ListAPIView(generics.ListAPIView, GenericAPIView):
     def get_queryset(self):
         return self.request.user.get_queryset(self.model)
 
-    def paginate_queryset(self, queryset):
-        page = super(ListAPIView, self).paginate_queryset(queryset)
-        # Queries RBAC info & stores into list objects
-        if hasattr(self, 'capabilities_prefetch') and page is not None:
-            cache_list_capabilities(page, self.capabilities_prefetch, self.model, self.request.user)
-        return page
-
     def get_description_context(self):
         if 'username' in get_all_field_names(self.model):
             order_field = 'username'
@@ -747,3 +776,152 @@ class ResourceAccessList(ParentMixin, ListAPIView):
         for r in roles:
             ancestors.update(set(r.ancestors.all()))
         return User.objects.filter(roles__in=list(ancestors)).distinct()
+
+
+def trigger_delayed_deep_copy(*args, **kwargs):
+    from awx.main.tasks import deep_copy_model_obj
+    connection.on_commit(lambda: deep_copy_model_obj.delay(*args, **kwargs))
+
+
+class CopyAPIView(GenericAPIView):
+
+    serializer_class = CopySerializer
+    permission_classes = (AllowAny,)
+    copy_return_serializer_class = None
+    new_in_330 = True
+    new_in_api_v2 = True
+
+    def _get_copy_return_serializer(self, *args, **kwargs):
+        if not self.copy_return_serializer_class:
+            return self.get_serializer(*args, **kwargs)
+        serializer_class_store = self.serializer_class
+        self.serializer_class = self.copy_return_serializer_class
+        ret = self.get_serializer(*args, **kwargs)
+        self.serializer_class = serializer_class_store
+        return ret
+
+    @staticmethod
+    def _decrypt_model_field_if_needed(obj, field_name, field_val):
+        if field_name in getattr(type(obj), 'REENCRYPTION_BLACKLIST_AT_COPY', []):
+            return field_val
+        if isinstance(field_val, dict):
+            for sub_field in field_val:
+                if isinstance(sub_field, six.string_types) \
+                        and isinstance(field_val[sub_field], six.string_types):
+                    try:
+                        field_val[sub_field] = decrypt_field(obj, field_name, sub_field)
+                    except InvalidToken:
+                        # Catching the corner case with v1 credential fields
+                        field_val[sub_field] = decrypt_field(obj, sub_field)
+        elif isinstance(field_val, six.string_types):
+            field_val = decrypt_field(obj, field_name)
+        return field_val
+
+    def _build_create_dict(self, obj):
+        ret = {}
+        if self.copy_return_serializer_class:
+            all_fields = Metadata().get_serializer_info(
+                self._get_copy_return_serializer(), method='POST'
+            )
+            for field_name, field_info in all_fields.items():
+                if not hasattr(obj, field_name) or field_info.get('read_only', True):
+                    continue
+                ret[field_name] = CopyAPIView._decrypt_model_field_if_needed(
+                    obj, field_name, getattr(obj, field_name)
+                )
+        return ret
+
+    @staticmethod
+    def copy_model_obj(old_parent, new_parent, model, obj, creater, copy_name='', create_kwargs=None):
+        fields_to_preserve = set(getattr(model, 'FIELDS_TO_PRESERVE_AT_COPY', []))
+        fields_to_discard = set(getattr(model, 'FIELDS_TO_DISCARD_AT_COPY', []))
+        m2m_to_preserve = {}
+        o2m_to_preserve = {}
+        create_kwargs = create_kwargs or {}
+        for field_name in fields_to_discard:
+            create_kwargs.pop(field_name, None)
+        for field in model._meta.get_fields():
+            try:
+                field_val = getattr(obj, field.name)
+            except AttributeError:
+                continue
+            # Adjust copy blacklist fields here.
+            if field.name in fields_to_discard or field.name in [
+                'id', 'pk', 'polymorphic_ctype', 'unifiedjobtemplate_ptr', 'created_by', 'modified_by'
+            ] or field.name.endswith('_role'):
+                create_kwargs.pop(field.name, None)
+                continue
+            if field.one_to_many:
+                if field.name in fields_to_preserve:
+                    o2m_to_preserve[field.name] = field_val
+            elif field.many_to_many:
+                if field.name in fields_to_preserve and not old_parent:
+                    m2m_to_preserve[field.name] = field_val
+            elif field.many_to_one and not field_val:
+                create_kwargs.pop(field.name, None)
+            elif field.many_to_one and field_val == old_parent:
+                create_kwargs[field.name] = new_parent
+            elif field.name == 'name' and not old_parent:
+                create_kwargs[field.name] = copy_name or field_val + ' copy'
+            elif field.name in fields_to_preserve:
+                create_kwargs[field.name] = CopyAPIView._decrypt_model_field_if_needed(
+                    obj, field.name, field_val
+                )
+        new_obj = model.objects.create(**create_kwargs)
+        # Need to save separatedly because Djang-crum get_current_user would
+        # not work properly in non-request-response-cycle context.
+        new_obj.created_by = creater
+        new_obj.save()
+        for m2m in m2m_to_preserve:
+            for related_obj in m2m_to_preserve[m2m].all():
+                getattr(new_obj, m2m).add(related_obj)
+        if not old_parent:
+            sub_objects = []
+            for o2m in o2m_to_preserve:
+                for sub_obj in o2m_to_preserve[o2m].all():
+                    sub_model = type(sub_obj)
+                    sub_objects.append((sub_model.__module__, sub_model.__name__, sub_obj.pk))
+            return new_obj, sub_objects
+        ret = {obj: new_obj}
+        for o2m in o2m_to_preserve:
+            for sub_obj in o2m_to_preserve[o2m].all():
+                ret.update(CopyAPIView.copy_model_obj(obj, new_obj, type(sub_obj), sub_obj, creater))
+        return ret
+
+    def get(self, request, *args, **kwargs):
+        obj = self.get_object()
+        create_kwargs = self._build_create_dict(obj)
+        for key in create_kwargs:
+            create_kwargs[key] = getattr(create_kwargs[key], 'pk', None) or create_kwargs[key]
+        return Response({'can_copy': request.user.can_access(self.model, 'add', create_kwargs)})
+
+    def post(self, request, *args, **kwargs):
+        obj = self.get_object()
+        create_kwargs = self._build_create_dict(obj)
+        create_kwargs_check = {}
+        for key in create_kwargs:
+            create_kwargs_check[key] = getattr(create_kwargs[key], 'pk', None) or create_kwargs[key]
+        if not request.user.can_access(self.model, 'add', create_kwargs_check):
+            raise PermissionDenied()
+        serializer = self.get_serializer(data=request.data)
+        if not serializer.is_valid():
+            return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
+        new_obj, sub_objs = CopyAPIView.copy_model_obj(
+            None, None, self.model, obj, request.user, create_kwargs=create_kwargs,
+            copy_name=serializer.validated_data.get('name', '')
+        )
+        if hasattr(new_obj, 'admin_role') and request.user not in new_obj.admin_role:
+            new_obj.admin_role.members.add(request.user)
+        if sub_objs:
+            permission_check_func = None
+            if hasattr(type(self), 'deep_copy_permission_check_func'):
+                permission_check_func = (
+                    type(self).__module__, type(self).__name__, 'deep_copy_permission_check_func'
+                )
+            trigger_delayed_deep_copy(
+                self.model.__module__, self.model.__name__,
+                obj.pk, new_obj.pk, request.user.pk, sub_objs,
+                permission_check_func=permission_check_func
+            )
+        serializer = self._get_copy_return_serializer(new_obj)
+        return Response(serializer.data, status=status.HTTP_201_CREATED)

awx/api/metadata.py

@@ -44,9 +44,9 @@ class Metadata(metadata.SimpleMetadata):
         if placeholder is not serializers.empty:
             field_info['placeholder'] = placeholder
 
-        # Update help text for common fields.
         serializer = getattr(field, 'parent', None)
-        if serializer:
+        if serializer and hasattr(serializer, 'Meta') and hasattr(serializer.Meta, 'model'):
+            # Update help text for common fields.
             field_help_text = {
                 'id': _('Database ID for this {}.'),
                 'name': _('Name of this {}.'),
@@ -59,10 +59,18 @@ class Metadata(metadata.SimpleMetadata):
                 'modified': _('Timestamp when this {} was last modified.'),
             }
             if field.field_name in field_help_text:
-                if hasattr(serializer, 'Meta') and hasattr(serializer.Meta, 'model'):
-                    opts = serializer.Meta.model._meta.concrete_model._meta
-                    verbose_name = smart_text(opts.verbose_name)
-                    field_info['help_text'] = field_help_text[field.field_name].format(verbose_name)
+                opts = serializer.Meta.model._meta.concrete_model._meta
+                verbose_name = smart_text(opts.verbose_name)
+                field_info['help_text'] = field_help_text[field.field_name].format(verbose_name)
+            # If field is not part of the model, then show it as non-filterable
+            else:
+                is_model_field = False
+                for model_field in serializer.Meta.model._meta.fields:
+                    if field.field_name == model_field.name:
+                        is_model_field = True
+                        break
+                if not is_model_field:
+                    field_info['filterable'] = False
 
         # Indicate if a field has a default value.
         # FIXME: Still isn't showing all default values?
@@ -190,23 +198,6 @@ class Metadata(metadata.SimpleMetadata):
         finally:
             delattr(view, '_request')
 
-        # Add version number in which view was added to Tower.
-        added_in_version = '1.2'
-        for version in ('3.2.0', '3.1.0', '3.0.0', '2.4.0', '2.3.0', '2.2.0',
-                        '2.1.0', '2.0.0', '1.4.8', '1.4.5', '1.4', '1.3'):
-            if getattr(view, 'new_in_%s' % version.replace('.', ''), False):
-                added_in_version = version
-                break
-        metadata['added_in_version'] = added_in_version
-
-        # Add API version number in which view was added to Tower.
-        added_in_api_version = 'v1'
-        for version in ('v2',):
-            if getattr(view, 'new_in_api_%s' % version, False):
-                added_in_api_version = version
-                break
-        metadata['added_in_api_version'] = added_in_api_version
-
         # Add type(s) handled by this view/serializer.
         if hasattr(view, 'get_serializer'):
             serializer = view.get_serializer()

awx/api/parsers.py

@@ -1,7 +1,6 @@
 # Python
 from collections import OrderedDict
 import json
-import yaml
 
 # Django
 from django.conf import settings
@@ -13,36 +12,6 @@ from rest_framework import parsers
 from rest_framework.exceptions import ParseError
 
 
-class OrderedDictLoader(yaml.SafeLoader):
-    """
-    This yaml loader is used to deal with current pyYAML (3.12) not supporting
-    custom object pairs hook. Remove it when new version adds that support.
-    """
-
-    def construct_mapping(self, node, deep=False):
-        if isinstance(node, yaml.nodes.MappingNode):
-            self.flatten_mapping(node)
-        else:
-            raise yaml.constructor.ConstructorError(
-                None, None,
-                "expected a mapping node, but found %s" % node.id,
-                node.start_mark
-            )
-        mapping = OrderedDict()
-        for key_node, value_node in node.value:
-            key = self.construct_object(key_node, deep=deep)
-            try:
-                hash(key)
-            except TypeError, exc:
-                raise yaml.constructor.ConstructorError(
-                    "while constructing a mapping", node.start_mark,
-                    "found unacceptable key (%s)" % exc, key_node.start_mark
-                )
-            value = self.construct_object(value_node, deep=deep)
-            mapping[key] = value
-        return mapping
-
-
 class JSONParser(parsers.JSONParser):
     """
     Parses JSON-serialized data, preserving order of dictionary keys.

awx/api/permissions.py

@@ -17,7 +17,7 @@ logger = logging.getLogger('awx.api.permissions')
 
 __all__ = ['ModelAccessPermission', 'JobTemplateCallbackPermission',
            'TaskPermission', 'ProjectUpdatePermission', 'InventoryInventorySourcesUpdatePermission',
-           'UserPermission', 'IsSuperUser']
+           'UserPermission', 'IsSuperUser', 'InstanceGroupTowerPermission',]
 
 
 class ModelAccessPermission(permissions.BasePermission):
@@ -103,7 +103,8 @@ class ModelAccessPermission(permissions.BasePermission):
             return False
 
         # Always allow superusers
-        if getattr(view, 'always_allow_superuser', True) and request.user.is_superuser:
+        if getattr(view, 'always_allow_superuser', True) and request.user.is_superuser \
+                and not hasattr(request.user, 'oauth_scopes'):
             return True
 
         # Check if view supports the request method before checking permission
@@ -226,3 +227,11 @@ class IsSuperUser(permissions.BasePermission):
 
     def has_permission(self, request, view):
         return request.user and request.user.is_superuser
+
+
+class InstanceGroupTowerPermission(ModelAccessPermission):
+    def has_object_permission(self, request, view, obj):
+        if request.method == 'DELETE' and obj.name == "tower":
+            return False
+        return super(InstanceGroupTowerPermission, self).has_object_permission(request, view, obj)
+

awx/api/renderers.py

@@ -5,6 +5,8 @@
 from rest_framework import renderers
 from rest_framework.request import override_method
 
+import six
+
 
 class BrowsableAPIRenderer(renderers.BrowsableAPIRenderer):
     '''
@@ -69,8 +71,8 @@ class PlainTextRenderer(renderers.BaseRenderer):
     format = 'txt'
 
     def render(self, data, media_type=None, renderer_context=None):
-        if not isinstance(data, basestring):
-            data = unicode(data)
+        if not isinstance(data, six.string_types):
+            data = six.text_type(data)
         return data.encode(self.charset)
 
 

awx/api/swagger.py

@@ -0,0 +1,103 @@
+import json
+import warnings
+
+from coreapi.document import Object, Link
+
+from rest_framework import exceptions
+from rest_framework.permissions import AllowAny
+from rest_framework.renderers import CoreJSONRenderer
+from rest_framework.response import Response
+from rest_framework.schemas import SchemaGenerator, AutoSchema as DRFAuthSchema
+from rest_framework.views import APIView
+
+from rest_framework_swagger import renderers
+
+
+class AutoSchema(DRFAuthSchema):
+
+    def get_link(self, path, method, base_url):
+        link = super(AutoSchema, self).get_link(path, method, base_url)
+        try:
+            serializer = self.view.get_serializer()
+        except Exception:
+            serializer = None
+            warnings.warn('{}.get_serializer() raised an exception during '
+                          'schema generation. Serializer fields will not be '
+                          'generated for {} {}.'
+                          .format(self.view.__class__.__name__, method, path))
+
+        link.__dict__['deprecated'] = getattr(self.view, 'deprecated', False)
+
+        # auto-generate a topic/tag for the serializer based on its model
+        if hasattr(self.view, 'swagger_topic'):
+            link.__dict__['topic'] = str(self.view.swagger_topic).title()
+        elif serializer and hasattr(serializer, 'Meta'):
+            link.__dict__['topic'] = str(
+                serializer.Meta.model._meta.verbose_name_plural
+            ).title()
+        elif hasattr(self.view, 'model'):
+            link.__dict__['topic'] = str(self.view.model._meta.verbose_name_plural).title()
+        else:
+            warnings.warn('Could not determine a Swagger tag for path {}'.format(path))
+        return link
+
+    def get_description(self, path, method):
+        self.view._request = self.view.request
+        setattr(self.view.request, 'swagger_method', method)
+        description = super(AutoSchema, self).get_description(path, method)
+        return description
+
+
+class SwaggerSchemaView(APIView):
+    _ignore_model_permissions = True
+    exclude_from_schema = True
+    permission_classes = [AllowAny]
+    renderer_classes = [
+        CoreJSONRenderer,
+        renderers.OpenAPIRenderer,
+        renderers.SwaggerUIRenderer
+    ]
+
+    def get(self, request):
+        generator = SchemaGenerator(
+            title='Ansible Tower API',
+            patterns=None,
+            urlconf=None
+        )
+        schema = generator.get_schema(request=request)
+        # python core-api doesn't support the deprecation yet, so track it
+        # ourselves and return it in a response header
+        _deprecated = []
+
+        # By default, DRF OpenAPI serialization places all endpoints in
+        # a single node based on their root path (/api).  Instead, we want to
+        # group them by topic/tag so that they're categorized in the rendered
+        # output
+        document = schema._data.pop('api')
+        for path, node in document.items():
+            if isinstance(node, Object):
+                for action in node.values():
+                    topic = getattr(action, 'topic', None)
+                    if topic:
+                        schema._data.setdefault(topic, Object())
+                        schema._data[topic]._data[path] = node
+
+                    if isinstance(action, Object):
+                        for link in action.links.values():
+                            if link.deprecated:
+                                _deprecated.append(link.url)
+            elif isinstance(node, Link):
+                topic = getattr(node, 'topic', None)
+                if topic:
+                    schema._data.setdefault(topic, Object())
+                    schema._data[topic]._data[path] = node
+
+        if not schema:
+            raise exceptions.ValidationError(
+                'The schema generator did not return a schema Document'
+            )
+
+        return Response(
+            schema,
+            headers={'X-Deprecated-Paths': json.dumps(_deprecated)}
+        )

awx/api/templates/api/_list_common.md

@@ -60,9 +60,10 @@ _Added in AWX 1.4_
 
     ?related__search=findme
 
-Note: If you want to provide more than one search terms, please use multiple
+Note: If you want to provide more than one search term, multiple
 search fields with the same key, like `?related__search=foo&related__search=bar`,
-All search terms with the same key will be ORed together.
+will be ORed together. Terms separated by commas, like `?related__search=foo,bar`
+will be ANDed together.
 
 ## Filtering
 

awx/api/templates/api/_new_in_awx.md

@@ -1,14 +0,0 @@
-{% if not version_label_flag or version_label_flag == 'true' %}
-{% if new_in_13 %}> _Added in AWX 1.3_{% endif %}
-{% if new_in_14 %}> _Added in AWX 1.4_{% endif %}
-{% if new_in_145 %}> _Added in Ansible Tower 1.4.5_{% endif %}
-{% if new_in_148 %}> _Added in Ansible Tower 1.4.8_{% endif %}
-{% if new_in_200 %}> _Added in Ansible Tower 2.0.0_{% endif %}
-{% if new_in_220 %}> _Added in Ansible Tower 2.2.0_{% endif %}
-{% if new_in_230 %}> _Added in Ansible Tower 2.3.0_{% endif %}
-{% if new_in_240 %}> _Added in Ansible Tower 2.4.0_{% endif %}
-{% if new_in_300 %}> _Added in Ansible Tower 3.0.0_{% endif %}
-{% if new_in_310 %}> _New in Ansible Tower 3.1.0_{% endif %}
-{% if new_in_320 %}> _New in Ansible Tower 3.2.0_{% endif %}
-{% if new_in_330 %}> _New in Ansible Tower 3.3.0_{% endif %}
-{% endif %}

awx/api/templates/api/ad_hoc_command_relaunch.md

@@ -0,0 +1,3 @@
+Relaunch an Ad Hoc Command:
+
+Make a POST request to this resource to launch a job. If any passwords or variables are required then they should be passed in via POST data.   In order to determine what values are required in order to launch a job based on this job template you may make a GET request to this endpoint.

awx/api/templates/api/api_o_auth_authorization_root_view.md

@@ -0,0 +1,122 @@
+# Token Handling using OAuth2
+
+This page lists OAuth 2 utility endpoints used for authorization, token refresh and revoke.
+Note endpoints other than `/api/o/authorize/` are not meant to be used in browsers and do not
+support HTTP GET. The endpoints here strictly follow
+[RFC specs for OAuth2](https://tools.ietf.org/html/rfc6749), so please use that for detailed
+reference. Note AWX net location default to `http://localhost:8013` in examples:
+
+
+## Create Token for an Application using Authorization code grant type
+Given an application "AuthCodeApp" of grant type `authorization-code`, 
+from the client app, the user makes a GET to the Authorize endpoint with 
+
+* `response_type`
+* `client_id`
+* `redirect_uris`
+* `scope`  
+
+AWX will respond with the authorization `code` and `state`
+to the redirect_uri specified in the application. The client application will then make a POST to the
+`api/o/token/` endpoint on AWX with
+
+* `code`
+* `client_id`
+* `client_secret`
+* `grant_type`
+* `redirect_uri`
+
+AWX will respond with the `access_token`, `token_type`, `refresh_token`, and `expires_in`. For more
+information on testing this flow, refer to [django-oauth-toolkit](http://django-oauth-toolkit.readthedocs.io/en/latest/tutorial/tutorial_01.html#test-your-authorization-server).
+
+## Create Token for an Application using Implicit grant type
+Suppose we have an application "admin's app" of grant type `implicit`.
+In API browser, first make sure the user is logged in via session auth, then visit authorization
+endpoint with given parameters:
+```text
+http://localhost:8013/api/o/authorize/?response_type=token&client_id=L0uQQWW8pKX51hoqIRQGsuqmIdPi2AcXZ9EJRGmj&scope=read
+```
+Here the value of `client_id` should be the same as that of `client_id` field of underlying application.
+On success, an authorization page should be displayed asking the logged in user to grant/deny the access token.
+Once the user clicks on 'grant', the API browser will try POSTing to the same endpoint with the same parameters 
+in POST body, on success a 302 redirect will be returned.  
+
+## Create Token for an Application using Password grant type
+
+Log in is not required for `password` grant type, so a simple `curl` can be used to acquire a personal access token
+via `/api/o/token/` with 
+
+* `grant_type`: Required to be "password"
+* `username`
+* `password`
+* `client_id`: Associated application must have grant_type "password"
+* `client_secret`
+
+For example:
+
+```bash
+curl -X POST \
+  -d "grant_type=password&username=<username>&password=<password>&scope=read" \
+  -u "gwSPoasWSdNkMDtBN3Hu2WYQpPWCO9SwUEsKK22l:fI6ZpfocHYBGfm1tP92r0yIgCyfRdDQt0Tos9L8a4fNsJjQQMwp9569e
+IaUBsaVDgt2eiwOGe0bg5m5vCSstClZmtdy359RVx2rQK5YlIWyPlrolpt2LEpVeKXWaiybo" \
+  http://localhost:8013/api/o/token/ -i
+```
+In the above post request, parameters `username` and `password` are username and password of the related
+AWX user of the underlying application, and the authentication information is of format
+`<client_id>:<client_secret>`, where `client_id` and `client_secret` are the corresponding fields of
+underlying application.
+
+Upon success, access token, refresh token and other information are given in the response body in JSON
+format:
+
+```text
+{
+"access_token": "9epHOqHhnXUcgYK8QanOmUQPSgX92g", 
+"token_type": "Bearer", 
+"expires_in": 31536000000, 
+"refresh_token": "jMRX6QvzOTf046KHee3TU5mT3nyXsz", 
+"scope": "read"
+}
+```
+
+
+## Refresh an existing access token
+
+The `/api/o/token/` endpoint is used for refreshing access token:
+```bash
+curl -X POST \
+  -d "grant_type=refresh_token&refresh_token=AL0NK9TTpv0qp54dGbC4VUZtsZ9r8z" \
+  -u "gwSPoasWSdNkMDtBN3Hu2WYQpPWCO9SwUEsKK22l:fI6ZpfocHYBGfm1tP92r0yIgCyfRdDQt0Tos9L8a4fNsJjQQMwp9569eIaUBsaVDgt2eiwOGe0bg5m5vCSstClZmtdy359RVx2rQK5YlIWyPlrolpt2LEpVeKXWaiybo" \
+  http://localhost:8013/api/o/token/ -i
+```
+In the above post request, `refresh_token` is provided by `refresh_token` field of the access token
+above. The authentication information is of format `<client_id>:<client_secret>`, where `client_id`
+and `client_secret` are the corresponding fields of underlying related application of the access token.
+
+Upon success, the new (refreshed) access token with the same scope information as the previous one is
+given in the response body in JSON format:
+```text
+{
+"access_token": "NDInWxGJI4iZgqpsreujjbvzCfJqgR", 
+"token_type": "Bearer", 
+"expires_in": 31536000000, 
+"refresh_token": "DqOrmz8bx3srlHkZNKmDpqA86bnQkT", 
+"scope": "read write"
+}
+```
+Internally, the refresh operation deletes the existing token and a new token is created immediately
+after, with information like scope and related application identical to the original one. We can
+verify by checking the new token is present at the `api/v2/tokens` endpoint.  
+
+## Revoke an access token
+Revoking an access token is the same as deleting the token resource object. 
+Revoking is done by POSTing to `/api/o/revoke_token/` with the token to revoke as parameter:
+
+```bash
+curl -X POST -d "token=rQONsve372fQwuc2pn76k3IHDCYpi7" \
+  -u "gwSPoasWSdNkMDtBN3Hu2WYQpPWCO9SwUEsKK22l:fI6ZpfocHYBGfm1tP92r0yIgCyfRdDQt0Tos9L8a4fNsJjQQMwp9569eIaUBsaVDgt2eiwOGe0bg5m5vCSstClZmtdy359RVx2rQK5YlIWyPlrolpt2LEpVeKXWaiybo" \
+  http://localhost:8013/api/o/revoke_token/ -i
+```
+`200 OK` means a successful delete.
+
+

awx/api/templates/api/api_v1_config_view.md

@@ -1,4 +1,5 @@
-Site configuration settings and general information.
+{% ifmeth GET %}
+# Site configuration settings and general information
 
 Make a GET request to this resource to retrieve the configuration containing
 the following fields (some fields may not be visible to all users):
@@ -11,6 +12,10 @@ the following fields (some fields may not be visible to all users):
 * `license_info`: Information about the current license.
 * `version`: Version of Ansible Tower package installed.
 * `eula`: The current End-User License Agreement
+{% endifmeth %}
+
+{% ifmeth POST %}
+# Install or update an existing license
 
 (_New in Ansible Tower 2.0.0_) Make a POST request to this resource as a super
 user to install or update the existing license.  The license data itself can
@@ -18,3 +23,11 @@ be POSTed as a normal json data structure.
 
 (_New in Ansible Tower 2.1.1_) The POST must include a `eula_accepted` boolean
 element indicating acceptance of the End-User License Agreement.
+{% endifmeth %}
+
+{% ifmeth DELETE %}
+# Delete an existing license
+
+(_New in Ansible Tower 2.0.0_) Make a DELETE request to this resource as a super
+user to delete the existing license
+{% endifmeth %}

awx/api/templates/api/api_view.md

@@ -1,3 +1 @@
 {{ docstring }}
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/auth_token_view.md

@@ -1,37 +0,0 @@
-Make a POST request to this resource with `username` and `password` fields to
-obtain an authentication token to use for subsequent requests.
-
-Example JSON to POST (content type is `application/json`):
-
-    {"username": "user", "password": "my pass"}
-
-Example form data to post (content type is `application/x-www-form-urlencoded`):
-
-    username=user&password=my%20pass
-
-If the username and password provided are valid, the response will contain a
-`token` field with the authentication token to use and an `expires` field with
-the timestamp when the token will expire:
-
-    {
-        "token": "8f17825cf08a7efea124f2638f3896f6637f8745",
-        "expires": "2013-09-05T21:46:35.729Z"
-    }
-
-Otherwise, the response will indicate the error that occurred and return a 4xx
-status code.
-
-For subsequent requests, pass the token via the HTTP `Authorization` request
-header:
-
-    Authorization: Token 8f17825cf08a7efea124f2638f3896f6637f8745
-
-The auth token is only valid when used from the same remote address and user
-agent that originally obtained it.
-
-Each request that uses the token for authentication will refresh its expiration
-timestamp and keep it from expiring.  A token only expires when it is not used
-for the configured timeout interval (default 1800 seconds).
-
-A DELETE request with the token set will cause the token to be invalidated and
-no further requests can be made with it.

awx/api/templates/api/base_variable_data.md

@@ -1,9 +1,13 @@
+{% ifmeth GET %}
 # Retrieve {{ model_verbose_name|title }} Variable Data:
 
-Make a GET request to this resource to retrieve all variables defined for this
+Make a GET request to this resource to retrieve all variables defined for a
 {{ model_verbose_name }}.
+{% endifmeth %}
 
+{% ifmeth PUT PATCH %}
 # Update {{ model_verbose_name|title }} Variable Data:
 
-Make a PUT request to this resource to update variables defined for this
+Make a PUT or PATCH request to this resource to update variables defined for a
 {{ model_verbose_name }}.
+{% endifmeth %}

awx/api/templates/api/dashboard_inventory_graph_view.md

@@ -38,5 +38,3 @@ Data about failed and successfull hosts by inventory will be given as:
             "id": 2, 
             "name": "Test Inventory"
      },
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/dashboard_jobs_graph_view.md

@@ -1,3 +1,5 @@
+# View Statistics for Job Runs
+
 Make a GET request to this resource to retrieve aggregate statistics about job runs suitable for graphing.
 
 ## Parmeters and Filtering
@@ -33,5 +35,3 @@ Data will be returned in the following format:
 
 Each element contains an epoch timestamp represented in seconds and a numerical value indicating
 the number of events during that time period
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/dashboard_view.md

@@ -1,3 +1 @@
 Make a GET request to this resource to retrieve aggregate statistics for Tower.
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/group_all_hosts_list.md

@@ -1,4 +1,4 @@
-# List All {{ model_verbose_name_plural|title }} for this {{ parent_model_verbose_name|title }}:
+# List All {{ model_verbose_name_plural|title }} for {{ parent_model_verbose_name|title|anora }}:
 
 Make a GET request to this resource to retrieve a list of all
 {{ model_verbose_name_plural }} directly or indirectly belonging to this

awx/api/templates/api/group_potential_children_list.md

@@ -1,9 +1,7 @@
-# List Potential Child Groups for this {{ parent_model_verbose_name|title }}:
+# List Potential Child Groups for {{ parent_model_verbose_name|title|anora }}:
 
 Make a GET request to this resource to retrieve a list of
 {{ model_verbose_name_plural }} available to be added as children of the
 current {{ parent_model_verbose_name }}.
 
 {% include "api/_list_common.md" %}
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/host_all_groups_list.md

@@ -1,4 +1,4 @@
-# List All {{ model_verbose_name_plural|title }} for this {{ parent_model_verbose_name|title }}:
+# List All {{ model_verbose_name_plural|title }} for {{ parent_model_verbose_name|title|anora }}:
 
 Make a GET request to this resource to retrieve a list of all
 {{ model_verbose_name_plural }} of which the selected

awx/api/templates/api/host_fact_compare_view.md

@@ -1,3 +1,5 @@
+# List Fact Scans for a Host Specific Host Scan
+
 Make a GET request to this resource to retrieve system tracking data for a particular scan
 
 You may filter by datetime:
@@ -7,5 +9,3 @@ You may filter by datetime:
 and module
 
 `?datetime=2015-06-01&module=ansible`
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/host_fact_versions_list.md

@@ -1,3 +1,5 @@
+# List Fact Scans for a Host by Module and Date
+
 Make a GET request to this resource to retrieve system tracking scans by module and date/time
 
 You may filter scan runs using the `from` and `to` properties:
@@ -7,5 +9,3 @@ You may filter scan runs using the `from` and `to` properties:
 You may also filter by module
 
 `?module=packages`
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/host_insights.md

@@ -0,0 +1 @@
+# List Red Hat Insights for a Host

awx/api/templates/api/inventory_inventory_sources_update.md

@@ -29,5 +29,3 @@ Response code from this action will be:
  - 202 if some inventory source updates were successful, but some failed
  - 400 if all of the inventory source updates failed
  - 400 if there are no inventory sources in the inventory
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/inventory_root_groups_list.md

@@ -1,7 +1,9 @@
-# List Root {{ model_verbose_name_plural|title }} for this {{ parent_model_verbose_name|title }}:
+{% ifmeth GET %}
+# List Root {{ model_verbose_name_plural|title }} for {{ parent_model_verbose_name|title|anora }}:
 
 Make a GET request to this resource to retrieve a list of root (top-level)
 {{ model_verbose_name_plural }} associated with this
 {{ parent_model_verbose_name }}.
 
 {% include "api/_list_common.md" %}
+{% endifmeth %}

awx/api/templates/api/inventory_source_cancel.md

@@ -9,5 +9,3 @@ cancelled.  The response will include the following field:
 Make a POST request to this resource to cancel a pending or running inventory
 update.  The response status code will be 202 if successful, or 405 if the
 update cannot be canceled.
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/inventory_source_update_view.md

@@ -9,5 +9,3 @@ from its inventory source.  The response will include the following field:
 Make a POST request to this resource to update the inventory source.  If
 successful, the response status code will be 202.  If the inventory source is
 not defined or cannot be updated, a 405 status code will be returned.
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/inventory_tree_view.md

@@ -1,4 +1,4 @@
-# Group Tree for this {{ model_verbose_name|title }}:
+# Group Tree for {{ model_verbose_name|title|anora }}:
 
 Make a GET request to this resource to retrieve a hierarchical view of groups
 associated with the selected {{ model_verbose_name }}.
@@ -11,5 +11,3 @@ also containing a list of its children.
 Each group data structure includes the following fields:
 
 {% include "api/_result_fields_common.md" %}
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/job_cancel.md

@@ -1,10 +1,15 @@
-# Cancel Job
+{% ifmeth GET %}
+# Determine if a Job can be cancelled
 
 Make a GET request to this resource to determine if the job can be cancelled.
 The response will include the following field:
 
 * `can_cancel`: Indicates whether this job can be canceled (boolean, read-only)
+{% endifmeth %}
 
+{% ifmeth POST %}
+# Cancel a Job
 Make a POST request to this resource to cancel a pending or running job.  The
 response status code will be 202 if successful, or 405 if the job cannot be
 canceled.
+{% endifmeth %}

awx/api/templates/api/job_job_plays_list.md

@@ -23,5 +23,3 @@ Will show only failed plays.  Alternatively `false` may be used.
     ?play__icontains=test
 
 Will filter plays matching the substring `test`
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/job_job_tasks_list.md

@@ -25,5 +25,3 @@ Will show only failed plays.  Alternatively `false` may be used.
     ?task__icontains=test
 
 Will filter tasks matching the substring `test`
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/job_relaunch.md

@@ -1,3 +1,3 @@
-Relaunch a job:
+Relaunch a Job:
 
-Make a POST request to this resource to launch a job. If any passwords or variables are required then they should be passed in via POST data.   In order to determine what values are required in order to launch a job based on this job template you may make a GET request to this endpoint.
+Make a POST request to this resource to launch a job. If any passwords or variables are required then they should be passed in via POST data.   In order to determine what values are required in order to launch a job based on this job template you may make a GET request to this endpoint.

awx/api/templates/api/job_start.md

@@ -1,4 +1,5 @@
-# Start Job
+{% ifmeth GET %}
+# Determine if a Job can be started
 
 Make a GET request to this resource to determine if the job can be started and
 whether any passwords are required to start the job.  The response will include
@@ -7,10 +8,14 @@ the following fields:
 * `can_start`: Flag indicating if this job can be started (boolean, read-only)
 * `passwords_needed_to_start`: Password names required to start the job (array,
   read-only)
+{% endifmeth %}
 
+{% ifmeth POST %}
+# Start a Job
 Make a POST request to this resource to start the job.  If any passwords are
 required, they must be passed via POST data.
 
 If successful, the response status code will be 202.  If any required passwords
 are not provided, a 400 status code will be returned.  If the job cannot be
 started, a 405 status code will be returned.
+{% endifmeth %}

awx/api/templates/api/job_template_label_list.md

@@ -1,13 +1,7 @@
-{% with 'false' as version_label_flag %}
 {% include "api/sub_list_create_api_view.md" %}
-{% endwith %}
 
 Labels not associated with any other resources are deleted. A label can become disassociated with a resource as a result of 3 events.
 
 1. A label is explicitly disassociated with a related job template
 2. A job is deleted with labels
 3. A cleanup job deletes a job with labels
-
-{% with 'true' as version_label_flag %}
-{% include "api/_new_in_awx.md" %}
-{% endwith %}

awx/api/templates/api/list_api_view.md

@@ -1,8 +1,8 @@
+{% ifmeth GET %}
 # List {{ model_verbose_name_plural|title }}:
 
 Make a GET request to this resource to retrieve the list of
 {{ model_verbose_name_plural }}.
 
 {% include "api/_list_common.md" %}
-
-{% include "api/_new_in_awx.md" %}
+{% endifmeth %}

awx/api/templates/api/list_create_api_view.md

@@ -1,6 +1,6 @@
 {% include "api/list_api_view.md" %}
 
-# Create {{ model_verbose_name_plural|title }}:
+# Create {{ model_verbose_name|title|anora }}:
 
 Make a POST request to this resource with the following {{ model_verbose_name }}
 fields to create a new {{ model_verbose_name }}:
@@ -8,5 +8,3 @@ fields to create a new {{ model_verbose_name }}:
 {% with write_only=1 %}
 {% include "api/_result_fields_common.md" with serializer_fields=serializer_create_fields %}
 {% endwith %}
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/project_playbooks.md

@@ -1,4 +1,4 @@
 # Retrieve {{ model_verbose_name|title }} Playbooks:
 
 Make GET request to this resource to retrieve a list of playbooks available
-for this {{ model_verbose_name }}.
+for {{ model_verbose_name|anora }}.

awx/api/templates/api/project_update_cancel.md

@@ -9,5 +9,3 @@ cancelled.  The response will include the following field:
 Make a POST request to this resource to cancel a pending or running project
 update.  The response status code will be 202 if successful, or 405 if the
 update cannot be canceled.
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/project_update_view.md

@@ -8,5 +8,3 @@ from its SCM source.  The response will include the following field:
 
 Make a POST request to this resource to update the project.  If the project
 cannot be updated, a 405 status code will be returned.
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/retrieve_api_view.md

@@ -2,11 +2,9 @@
 ### Note: starting from api v2, this resource object can be accessed via its named URL.
 {% endif %}
 
-# Retrieve {{ model_verbose_name|title }}:
+# Retrieve {{ model_verbose_name|title|anora }}:
 
 Make GET request to this resource to retrieve a single {{ model_verbose_name }}
 record containing the following fields:
 
 {% include "api/_result_fields_common.md" %}
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/retrieve_destroy_api_view.md

@@ -2,15 +2,17 @@
 ### Note: starting from api v2, this resource object can be accessed via its named URL.
 {% endif %}
 
-# Retrieve {{ model_verbose_name|title }}:
+{% ifmeth GET %}
+# Retrieve {{ model_verbose_name|title|anora }}:
 
 Make GET request to this resource to retrieve a single {{ model_verbose_name }}
 record containing the following fields:
 
 {% include "api/_result_fields_common.md" %}
+{% endifmeth %}
 
-# Delete {{ model_verbose_name|title }}:
+{% ifmeth DELETE %}
+# Delete {{ model_verbose_name|title|anora }}:
 
 Make a DELETE request to this resource to delete this {{ model_verbose_name }}.
-
-{% include "api/_new_in_awx.md" %}
+{% endifmeth %}

awx/api/templates/api/retrieve_update_api_view.md

@@ -2,14 +2,17 @@
 ### Note: starting from api v2, this resource object can be accessed via its named URL.
 {% endif %}
 
-# Retrieve {{ model_verbose_name|title }}:
+{% ifmeth GET %}
+# Retrieve {{ model_verbose_name|title|anora }}:
 
 Make GET request to this resource to retrieve a single {{ model_verbose_name }}
 record containing the following fields:
 
 {% include "api/_result_fields_common.md" %}
+{% endifmeth %}
 
-# Update {{ model_verbose_name|title }}:
+{% ifmeth PUT PATCH %}
+# Update {{ model_verbose_name|title|anora }}:
 
 Make a PUT or PATCH request to this resource to update this
 {{ model_verbose_name }}.  The following fields may be modified:
@@ -17,9 +20,12 @@ Make a PUT or PATCH request to this resource to update this
 {% with write_only=1 %}
 {% include "api/_result_fields_common.md" with serializer_fields=serializer_update_fields %}
 {% endwith %}
+{% endifmeth %}
 
+{% ifmeth PUT %}
 For a PUT request, include **all** fields in the request.
+{% endifmeth %}
 
+{% ifmeth PATCH %}
 For a PATCH request, include only the fields that are being modified.
-
-{% include "api/_new_in_awx.md" %}
+{% endifmeth %}

awx/api/templates/api/retrieve_update_destroy_api_view.md

@@ -2,14 +2,17 @@
 ### Note: starting from api v2, this resource object can be accessed via its named URL.
 {% endif %}
 
-# Retrieve {{ model_verbose_name|title }}:
+{% ifmeth GET %}
+# Retrieve {{ model_verbose_name|title|anora }}:
 
 Make GET request to this resource to retrieve a single {{ model_verbose_name }}
 record containing the following fields:
 
 {% include "api/_result_fields_common.md" %}
+{% endifmeth %}
 
-# Update {{ model_verbose_name|title }}:
+{% ifmeth PUT PATCH %}
+# Update {{ model_verbose_name|title|anora }}:
 
 Make a PUT or PATCH request to this resource to update this
 {{ model_verbose_name }}.  The following fields may be modified:
@@ -17,13 +20,18 @@ Make a PUT or PATCH request to this resource to update this
 {% with write_only=1 %}
 {% include "api/_result_fields_common.md" with serializer_fields=serializer_update_fields %}
 {% endwith %}
+{% endifmeth %}
 
+{% ifmeth PUT %}
 For a PUT request, include **all** fields in the request.
+{% endifmeth %}
 
+{% ifmeth PATCH %}
 For a PATCH request, include only the fields that are being modified.
+{% endifmeth %}
 
-# Delete {{ model_verbose_name|title }}:
+{% ifmeth DELETE %}
+# Delete {{ model_verbose_name|title|anora }}:
 
 Make a DELETE request to this resource to delete this {{ model_verbose_name }}.
-
-{% include "api/_new_in_awx.md" %}
+{% endifmeth %}

awx/api/templates/api/setting_logging_test.md

@@ -0,0 +1 @@
+# Test Logging Configuration

awx/api/templates/api/sub_list_api_view.md

@@ -1,9 +1,9 @@
-# List {{ model_verbose_name_plural|title }} for this {{ parent_model_verbose_name|title }}:
+{% ifmeth GET %}
+# List {{ model_verbose_name_plural|title }} for {{ parent_model_verbose_name|title|anora }}:
 
 Make a GET request to this resource to retrieve a list of
 {{ model_verbose_name_plural }} associated with the selected
 {{ parent_model_verbose_name }}.
 
 {% include "api/_list_common.md" %}
-
-{% include "api/_new_in_awx.md" %}
+{% endifmeth %}

awx/api/templates/api/sub_list_create_api_view.md

@@ -1,6 +1,6 @@
 {% include "api/sub_list_api_view.md" %}
 
-# Create {{ model_verbose_name_plural|title }} for this {{ parent_model_verbose_name|title }}:
+# Create {{ model_verbose_name|title|anora }} for {{ parent_model_verbose_name|title|anora }}:
 
 Make a POST request to this resource with the following {{ model_verbose_name }}
 fields to create a new {{ model_verbose_name }} associated with this
@@ -25,7 +25,7 @@ delete the associated {{ model_verbose_name }}.
     }
 
 {% else %}
-# Add {{ model_verbose_name_plural|title }} for this {{ parent_model_verbose_name|title }}:
+# Add {{ model_verbose_name_plural|title }} for {{ parent_model_verbose_name|title|anora }}:
 
 Make a POST request to this resource with only an `id` field to associate an
 existing {{ model_verbose_name }} with this {{ parent_model_verbose_name }}.
@@ -37,5 +37,3 @@ remove the {{ model_verbose_name }} from this {{ parent_model_verbose_name }}
 {% if model_verbose_name != "label" %} without deleting the {{ model_verbose_name }}{% endif %}.
 {% endif %}
 {% endif %}
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/system_job_template_launch.md

@@ -12,12 +12,6 @@ For example on `cleanup_jobs` and `cleanup_activitystream`:
 
 Which will act on data older than 30 days.
 
-For `cleanup_facts`:
-
-`{"extra_vars": {"older_than": "4w", "granularity": "3d"}}`
-
-Which will reduce the granularity of scan data to one scan per 3 days when the data is older than 4w.
-
 For `cleanup_activitystream` and `cleanup_jobs` commands, providing
 `"dry_run": true` inside of `extra_vars` will show items that will be
 removed without deleting them.
@@ -27,7 +21,6 @@ applicable either when running it from the command line or launching its
 system job template with empty `extra_vars`.
 
  - Defaults for `cleanup_activitystream`: days=90
- - Defaults for `cleanup_facts`: older_than="30d", granularity="1w"
  - Defaults for `cleanup_jobs`: days=90
 
 If successful, the response status code will be 202.  If the job cannot be

awx/api/templates/api/team_roles_list.md

@@ -1,12 +1,16 @@
-# List Roles for this Team:
+# List Roles for a Team:
 
+{% ifmeth GET %}
 Make a GET request to this resource to retrieve a list of roles associated with the selected team.
 
 {% include "api/_list_common.md" %}
+{% endifmeth %}
 
+{% ifmeth POST %}
 # Associate Roles with this Team:
 
 Make a POST request to this resource to add or remove a role from this team. The following fields may be modified:
 
    * `id`: The Role ID to add to the team. (int, required)
    * `disassociate`: Provide if you want to remove the role. (any value, optional)
+{% endifmeth %}

awx/api/templates/api/unified_job_stdout.md

@@ -25,5 +25,3 @@ dark background.
 Files over {{ settings.STDOUT_MAX_BYTES_DISPLAY|filesizeformat }} (configurable)
 will not display in the browser. Use the `txt_download` or `ansi_download`
 formats to download the file directly to view it.
-
-{% include "api/_new_in_awx.md" %}

awx/api/templates/api/user_me_list.md

@@ -1,3 +1,5 @@
+# Retrieve Information about the current User
+
 Make a GET request to retrieve user information about the current user.
 
 One result should be returned containing the following fields:

awx/api/templates/api/user_roles_list.md

@@ -1,12 +1,16 @@
-# List Roles for this User:
+# List Roles for a User:
 
+{% ifmeth GET %}
 Make a GET request to this resource to retrieve a list of roles associated with the selected user.
 
 {% include "api/_list_common.md" %}
+{% endifmeth %}
 
+{% ifmeth POST %}
 # Associate Roles with this User:
 
 Make a POST request to this resource to add or remove a role from this user. The following fields may be modified:
 
    * `id`: The Role ID to add to the user. (int, required)
    * `disassociate`: Provide if you want to remove the role. (any value, optional)
+{% endifmeth %}

awx/api/urls/credential.py

@@ -11,6 +11,7 @@ from awx.api.views import (
     CredentialObjectRolesList,
     CredentialOwnerUsersList,
     CredentialOwnerTeamsList,
+    CredentialCopy,
 )
 
 
@@ -22,6 +23,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/object_roles/$', CredentialObjectRolesList.as_view(), name='credential_object_roles_list'),
     url(r'^(?P<pk>[0-9]+)/owner_users/$', CredentialOwnerUsersList.as_view(), name='credential_owner_users_list'),
     url(r'^(?P<pk>[0-9]+)/owner_teams/$', CredentialOwnerTeamsList.as_view(), name='credential_owner_teams_list'),
+    url(r'^(?P<pk>[0-9]+)/copy/$', CredentialCopy.as_view(), name='credential_copy'),
 ]
 
 __all__ = ['urls']

awx/api/urls/inventory.py

@@ -20,6 +20,7 @@ from awx.api.views import (
     InventoryAccessList,
     InventoryObjectRolesList,
     InventoryInstanceGroupsList,
+    InventoryCopy,
 )
 
 
@@ -40,6 +41,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/access_list/$', InventoryAccessList.as_view(), name='inventory_access_list'),
     url(r'^(?P<pk>[0-9]+)/object_roles/$', InventoryObjectRolesList.as_view(), name='inventory_object_roles_list'),
     url(r'^(?P<pk>[0-9]+)/instance_groups/$', InventoryInstanceGroupsList.as_view(), name='inventory_instance_groups_list'),
+    url(r'^(?P<pk>[0-9]+)/copy/$', InventoryCopy.as_view(), name='inventory_copy'),
 ]
 
 __all__ = ['urls']

awx/api/urls/inventory_script.py

@@ -7,6 +7,7 @@ from awx.api.views import (
     InventoryScriptList,
     InventoryScriptDetail,
     InventoryScriptObjectRolesList,
+    InventoryScriptCopy,
 )
 
 
@@ -14,6 +15,7 @@ urls = [
     url(r'^$', InventoryScriptList.as_view(), name='inventory_script_list'),
     url(r'^(?P<pk>[0-9]+)/$', InventoryScriptDetail.as_view(), name='inventory_script_detail'),
     url(r'^(?P<pk>[0-9]+)/object_roles/$', InventoryScriptObjectRolesList.as_view(), name='inventory_script_object_roles_list'),
+    url(r'^(?P<pk>[0-9]+)/copy/$', InventoryScriptCopy.as_view(), name='inventory_script_copy'),
 ]
 
 __all__ = ['urls']

awx/api/urls/inventory_source.py

@@ -10,6 +10,7 @@ from awx.api.views import (
     InventorySourceUpdatesList,
     InventorySourceActivityStreamList,
     InventorySourceSchedulesList,
+    InventorySourceCredentialsList,
     InventorySourceGroupsList,
     InventorySourceHostsList,
     InventorySourceNotificationTemplatesAnyList,
@@ -25,6 +26,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/inventory_updates/$', InventorySourceUpdatesList.as_view(), name='inventory_source_updates_list'),
     url(r'^(?P<pk>[0-9]+)/activity_stream/$', InventorySourceActivityStreamList.as_view(), name='inventory_source_activity_stream_list'),
     url(r'^(?P<pk>[0-9]+)/schedules/$', InventorySourceSchedulesList.as_view(), name='inventory_source_schedules_list'),
+    url(r'^(?P<pk>[0-9]+)/credentials/$', InventorySourceCredentialsList.as_view(), name='inventory_source_credentials_list'),
     url(r'^(?P<pk>[0-9]+)/groups/$', InventorySourceGroupsList.as_view(), name='inventory_source_groups_list'),
     url(r'^(?P<pk>[0-9]+)/hosts/$', InventorySourceHostsList.as_view(), name='inventory_source_hosts_list'),
     url(r'^(?P<pk>[0-9]+)/notification_templates_any/$', InventorySourceNotificationTemplatesAnyList.as_view(),

awx/api/urls/inventory_update.py

@@ -9,6 +9,7 @@ from awx.api.views import (
     InventoryUpdateCancel,
     InventoryUpdateStdout,
     InventoryUpdateNotificationsList,
+    InventoryUpdateCredentialsList,
     InventoryUpdateEventsList,
 )
 
@@ -19,6 +20,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/cancel/$', InventoryUpdateCancel.as_view(), name='inventory_update_cancel'),
     url(r'^(?P<pk>[0-9]+)/stdout/$', InventoryUpdateStdout.as_view(), name='inventory_update_stdout'),
     url(r'^(?P<pk>[0-9]+)/notifications/$', InventoryUpdateNotificationsList.as_view(), name='inventory_update_notifications_list'),
+    url(r'^(?P<pk>[0-9]+)/credentials/$', InventoryUpdateCredentialsList.as_view(), name='inventory_update_credentials_list'),
     url(r'^(?P<pk>[0-9]+)/events/$', InventoryUpdateEventsList.as_view(), name='inventory_update_events_list'),
 ]
 

awx/api/urls/job_template.py

@@ -19,6 +19,7 @@ from awx.api.views import (
     JobTemplateAccessList,
     JobTemplateObjectRolesList,
     JobTemplateLabelList,
+    JobTemplateCopy,
 )
 
 
@@ -41,6 +42,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/access_list/$', JobTemplateAccessList.as_view(), name='job_template_access_list'),
     url(r'^(?P<pk>[0-9]+)/object_roles/$', JobTemplateObjectRolesList.as_view(), name='job_template_object_roles_list'),
     url(r'^(?P<pk>[0-9]+)/labels/$', JobTemplateLabelList.as_view(), name='job_template_label_list'),
+    url(r'^(?P<pk>[0-9]+)/copy/$', JobTemplateCopy.as_view(), name='job_template_copy'),
 ]
 
 __all__ = ['urls']

awx/api/urls/notification_template.py

@@ -8,6 +8,7 @@ from awx.api.views import (
     NotificationTemplateDetail,
     NotificationTemplateTest,
     NotificationTemplateNotificationList,
+    NotificationTemplateCopy,
 )
 
 
@@ -16,6 +17,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/$', NotificationTemplateDetail.as_view(), name='notification_template_detail'),
     url(r'^(?P<pk>[0-9]+)/test/$', NotificationTemplateTest.as_view(), name='notification_template_test'),
     url(r'^(?P<pk>[0-9]+)/notifications/$', NotificationTemplateNotificationList.as_view(), name='notification_template_notification_list'),
+    url(r'^(?P<pk>[0-9]+)/copy/$', NotificationTemplateCopy.as_view(), name='notification_template_copy'),
 ]
 
 __all__ = ['urls']

awx/api/urls/oauth.py

@@ -0,0 +1,18 @@
+# Copyright (c) 2017 Ansible, Inc.
+# All Rights Reserved.
+
+from django.conf.urls import url
+
+from oauth2_provider.urls import base_urlpatterns
+
+from awx.api.views import (
+    ApiOAuthAuthorizationRootView,
+)
+
+
+urls = [
+    url(r'^$', ApiOAuthAuthorizationRootView.as_view(), name='oauth_authorization_root_view'),
+] + base_urlpatterns
+
+
+__all__ = ['urls']

awx/api/urls/organization.py

@@ -21,6 +21,7 @@ from awx.api.views import (
     OrganizationInstanceGroupsList,
     OrganizationObjectRolesList,
     OrganizationAccessList,
+    OrganizationApplicationList,
 )
 
 
@@ -45,6 +46,7 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/instance_groups/$', OrganizationInstanceGroupsList.as_view(), name='organization_instance_groups_list'),
     url(r'^(?P<pk>[0-9]+)/object_roles/$', OrganizationObjectRolesList.as_view(), name='organization_object_roles_list'),
     url(r'^(?P<pk>[0-9]+)/access_list/$', OrganizationAccessList.as_view(), name='organization_access_list'),
+    url(r'^(?P<pk>[0-9]+)/applications/$', OrganizationApplicationList.as_view(), name='organization_applications_list'),
 ]
 
 __all__ = ['urls']

awx/api/urls/project.py

@@ -19,10 +19,11 @@ from awx.api.views import (
     ProjectNotificationTemplatesSuccessList,
     ProjectObjectRolesList,
     ProjectAccessList,
+    ProjectCopy,
 )
 
 
-urls = [ 
+urls = [
     url(r'^$', ProjectList.as_view(), name='project_list'),
     url(r'^(?P<pk>[0-9]+)/$', ProjectDetail.as_view(), name='project_detail'),
     url(r'^(?P<pk>[0-9]+)/playbooks/$', ProjectPlaybooks.as_view(), name='project_playbooks'),
@@ -39,6 +40,7 @@ urls = [
         name='project_notification_templates_success_list'),
     url(r'^(?P<pk>[0-9]+)/object_roles/$', ProjectObjectRolesList.as_view(), name='project_object_roles_list'),
     url(r'^(?P<pk>[0-9]+)/access_list/$', ProjectAccessList.as_view(), name='project_access_list'),
+    url(r'^(?P<pk>[0-9]+)/copy/$', ProjectCopy.as_view(), name='project_copy'),
 ]
 
 __all__ = ['urls']

awx/api/urls/urls.py

@@ -2,8 +2,13 @@
 # All Rights Reserved.
 
 from __future__ import absolute_import, unicode_literals
+from django.conf import settings
 from django.conf.urls import include, url
 
+from awx.api.generics import (
+    LoggedLoginView,
+    LoggedLogoutView,
+)
 from awx.api.views import (
     ApiRootView,
     ApiV1RootView,
@@ -11,7 +16,6 @@ from awx.api.views import (
     ApiV1PingView,
     ApiV1ConfigView,
     AuthView,
-    AuthTokenView,
     UserMeList,
     DashboardView,
     DashboardJobsGraphView,
@@ -24,6 +28,10 @@ from awx.api.views import (
     JobTemplateExtraCredentialsList,
     SchedulePreview,
     ScheduleZoneInfo,
+    OAuth2ApplicationList,
+    OAuth2TokenList,
+    ApplicationOAuth2TokenList,
+    OAuth2ApplicationDetail,
 )
 
 from .organization import urls as organization_urls
@@ -59,6 +67,8 @@ from .schedule import urls as schedule_urls
 from .activity_stream import urls as activity_stream_urls
 from .instance import urls as instance_urls
 from .instance_group import urls as instance_group_urls
+from .user_oauth import urls as user_oauth_urls
+from .oauth import urls as oauth_urls
 
 
 v1_urls = [
@@ -66,7 +76,6 @@ v1_urls = [
     url(r'^ping/$', ApiV1PingView.as_view(), name='api_v1_ping_view'),
     url(r'^config/$', ApiV1ConfigView.as_view(), name='api_v1_config_view'),
     url(r'^auth/$', AuthView.as_view()),
-    url(r'^authtoken/$', AuthTokenView.as_view(), name='auth_token_view'),
     url(r'^me/$', UserMeList.as_view(), name='user_me_list'),
     url(r'^dashboard/$', DashboardView.as_view(), name='dashboard_view'),
     url(r'^dashboard/graphs/jobs/$', DashboardJobsGraphView.as_view(), name='dashboard_jobs_graph_view'),
@@ -117,11 +126,29 @@ v2_urls = [
     url(r'^job_templates/(?P<pk>[0-9]+)/credentials/$', JobTemplateCredentialsList.as_view(), name='job_template_credentials_list'),
     url(r'^schedules/preview/$', SchedulePreview.as_view(), name='schedule_rrule'),
     url(r'^schedules/zoneinfo/$', ScheduleZoneInfo.as_view(), name='schedule_zoneinfo'),
+    url(r'^applications/$', OAuth2ApplicationList.as_view(), name='o_auth2_application_list'),
+    url(r'^applications/(?P<pk>[0-9]+)/$', OAuth2ApplicationDetail.as_view(), name='o_auth2_application_detail'),
+    url(r'^applications/(?P<pk>[0-9]+)/tokens/$', ApplicationOAuth2TokenList.as_view(), name='application_o_auth2_token_list'),
+    url(r'^tokens/$', OAuth2TokenList.as_view(), name='o_auth2_token_list'),
+    url(r'^', include(user_oauth_urls)),
 ]
 
 app_name = 'api'
 urlpatterns = [
     url(r'^$', ApiRootView.as_view(), name='api_root_view'),
     url(r'^(?P<version>(v2))/', include(v2_urls)),
-    url(r'^(?P<version>(v1|v2))/', include(v1_urls))
+    url(r'^(?P<version>(v1|v2))/', include(v1_urls)),
+    url(r'^login/$', LoggedLoginView.as_view(
+        template_name='rest_framework/login.html',
+        extra_context={'inside_login_context': True}
+    ), name='login'),
+    url(r'^logout/$', LoggedLogoutView.as_view(
+        next_page='/api/', redirect_field_name='next'
+    ), name='logout'),
+    url(r'^o/', include(oauth_urls)),
 ]
+if settings.SETTINGS_MODULE == 'awx.settings.development':
+    from awx.api.swagger import SwaggerSchemaView
+    urlpatterns += [
+        url(r'^swagger/$', SwaggerSchemaView.as_view(), name='swagger_view'),
+    ]

awx/api/urls/user.py

@@ -14,9 +14,12 @@ from awx.api.views import (
     UserRolesList,
     UserActivityStreamList,
     UserAccessList,
+    OAuth2ApplicationList,
+    OAuth2TokenList,
+    OAuth2PersonalTokenList,
+    UserAuthorizedTokenList,
 )
 
-
 urls = [ 
     url(r'^$', UserList.as_view(), name='user_list'),
     url(r'^(?P<pk>[0-9]+)/$', UserDetail.as_view(), name='user_detail'),
@@ -28,6 +31,11 @@ urls = [
     url(r'^(?P<pk>[0-9]+)/roles/$', UserRolesList.as_view(), name='user_roles_list'),
     url(r'^(?P<pk>[0-9]+)/activity_stream/$', UserActivityStreamList.as_view(), name='user_activity_stream_list'),
     url(r'^(?P<pk>[0-9]+)/access_list/$', UserAccessList.as_view(), name='user_access_list'),
+    url(r'^(?P<pk>[0-9]+)/applications/$', OAuth2ApplicationList.as_view(), name='o_auth2_application_list'),
+    url(r'^(?P<pk>[0-9]+)/tokens/$', OAuth2TokenList.as_view(), name='o_auth2_token_list'),
+    url(r'^(?P<pk>[0-9]+)/authorized_tokens/$', UserAuthorizedTokenList.as_view(), name='user_authorized_token_list'),
+    url(r'^(?P<pk>[0-9]+)/personal_tokens/$', OAuth2PersonalTokenList.as_view(), name='o_auth2_personal_token_list'),
+    
 ] 
 
 __all__ = ['urls']

awx/api/urls/user_oauth.py

@@ -0,0 +1,49 @@
+# Copyright (c) 2017 Ansible, Inc.
+# All Rights Reserved.
+
+from django.conf.urls import url
+
+from awx.api.views import (
+    OAuth2ApplicationList,
+    OAuth2ApplicationDetail,
+    ApplicationOAuth2TokenList,
+    OAuth2ApplicationActivityStreamList,
+    OAuth2TokenList,
+    OAuth2TokenDetail,
+    OAuth2TokenActivityStreamList,
+    OAuth2PersonalTokenList
+)
+
+
+urls = [
+    url(r'^applications/$', OAuth2ApplicationList.as_view(), name='o_auth2_application_list'),
+    url(
+        r'^applications/(?P<pk>[0-9]+)/$',
+        OAuth2ApplicationDetail.as_view(),
+        name='o_auth2_application_detail'
+    ),
+    url(
+        r'^applications/(?P<pk>[0-9]+)/tokens/$',
+        ApplicationOAuth2TokenList.as_view(),
+        name='o_auth2_application_token_list'
+    ),
+    url(
+        r'^applications/(?P<pk>[0-9]+)/activity_stream/$',
+        OAuth2ApplicationActivityStreamList.as_view(),
+        name='o_auth2_application_activity_stream_list'
+    ),
+    url(r'^tokens/$', OAuth2TokenList.as_view(), name='o_auth2_token_list'),
+    url(
+        r'^tokens/(?P<pk>[0-9]+)/$',
+        OAuth2TokenDetail.as_view(),
+        name='o_auth2_token_detail'
+    ),
+    url(
+        r'^tokens/(?P<pk>[0-9]+)/activity_stream/$',
+        OAuth2TokenActivityStreamList.as_view(),
+        name='o_auth2_token_activity_stream_list'
+    ),
+    url(r'^personal_tokens/$', OAuth2PersonalTokenList.as_view(), name='o_auth2_personal_token_list'),    
+]
+
+__all__ = ['urls']

awx/celery.py

@@ -1,3 +1,4 @@
+
 # Copyright (c) 2017 Ansible, Inc.
 # All Rights Reserved.
 
@@ -5,6 +6,7 @@ from __future__ import absolute_import, unicode_literals
 
 import os
 from celery import Celery
+from django.conf import settings # noqa
 
 
 try:
@@ -16,8 +18,8 @@ except ImportError: # pragma: no cover
 os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'awx.settings.%s' % MODE)
 
 app = Celery('awx')
-app.config_from_object('django.conf:settings', namespace='CELERY')
-app.autodiscover_tasks()
+app.config_from_object('django.conf:settings')
+app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
 
 if __name__ == '__main__':
     app.start()

awx/conf/apps.py

@@ -11,8 +11,16 @@ class ConfConfig(AppConfig):
     name = 'awx.conf'
     verbose_name = _('Configuration')
 
+    def configure_oauth2_provider(self, settings):
+        from oauth2_provider import settings as o_settings
+        o_settings.oauth2_settings = o_settings.OAuth2ProviderSettings(
+            settings.OAUTH2_PROVIDER, o_settings.DEFAULTS,
+            o_settings.IMPORT_STRINGS, o_settings.MANDATORY
+        )
+
     def ready(self):
         self.module.autodiscover()
         from .settings import SettingsWrapper
         SettingsWrapper.initialize()
         configure_external_logger(settings)
+        self.configure_oauth2_provider(settings)

awx/conf/serializers.py

@@ -1,6 +1,8 @@
 # Django REST Framework
 from rest_framework import serializers
 
+import six
+
 # Tower
 from awx.api.fields import VerbatimField
 from awx.api.serializers import BaseSerializer
@@ -45,12 +47,12 @@ class SettingFieldMixin(object):
     """Mixin to use a registered setting field class for API display/validation."""
 
     def to_representation(self, obj):
-        if getattr(self, 'encrypted', False) and isinstance(obj, basestring) and obj:
+        if getattr(self, 'encrypted', False) and isinstance(obj, six.string_types) and obj:
             return '$encrypted$'
         return obj
 
     def to_internal_value(self, value):
-        if getattr(self, 'encrypted', False) and isinstance(value, basestring) and value.startswith('$encrypted$'):
+        if getattr(self, 'encrypted', False) and isinstance(value, six.string_types) and value.startswith('$encrypted$'):
             raise serializers.SkipField()
         obj = super(SettingFieldMixin, self).to_internal_value(value)
         return super(SettingFieldMixin, self).to_representation(obj)

awx/conf/settings.py

@@ -275,7 +275,7 @@ class SettingsWrapper(UserSettingsHolder):
                 setting_ids[setting.key] = setting.id
                 try:
                     value = decrypt_field(setting, 'value')
-                except ValueError, e:
+                except ValueError as e:
                     #TODO: Remove in Tower 3.3
                     logger.debug('encountered error decrypting field: %s - attempting fallback to old', e)
                     value = old_decrypt_field(setting, 'value')
@@ -305,7 +305,7 @@ class SettingsWrapper(UserSettingsHolder):
         settings_to_cache['_awx_conf_preload_expires'] = self._awx_conf_preload_expires
         self.cache.set_many(settings_to_cache, timeout=SETTING_CACHE_TIMEOUT)
 
-    def _get_local(self, name):
+    def _get_local(self, name, validate=True):
         self._preload_cache()
         cache_key = Setting.get_cache_key(name)
         try:
@@ -368,7 +368,10 @@ class SettingsWrapper(UserSettingsHolder):
                     field.run_validators(internal_value)
                     return internal_value
                 else:
-                    return field.run_validation(value)
+                    if validate:
+                        return field.run_validation(value)
+                    else:
+                        return value
             except Exception:
                 logger.warning(
                     'The current value "%r" for setting "%s" is invalid.',

awx/conf/utils.py

@@ -6,6 +6,8 @@ import glob
 import os
 import shutil
 
+import six
+
 # AWX
 from awx.conf.registry import settings_registry
 
@@ -13,7 +15,7 @@ __all__ = ['comment_assignments', 'conf_to_dict']
 
 
 def comment_assignments(patterns, assignment_names, dry_run=True, backup_suffix='.old'):
-    if isinstance(patterns, basestring):
+    if isinstance(patterns, six.string_types):
         patterns = [patterns]
     diffs = []
     for pattern in patterns:
@@ -32,7 +34,7 @@ def comment_assignments(patterns, assignment_names, dry_run=True, backup_suffix=
 def comment_assignments_in_file(filename, assignment_names, dry_run=True, backup_filename=None):
     from redbaron import RedBaron, indent
 
-    if isinstance(assignment_names, basestring):
+    if isinstance(assignment_names, six.string_types):
         assignment_names = [assignment_names]
     else:
         assignment_names = assignment_names[:]

awx/conf/views.py

@@ -21,7 +21,7 @@ from awx.api.generics import *  # noqa
 from awx.api.permissions import IsSuperUser
 from awx.api.versioning import reverse, get_request_version
 from awx.main.utils import *  # noqa
-from awx.main.utils.handlers import BaseHTTPSHandler, LoggingConnectivityException
+from awx.main.utils.handlers import BaseHTTPSHandler, UDPHandler, LoggingConnectivityException
 from awx.main.tasks import handle_setting_changes
 from awx.conf.license import get_licensed_features
 from awx.conf.models import Setting
@@ -44,7 +44,6 @@ class SettingCategoryList(ListAPIView):
     model = Setting  # Not exactly, but needed for the view.
     serializer_class = SettingCategorySerializer
     filter_backends = []
-    new_in_310 = True
     view_name = _('Setting Categories')
 
     def get_queryset(self):
@@ -69,7 +68,6 @@ class SettingSingletonDetail(RetrieveUpdateDestroyAPIView):
     model = Setting  # Not exactly, but needed for the view.
     serializer_class = SettingSingletonSerializer
     filter_backends = []
-    new_in_310 = True
     view_name = _('Setting Detail')
 
     def get_queryset(self):
@@ -170,7 +168,6 @@ class SettingLoggingTest(GenericAPIView):
     serializer_class = SettingSingletonSerializer
     permission_classes = (IsSuperUser,)
     filter_backends = []
-    new_in_320 = True
 
     def post(self, request, *args, **kwargs):
         defaults = dict()
@@ -202,7 +199,11 @@ class SettingLoggingTest(GenericAPIView):
             for k, v in serializer.validated_data.items():
                 setattr(mock_settings, k, v)
             mock_settings.LOG_AGGREGATOR_LEVEL = 'DEBUG'
-            BaseHTTPSHandler.perform_test(mock_settings)
+            if mock_settings.LOG_AGGREGATOR_PROTOCOL.upper() == 'UDP':
+                UDPHandler.perform_test(mock_settings)
+                return Response(status=status.HTTP_201_CREATED)
+            else:
+                BaseHTTPSHandler.perform_test(mock_settings)
         except LoggingConnectivityException as e:
             return Response({'error': str(e)}, status=status.HTTP_500_INTERNAL_SERVER_ERROR)
         return Response(status=status.HTTP_200_OK)

awx/lib/awx_display_callback/events.py

@@ -29,6 +29,8 @@ import threading
 import uuid
 import memcache
 
+from six.moves import xrange
+
 __all__ = ['event_context']
 
 

awx/lib/awx_display_callback/minimal.py

@@ -25,4 +25,5 @@ import ansible
 
 # Because of the way Ansible loads plugins, it's not possible to import
 # ansible.plugins.callback.minimal when being loaded as the minimal plugin. Ugh.
-execfile(os.path.join(os.path.dirname(ansible.__file__), 'plugins', 'callback', 'minimal.py'))
+with open(os.path.join(os.path.dirname(ansible.__file__), 'plugins', 'callback', 'minimal.py')) as in_file:
+    exec(in_file.read())

awx/lib/awx_display_callback/module.py

@@ -18,12 +18,17 @@
 from __future__ import (absolute_import, division, print_function)
 
 # Python
+import codecs
 import contextlib
+import json
+import os
+import stat
 import sys
 import uuid
 from copy import copy
 
 # Ansible
+from ansible import constants as C
 from ansible.plugins.callback import CallbackBase
 from ansible.plugins.callback.default import CallbackModule as DefaultCallbackModule
 
@@ -122,16 +127,19 @@ class BaseCallbackModule(CallbackBase):
             task=(task.name or task.action),
             task_uuid=str(task._uuid),
             task_action=task.action,
+            task_args='',
         )
         try:
             task_ctx['task_path'] = task.get_path()
         except AttributeError:
             pass
-        if task.no_log:
-            task_ctx['task_args'] = "the output has been hidden due to the fact that 'no_log: true' was specified for this result"
-        else:
-            task_args = ', '.join(('%s=%s' % a for a in task.args.items()))
-            task_ctx['task_args'] = task_args
+
+        if C.DISPLAY_ARGS_TO_STDOUT:
+            if task.no_log:
+                task_ctx['task_args'] = "the output has been hidden due to the fact that 'no_log: true' was specified for this result"
+            else:
+                task_args = ', '.join(('%s=%s' % a for a in task.args.items()))
+                task_ctx['task_args'] = task_args
         if getattr(task, '_role', None):
             task_role = task._role._role_name
         else:
@@ -270,15 +278,14 @@ class BaseCallbackModule(CallbackBase):
         with self.capture_event_data('playbook_on_no_hosts_remaining'):
             super(BaseCallbackModule, self).v2_playbook_on_no_hosts_remaining()
 
-    def v2_playbook_on_notify(self, result, handler):
-        # NOTE: Not used by Ansible 2.x.
+    def v2_playbook_on_notify(self, handler, host):
+        # NOTE: Not used by Ansible < 2.5.
         event_data = dict(
-            host=result._host.get_name(),
-            task=result._task,
-            handler=handler,
+            host=host.get_name(),
+            handler=handler.get_name(),
         )
         with self.capture_event_data('playbook_on_notify', **event_data):
-            super(BaseCallbackModule, self).v2_playbook_on_notify(result, handler)
+            super(BaseCallbackModule, self).v2_playbook_on_notify(handler, host)
 
     '''
     ansible_stats is, retoractively, added in 2.2
@@ -292,13 +299,33 @@ class BaseCallbackModule(CallbackBase):
             failures=stats.failures,
             ok=stats.ok,
             processed=stats.processed,
-            skipped=stats.skipped,
-            artifact_data=stats.custom.get('_run', {}) if hasattr(stats, 'custom') else {}
+            skipped=stats.skipped
         )
 
+        # write custom set_stat artifact data to the local disk so that it can
+        # be persisted by awx after the process exits
+        custom_artifact_data = stats.custom.get('_run', {}) if hasattr(stats, 'custom') else {}
+        if custom_artifact_data:
+            # create the directory for custom stats artifacts to live in (if it doesn't exist)
+            custom_artifacts_dir = os.path.join(os.getenv('AWX_PRIVATE_DATA_DIR'), 'artifacts')
+            os.makedirs(custom_artifacts_dir, mode=stat.S_IXUSR + stat.S_IWUSR + stat.S_IRUSR)
+
+            custom_artifacts_path = os.path.join(custom_artifacts_dir, 'custom')
+            with codecs.open(custom_artifacts_path, 'w', encoding='utf-8') as f:
+                os.chmod(custom_artifacts_path, stat.S_IRUSR | stat.S_IWUSR)
+                json.dump(custom_artifact_data, f)
+
         with self.capture_event_data('playbook_on_stats', **event_data):
             super(BaseCallbackModule, self).v2_playbook_on_stats(stats)
 
+    @staticmethod
+    def _get_event_loop(task):
+        if hasattr(task, 'loop_with'):  # Ansible >=2.5
+            return task.loop_with
+        elif hasattr(task, 'loop'):  # Ansible <2.4
+            return task.loop
+        return None
+
     def v2_runner_on_ok(self, result):
         # FIXME: Display detailed results or not based on verbosity.
 
@@ -312,7 +339,7 @@ class BaseCallbackModule(CallbackBase):
             remote_addr=result._host.address,
             task=result._task,
             res=result._result,
-            event_loop=result._task.loop if hasattr(result._task, 'loop') else None,
+            event_loop=self._get_event_loop(result._task),
         )
         with self.capture_event_data('runner_on_ok', **event_data):
             super(BaseCallbackModule, self).v2_runner_on_ok(result)
@@ -325,7 +352,7 @@ class BaseCallbackModule(CallbackBase):
             res=result._result,
             task=result._task,
             ignore_errors=ignore_errors,
-            event_loop=result._task.loop if hasattr(result._task, 'loop') else None,
+            event_loop=self._get_event_loop(result._task),
         )
         with self.capture_event_data('runner_on_failed', **event_data):
             super(BaseCallbackModule, self).v2_runner_on_failed(result, ignore_errors)
@@ -335,7 +362,7 @@ class BaseCallbackModule(CallbackBase):
             host=result._host.get_name(),
             remote_addr=result._host.address,
             task=result._task,
-            event_loop=result._task.loop if hasattr(result._task, 'loop') else None,
+            event_loop=self._get_event_loop(result._task),
         )
         with self.capture_event_data('runner_on_skipped', **event_data):
             super(BaseCallbackModule, self).v2_runner_on_skipped(result)

awx/lib/tests/test_display_callback.py

@@ -7,7 +7,9 @@ from collections import OrderedDict
 import json
 import mock
 import os
+import shutil
 import sys
+import tempfile
 
 import pytest
 
@@ -26,6 +28,7 @@ CALLBACK = os.path.splitext(os.path.basename(__file__))[0]
 PLUGINS = os.path.dirname(__file__)
 with mock.patch.dict(os.environ, {'ANSIBLE_STDOUT_CALLBACK': CALLBACK,
                                   'ANSIBLE_CALLBACK_PLUGINS': PLUGINS}):
+    from ansible import __version__ as ANSIBLE_VERSION
     from ansible.cli.playbook import PlaybookCLI
     from ansible.executor.playbook_executor import PlaybookExecutor
     from ansible.inventory.manager import InventoryManager
@@ -33,7 +36,7 @@ with mock.patch.dict(os.environ, {'ANSIBLE_STDOUT_CALLBACK': CALLBACK,
     from ansible.vars.manager import VariableManager
 
     # Add awx/lib to sys.path so we can use the plugin
-    path = os.path.abspath(os.path.join(PLUGINS, '..', '..'))
+    path = os.path.abspath(os.path.join(PLUGINS, '..', '..', 'lib'))
     if path not in sys.path:
         sys.path.insert(0, path)
 
@@ -174,6 +177,19 @@ def test_callback_plugin_receives_events(executor, cache, event, playbook):
         when: item != "SENSITIVE-SKIPPED"
         failed_when: item == "SENSITIVE-FAILED"
         ignore_errors: yes
+'''},  # noqa, NOTE: with_items will be deprecated in 2.9
+{'loop.yml': '''
+- name: loop tasks should be suppressed with no_log
+  connection: local
+  hosts: all
+  gather_facts: no
+  tasks:
+      - shell: echo {{ item }}
+        no_log: true
+        loop: [ "SENSITIVE", "SENSITIVE-SKIPPED", "SENSITIVE-FAILED" ]
+        when: item != "SENSITIVE-SKIPPED"
+        failed_when: item == "SENSITIVE-FAILED"
+        ignore_errors: yes
 '''},  # noqa
 ])
 def test_callback_plugin_no_log_filters(executor, cache, playbook):
@@ -184,14 +200,16 @@ def test_callback_plugin_no_log_filters(executor, cache, playbook):
 
 @pytest.mark.parametrize('playbook', [
 {'no_log_on_ok.yml': '''
-- name: args should not be logged when task-level no_log is set
+- name: args should not be logged when no_log is set at the task or module level
   connection: local
   hosts: all
   gather_facts: no
   tasks:
-    - shell: echo "SENSITIVE"
+    - shell: echo "PUBLIC"
     - shell: echo "PRIVATE"
       no_log: true
+    - uri: url=https://example.org username="PUBLIC" password="PRIVATE"
+    - copy: content="PRIVATE" dest="/tmp/tmp_no_log"
 '''},  # noqa
 ])
 def test_callback_plugin_task_args_leak(executor, cache, playbook):
@@ -202,15 +220,15 @@ def test_callback_plugin_task_args_leak(executor, cache, playbook):
 
     # task 1
     assert events[2]['event'] == 'playbook_on_task_start'
-    assert 'SENSITIVE' in events[2]['event_data']['task_args']
     assert events[3]['event'] == 'runner_on_ok'
-    assert 'SENSITIVE' in events[3]['event_data']['task_args']
 
     # task 2 no_log=True
     assert events[4]['event'] == 'playbook_on_task_start'
-    assert events[4]['event_data']['task_args'] == "the output has been hidden due to the fact that 'no_log: true' was specified for this result"  # noqa
     assert events[5]['event'] == 'runner_on_ok'
-    assert events[5]['event_data']['task_args'] == "the output has been hidden due to the fact that 'no_log: true' was specified for this result"  # noqa
+    assert 'PUBLIC' in json.dumps(cache.items())
+    assert 'PRIVATE' not in json.dumps(cache.items())
+    # make sure playbook was successful, so all tasks were hit
+    assert not events[-1]['event_data']['failures'], 'Unexpected playbook execution failure'
 
 
 @pytest.mark.parametrize('playbook', [
@@ -259,3 +277,77 @@ def test_callback_plugin_strips_task_environ_variables(executor, cache, playbook
     assert len(cache)
     for event in cache.values():
         assert os.environ['PATH'] not in json.dumps(event)
+
+
+@pytest.mark.parametrize('playbook', [
+{'custom_set_stat.yml': '''
+- name: custom set_stat calls should persist to the local disk so awx can save them
+  connection: local
+  hosts: all
+  tasks:
+    - set_stats:
+        data:
+          foo: "bar"
+'''},  # noqa
+])
+def test_callback_plugin_saves_custom_stats(executor, cache, playbook):
+    try:
+        private_data_dir = tempfile.mkdtemp()
+        with mock.patch.dict(os.environ, {'AWX_PRIVATE_DATA_DIR': private_data_dir}):
+            executor.run()
+            artifacts_path = os.path.join(private_data_dir, 'artifacts', 'custom')
+            with open(artifacts_path, 'r') as f:
+                assert json.load(f) == {'foo': 'bar'}
+    finally:
+        shutil.rmtree(os.path.join(private_data_dir))
+
+
+@pytest.mark.parametrize('playbook', [
+{'handle_playbook_on_notify.yml': '''
+- name: handle playbook_on_notify events properly
+  connection: local
+  hosts: all
+  handlers:
+    - name: my_handler
+      debug: msg="My Handler"
+  tasks:
+    - debug: msg="My Task"
+      changed_when: true
+      notify:
+        - my_handler
+'''},  # noqa
+])
+@pytest.mark.skipif(ANSIBLE_VERSION < '2.5', reason="v2_playbook_on_notify doesn't work before ansible 2.5")
+def test_callback_plugin_records_notify_events(executor, cache, playbook):
+    executor.run()
+    assert len(cache)
+    notify_events = [x[1] for x in cache.items() if x[1]['event'] == 'playbook_on_notify']
+    assert len(notify_events) == 1
+    assert notify_events[0]['event_data']['handler'] == 'my_handler'
+    assert notify_events[0]['event_data']['host'] == 'localhost'
+    assert notify_events[0]['event_data']['task'] == 'debug'
+
+
+@pytest.mark.parametrize('playbook', [
+{'no_log_module_with_var.yml': '''
+- name: ensure that module-level secrets are redacted
+  connection: local
+  hosts: all
+  vars:
+    - pw: SENSITIVE
+  tasks:
+    - uri:
+        url: https://example.org
+        user: john-jacob-jingleheimer-schmidt
+        password: "{{ pw }}"
+'''},  # noqa
+])
+def test_module_level_no_log(executor, cache, playbook):
+    # https://github.com/ansible/tower/issues/1101
+    # It's possible for `no_log=True` to be defined at the _module_ level,
+    # e.g., for the URI module password parameter
+    # This test ensures that we properly redact those
+    executor.run()
+    assert len(cache)
+    assert 'john-jacob-jingleheimer-schmidt' in json.dumps(cache.items())
+    assert 'SENSITIVE' not in json.dumps(cache.items())

awx/main/access.py

@@ -15,7 +15,10 @@ from django.utils.translation import ugettext_lazy as _
 from django.core.exceptions import ObjectDoesNotExist
 
 # Django REST Framework
-from rest_framework.exceptions import ParseError, PermissionDenied, ValidationError
+from rest_framework.exceptions import ParseError, PermissionDenied
+
+# Django OAuth Toolkit
+from awx.main.models.oauth import OAuth2Application, OAuth2AccessToken
 
 # AWX
 from awx.main.utils import (
@@ -25,14 +28,12 @@ from awx.main.utils import (
     get_licenser,
 )
 from awx.main.models import * # noqa
-from awx.main.models.unified_jobs import ACTIVE_STATES
 from awx.main.models.mixins import ResourceMixin
 
 from awx.conf.license import LicenseForbids, feature_enabled
 
 __all__ = ['get_user_queryset', 'check_user_access', 'check_user_access_with_errors',
-           'user_accessible_objects', 'consumer_access',
-           'user_admin_role', 'ActiveJobConflict',]
+           'user_accessible_objects', 'consumer_access',]
 
 logger = logging.getLogger('awx.main.access')
 
@@ -72,32 +73,10 @@ def get_object_from_data(field, Model, data, obj=None):
         raise ParseError(_("Bad data found in related field %s." % field))
 
 
-class ActiveJobConflict(ValidationError):
-    status_code = 409
-
-    def __init__(self, active_jobs):
-        super(ActiveJobConflict, self).__init__({
-            "conflict": _("Resource is being used by running jobs."),
-            "active_jobs": active_jobs
-        })
-
-
 def register_access(model_class, access_class):
     access_registry[model_class] = access_class
 
 
-@property
-def user_admin_role(self):
-    role = Role.objects.get(
-        content_type=ContentType.objects.get_for_model(User),
-        object_id=self.id,
-        role_field='admin_role'
-    )
-    # Trick the user.admin_role so that the signal filtering for RBAC activity stream works as intended.
-    role.parents = [org.admin_role.pk for org in self.organizations]
-    return role
-
-
 def user_accessible_objects(user, role_name):
     return ResourceMixin._accessible_objects(User, user, role_name)
 
@@ -117,6 +96,8 @@ def check_user_access(user, model_class, action, *args, **kwargs):
     Return True if user can perform action against model_class with the
     provided parameters.
     '''
+    if 'write' not in getattr(user, 'oauth_scopes', ['write']) and action != 'read':
+        return False
     access_class = access_registry[model_class]
     access_instance = access_class(user)
     access_method = getattr(access_instance, 'can_%s' % action)
@@ -233,6 +214,9 @@ class BaseAccess(object):
     def can_delete(self, obj):
         return self.user.is_superuser
 
+    def can_copy(self, obj):
+        return self.can_add({'reference_obj': obj})
+
     def can_attach(self, obj, sub_obj, relationship, data,
                    skip_sub_obj_read_check=False):
         if skip_sub_obj_read_check:
@@ -328,7 +312,7 @@ class BaseAccess(object):
             elif "features" not in validation_info:
                 raise LicenseForbids(_("Features not found in active license."))
 
-    def get_user_capabilities(self, obj, method_list=[], parent_obj=None):
+    def get_user_capabilities(self, obj, method_list=[], parent_obj=None, capabilities_cache={}):
         if obj is None:
             return {}
         user_capabilities = {}
@@ -338,19 +322,29 @@ class BaseAccess(object):
             if display_method not in method_list:
                 continue
 
+            if not settings.MANAGE_ORGANIZATION_AUTH and isinstance(obj, (Team, User)):
+                user_capabilities[display_method] = self.user.is_superuser
+                continue
+
             # Actions not possible for reason unrelated to RBAC
             # Cannot copy with validation errors, or update a manual group/project
-            if display_method == 'copy' and isinstance(obj, JobTemplate):
+            if 'write' not in getattr(self.user, 'oauth_scopes', ['write']):
+                user_capabilities[display_method] = False  # Read tokens cannot take any actions
+                continue
+            elif display_method in ['copy', 'start', 'schedule'] and isinstance(obj, JobTemplate):
                 if obj.validation_errors:
                     user_capabilities[display_method] = False
                     continue
-            elif isinstance(obj, (WorkflowJobTemplate, WorkflowJob)):
-                if not feature_enabled('workflows'):
-                    user_capabilities[display_method] = (display_method == 'delete')
-                    continue
+            elif isinstance(obj, (WorkflowJobTemplate, WorkflowJob)) and (not feature_enabled('workflows')):
+                user_capabilities[display_method] = (display_method == 'delete')
+                continue
             elif display_method == 'copy' and isinstance(obj, WorkflowJobTemplate) and obj.organization_id is None:
                 user_capabilities[display_method] = self.user.is_superuser
                 continue
+            elif display_method == 'copy' and isinstance(obj, Project) and obj.scm_type == '':
+                # Connot copy manual project without errors
+                user_capabilities[display_method] = False
+                continue
             elif display_method in ['start', 'schedule'] and isinstance(obj, Group):  # TODO: remove in 3.3
                 try:
                     if obj.deprecated_inventory_source and not obj.deprecated_inventory_source._can_update():
@@ -365,8 +359,8 @@ class BaseAccess(object):
                     continue
 
             # Grab the answer from the cache, if available
-            if hasattr(obj, 'capabilities_cache') and display_method in obj.capabilities_cache:
-                user_capabilities[display_method] = obj.capabilities_cache[display_method]
+            if display_method in capabilities_cache:
+                user_capabilities[display_method] = capabilities_cache[display_method]
                 if self.user.is_superuser and not user_capabilities[display_method]:
                     # Cache override for models with bad orphaned state
                     user_capabilities[display_method] = True
@@ -384,7 +378,7 @@ class BaseAccess(object):
             if display_method == 'schedule':
                 user_capabilities['schedule'] = user_capabilities['start']
                 continue
-            elif display_method == 'delete' and not isinstance(obj, (User, UnifiedJob)):
+            elif display_method == 'delete' and not isinstance(obj, (User, UnifiedJob, CustomInventoryScript)):
                 user_capabilities['delete'] = user_capabilities['edit']
                 continue
             elif display_method == 'copy' and isinstance(obj, (Group, Host)):
@@ -424,6 +418,18 @@ class InstanceAccess(BaseAccess):
         return Instance.objects.filter(
             rampart_groups__in=self.user.get_queryset(InstanceGroup)).distinct()
 
+
+    def can_attach(self, obj, sub_obj, relationship, data,
+                   skip_sub_obj_read_check=False):
+        if relationship == 'rampart_groups' and isinstance(sub_obj, InstanceGroup):
+            return self.user.is_superuser
+        return super(InstanceAccess, self).can_attach(obj, sub_obj, relationship, *args, **kwargs)
+
+    def can_unattach(self, obj, sub_obj, relationship, data=None):
+        if relationship == 'rampart_groups' and isinstance(sub_obj, InstanceGroup):
+            return self.user.is_superuser
+        return super(InstanceAccess, self).can_unattach(obj, sub_obj, relationship, *args, **kwargs)
+
     def can_add(self, data):
         return False
 
@@ -444,19 +450,16 @@ class InstanceGroupAccess(BaseAccess):
             organization__in=Organization.accessible_pk_qs(self.user, 'admin_role'))
 
     def can_add(self, data):
-        return False
+        return self.user.is_superuser
 
     def can_change(self, obj, data):
-        return False
-
-    def can_delete(self, obj):
-        return False
+        return self.user.is_superuser
 
 
 class UserAccess(BaseAccess):
     '''
     I can see user records when:
-     - I'm a useruser
+     - I'm a superuser
      - I'm in a role with them (such as in an organization or team)
      - They are in a role which includes a role of mine
      - I am in a role that includes a role of theirs
@@ -495,6 +498,8 @@ class UserAccess(BaseAccess):
                 return False
         if self.user.is_superuser:
             return True
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
         return Organization.accessible_objects(self.user, 'admin_role').exists()
 
     def can_change(self, obj, data):
@@ -507,12 +512,46 @@ class UserAccess(BaseAccess):
         # A user can be changed if they are themselves, or by org admins or
         # superusers.  Change permission implies changing only certain fields
         # that a user should be able to edit for themselves.
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
         return bool(self.user == obj or self.can_admin(obj, data))
 
+    def user_membership_roles(self, u):
+        return Role.objects.filter(
+            content_type=ContentType.objects.get_for_model(Organization),
+            role_field__in=[
+                'admin_role', 'member_role',
+                'execute_role', 'project_admin_role', 'inventory_admin_role',
+                'credential_admin_role', 'workflow_admin_role',
+                'notification_admin_role'
+            ],
+            members=u
+        )
+
+    def is_all_org_admin(self, u):
+        return not self.user_membership_roles(u).exclude(
+            ancestors__in=self.user.roles.filter(role_field='admin_role')
+        ).exists()
+
+    def user_is_orphaned(self, u):
+        return not self.user_membership_roles(u).exists()
+
     @check_superuser
-    def can_admin(self, obj, data):
-        return Organization.objects.filter(Q(member_role__members=obj) | Q(admin_role__members=obj),
-                                           Q(admin_role__members=self.user)).exists()
+    def can_admin(self, obj, data, allow_orphans=False):
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
+        if obj.is_superuser or obj.is_system_auditor:
+            # must be superuser to admin users with system roles
+            return False
+        if self.user_is_orphaned(obj):
+            if not allow_orphans:
+                # in these cases only superusers can modify orphan users
+                return False
+            return not obj.roles.all().exclude(
+                content_type=ContentType.objects.get_for_model(User)
+            ).filter(ancestors__in=self.user.roles.all()).exists()
+        else:
+            return self.is_all_org_admin(obj)
 
     def can_delete(self, obj):
         if obj == self.user:
@@ -527,19 +566,100 @@ class UserAccess(BaseAccess):
         return False
 
     def can_attach(self, obj, sub_obj, relationship, *args, **kwargs):
-        "Reverse obj and sub_obj, defer to RoleAccess if this is a role assignment."
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
+
+        # Reverse obj and sub_obj, defer to RoleAccess if this is a role assignment.
         if relationship == 'roles':
             role_access = RoleAccess(self.user)
             return role_access.can_attach(sub_obj, obj, 'members', *args, **kwargs)
         return super(UserAccess, self).can_attach(obj, sub_obj, relationship, *args, **kwargs)
 
     def can_unattach(self, obj, sub_obj, relationship, *args, **kwargs):
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
+
         if relationship == 'roles':
             role_access = RoleAccess(self.user)
             return role_access.can_unattach(sub_obj, obj, 'members', *args, **kwargs)
         return super(UserAccess, self).can_unattach(obj, sub_obj, relationship, *args, **kwargs)
 
 
+class OAuth2ApplicationAccess(BaseAccess):
+    '''
+    I can read, change or delete OAuth 2 applications when:
+     - I am a superuser.
+     - I am the admin of the organization of the user of the application.
+     - I am a user in the organization of the application.
+    I can create OAuth 2 applications when:
+     - I am a superuser.
+     - I am the admin of the organization of the application.
+    '''
+
+    model = OAuth2Application
+    select_related = ('user',)
+
+    def filtered_queryset(self):
+        return self.model.objects.filter(organization__in=self.user.organizations)
+
+    def can_change(self, obj, data):
+        return self.user.is_superuser or self.check_related('organization', Organization, data, obj=obj, 
+                                                            role_field='admin_role', mandatory=True)
+
+    def can_delete(self, obj):
+        return self.user.is_superuser or obj.organization in self.user.admin_of_organizations
+
+    def can_add(self, data):
+        if self.user.is_superuser:
+            return True    
+        if not data:
+            return Organization.accessible_objects(self.user, 'admin_role').exists()
+        return self.check_related('organization', Organization, data, role_field='admin_role', mandatory=True)
+
+
+class OAuth2TokenAccess(BaseAccess):
+    '''
+    I can read, change or delete an app token when:
+     - I am a superuser.
+     - I am the admin of the organization of the application of the token.
+     - I am the user of the token.
+    I can create an OAuth2 app token when:
+     - I have the read permission of the related application.
+    I can read, change or delete a personal token when: 
+     - I am the user of the token
+     - I am the superuser
+    I can create an OAuth2 Personal Access Token when:
+     - I am a user.  But I can only create a PAT for myself.  
+    '''
+
+    model = OAuth2AccessToken
+    
+    select_related = ('user', 'application')
+    
+    def filtered_queryset(self):        
+        org_access_qs = Organization.objects.filter(
+            Q(admin_role__members=self.user) | Q(auditor_role__members=self.user))
+        return self.model.objects.filter(application__organization__in=org_access_qs)  | self.model.objects.filter(user__id=self.user.pk)
+        
+    def can_delete(self, obj):
+        if (self.user.is_superuser) | (obj.user == self.user):
+            return True
+        elif not obj.application:
+            return False
+        return self.user in obj.application.organization.admin_role
+        
+    def can_change(self, obj, data):
+        return self.can_delete(obj)
+
+    def can_add(self, data):
+        if 'application' in data:
+            app = get_object_from_data('application', OAuth2Application, data)
+            if app is None:
+                return True
+            return OAuth2ApplicationAccess(self.user).can_read(app)
+        return True
+
+
 class OrganizationAccess(BaseAccess):
     '''
     I can see organizations when:
@@ -567,15 +687,6 @@ class OrganizationAccess(BaseAccess):
         is_change_possible = self.can_change(obj, None)
         if not is_change_possible:
             return False
-        active_jobs = []
-        active_jobs.extend([dict(type="job", id=o.id)
-                            for o in Job.objects.filter(project__in=obj.projects.all(), status__in=ACTIVE_STATES)])
-        active_jobs.extend([dict(type="project_update", id=o.id)
-                            for o in ProjectUpdate.objects.filter(project__in=obj.projects.all(), status__in=ACTIVE_STATES)])
-        active_jobs.extend([dict(type="inventory_update", id=o.id)
-                            for o in InventoryUpdate.objects.filter(inventory_source__inventory__organization=obj, status__in=ACTIVE_STATES)])
-        if len(active_jobs) > 0:
-            raise ActiveJobConflict(active_jobs)
         return True
 
     def can_attach(self, obj, sub_obj, relationship, *args, **kwargs):
@@ -596,6 +707,7 @@ class InventoryAccess(BaseAccess):
     I can see inventory when:
      - I'm a superuser.
      - I'm an org admin of the inventory's org.
+     - I'm an inventory admin of the inventory's org.
      - I have read, write or admin permissions on it.
     I can change inventory when:
      - I'm a superuser.
@@ -629,9 +741,9 @@ class InventoryAccess(BaseAccess):
     def can_add(self, data):
         # If no data is specified, just checking for generic add permission?
         if not data:
-            return Organization.accessible_objects(self.user, 'admin_role').exists()
+            return Organization.accessible_objects(self.user, 'inventory_admin_role').exists()
 
-        return self.check_related('organization', Organization, data)
+        return self.check_related('organization', Organization, data, role_field='inventory_admin_role')
 
     @check_superuser
     def can_change(self, obj, data):
@@ -647,7 +759,7 @@ class InventoryAccess(BaseAccess):
         # Verify that the user has access to the new organization if moving an
         # inventory to a new organization.  Otherwise, just check for admin permission.
         return (
-            self.check_related('organization', Organization, data, obj=obj,
+            self.check_related('organization', Organization, data, obj=obj, role_field='inventory_admin_role',
                                mandatory=org_admin_mandatory) and
             self.user in obj.admin_role
         )
@@ -657,19 +769,7 @@ class InventoryAccess(BaseAccess):
         return self.user in obj.update_role
 
     def can_delete(self, obj):
-        is_can_admin = self.can_admin(obj, None)
-        if not is_can_admin:
-            return False
-        active_jobs = []
-        active_jobs.extend([dict(type="job", id=o.id)
-                           for o in Job.objects.filter(inventory=obj, status__in=ACTIVE_STATES)])
-        active_jobs.extend([dict(type="inventory_update", id=o.id)
-                            for o in InventoryUpdate.objects.filter(inventory_source__inventory=obj, status__in=ACTIVE_STATES)])
-        active_jobs.extend([dict(type="ad_hoc_command", id=o.id)
-                            for o in AdHocCommand.objects.filter(inventory=obj, status__in=ACTIVE_STATES)])
-        if len(active_jobs) > 0:
-            raise ActiveJobConflict(active_jobs)
-        return True
+        return self.can_admin(obj, None)
 
     def can_run_ad_hoc_commands(self, obj):
         return self.user in obj.adhoc_role
@@ -786,15 +886,7 @@ class GroupAccess(BaseAccess):
         return True
 
     def can_delete(self, obj):
-        is_delete_allowed = bool(obj and self.user in obj.inventory.admin_role)
-        if not is_delete_allowed:
-            return False
-        active_jobs = []
-        active_jobs.extend([dict(type="inventory_update", id=o.id)
-                            for o in InventoryUpdate.objects.filter(inventory_source__in=obj.inventory_sources.all(), status__in=ACTIVE_STATES)])
-        if len(active_jobs) > 0:
-            raise ActiveJobConflict(active_jobs)
-        return True
+        return bool(obj and self.user in obj.inventory.admin_role)
 
     def can_start(self, obj, validate_license=True):
         # TODO: Delete for 3.3, only used by v1 serializer
@@ -817,7 +909,9 @@ class InventorySourceAccess(BaseAccess):
     '''
 
     model = InventorySource
-    select_related = ('created_by', 'modified_by', 'inventory',)
+    select_related = ('created_by', 'modified_by', 'inventory')
+    prefetch_related = ('credentials__credential_type', 'last_job',
+                        'source_script', 'source_project')
 
     def filtered_queryset(self):
         return self.model.objects.filter(inventory__in=Inventory.accessible_pk_qs(self.user, 'read_role'))
@@ -841,9 +935,6 @@ class InventorySourceAccess(BaseAccess):
         if not self.user.is_superuser and \
                 not (obj and obj.inventory and self.user.can_access(Inventory, 'admin', obj.inventory, None)):
             return False
-        active_jobs_qs = InventoryUpdate.objects.filter(inventory_source=obj, status__in=ACTIVE_STATES)
-        if active_jobs_qs.exists():
-            raise ActiveJobConflict([dict(type="inventory_update", id=o.id) for o in active_jobs_qs.all()])
         return True
 
     @check_superuser
@@ -852,7 +943,6 @@ class InventorySourceAccess(BaseAccess):
         if obj and obj.inventory:
             return (
                 self.user.can_access(Inventory, 'change', obj.inventory, None) and
-                self.check_related('credential', Credential, data, obj=obj, role_field='use_role') and
                 self.check_related('source_project', Project, data, obj=obj, role_field='use_role')
             )
         # Can't change inventory sources attached to only the inventory, since
@@ -865,6 +955,21 @@ class InventorySourceAccess(BaseAccess):
             return self.user in obj.inventory.update_role
         return False
 
+    @check_superuser
+    def can_attach(self, obj, sub_obj, relationship, data, skip_sub_obj_read_check=False):
+        if relationship == 'credentials' and isinstance(sub_obj, Credential):
+            return (
+                obj and obj.inventory and self.user in obj.inventory.admin_role and
+                self.user in sub_obj.use_role)
+        return super(InventorySourceAccess, self).can_attach(
+            obj, sub_obj, relationship, data, skip_sub_obj_read_check=skip_sub_obj_read_check)
+
+    @check_superuser
+    def can_unattach(self, obj, sub_obj, relationship, *args, **kwargs):
+        if relationship == 'credentials' and isinstance(sub_obj, Credential):
+            return obj and obj.inventory and self.user in obj.inventory.admin_role
+        return super(InventorySourceAccess, self).can_attach(obj, sub_obj, relationship, *args, **kwargs)
+
 
 class InventoryUpdateAccess(BaseAccess):
     '''
@@ -875,7 +980,7 @@ class InventoryUpdateAccess(BaseAccess):
 
     model = InventoryUpdate
     select_related = ('created_by', 'modified_by', 'inventory_source__inventory',)
-    prefetch_related = ('unified_job_template', 'instance_group',)
+    prefetch_related = ('unified_job_template', 'instance_group', 'credentials',)
 
     def filtered_queryset(self):
         return self.model.objects.filter(inventory_source__inventory__in=Inventory.accessible_pk_qs(self.user, 'read_role'))
@@ -933,8 +1038,12 @@ class CredentialAccess(BaseAccess):
      - I'm a superuser.
      - It's a user credential and it's my credential.
      - It's a user credential and I'm an admin of an organization where that
-       user is a member of admin of the organization.
+       user is a member.
+     - It's a user credential and I'm a credential_admin of an organization
+       where that user is a member.
      - It's a team credential and I'm an admin of the team's organization.
+     - It's a team credential and I'm a credential admin of the team's
+       organization.
      - It's a team credential and I'm a member of the team.
     I can change/delete when:
      - I'm a superuser.
@@ -968,7 +1077,8 @@ class CredentialAccess(BaseAccess):
             return check_user_access(self.user, Team, 'change', team_obj, None)
         if data and data.get('organization', None):
             organization_obj = get_object_from_data('organization', Organization, data)
-            return check_user_access(self.user, Organization, 'change', organization_obj, None)
+            return any([check_user_access(self.user, Organization, 'change', organization_obj, None),
+                        self.user in organization_obj.credential_admin_role])
         return False
 
     @check_superuser
@@ -979,7 +1089,7 @@ class CredentialAccess(BaseAccess):
     def can_change(self, obj, data):
         if not obj:
             return False
-        return self.user in obj.admin_role and self.check_related('organization', Organization, data, obj=obj)
+        return self.user in obj.admin_role and self.check_related('organization', Organization, data, obj=obj, role_field='credential_admin_role')
 
     def can_delete(self, obj):
         # Unassociated credentials may be marked deleted by anyone, though we
@@ -1010,6 +1120,8 @@ class TeamAccess(BaseAccess):
     def can_add(self, data):
         if not data:  # So the browseable API will work
             return Organization.accessible_objects(self.user, 'admin_role').exists()
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
         return self.check_related('organization', Organization, data)
 
     def can_change(self, obj, data):
@@ -1019,6 +1131,8 @@ class TeamAccess(BaseAccess):
             raise PermissionDenied(_('Unable to change organization on a team.'))
         if self.user.is_superuser:
             return True
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
         return self.user in obj.admin_role
 
     def can_delete(self, obj):
@@ -1027,6 +1141,8 @@ class TeamAccess(BaseAccess):
     def can_attach(self, obj, sub_obj, relationship, *args, **kwargs):
         """Reverse obj and sub_obj, defer to RoleAccess if this is an assignment
         of a resource role to the team."""
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
         if isinstance(sub_obj, Role):
             if sub_obj.content_object is None:
                 raise PermissionDenied(_("The {} role cannot be assigned to a team").format(sub_obj.name))
@@ -1037,10 +1153,15 @@ class TeamAccess(BaseAccess):
                 role_access = RoleAccess(self.user)
                 return role_access.can_attach(sub_obj, obj, 'member_role.parents',
                                               *args, **kwargs)
+        if self.user.is_superuser:
+            return True
         return super(TeamAccess, self).can_attach(obj, sub_obj, relationship,
                                                   *args, **kwargs)
 
     def can_unattach(self, obj, sub_obj, relationship, *args, **kwargs):
+        if not settings.MANAGE_ORGANIZATION_AUTH:
+            return False
+
         if isinstance(sub_obj, Role):
             if isinstance(sub_obj.content_object, ResourceMixin):
                 role_access = RoleAccess(self.user)
@@ -1055,6 +1176,7 @@ class ProjectAccess(BaseAccess):
     I can see projects when:
      - I am a superuser.
      - I am an admin in an organization associated with the project.
+     - I am a project admin in an organization associated with the project.
      - I am a user in an organization associated with the project.
      - I am on a team associated with the project.
      - I have been explicitly granted permission to run/check jobs using the
@@ -1075,32 +1197,22 @@ class ProjectAccess(BaseAccess):
     @check_superuser
     def can_add(self, data):
         if not data:  # So the browseable API will work
-            return Organization.accessible_objects(self.user, 'admin_role').exists()
-        return self.check_related('organization', Organization, data, mandatory=True)
+            return Organization.accessible_objects(self.user, 'project_admin_role').exists()
+        return self.check_related('organization', Organization, data, role_field='project_admin_role', mandatory=True)
 
     @check_superuser
     def can_change(self, obj, data):
-        if not self.check_related('organization', Organization, data, obj=obj):
+        if not self.check_related('organization', Organization, data, obj=obj, role_field='project_admin_role'):
             return False
         return self.user in obj.admin_role
 
-    def can_delete(self, obj):
-        is_change_allowed = self.can_change(obj, None)
-        if not is_change_allowed:
-            return False
-        active_jobs = []
-        active_jobs.extend([dict(type="job", id=o.id)
-                            for o in Job.objects.filter(project=obj, status__in=ACTIVE_STATES)])
-        active_jobs.extend([dict(type="project_update", id=o.id)
-                            for o in ProjectUpdate.objects.filter(project=obj, status__in=ACTIVE_STATES)])
-        if len(active_jobs) > 0:
-            raise ActiveJobConflict(active_jobs)
-        return True
-
     @check_superuser
     def can_start(self, obj, validate_license=True):
         return obj and self.user in obj.update_role
 
+    def can_delete(self, obj):
+        return self.can_change(obj, None)
+
 
 class ProjectUpdateAccess(BaseAccess):
     '''
@@ -1162,6 +1274,7 @@ class JobTemplateAccess(BaseAccess):
         a user can create a job template if
          - they are a superuser
          - an org admin of any org that the project is a member
+         - if they are a project_admin for any org that project is a member of
          - if they have user or team
         based permissions tying the project to the inventory source for the
         given action as well as the 'create' deploy permission.
@@ -1208,9 +1321,6 @@ class JobTemplateAccess(BaseAccess):
         else:
             return False
 
-    def can_copy(self, obj):
-        return self.can_add({'reference_obj': obj})
-
     def can_start(self, obj, validate_license=True):
         # Check license.
         if validate_license:
@@ -1269,14 +1379,7 @@ class JobTemplateAccess(BaseAccess):
         return True
 
     def can_delete(self, obj):
-        is_delete_allowed = self.user.is_superuser or self.user in obj.admin_role
-        if not is_delete_allowed:
-            return False
-        active_jobs = [dict(type="job", id=o.id)
-                       for o in obj.jobs.filter(status__in=ACTIVE_STATES)]
-        if len(active_jobs) > 0:
-            raise ActiveJobConflict(active_jobs)
-        return True
+        return self.user.is_superuser or self.user in obj.admin_role
 
     @check_superuser
     def can_attach(self, obj, sub_obj, relationship, data, skip_sub_obj_read_check=False):
@@ -1362,24 +1465,7 @@ class JobAccess(BaseAccess):
 
         if not data:  # So the browseable API will work
             return True
-        if not self.user.is_superuser:
-            return False
-
-
-        add_data = dict(data.items())
-
-        # If a job template is provided, the user should have read access to it.
-        if data and data.get('job_template', None):
-            job_template = get_object_from_data('job_template', JobTemplate, data)
-            add_data.setdefault('inventory', job_template.inventory.pk)
-            add_data.setdefault('project', job_template.project.pk)
-            add_data.setdefault('job_type', job_template.job_type)
-            if job_template.credential:
-                add_data.setdefault('credential', job_template.credential.pk)
-        else:
-            job_template = None
-
-        return True
+        return self.user.is_superuser
 
     def can_change(self, obj, data):
         return (obj.status == 'new' and
@@ -1420,7 +1506,7 @@ class JobAccess(BaseAccess):
             elif not jt_access:
                 return False
 
-        org_access = obj.inventory and self.user in obj.inventory.organization.admin_role
+        org_access = obj.inventory and self.user in obj.inventory.organization.inventory_admin_role
         project_access = obj.project is None or self.user in obj.project.admin_role
         credential_access = all([self.user in cred.use_role for cred in obj.credentials.all()])
 
@@ -1713,13 +1799,14 @@ class WorkflowJobTemplateAccess(BaseAccess):
         Users who are able to create deploy jobs can also run normal and check (dry run) jobs.
         '''
         if not data:  # So the browseable API will work
-            return Organization.accessible_objects(self.user, 'admin_role').exists()
+            return Organization.accessible_objects(self.user, 'workflow_admin_role').exists()
 
         # will check this if surveys are added to WFJT
         if 'survey_enabled' in data and data['survey_enabled']:
             self.check_license(feature='surveys')
 
-        return self.check_related('organization', Organization, data, mandatory=True)
+        return self.check_related('organization', Organization, data, role_field='workflow_admin_role',
+                                  mandatory=True)
 
     def can_copy(self, obj):
         if self.save_messages:
@@ -1746,7 +1833,8 @@ class WorkflowJobTemplateAccess(BaseAccess):
             if missing_inventories:
                 self.messages['inventories_unable_to_copy'] = missing_inventories
 
-        return self.check_related('organization', Organization, {'reference_obj': obj}, mandatory=True)
+        return self.check_related('organization', Organization, {'reference_obj': obj}, role_field='workflow_admin_role',
+                                  mandatory=True)
 
     def can_start(self, obj, validate_license=True):
         if validate_license:
@@ -1771,17 +1859,11 @@ class WorkflowJobTemplateAccess(BaseAccess):
         if self.user.is_superuser:
             return True
 
-        return self.check_related('organization', Organization, data, obj=obj) and self.user in obj.admin_role
+        return (self.check_related('organization', Organization, data, role_field='workflow_admin_role', obj=obj) and
+                self.user in obj.admin_role)
 
     def can_delete(self, obj):
-        is_delete_allowed = self.user.is_superuser or self.user in obj.admin_role
-        if not is_delete_allowed:
-            return False
-        active_jobs = [dict(type="workflow_job", id=o.id)
-                       for o in obj.workflow_jobs.filter(status__in=ACTIVE_STATES)]
-        if len(active_jobs) > 0:
-            raise ActiveJobConflict(active_jobs)
-        return True
+        return self.user.is_superuser or self.user in obj.admin_role
 
 
 class WorkflowJobAccess(BaseAccess):
@@ -1812,7 +1894,7 @@ class WorkflowJobAccess(BaseAccess):
     def can_delete(self, obj):
         return (obj.workflow_job_template and
                 obj.workflow_job_template.organization and
-                self.user in obj.workflow_job_template.organization.admin_role)
+                self.user in obj.workflow_job_template.organization.workflow_admin_role)
 
     def get_method_capability(self, method, obj, parent_obj):
         if method == 'start':
@@ -1996,7 +2078,7 @@ class ProjectUpdateEventAccess(BaseAccess):
 
     def filtered_queryset(self):
         return self.model.objects.filter(
-            Q(project_update__in=ProjectUpdate.accessible_pk_qs(self.user, 'read_role')))
+            Q(project_update__project__in=Project.accessible_pk_qs(self.user, 'read_role')))
 
     def can_add(self, data):
         return False
@@ -2017,7 +2099,7 @@ class InventoryUpdateEventAccess(BaseAccess):
 
     def filtered_queryset(self):
         return self.model.objects.filter(
-            Q(inventory_update__in=InventoryUpdate.accessible_pk_qs(self.user, 'read_role')))
+            Q(inventory_update__inventory_source__inventory__in=Inventory.accessible_pk_qs(self.user, 'read_role')))
 
     def can_add(self, data):
         return False
@@ -2147,13 +2229,9 @@ class ScheduleAccess(BaseAccess):
     prefetch_related = ('unified_job_template', 'credentials',)
 
     def filtered_queryset(self):
-        qs = self.model.objects.all()
-
-        unified_pk_qs = UnifiedJobTemplate.accessible_pk_qs(self.user, 'read_role')
-        inv_src_qs = InventorySource.objects.filter(inventory_id=Inventory._accessible_pk_qs(Inventory, self.user, 'read_role'))
-        return qs.filter(
-            Q(unified_job_template_id__in=unified_pk_qs) |
-            Q(unified_job_template_id__in=inv_src_qs.values_list('pk', flat=True)))
+        return self.model.objects.filter(
+            unified_job_template__in=UnifiedJobTemplateAccess(self.user).filtered_queryset()
+        )
 
     @check_superuser
     def can_add(self, data):
@@ -2196,7 +2274,7 @@ class NotificationTemplateAccess(BaseAccess):
 
     def filtered_queryset(self):
         return self.model.objects.filter(
-            Q(organization__in=self.user.admin_of_organizations) |
+            Q(organization__in=Organization.accessible_objects(self.user, 'notification_admin_role')) |
             Q(organization__in=self.user.auditor_of_organizations)
         ).distinct()
 
@@ -2204,22 +2282,22 @@ class NotificationTemplateAccess(BaseAccess):
         if self.user.is_superuser or self.user.is_system_auditor:
             return True
         if obj.organization is not None:
-            if self.user in obj.organization.admin_role or self.user in obj.organization.auditor_role:
+            if self.user in obj.organization.notification_admin_role or self.user in obj.organization.auditor_role:
                 return True
         return False
 
     @check_superuser
     def can_add(self, data):
         if not data:
-            return Organization.accessible_objects(self.user, 'admin_role').exists()
-        return self.check_related('organization', Organization, data, mandatory=True)
+            return Organization.accessible_objects(self.user, 'notification_admin_role').exists()
+        return self.check_related('organization', Organization, data, role_field='notification_admin_role', mandatory=True)
 
     @check_superuser
     def can_change(self, obj, data):
         if obj.organization is None:
             # only superusers are allowed to edit orphan notification templates
             return False
-        return self.check_related('organization', Organization, data, obj=obj, mandatory=True)
+        return self.check_related('organization', Organization, data, obj=obj, role_field='notification_admin_role', mandatory=True)
 
     def can_admin(self, obj, data):
         return self.can_change(obj, data)
@@ -2231,7 +2309,7 @@ class NotificationTemplateAccess(BaseAccess):
     def can_start(self, obj, validate_license=True):
         if obj.organization is None:
             return False
-        return self.user in obj.organization.admin_role
+        return self.user in obj.organization.notification_admin_role
 
 
 class NotificationAccess(BaseAccess):
@@ -2243,7 +2321,7 @@ class NotificationAccess(BaseAccess):
 
     def filtered_queryset(self):
         return self.model.objects.filter(
-            Q(notification_template__organization__in=self.user.admin_of_organizations) |
+            Q(notification_template__organization__in=Organization.accessible_objects(self.user, 'notification_admin_role')) |
             Q(notification_template__organization__in=self.user.auditor_of_organizations)
         ).distinct()
 
@@ -2295,7 +2373,7 @@ class ActivityStreamAccess(BaseAccess):
     model = ActivityStream
     prefetch_related = ('organization', 'user', 'inventory', 'host', 'group',
                         'inventory_update', 'credential', 'credential_type', 'team',
-                        'ad_hoc_command',
+                        'ad_hoc_command', 'o_auth2_application', 'o_auth2_access_token', 
                         'notification_template', 'notification', 'label', 'role', 'actor',
                         'schedule', 'custom_inventory_script', 'unified_job_template',
                         'workflow_job_template_node',)
@@ -2338,9 +2416,13 @@ class ActivityStreamAccess(BaseAccess):
         jt_set = JobTemplate.accessible_objects(self.user, 'read_role')
         team_set = Team.accessible_objects(self.user, 'read_role')
         wfjt_set = WorkflowJobTemplate.accessible_objects(self.user, 'read_role')
+        app_set = OAuth2ApplicationAccess(self.user).filtered_queryset()
+        token_set = OAuth2TokenAccess(self.user).filtered_queryset()
 
         return qs.filter(
             Q(ad_hoc_command__inventory__in=inventory_set) |
+            Q(o_auth2_application__in=app_set) |
+            Q(o_auth2_access_token__in=token_set) |
             Q(user__in=auditing_orgs.values('member_role__members')) |
             Q(user=self.user) |
             Q(organization__in=auditing_orgs) |
@@ -2430,6 +2512,10 @@ class RoleAccess(BaseAccess):
 
     @check_superuser
     def can_unattach(self, obj, sub_obj, relationship, data=None, skip_sub_obj_read_check=False):
+        if isinstance(obj.content_object, Team):
+            if not settings.MANAGE_ORGANIZATION_AUTH:
+                return False
+
         if not skip_sub_obj_read_check and relationship in ['members', 'member_role.parents', 'parents']:
             # If we are unattaching a team Role, check the Team read access
             if relationship == 'parents':
@@ -2439,6 +2525,14 @@ class RoleAccess(BaseAccess):
             if not check_user_access(self.user, sub_obj_resource.__class__, 'read', sub_obj_resource):
                 return False
 
+        # Being a user in the member_role or admin_role of an organization grants
+        # administrators of that Organization the ability to edit that user. To prevent
+        # unwanted escalations lets ensure that the Organization administartor has the abilty
+        # to admin the user being added to the role.
+        if isinstance(obj.content_object, Organization) and obj.role_field in ['member_role', 'admin_role']:
+            if not UserAccess(self.user).can_admin(sub_obj, None, allow_orphans=True):
+                return False
+
         if isinstance(obj.content_object, ResourceMixin) and \
            self.user in obj.content_object.admin_role:
             return True

awx/main/conf.py

@@ -43,6 +43,16 @@ register(
     category_slug='system',
 )
 
+register(
+    'MANAGE_ORGANIZATION_AUTH',
+    field_class=fields.BooleanField,
+    label=_('Organization Admins Can Manage Users and Teams'),
+    help_text=_('Controls whether any Organization Admin has the privileges to create and manage users and teams. '
+                'You may want to disable this ability if you are using an LDAP or SAML integration.'),
+    category=_('System'),
+    category_slug='system',
+)
+
 register(
     'TOWER_ADMIN_ALERTS',
     field_class=fields.BooleanField,
@@ -125,6 +135,27 @@ register(
     required=False,
 )
 
+register(
+    'ALLOW_JINJA_IN_EXTRA_VARS',
+    field_class=fields.ChoiceField,
+    choices=[
+        ('always', _('Always')),
+        ('never', _('Never')),
+        ('template', _('Only On Job Template Definitions')),
+    ],
+    required=True,
+    label=_('When can extra variables contain Jinja templates?'),
+    help_text=_(
+        'Ansible allows variable substitution via the Jinja2 templating '
+        'language for --extra-vars. This poses a potential security '
+        'risk where Tower users with the ability to specify extra vars at job '
+        'launch time can use Jinja2 templates to run arbitrary Python.  It is '
+        'recommended that this value be set to "template" or "never".'
+    ),
+    category=_('Jobs'),
+    category_slug='jobs',
+)
+
 register(
     'AWX_PROOT_ENABLED',
     field_class=fields.BooleanField,
@@ -331,7 +362,8 @@ register(
     label=_('Per-Host Ansible Fact Cache Timeout'),
     help_text=_('Maximum time, in seconds, that stored Ansible facts are considered valid since '
                 'the last time they were modified. Only valid, non-stale, facts will be accessible by '
-                'a playbook. Note, this does not influence the deletion of ansible_facts from the database.'),
+                'a playbook. Note, this does not influence the deletion of ansible_facts from the database. '
+                'Use a value of 0 to indicate that no timeout should be imposed.'),
     category=_('Jobs'),
     category_slug='jobs',
 )
@@ -411,7 +443,7 @@ register(
     field_class=fields.BooleanField,
     default=False,
     label=_('Log System Tracking Facts Individually'),
-    help_text=_('If set, system tracking facts will be sent for each package, service, or'
+    help_text=_('If set, system tracking facts will be sent for each package, service, or '
                 'other item found in a scan, allowing for greater search query granularity. '
                 'If unset, facts will be sent as a single dictionary, allowing for greater '
                 'efficiency in fact processing.'),

awx/main/constants.py

@@ -5,9 +5,21 @@ import re
 
 from django.utils.translation import ugettext_lazy as _
 
+__all__ = [
+    'CLOUD_PROVIDERS', 'SCHEDULEABLE_PROVIDERS', 'PRIVILEGE_ESCALATION_METHODS',
+    'ANSI_SGR_PATTERN', 'CAN_CANCEL', 'ACTIVE_STATES'
+]
+
+
 CLOUD_PROVIDERS = ('azure_rm', 'ec2', 'gce', 'vmware', 'openstack', 'rhv', 'satellite6', 'cloudforms', 'tower')
 SCHEDULEABLE_PROVIDERS = CLOUD_PROVIDERS + ('custom', 'scm',)
 PRIVILEGE_ESCALATION_METHODS = [
     ('sudo', _('Sudo')), ('su', _('Su')), ('pbrun', _('Pbrun')), ('pfexec', _('Pfexec')),
-    ('dzdo', _('DZDO')), ('pmrun', _('Pmrun')), ('runas', _('Runas'))]
+    ('dzdo', _('DZDO')), ('pmrun', _('Pmrun')), ('runas', _('Runas')),
+    ('enable', _('Enable')), ('doas', _('Doas')),
+]
+CHOICES_PRIVILEGE_ESCALATION_METHODS = [('', _('None'))] + PRIVILEGE_ESCALATION_METHODS
 ANSI_SGR_PATTERN = re.compile(r'\x1b\[[0-9;]*m')
+CAN_CANCEL = ('new', 'pending', 'waiting', 'running')
+ACTIVE_STATES = CAN_CANCEL
+TOKEN_CENSOR = '************'

awx/main/consumers.py

@@ -1,17 +1,11 @@
 import json
 import logging
-import urllib
 
-from channels import Group, channel_layers
-from channels.sessions import channel_session
-from channels.handler import AsgiRequest
+from channels import Group
+from channels.auth import channel_session_user_from_http, channel_session_user
 
-from django.conf import settings
 from django.core.serializers.json import DjangoJSONEncoder
 
-from django.contrib.auth.models import User
-from awx.main.models.organization import AuthToken
-
 
 logger = logging.getLogger('awx.main.consumers')
 
@@ -22,51 +16,29 @@ def discard_groups(message):
             Group(group).discard(message.reply_channel)
 
 
-@channel_session
+@channel_session_user_from_http
 def ws_connect(message):
     message.reply_channel.send({"accept": True})
-
     message.content['method'] = 'FAKE'
-    request = AsgiRequest(message)
-    token = request.COOKIES.get('token', None)
-    if token is not None:
-        token = urllib.unquote(token).strip('"')
-        try:
-            auth_token = AuthToken.objects.get(key=token)
-            if auth_token.in_valid_tokens:
-                message.channel_session['user_id'] = auth_token.user_id
-                message.reply_channel.send({"text": json.dumps({"accept": True, "user": auth_token.user_id})})
-                return None
-        except AuthToken.DoesNotExist:
-            logger.error("auth_token provided was invalid.")
-    message.reply_channel.send({"close": True})
+    if message.user.is_authenticated():
+        message.reply_channel.send(
+            {"text": json.dumps({"accept": True, "user": message.user.id})}
+        )
+    else:
+        logger.error("Request user is not authenticated to use websocket.")
+        message.reply_channel.send({"close": True})
     return None
 
 
-@channel_session
+@channel_session_user
 def ws_disconnect(message):
     discard_groups(message)
 
 
-@channel_session
+@channel_session_user
 def ws_receive(message):
     from awx.main.access import consumer_access
-    channel_layer_settings = channel_layers.configs[message.channel_layer.alias]
-    max_retries = channel_layer_settings.get('RECEIVE_MAX_RETRY', settings.CHANNEL_LAYER_RECEIVE_MAX_RETRY)
-
-    user_id = message.channel_session.get('user_id', None)
-    if user_id is None:
-        retries = message.content.get('connect_retries', 0) + 1
-        message.content['connect_retries'] = retries
-        message.reply_channel.send({"text": json.dumps({"error": "no valid user"})})
-        retries_left = max_retries - retries
-        if retries_left > 0:
-            message.channel_layer.send(message.channel.name, message.content)
-        else:
-            logger.error("No valid user found for websocket.")
-        return None
-
-    user = User.objects.get(pk=user_id)
+    user = message.user
     raw_data = message.content['text']
     data = json.loads(raw_data)
 

awx/main/exceptions.py

@@ -1,6 +1,9 @@
 # Copyright (c) 2018 Ansible by Red Hat
 # All Rights Reserved.
 
+import six
+
+
 # Celery does not respect exception type when using a serializer different than pickle;
 # and awx uses the json serializer
 # https://github.com/celery/celery/issues/3586
@@ -9,7 +12,7 @@
 class _AwxTaskError():
     def build_exception(self, task, message=None):
         if message is None:
-            message = "Execution error running {}".format(task.log_format)
+            message = six.text_type("Execution error running {}").format(task.log_format)
         e = Exception(message)
         e.task = task
         e.is_awx_task_error = True
@@ -17,7 +20,7 @@ class _AwxTaskError():
 
     def TaskCancel(self, task, rc):
         """Canceled flag caused run_pexpect to kill the job run"""
-        message="{} was canceled (rc={})".format(task.log_format, rc)
+        message=six.text_type("{} was canceled (rc={})").format(task.log_format, rc)
         e = self.build_exception(task, message)
         e.rc = rc
         e.awx_task_error_type = "TaskCancel"
@@ -25,7 +28,7 @@ class _AwxTaskError():
 
     def TaskError(self, task, rc):
         """Userspace error (non-zero exit code) in run_pexpect subprocess"""
-        message = "{} encountered an error (rc={}), please see task stdout for details.".format(task.log_format, rc)
+        message = six.text_type("{} encountered an error (rc={}), please see task stdout for details.").format(task.log_format, rc)
         e = self.build_exception(task, message)
         e.rc = rc
         e.awx_task_error_type = "TaskError"
@@ -34,3 +37,4 @@ class _AwxTaskError():
 
 AwxTaskError = _AwxTaskError()
 
+

awx/main/expect/isolated_manager.py

@@ -14,7 +14,7 @@ from django.conf import settings
 
 import awx
 from awx.main.expect import run
-from awx.main.utils import OutputEventFilter
+from awx.main.utils import OutputEventFilter, get_system_task_capacity
 from awx.main.queue import CallbackQueueDispatcher
 
 logger = logging.getLogger('awx.isolated.manager')
@@ -381,10 +381,14 @@ class IsolatedManager(object):
             logger.error(err_template.format(instance.hostname, instance.version, awx_application_version))
             instance.capacity = 0
         else:
-            if instance.capacity == 0 and task_result['capacity']:
+            if instance.capacity == 0 and task_result['capacity_cpu']:
                 logger.warning('Isolated instance {} has re-joined.'.format(instance.hostname))
-            instance.capacity = int(task_result['capacity'])
-        instance.save(update_fields=['capacity', 'version', 'modified'])
+            instance.cpu_capacity = int(task_result['capacity_cpu'])
+            instance.mem_capacity = int(task_result['capacity_mem'])
+            instance.capacity = get_system_task_capacity(scale=instance.capacity_adjustment,
+                                                         cpu_capacity=int(task_result['capacity_cpu']),
+                                                         mem_capacity=int(task_result['capacity_mem']))
+        instance.save(update_fields=['cpu_capacity', 'mem_capacity', 'capacity', 'version', 'modified'])
 
     @classmethod
     def health_check(cls, instance_qs, awx_application_version):
@@ -428,7 +432,7 @@ class IsolatedManager(object):
                 task_result = result['plays'][0]['tasks'][0]['hosts'][instance.hostname]
             except (KeyError, IndexError):
                 task_result = {}
-            if 'capacity' in task_result:
+            if 'capacity_cpu' in task_result and 'capacity_mem' in task_result:
                 cls.update_capacity(instance, task_result, awx_application_version)
             elif instance.capacity == 0:
                 logger.debug('Isolated instance {} previously marked as lost, could not re-join.'.format(

awx/main/expect/run.py

@@ -47,7 +47,7 @@ def open_fifo_write(path, data):
     This blocks the thread until an external process (such as ssh-agent)
     reads data from the pipe.
     '''
-    os.mkfifo(path, 0600)
+    os.mkfifo(path, 0o600)
     thread.start_new_thread(lambda p, d: open(p, 'w').write(d), (path, data))
 
 
@@ -101,7 +101,7 @@ def run_pexpect(args, cwd, env, logfile,
 
     child = pexpect.spawn(
         args[0], args[1:], cwd=cwd, env=env, ignore_sighup=True,
-        encoding='utf-8', echo=False,
+        encoding='utf-8', echo=False, use_poll=True
     )
     child.logfile_read = logfile
     canceled = False

awx/main/fields.py

@@ -4,12 +4,13 @@
 # Python
 import copy
 import json
+import operator
 import re
 import six
 import urllib
 
 from jinja2 import Environment, StrictUndefined
-from jinja2.exceptions import UndefinedError
+from jinja2.exceptions import UndefinedError, TemplateSyntaxError
 
 # Django
 from django.core import exceptions as django_exceptions
@@ -42,19 +43,24 @@ from rest_framework import serializers
 
 # AWX
 from awx.main.utils.filters import SmartFilter
+from awx.main.utils.encryption import encrypt_value, decrypt_value, get_encryption_key
 from awx.main.validators import validate_ssh_private_key
 from awx.main.models.rbac import batch_role_ancestor_rebuilding, Role
+from awx.main.constants import CHOICES_PRIVILEGE_ESCALATION_METHODS
 from awx.main import utils
 
 
-__all__ = ['AutoOneToOneField', 'ImplicitRoleField', 'JSONField', 'SmartFilterField']
+__all__ = ['AutoOneToOneField', 'ImplicitRoleField', 'JSONField',
+           'SmartFilterField', 'update_role_parentage_for_instance',
+           'is_implicit_parent']
 
 
 # Provide a (better) custom error message for enum jsonschema validation
 def __enum_validate__(validator, enums, instance, schema):
     if instance not in enums:
         yield jsonschema.exceptions.ValidationError(
-            _("'%s' is not one of ['%s']") % (instance, "', '".join(enums))
+            _("'{value}' is not one of ['{allowed_values}']").format(
+                value=instance, allowed_values="', '".join(enums))
         )
 
 
@@ -74,7 +80,7 @@ class JSONField(upstream_JSONField):
 
 class JSONBField(upstream_JSONBField):
     def get_prep_lookup(self, lookup_type, value):
-        if isinstance(value, basestring) and value == "null":
+        if isinstance(value, six.string_types) and value == "null":
             return 'null'
         return super(JSONBField, self).get_prep_lookup(lookup_type, value)
 
@@ -180,6 +186,23 @@ def is_implicit_parent(parent_role, child_role):
     return False
 
 
+def update_role_parentage_for_instance(instance):
+    '''update_role_parentage_for_instance
+    updates the parents listing for all the roles
+    of a given instance if they have changed
+    '''
+    for implicit_role_field in getattr(instance.__class__, '__implicit_role_fields'):
+        cur_role = getattr(instance, implicit_role_field.name)
+        new_parents = implicit_role_field._resolve_parent_roles(instance)
+        cur_role.parents.set(new_parents)
+        new_parents_list = list(new_parents)
+        new_parents_list.sort()
+        new_parents_json = json.dumps(new_parents_list)
+        if cur_role.implicit_parents != new_parents_json:
+            cur_role.implicit_parents = new_parents_json
+            cur_role.save()
+
+
 class ImplicitRoleDescriptor(ForwardManyToOneDescriptor):
     pass
 
@@ -273,43 +296,37 @@ class ImplicitRoleField(models.ForeignKey):
         Role_ = utils.get_current_apps().get_model('main', 'Role')
         ContentType_ = utils.get_current_apps().get_model('contenttypes', 'ContentType')
         ct_id = ContentType_.objects.get_for_model(instance).id
+
+        Model = utils.get_current_apps().get_model('main', instance.__class__.__name__)
+        latest_instance = Model.objects.get(pk=instance.pk)
+
         with batch_role_ancestor_rebuilding():
             # Create any missing role objects
             missing_roles = []
-            for implicit_role_field in getattr(instance.__class__, '__implicit_role_fields'):
-                cur_role = getattr(instance, implicit_role_field.name, None)
+            for implicit_role_field in getattr(latest_instance.__class__, '__implicit_role_fields'):
+                cur_role = getattr(latest_instance, implicit_role_field.name, None)
                 if cur_role is None:
                     missing_roles.append(
                         Role_(
                             role_field=implicit_role_field.name,
                             content_type_id=ct_id,
-                            object_id=instance.id
+                            object_id=latest_instance.id
                         )
                     )
+
             if len(missing_roles) > 0:
                 Role_.objects.bulk_create(missing_roles)
                 updates = {}
                 role_ids = []
-                for role in Role_.objects.filter(content_type_id=ct_id, object_id=instance.id):
-                    setattr(instance, role.role_field, role)
+                for role in Role_.objects.filter(content_type_id=ct_id, object_id=latest_instance.id):
+                    setattr(latest_instance, role.role_field, role)
                     updates[role.role_field] = role.id
                     role_ids.append(role.id)
-                type(instance).objects.filter(pk=instance.pk).update(**updates)
+                type(latest_instance).objects.filter(pk=latest_instance.pk).update(**updates)
                 Role.rebuild_role_ancestor_list(role_ids, [])
 
-            # Update parentage if necessary
-            for implicit_role_field in getattr(instance.__class__, '__implicit_role_fields'):
-                cur_role = getattr(instance, implicit_role_field.name)
-                original_parents = set(json.loads(cur_role.implicit_parents))
-                new_parents = implicit_role_field._resolve_parent_roles(instance)
-                cur_role.parents.remove(*list(original_parents - new_parents))
-                cur_role.parents.add(*list(new_parents - original_parents))
-                new_parents_list = list(new_parents)
-                new_parents_list.sort()
-                new_parents_json = json.dumps(new_parents_list)
-                if cur_role.implicit_parents != new_parents_json:
-                    cur_role.implicit_parents = new_parents_json
-                    cur_role.save()
+            update_role_parentage_for_instance(latest_instance)
+            instance.refresh_from_db()
 
 
     def _resolve_parent_roles(self, instance):
@@ -356,7 +373,7 @@ class SmartFilterField(models.TextField):
         value = urllib.unquote(value)
         try:
             SmartFilter().query_from_string(value)
-        except RuntimeError, e:
+        except RuntimeError as e:
             raise models.base.ValidationError(e)
         return super(SmartFilterField, self).get_prep_value(value)
 
@@ -391,7 +408,25 @@ class JSONSchemaField(JSONBField):
             error.message = re.sub(r'\bu(\'|")', r'\1', error.message)
 
             if error.validator == 'pattern' and 'error' in error.schema:
-                error.message = error.schema['error'] % error.instance
+                error.message = six.text_type(error.schema['error']).format(instance=error.instance)
+            elif error.validator == 'type':
+                expected_type = error.validator_value
+                if expected_type == 'object':
+                    expected_type = 'dict'
+                if error.path:
+                    error.message = _(
+                        '{type} provided in relative path {path}, expected {expected_type}'
+                    ).format(path=list(error.path), type=type(error.instance).__name__,
+                             expected_type=expected_type)
+                else:
+                    error.message = _(
+                        '{type} provided, expected {expected_type}'
+                    ).format(path=list(error.path), type=type(error.instance).__name__,
+                             expected_type=expected_type)
+            elif error.validator == 'additionalProperties' and hasattr(error, 'path'):
+                error.message = _(
+                    'Schema validation error in relative path {path} ({error})'
+                ).format(path=list(error.path), error=error.message)
             errors.append(error)
 
         if errors:
@@ -474,6 +509,9 @@ class CredentialInputField(JSONSchemaField):
         properties = {}
         for field in model_instance.credential_type.inputs.get('fields', []):
             field = field.copy()
+            if field['type'] == 'become_method':
+                field.pop('type')
+                field['choices'] = map(operator.itemgetter(0), CHOICES_PRIVILEGE_ESCALATION_METHODS)
             properties[field['id']] = field
             if field.get('choices', []):
                 field['enum'] = field['choices'][:]
@@ -506,6 +544,12 @@ class CredentialInputField(JSONSchemaField):
                 v != '$encrypted$',
                 model_instance.pk
             ]):
+                if not isinstance(getattr(model_instance, k), six.string_types):
+                    raise django_exceptions.ValidationError(
+                        _('secret values must be of type string, not {}').format(type(v).__name__),
+                        code='invalid',
+                        params={'value': v},
+                    )
                 decrypted_values[k] = utils.decrypt_field(model_instance, k)
             else:
                 decrypted_values[k] = v
@@ -517,7 +561,7 @@ class CredentialInputField(JSONSchemaField):
             format_checker=self.format_checker
         ).iter_errors(decrypted_values):
             if error.validator == 'pattern' and 'error' in error.schema:
-                error.message = error.schema['error'] % error.instance
+                error.message = six.text_type(error.schema['error']).format(instance=error.instance)
             if error.validator == 'dependencies':
                 # replace the default error messaging w/ a better i18n string
                 # I wish there was a better way to determine the parameters of
@@ -611,7 +655,7 @@ class CredentialTypeInputField(JSONSchemaField):
                     'items': {
                         'type': 'object',
                         'properties': {
-                            'type': {'enum': ['string', 'boolean']},
+                            'type': {'enum': ['string', 'boolean', 'become_method']},
                             'format': {'enum': ['ssh_private_key']},
                             'choices': {
                                 'type': 'array',
@@ -622,7 +666,7 @@ class CredentialTypeInputField(JSONSchemaField):
                             'id': {
                                 'type': 'string',
                                 'pattern': '^[a-zA-Z_]+[a-zA-Z0-9_]*$',
-                                'error': '%s is an invalid variable name',
+                                'error': '{instance} is an invalid variable name',
                             },
                             'label': {'type': 'string'},
                             'help_text': {'type': 'string'},
@@ -672,10 +716,22 @@ class CredentialTypeInputField(JSONSchemaField):
                 # If no type is specified, default to string
                 field['type'] = 'string'
 
+            if field['type'] == 'become_method':
+                if not model_instance.managed_by_tower:
+                    raise django_exceptions.ValidationError(
+                        _('become_method is a reserved type name'),
+                        code='invalid',
+                        params={'value': value},
+                    )
+                else:
+                    field.pop('type')
+                    field['choices'] = CHOICES_PRIVILEGE_ESCALATION_METHODS
+
             for key in ('choices', 'multiline', 'format', 'secret',):
                 if key in field and field['type'] != 'string':
                     raise django_exceptions.ValidationError(
-                        _('%s not allowed for %s type (%s)' % (key, field['type'], field['id'])),
+                        _('{sub_key} not allowed for {element_type} type ({element_id})'.format(
+                            sub_key=key, element_type=field['type'], element_id=field['id'])),
                         code='invalid',
                         params={'value': value},
                     )
@@ -695,11 +751,10 @@ class CredentialTypeInjectorField(JSONSchemaField):
             'properties': {
                 'file': {
                     'type': 'object',
-                    'properties': {
-                        'template': {'type': 'string'},
+                    'patternProperties': {
+                        '^template(\.[a-zA-Z_]+[a-zA-Z0-9_]*)?$': {'type': 'string'},
                     },
                     'additionalProperties': False,
-                    'required': ['template'],
                 },
                 'env': {
                     'type': 'object',
@@ -749,8 +804,22 @@ class CredentialTypeInjectorField(JSONSchemaField):
 
         class TowerNamespace:
             filename = None
-
         valid_namespace['tower'] = TowerNamespace()
+
+        # ensure either single file or multi-file syntax is used (but not both)
+        template_names = [x for x in value.get('file', {}).keys() if x.startswith('template')]
+        if 'template' in template_names and len(template_names) > 1:
+            raise django_exceptions.ValidationError(
+                _('Must use multi-file syntax when injecting multiple files'),
+                code='invalid',
+                params={'value': value},
+            )
+        if 'template' not in template_names:
+            valid_namespace['tower'].filename = TowerNamespace()
+            for template_name in template_names:
+                template_name = template_name.split('.')[1]
+                setattr(valid_namespace['tower'].filename, template_name, 'EXAMPLE')
+
         for type_, injector in value.items():
             for key, tmpl in injector.items():
                 try:
@@ -759,7 +828,15 @@ class CredentialTypeInjectorField(JSONSchemaField):
                     ).from_string(tmpl).render(valid_namespace)
                 except UndefinedError as e:
                     raise django_exceptions.ValidationError(
-                        _('%s uses an undefined field (%s)') % (key, e),
+                        _('{sub_key} uses an undefined field ({error_msg})').format(
+                            sub_key=key, error_msg=e),
+                        code='invalid',
+                        params={'value': value},
+                    )
+                except TemplateSyntaxError as e:
+                    raise django_exceptions.ValidationError(
+                        _('Syntax error rendering template for {sub_key} inside of {type} ({error_msg})').format(
+                            sub_key=key, type=type_, error_msg=e),
                         code='invalid',
                         params={'value': value},
                     )
@@ -782,3 +859,16 @@ class AskForField(models.BooleanField):
                 # self.name will be set by the model metaclass, not this field
                 raise Exception('Corresponding allows_field cannot be accessed until model is initialized.')
         return self._allows_field
+
+
+class OAuth2ClientSecretField(models.CharField):
+
+    def get_db_prep_value(self, value, connection, prepared=False):
+        return super(OAuth2ClientSecretField, self).get_db_prep_value(
+            encrypt_value(value), connection, prepared
+        )
+
+    def from_db_value(self, value, expression, connection, context):
+        if value and value.startswith('$encrypted$'):
+            return decrypt_value(get_encryption_key('value', pk=None), value)
+        return value

awx/main/management/commands/cleanup_activitystream.py

@@ -5,6 +5,8 @@
 import datetime
 import logging
 
+import six
+
 # Django
 from django.core.management.base import BaseCommand
 from django.utils.timezone import now
@@ -41,7 +43,7 @@ class Command(BaseCommand):
         n_deleted_items = 0
         pks_to_delete = set()
         for asobj in ActivityStream.objects.iterator():
-            asobj_disp = '"%s" id: %s' % (unicode(asobj), asobj.id)
+            asobj_disp = '"%s" id: %s' % (six.text_type(asobj), asobj.id)
             if asobj.timestamp >= self.cutoff:
                 if self.dry_run:
                     self.logger.info("would skip %s" % asobj_disp)

awx/main/management/commands/cleanup_authtokens.py

@@ -1,35 +0,0 @@
-# Copyright (c) 2015 Ansible, Inc.
-# All Rights Reserved.
-
-# Python
-import logging
-
-# Django
-from django.db import transaction
-from django.core.management.base import BaseCommand
-from django.utils.timezone import now
-
-# AWX
-from awx.main.models import * # noqa
-
-
-class Command(BaseCommand):
-    '''
-    Management command to cleanup expired auth tokens
-    '''
-
-    help = 'Cleanup expired auth tokens.'
-
-    def init_logging(self):
-        self.logger = logging.getLogger('awx.main.commands.cleanup_authtokens')
-        handler = logging.StreamHandler()
-        handler.setFormatter(logging.Formatter('%(message)s'))
-        self.logger.addHandler(handler)
-        self.logger.propagate = False
-
-    @transaction.atomic
-    def handle(self, *args, **options):
-        self.init_logging()
-        tokens_removed = AuthToken.objects.filter(expires__lt=now())
-        self.logger.log(99, "Removing %d expired auth tokens" % tokens_removed.count())
-        tokens_removed.delete()