Feature: custom virtual environment directories

Currently, users are allowed to define virtual environments in
`settings.BASE_VENV_PATH` only, because that's the only place
Tower looks for virtual environments. This feature allows users
to custom define the directory paths, using API or UI, to look
for virtual environments. Tower aggregates virtual environments
from all these paths, except environments with special name `awx`.

Signed-off-by: Vismay Golwala <vgolwala@redhat.com>

docs/custom_virtualenvs.md

@@ -16,29 +16,48 @@ virtualenv; this documentation describes the supported way to do so.
 Preparing a New Custom Virtualenv
 =================================
 awx allows a _different_ virtualenv to be specified and used on Job Template
-runs.  To choose a custom virtualenv, first create one in `/var/lib/awx/venv`:
+runs.  To choose a custom virtualenv, first we need to create one. Here, we are
+using `/opt/my-envs/` as the directory to hold custom venvs. But you can use any
+other directory and replace `/opt/my-envs/` with that. Let's create the directory
+first if absent:
 
-    $ sudo virtualenv /var/lib/awx/venv/my-custom-venv
+    $ sudo mkdir /opt/my-envs
+
+Now, we need to tell Tower to look into this directory for custom venvs. For that,
+we can add this directory to the `CUSTOM_VENV_PATHS` setting as:
+
+    $ HTTP PATCH /api/v2/settings/system {'CUSTOM_VENV_PATHS': ["/opt/my-envs/"]}
+
+If we have venvs spanned over multiple directories, we can add all the paths and
+Tower will aggregate venvs from them:
+
+    $ HTTP PATCH /api/v2/settings/system {'CUSTOM_VENV_PATHS': ["/path/1/to/venv/",
+                                                                "/path/2/to/venv/",
+                                                                "/path/3/to/venv/"]}
+
+Now that we have the directory setup, we can create a virtual environment in that using:
+
+    $ sudo virtualenv /opt/my-envs/custom-venv
 
 Multiple versions of Python are supported, though it's important to note that
 the semantics for creating virtualenvs in Python 3 has changed slightly:
 
-    $ sudo python3 -m venv /var/lib/awx/venv/my-custom-venv
+    $ sudo python3 -m venv /opt/my-envs/custom-venv
 
 Your newly created virtualenv needs a few base dependencies to properly run
 playbooks:
 fact gathering):
 
-    $ sudo /var/lib/awx/venv/my-custom-venv/bin/pip install psutil
+    $ sudo /opt/my-envs/custom-venv/bin/pip install psutil
 
 From here, you can install _additional_ Python dependencies that you care
 about, such as a per-virtualenv version of ansible itself:
 
-    $ sudo /var/lib/awx/venv/my-custom-venv/bin/pip install -U "ansible == X.Y.Z"
+    $ sudo /opt/my-envs/custom-venv/bin/pip install -U "ansible == X.Y.Z"
 
 ...or an additional third-party SDK that's not included with the base awx installation:
 
-    $ sudo /var/lib/awx/venv/my-custom-venv/bin/pip install -U python-digitalocean
+    $ sudo /opt/my-envs/custom-venv/bin/pip install -U python-digitalocean
 
 If you want to copy them, the libraries included in awx's default virtualenv
 can be found using `pip freeze`:
@@ -47,7 +66,7 @@ can be found using `pip freeze`:
 
 One important item to keep in mind is that in a clustered awx installation,
 you need to ensure that the same custom virtualenv exists on _every_ local file
-system at `/var/lib/awx/venv/`.  For container-based deployments, this likely
+system at `/opt/my-envs/`.  For container-based deployments, this likely
 means building these steps into your own custom image building workflow, e.g.,
 
 ```diff
@@ -60,8 +79,9 @@ index aa8b304..eb05f91 100644
     fi
  
 +requirements_custom:
-+	virtualenv $(VENV_BASE)/my-custom-env
-+	$(VENV_BASE)/my-custom-env/bin/pip install psutil
++ mkdir -p /opt/my-envs
++	virtualenv /opt/my-envs/my-custom-env
++	/opt/my-envs/my-custom-env/bin/pip install psutil
 +
 diff --git a/installer/image_build/templates/Dockerfile.j2 b/installer/image_build/templates/Dockerfile.j2
 index d69e2c9..a08bae5 100644
@@ -74,6 +94,8 @@ index d69e2c9..a08bae5 100644
 +    VENV_BASE=/var/lib/awx/venv make requirements_custom && \
 ```
 
+Once the AWX API is available, update the `CUSTOM_VENV_PATHS` setting as described in `Preparing a New Custom Virtualenv`.
+
 Kubernetes Custom Virtualenvs
 =============================
 
@@ -87,7 +109,7 @@ Now create an initContainer stanza.  You can subsititute your own custom images
 
     initContainers:
         - image: 'centos:7'
-          name: init-my-custom-venv
+          name: init-custom-venv
           command:
             - sh
             - '-c'
@@ -95,19 +117,20 @@ Now create an initContainer stanza.  You can subsititute your own custom images
               yum install -y ansible python-pip curl python-setuptools epel-release openssl openssl-devel gcc python-devel &&
               curl 'https://bootstrap.pypa.io/get-pip.py' | python &&
               pip install virtualenv &&
-              virtualenv /var/lib/awx/venv/my-custom-venv &&
-              source /var/lib/awx/venv/my-custom-venv/bin/activate &&
-              /var/lib/awx/venv/my-custom-venv/bin/pip install psutil &&
-              /var/lib/awx/venv/my-custom-venv/bin/pip install -U "ansible == X.Y.Z" &&
-              /var/lib/awx/venv/my-custom-venv/bin/pip install -U custom-python-module
+              mkdir -p /opt/my-envs &&
+              virtualenv /opt/my-envs/custom-venv &&
+              source /opt/my-envs/custom-venv/bin/activate &&
+              /opt/my-envs/custom-venv/bin/pip install psutil &&
+              /opt/my-envs/custom-venv/bin/pip install -U "ansible == X.Y.Z" &&
+              /opt/my-envs/custom-venv/bin/pip install -U custom-python-module
           volumeMounts:
-            - mountPath: /var/lib/awx/venv/my-custom-venv
+            - mountPath: /opt/my-envs/custom-venv
               name: custom-venv
 
 Finally in the awx-celery and awx-web containers stanza add the shared volume as a mount.
 
     volumeMounts:
-        - mountPath: /var/lib/awx/venv/my-custom-venv
+        - mountPath: /opt/my-envs/custom-venv
           name: custom-venv
         - mountPath: /etc/tower
           name: awx-application-config
@@ -116,6 +139,8 @@ Finally in the awx-celery and awx-web containers stanza add the shared volume as
           name: awx-confd
           readOnly: true
 
+Once the AWX API is available, update the `CUSTOM_VENV_PATHS` setting as described in `Preparing a New Custom Virtualenv`.
+
 Assigning Custom Virtualenvs
 ============================
 Once you've created a custom virtualenv, you can assign it at the Organization,
@@ -127,7 +152,7 @@ Project, or Job Template level:
 
     Content-Type: application/json
     {
-        'custom_virtualenv': '/var/lib/awx/venv/my-custom-venv'
+        'custom_virtualenv': '/opt/my-envs/custom-venv'
     }
 
 An HTTP `GET` request to `/api/v2/config/` will provide a list of
@@ -135,8 +160,8 @@ detected installed virtualenvs:
 
     {
         "custom_virtualenvs": [
-            "/var/lib/awx/venv/my-custom-venv",
-            "/var/lib/awx/venv/my-other-custom-venv",
+            "/opt/my-envs/custom-venv",
+            "/opt/my-envs/my-other-custom-venv",
         ],
         ...
     }