acceptance docs for [AUDIC] Arbitrary User-Defined Inventory & Credential

docs/custom_credential_types.md

@@ -0,0 +1,316 @@
+Custom Credential Types Overview
+================================
+
+Prior to Tower 3.2, Tower included bundled credential types, such as
+"Machine", "Network", or "Amazon Web Services".  In 3.2, we have added support
+for custom types so that customers can extend Tower with support for
+third-party credential mechanisms.
+
+Important Changes
+-----------------
+* Tower has a new top-level resource, ``Credential Type``, which can fall into
+  one of several categories, or "kinds":
+
+    - SSH
+    - Vault
+    - Source Control
+    - Network
+    - Cloud
+
+  ``Credential Types`` are composed of a set of field ``inputs`` (for example,
+  ``"username"`` - which is a required string - and ``"password"`` - which is
+  a required string which should be encrypted at storage time) and custom
+  ``injectors`` which define how the inputs are applied to the environment when
+  a job is run (for example, the value for ``"username"`` should be injected
+  into an environment variable named ``"MY_USERNAME"``).
+
+  By utilizing these custom ``Credential Types``, customers have the ability to
+  define custom "Cloud" and "Network" ``Credential Types`` which
+  modify environment variables, extra vars, and generate file-based
+  credentials (such as file-based certificates or .ini files) at
+  `ansible-playbook` runtime.
+
+* Multiple ``Credentials`` can now be assigned to a ``Job Template`` as long as
+  the ``Credential Types`` are unique.  For example, you can now create a ``Job
+  Template`` that uses one SSH, one Vault, one EC2, and one Google Compute
+  Engine credential.  You cannot, however, create a ``Job Template`` that uses
+  two OpenStack credentials.
+
+* Custom inventory sources can now utilize a ``Credential``; you
+  can store third-party credentials encrypted within Tower and use their
+  values from within your custom inventory script (by - for example - reading
+  an environment variable or a file's contents).
+
+API Interaction for Credential Management
+-----------------------------------------
+
+``HTTP GET /api/v2/credential_types`` provides a listing of all supported
+``Credential Types``, including several read-only types that Tower provides
+support for out of the box (SSH, Vault, SCM, Network, Amazon Web Services,
+etc...)
+
+Superusers have the ability to extend Tower by creating, updating, and deleting
+new "custom" ``Credential Types``:
+
+    HTTP POST /api/v2/credential_types/
+
+    {
+        "name": "Third Party Cloud",
+        "description": "Integration with Third Party Cloud",
+        "kind": "cloud",
+        "inputs": {
+            "fields": [{
+                "id": "api_token",
+                "label": "API Token",
+                "type": "string",
+                "secret": True,
+            }]
+        },
+        "injectors": {
+            "env": {
+                "THIRD_PARTY_CLOUD_API_TOKEN": "{{api_token}}"
+            }
+        }
+    }
+
+In Tower 3.2, when users create or modify ``Credentials``, they specify the
+``credential_type``, and the inputs they pass in are dictated by the
+defined ``inputs`` for that ``Credential Type``:
+
+    HTTP POST /api/v2/credentials/
+
+    {
+        "name": "Joe's Third Party Cloud API Token",
+        "description": "",
+        "organization": <pk>,
+        "user": null,
+        "team": null,
+        "credential_type": <pk>,
+        "inputs": {
+            "api_token": "f239248b-97d0-431b-ae2f-091d80c3452e"
+        }
+    }
+
+    HTTP GET /api/v2/credentials/N
+
+    {
+        "name": "Joe's Third Party Cloud API Token",
+        "description": "",
+        "organization": <pk>,
+        "user": null,
+        "team": null,
+        "credential_type": <pk>,
+        "inputs": {
+            "api_token": "$encrypted$"
+        }
+    }
+
+Defining Custom Credential Type Inputs
+--------------------------------------
+
+A ``Credential Type`` specifies an ``inputs`` schema which defines a set of
+ordered fields for that type:
+
+    "inputs": {
+        "fields": [{
+            "id": "api_token",                                  # required - a unique name used to
+                                                                # reference the field value
+
+            "label": "API Token",                               # required - a unique label for the
+                                                                # field
+
+            "help_text": "User-facing short text describing the field.",
+
+            "type": ("string" | "number" | "ssh_private_key")   # required,
+
+            "secret": true,                                     # if true, the field will be treated
+                                                                # as sensitive and stored encrypted
+
+            "multiline": false                                  # if true, the field should be rendered
+                                                                # as multi-line for input entry
+        },{
+            # field 2...
+        },{
+            # field 3...
+        }]
+        "required": ["api_token"]  # optional; one or more fields can be marked as required
+    },
+
+As an alternative to static types, fields can also specify multiple choice
+strings:
+
+    "inputs": {
+        "fields": [{
+            "id": "api_token",          # required - a unique name used to reference the field value
+            "label": "API Token",       # required - a unique label for the field
+            "choices": ["A", "B", "C"]
+        }]
+    },
+
+Defining Custom Credential Type Injectors
+-----------------------------------------
+A ``Credential Type`` can inject ``Credential`` values through the use
+of the Jinja templating language (which should be familiar to users of Ansible):
+
+    "injectors": {
+        "env": {
+            "THIRD_PARTY_CLOUD_API_TOKEN": "{{api_token}}"
+        },
+        "extra_vars": {
+            "some_extra_var": "{{username}}:{{password}"
+        }
+    }
+
+``Credential Types`` can also generate temporary files to support .ini files or
+certificate/key data:
+
+    "injectors": {
+        "file": {
+            "template": "[mycloud]\ntoken={{api_token}}"
+        },
+        "env": {
+            "MY_CLOUD_INI_FILE": "{{tower.filename}"
+        }
+    }
+
+
+Job and Job Template Credential Assignment
+------------------------------------------
+
+In Tower 3.2, ``Jobs`` and ``Job Templates`` have a new many-to-many
+relationship with ``Credential`` that allows selection of multiple
+network/cloud credentials.  As such, the ``Job`` and ``JobTemplate``
+API resources in `/api/v2/` now have two credential related fields:
+
+    HTTP GET /api/v2/job_templates/N/
+
+    {
+      ...
+      'credential': <integer-or-null>
+      'vault_credential': <integer-or-null>
+      ...
+    }
+
+...and a new endpoint for fetching all "extra" credentials:
+
+    HTTP GET /api/v2/job_templates/N/extra_credentials/
+
+    {
+        'count': N,
+        'results': [{
+            'name': 'My Credential',
+            'credential_type': <pk>,
+            'inputs': {...},
+            ...
+        }]
+    }
+
+Similar to other list attachment/detachment API views, cloud and network
+credentials can be created and attached via an `HTTP POST` at this new
+endpoint:
+
+    HTTP POST /api/v2/job_templates/N/extra_credentials/
+
+    {
+        'id': <cloud_credential_primary_key>,
+        'associate': True,
+    }
+
+    HTTP POST /api/v2/job_templates/N/extra_credentials/
+
+    {
+        'id': <network_credential_primary_key>,
+        'disassociate': True,
+    }
+
+    HTTP POST /api/v2/job_templates/N/extra_credentials/
+
+    {
+        'name': 'My Credential',
+        'credential_type': <primary_key>,
+        'inputs': {...},
+        ...
+    }
+
+
+API Backwards Compatability
+---------------------------
+
+`/api/v1/credentials/` still exists in Tower 3.2, and it transparently works as
+before with minimal surprises by attempting to translate `/api/v1/` requests to
+the new ``Credential`` and ``Credential Type`` models.
+
+* When creating or modifying a ``Job Template`` through `v1` of the API,
+  old-style credential assignment will transparently map to the new model.  For
+  example, the following `POST`'ed payload:
+
+      {
+        credential: <pk>,
+        vault_credential: <pk>,
+        cloud_credential: <pk>,
+        network_credential: <pk>,
+      }
+
+  ...would transparently update ``JobTemplate.extra_credentials`` to a list
+  containing both the cloud and network ``Credentials``.
+
+  Similarly, an `HTTP GET /api/v1/job_credentials/N/` will populate
+  `cloud_credential`, and `network_credential` with the *most recently applied*
+  matching credential in the list.
+
+* Custom ``Credentials`` will not be returned in the ``v1`` API; if a user
+  defines their own ``Credential Type``, its credentials won't show up in the
+  ``v1`` API.
+
+* ``HTTP POST`` requests to ``/api/v1/credentials/`` will transparently map
+  old-style attributes (i.e., ``username``, ``password``, ``ssh_key_data``) to
+  the appropriate new-style model.  Similarly, ``HTTP GET
+  /api/v1/credentials/N/`` requests will continue to contain old-style
+  key-value mappings in their payloads.
+
+* Vault credentials are a new first-level type of credential in Tower 3.2.
+  As such, any ``Credentials`` pre-Tower 3.2 that contain *both* SSH and Vault
+  parameters will be migrated to separate distinct ``Credentials``
+  post-migration.
+
+  For example, if your Tower 3.1 installation has one ``Credential`` with
+  a defined ``username``, ``password``, and ``vault_password``, after migration
+  *two* ``Credentials`` will exist (one which contains the ``username`` and
+  ``password``, and another which contains only the ``vault_password``).
+
+
+Additional Criteria
+-------------------
+* Rackspace is being removed from official support in Tower 3.2.  Pre-existing
+  Rackspace Cloud credentials should be automatically migrated to "custom"
+  credentials.  If a customer has never created or used Rackspace Cloud
+  credentials, the only change they should notice in Tower 3.2 is that
+  Rackspace is no longer an option provided by Tower when creating/modifying
+  a Credential.
+
+
+Acceptance Criteria
+-------------------
+When verifying acceptance we should ensure the following statements are true:
+
+* `Credential` injection for playbook runs, SCM updates, inventory updates, and
+  ad-hoc runs should continue to function as they did prior to Tower 3.2 for the
+  `Credential Types` provided by Tower.
+* It should be possible to create and modify every type of `Credential` supported
+  prior to Tower 3.2 (SSH, SCM, EC2, etc..., with the exception of Rackspace).
+* Superusers (and only superusers) should be able to define custom `Credential
+  Types`.  They should properly inject environment variables, extra vars, and
+  files for playbook runs, SCM updates, inventory updates, and ad-hoc runs.
+* The default `Credential Types` included with Tower in 3.2 should be
+  non-editable/readonly and cannot be deleted by any user.
+* Stored `Credential` values for _all_ types should be consistent before and
+  after Tower 3.2 migration/upgrade.
+* `Job Templates` should be able to specify multiple extra `Credentials` as
+  defined in the constraints in this document.
+* Custom inventory sources should be able to specify a cloud/network
+  `Credential` and they should properly update the environment (environment
+  variables, extra vars, written files) when an inventory source update runs.
+* If a `Credential Type` is being used by one or more `Credentials`, the fields
+  defined in its ``inputs`` should be read-only.
+* `Credential Types` should support activity stream history for basic object
+  modification.