External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-02-01 01:28:09 -03:30
Code Issues Packages Projects Releases Wiki Activity

Edit AWX docs

Browse Source
This commit is contained in:
beeankha
2019-09-11 16:27:28 -04:00
parent 14cc203945
commit e2be392f31
16 changed files with 283 additions and 526 deletions
Download Patch File Download Diff File Expand all files Collapse all files

313
docs/credentials/credential_plugins.md Normal file
View File

@@ -0,0 +1,313 @@
Credential Plugins
==================
By default, sensitive credential values (such as SSH passwords, SSH private
keys, API tokens for cloud services, etc.) in AWX are stored in the AWX database
after being encrypted with a symmetric encryption cipher utilizing AES-256 in
CBC mode alongside a SHA-256 HMAC.
Alternatively, AWX supports retrieving secret values from third-party secret
management systems, such as HashiCorp Vault and Microsoft Azure Key Vault.
These external secret values will be fetched on demand every time they are
needed (generally speaking, immediately before running a playbook that needs
them).
Configuring Secret Lookups
--------------------------
When configuring AWX to pull a secret from a third party system, there are
generally three steps.
Here is an example of creating an (1) AWX Machine Credential with
a static username, `example-user` and (2) an externally-sourced secret from
HashiCorp Vault Key/Value system which will populate the (3) password field on
the Machine Credential:
1. Create the Machine Credential with a static username, `example-user`.
2. Create a second credential used to _authenticate_ with the external
secret management system (in this example, specifying a URL and an
OAuth2.0 token _to access_ HashiCorp Vault)
3. _Link_ the `password` field for the Machine cCredential to the external
system by specifying the source (in this example, the HashiCorp Credential)
and metadata about the path (e.g., `/some/path/to/my/password/`).
Note that you can perform these lookups on *any* field for any non-external
credential, including those with custom credential types. You could just as
easily create an AWS Credential and use lookups to retrieve the Access Key and
Secret Key from an external secret management system. External credentials
cannot have lookups applied to their fields.
Writing Custom Credential Plugins
---------------------------------
Credential Plugins in AWX are just importable Python functions that are
registered using setuptools entrypoints
(https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins)
Example plugins officially supported in AWX can be found in the source code at
`awx.main.credential_plugins`.
Credential plugins are any Python object which defines attribute lookups for `.name`, `.inputs`, and `.backend`:
```python
import collections
CredentialPlugin = collections.namedtuple('CredentialPlugin', ['name', 'inputs', 'backend'])
def some_callable(value_from_awx, **kwargs):
return some_libary.get_secret_key(
url=kwargs['url'],
token=kwargs['token'],
key=kwargs['secret_key']
)
some_fancy_plugin = CredentialPlugin(
'My Plugin Name',
# inputs will be used to create a new CredentialType() instance
#
# inputs.fields represents fields the user will specify *when they create*
# a credential of this type; they generally represent fields
# used for authentication (URL to the credential management system, any
# fields necessary for authentication, such as an OAuth2.0 token, or
# a username and password). They're the types of values you set up _once_
# in AWX
#
# inputs.metadata represents values the user will specify *every time
# they link two credentials together*
# this is generally _pathing_ information about _where_ in the external
# management system you can find the value you care about i.e.,
#
# "I would like Machine Credential A to retrieve its username using
# Credential-O-Matic B at secret_key=some_key"
inputs={
'fields': [{
'id': 'url',
'label': 'Server URL',
'type': 'string',
}, {
'id': 'token',
'label': 'Authentication Token',
'type': 'string',
'secret': True,
}],
'metadata': [{
'id': 'secret_key',
'label': 'Secret Key',
'type': 'string',
'help_text': 'The value of the key in My Credential System to fetch.'
}],
'required': ['url', 'token', 'secret_key'],
},
# backend is a callable function which will be passed all of the values
# defined in `inputs`; this function is responsible for taking the arguments,
# interacting with the third party credential management system in question
# using Python code, and returning the value from the third party
# credential management system
backend = some_callable
```
Plugins are registered by specifying an entry point in the `setuptools.setup()`
call (generally in the package's `setup.py` file - https://github.com/ansible/awx/blob/devel/setup.py):
```python
setuptools.setup(
...,
entry_points = {
...,
'awx.credential_plugins': [
'fancy_plugin = awx.main.credential_plugins.fancy:some_fancy_plugin',
]
}
)
```
Programmatic Secret Fetching
----------------------------
If you want to programmatically fetch secrets from a supported external secret
management system (for example, if you wanted to compose an AWX database connection
string in `/etc/tower/conf.d/postgres.py` using an external system rather than
storing the password in plaintext on your disk), doing so is fairly easy:
```python
from awx.main.credential_plugins import hashivault
hashivault.hashivault_kv_plugin.backend(
url='https://hcv.example.org',
token='some-valid-token',
api_version='v2',
secret_path='/path/to/secret',
secret_key='dbpass'
)
```
Supported Plugins
=================
HashiCorp Vault KV
------------------
AWX supports retrieving secret values from HashiCorp Vault KV
(https://www.vaultproject.io/api/secret/kv/)
The following example illustrates how to configure a Machine Credential to pull
its password from a HashiCorp Vault:
1. Look up the ID of the Machine and HashiCorp Vault Secret Lookup Credential
types (in this example, `1` and `15`):
```shell
~ curl -sik "https://awx.example.org/api/v2/credential_types/?name=Machine" \
-H "Authorization: Bearer <token>"
HTTP/1.1 200 OK
{
"results": [
{
"id": 1,
"url": "/api/v2/credential_types/1/",
"name": "Machine",
...
```
```shell
~ curl -sik "https://awx.example.org/api/v2/credential_types/?name__startswith=HashiCorp" \
-H "Authorization: Bearer <token>"
HTTP/1.1 200 OK
{
"results": [
{
"id": 15,
"url": "/api/v2/credential_types/15/",
"name": "HashiCorp Vault Secret Lookup",
...
```
2. Create a Machine and a HashiCorp Vault Credential:
```shell
~ curl -sik "https://awx.example.org/api/v2/credentials/" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-X POST \
-d '{"user": N, "credential_type": 1, "name": "My SSH", "inputs": {"username": "example"}}'
HTTP/1.1 201 Created
{
"credential_type": 1,
"description": "",
"id": 1,
...
```
```shell
~ curl -sik "https://awx.example.org/api/v2/credentials/" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-X POST \
-d '{"user": N, "credential_type": 15, "name": "My Hashi Credential", "inputs": {"url": "https://vault.example.org", "token": "vault-token", "api_version": "v2"}}'
HTTP/1.1 201 Created
{
"credential_type": 15,
"description": "",
"id": 2,
...
```
3. Link the Machine Credential to the HashiCorp Vault Credential:
```shell
~ curl -sik "https://awx.example.org/api/v2/credentials/1/input_sources/" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-X POST \
-d '{"source_credential": 2, "input_field_name": "password", "metadata": {"secret_path": "/kv/my-secret", "secret_key": "password"}}'
HTTP/1.1 201 Created
```
HashiCorp Vault SSH Secrets Engine
----------------------------------
AWX supports signing public keys via HashiCorp Vault's SSH Secrets Engine
(https://www.vaultproject.io/api/secret/ssh/)
The following example illustrates how to configure a Machine Credential to sign
a public key using HashiCorp Vault:
1. Look up the ID of the Machine and HashiCorp Vault Signed SSH Credential
types (in this example, `1` and `16`):
```shell
~ curl -sik "https://awx.example.org/api/v2/credential_types/?name=Machine" \
-H "Authorization: Bearer <token>"
HTTP/1.1 200 OK
{
"results": [
{
"id": 1,
"url": "/api/v2/credential_types/1/",
"name": "Machine",
...
```
```shell
~ curl -sik "https://awx.example.org/api/v2/credential_types/?name__startswith=HashiCorp" \
-H "Authorization: Bearer <token>"
HTTP/1.1 200 OK
{
"results": [
{
"id": 16,
"url": "/api/v2/credential_types/16/",
"name": "HashiCorp Vault Signed SSH",
```
2. Create a Machine and a HashiCorp Vault Credential:
```shell
~ curl -sik "https://awx.example.org/api/v2/credentials/" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-X POST \
-d '{"user": N, "credential_type": 1, "name": "My SSH", "inputs": {"username": "example", "ssh_key_data": "RSA KEY DATA"}}'
HTTP/1.1 201 Created
{
"credential_type": 1,
"description": "",
"id": 1,
...
```
```shell
~ curl -sik "https://awx.example.org/api/v2/credentials/" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-X POST \
-d '{"user": N, "credential_type": 16, "name": "My Hashi Credential", "inputs": {"url": "https://vault.example.org", "token": "vault-token"}}'
HTTP/1.1 201 Created
{
"credential_type": 16,
"description": "",
"id": 2,
...
```
3. Link the Machine Credential to the HashiCorp Vault Credential:
```shell
~ curl -sik "https://awx.example.org/api/v2/credentials/1/input_sources/" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-X POST \
-d '{"source_credential": 2, "input_field_name": "password", "metadata": {"public_key": "UNSIGNED PUBLIC KEY", "secret_path": "/ssh/", "role": "example-role"}}'
HTTP/1.1 201 Created
```
4. Associate the Machine Credential with a Job Template. When the Job Template
is run, AWX will use the provided HashiCorp URL and token to sign the
unsigned public key data using the HashiCorp Vault SSH Secrets API.
AWX will generate an `id_rsa` and `id_rsa-cert.pub` on the fly and
apply them using `ssh-add`.

305
docs/credentials/custom_credential_types.md Normal file
View File

@@ -0,0 +1,305 @@
Custom Credential Types Overview
================================
Prior to Tower 3.2, Tower included bundled credential types, such as
"Machine", "Network", or "Amazon Web Services". In 3.2, we have added support
for custom types so that customers can extend Tower with support for
third-party credential mechanisms.
Important Changes
-----------------
* Tower has a new top-level resource, ``Credential Type``, which can fall into
one of several categories, or "kinds":
- SSH
- Vault
- Source Control
- Network
- Cloud
``Credential Types`` are composed of a set of field ``inputs`` (for example,
``"username"`` - which is a required string - and ``"password"`` - which is
a required string which should be encrypted at storage time) and custom
``injectors`` which define how the inputs are applied to the environment when
a job is run (for example, the value for ``"username"`` should be injected
into an environment variable named ``"MY_USERNAME"``).
By utilizing these custom ``Credential Types``, customers have the ability to
define custom "Cloud" and "Network" ``Credential Types`` which
modify environment variables, extra vars, and generate file-based
credentials (such as file-based certificates or `.ini` files) at
`ansible-playbook` runtime.
* Multiple ``Credentials`` can now be assigned to a ``Job Template`` as long as
the ``Credential Types`` are unique. For example, you can now create a ``Job
Template`` that uses one SSH, one Vault, one EC2, and one Google Compute
Engine credential. You cannot, however, create a ``Job Template`` that uses
two OpenStack credentials.
* In the same manner as "promptable SSH credentials", when
``ask_credential_on_launch = true``, ``JobTemplate.extra_credentials`` can be
specified in the launch payload.
* Custom inventory sources can now utilize a ``Credential``; you
can store third-party credentials encrypted within Tower and use their
values from within your custom inventory script (by - for example - reading
an environment variable or a file's contents).
API Interaction for Credential Management
-----------------------------------------
``HTTP GET /api/v2/credential_types`` provides a listing of all supported
``Credential Types``, including several read-only types that Tower provides
support for out of the box (SSH, Vault, SCM, Network, Amazon Web Services,
etc...)
Superusers have the ability to extend Tower by creating, updating, and deleting
new "custom" ``Credential Types``:
HTTP POST /api/v2/credential_types/
{
"name": "Third Party Cloud",
"description": "Integration with Third Party Cloud",
"kind": "cloud",
"inputs": {
"fields": [{
"id": "api_token",
"label": "API Token",
"type": "string",
"secret": true
}]
},
"injectors": {
"env": {
"THIRD_PARTY_CLOUD_API_TOKEN": "{{api_token}}"
}
}
}
In Tower 3.2, when users create or modify ``Credentials``, they specify the
``credential_type``, and the inputs they pass in are dictated by the
defined ``inputs`` for that ``Credential Type``:
HTTP POST /api/v2/credentials/
{
"name": "Joe's Third Party Cloud API Token",
"description": "",
"organization": <pk>,
"user": null,
"team": null,
"credential_type": <pk>,
"inputs": {
"api_token": "f239248b-97d0-431b-ae2f-091d80c3452e"
}
}
HTTP GET /api/v2/credentials/N
{
"name": "Joe's Third Party Cloud API Token",
"description": "",
"organization": <pk>,
"user": null,
"team": null,
"credential_type": <pk>,
"inputs": {
"api_token": "$encrypted$"
}
}
Defining Custom Credential Type Inputs
--------------------------------------
A ``Credential Type`` specifies an ``inputs`` schema which defines a set of
ordered fields for that type:
"inputs": {
"fields": [{
"id": "api_token", # required - a unique name used to
# reference the field value
"label": "API Token", # required - a unique label for the
# field
"help_text": "User-facing short text describing the field.",
"type": ("string" | "boolean") # defaults to 'string'
"format": "ssh_private_key" # optional, can be used to enforce data
# format validity for SSH private key
# data (only applicable to `type=string`)
"secret": true, # if true, the field value will be encrypted
"multiline": false # if true, the field should be rendered
# as multi-line for input entry
# (only applicable to `type=string`)
"default": "default value" # optional, can be used to provide a
# default value if the field is left empty;
# when creating a credential of this type,
# credential forms will use this value
# as a prefill when making credentials of
# this type
},{
# field 2...
},{
# field 3...
}]
"required": ["api_token"] # optional; one or more fields can be marked as required
},
When `type=string`, fields can optionally specify multiple choice options:
"inputs": {
"fields": [{
"id": "api_token", # required - a unique name used to reference the field value
"label": "API Token", # required - a unique label for the field
"type": "string",
"choices": ["A", "B", "C"]
}]
},
Defining Custom Credential Type Injectors
-----------------------------------------
A ``Credential Type`` can inject ``Credential`` values through the use
of the [Jinja templating language](https://jinja.palletsprojects.com/en/2.10.x/) (which should be familiar to users of Ansible):
"injectors": {
"env": {
"THIRD_PARTY_CLOUD_API_TOKEN": "{{api_token}}"
},
"extra_vars": {
"some_extra_var": "{{username}}:{{password}"
}
}
``Credential Types`` can also generate temporary files to support `.ini` files or
certificate/key data:
"injectors": {
"file": {
"template": "[mycloud]\ntoken={{api_token}}"
},
"env": {
"MY_CLOUD_INI_FILE": "{{tower.filename}}"
}
}
3.3 adds the ability for a single ``Credential Type`` to inject multiple files:
"injectors": {
"file": {
"template.cert": "{{cert}}",
"template.key": "{{key}}"
},
"env": {
"MY_CERT": "{{tower.filename.cert}",
"MY_KEY": "{{tower.filename.key}}"
}
}
Note that the single and multi-file syntax cannot be mixed within the same
``Credential Type``.
Job and Job Template Credential Assignment
------------------------------------------
In Tower 3.2, ``Jobs`` and ``Job Templates`` have a new many-to-many
relationship with ``Credential`` that allows selection of multiple
network/cloud credentials. As such, the ``Job`` and ``JobTemplate``
API resources in `/api/v2/` now have two credential related fields:
HTTP GET /api/v2/job_templates/N/
{
...
'credential': <integer-or-null>
'vault_credential': <integer-or-null>
...
}
...and a new endpoint for fetching all "extra" credentials:
HTTP GET /api/v2/job_templates/N/extra_credentials/
{
'count': N,
'results': [{
'name': 'My Credential',
'credential_type': <pk>,
'inputs': {...},
...
}]
}
Similar to other list attachment/detachment API views, cloud and network
credentials can be created and attached via an `HTTP POST` at this new
endpoint:
HTTP POST /api/v2/job_templates/N/extra_credentials/
{
'id': <cloud_credential_primary_key>,
'associate': True,
}
HTTP POST /api/v2/job_templates/N/extra_credentials/
{
'id': <network_credential_primary_key>,
'disassociate': True,
}
HTTP POST /api/v2/job_templates/N/extra_credentials/
{
'name': 'My Credential',
'credential_type': <primary_key>,
'inputs': {...},
...
}
Additional Criteria
-------------------
* Rackspace is being removed from official support in Tower 3.2. Pre-existing
Rackspace Cloud credentials should be automatically migrated to "custom"
credentials. If a customer has never created or used Rackspace Cloud
credentials, the only change they should notice in Tower 3.2 is that
Rackspace is no longer an option provided by Tower when creating/modifying
a Credential.
Acceptance Criteria
-------------------
When verifying acceptance, the following statements should be true:
* `Credential` injection for playbook runs, SCM updates, inventory updates, and
ad-hoc runs should continue to function as they did prior to Tower 3.2 for the
`Credential Types` provided by Tower.
* It should be possible to create and modify every type of `Credential` supported
prior to Tower 3.2 (SSH, SCM, EC2, etc..., with the exception of Rackspace).
* Superusers (and only superusers) should be able to define custom `Credential
Types`. They should properly inject environment variables, extra vars, and
files for playbook runs, SCM updates, inventory updates, and ad-hoc runs.
* Custom `Credential Types` should support injecting both single and
multiple files. (Furthermore, the new syntax for injecting multiple files
should work properly even if only a single file is injected).
* Users should not be able to use the syntax for injecting single and
multiple files in the same custom credential.
* The default `Credential Types` included with Tower in 3.2 should be
non-editable/read-only and unable to be deleted by any user.
* Stored `Credential` values for _all_ types should be consistent before and
after a Tower 3.2 migration/upgrade.
* `Job Templates` should be able to specify multiple extra `Credentials` as
defined in the constraints in this document.
* Custom inventory sources should be able to specify a cloud/network
`Credential` and they should properly update the environment (environment
variables, extra vars, written files) when an inventory source update runs.
* If a `Credential Type` is being used by one or more `Credentials`, the fields
defined in its `inputs` should be read-only.
* `Credential Types` should support Activity Stream history for basic object
modification.