Commit Graph
100 Commits
Author SHA1 Message Date
Ryan Petrello 97562fc4df block job template launches when an inventory is being deleted
see: #4382
2017-06-22 15:33:55 -04:00
Ryan Petrello 14addae813 delete inventories in the background via a celery task
see: #4382
see: #6279
2017-06-22 14:32:33 -04:00
Ryan Petrello 5e908bfc50 more isolated acceptance doc updates 2017-06-22 12:32:20 -04:00
Ryan Petrello 0804b7336c make isolated sdist use the correct filename for unofficial builds 2017-06-22 10:51:29 -04:00
Ryan Petrello 45bdd9f747 improve readability of the honcho console logs in the dev environment
* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
  are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
2017-06-22 09:19:32 -04:00
Ryan Petrello 39525316ac prefix all /api/v1/ view descriptions with a deprecation message
additionally, fix a bug in /api/v1/credentials/ view descriptions that
causes v2 fields to display (see: #6116)
2017-06-21 16:27:07 -04:00
Ryan Petrello e832a56506 fix a bug in team-based credential validation
when a credential is created with `team` in the payload, set the
credential's `organization` *prior* to validation so that we don't miss
organization-oriented validators (like the org + name + kind unique
validation)

see: #3303
2017-06-21 14:25:51 -04:00
Ryan Petrello b8e0c91bd4 store stderr on successful inventory syncs
see: #4823
2017-06-21 10:20:50 -04:00
Ryan Petrello c35d83df13 fix a race condition in isolated stdout processing 2017-06-19 16:59:03 -04:00
Ryan Petrello 2ba9e56033 Include the Tower venv in the isolated bubblewrapped arguments.
This moves the container-based code location and venvs.
The goal here is that the paths of Tower source for isolated
vs normal nodes matches (both in prod and local development) so that we
don't have to add a bunch of additional bwrap argument logic for
<location-of-isolated-tower-venv>.
2017-06-19 15:52:54 -04:00
Ryan Petrello 1ea03aa4c9 more isolated task execution tweaking
* set a more reasonable default `AWX_ISOLATED_CHECK_INTERVAL`
* make manual cancellation work for high values of
  `AWX_ISOLATED_CHECK_INTERVAL`
* remove the `/tmp/ansible_tower/jobs/` symlink directory

see: #6616
2017-06-16 15:37:07 -04:00
Ryan Petrello 1b88349f1e fix a typo in the tower-manage instance_group_remove arg parsing 2017-06-16 11:21:35 -04:00
Ryan Petrello 44e0c8621a isolated ramparts: replace systemd unit with a tower-expect binary
instead of launching isolated tasks via `systemctl`, treat
`awx.main.isolated.run` as an executable that knows how to daemonize

additionally, add `setup.py isolated_build` for isolated Tower source
distribution
2017-06-16 09:59:21 -04:00
Ryan Petrello e95b254a3f provide a more helpful error message for secret decryption failures.
see: #6230
see: #6395
2017-06-13 14:35:15 -04:00
Ryan Petrello d4afa41acb key prompting of all credentials off of ask_credential_on_launch
see: #6371
2017-06-12 12:26:58 -04:00
Ryan Petrello 240d629128 fix a bug in ssh key unlock validation
see: #6553
2017-06-12 10:28:38 -04:00
Ryan Petrello 28ad576c90 properly validate ssh_key_unlock for Net and SCM credentials
see: #6460
2017-06-09 10:43:04 -04:00
Ryan Petrello ea0f4ce59d properly validate SSH key data for SCM, Net, GCE, and Azure Classic
see: #6384
2017-06-09 10:39:07 -04:00
Ryan Petrello 9bba3d9749 resolve a few CredentialType.inputs validation issues
see: #6363
2017-06-06 15:48:23 -04:00
Ryan Petrello 9445fed340 use str() - not repr() - for invalid SSH cert error messages
see: #6417
2017-06-06 13:53:13 -04:00
Ryan Petrello ad9fda9a06 add a boolean authorize field for the Network Credential Type
see: #6464
2017-06-06 11:13:10 -04:00
Ryan Petrello af457ad8eb clarify valid CredentialType field types
almost all of our current credential values are strings, but under the
v1 Credential model, `authorize` is boolean

additionally, if a field is specified with no type, fall back to
`string` as a default (this is almost always what people creating custom
types will want)

see: #6406
2017-06-01 13:23:51 -04:00
Ryan Petrello fd4b86349c fix a 500 error in /api/v1/credentials/ backwards compat
see: #6414
2017-06-01 11:26:18 -04:00
Ryan Petrello 71dda544ab fix a bug in /api/v1/credential detection of Vault payloads
see: #6390
2017-06-01 10:18:43 -04:00
Ryan Petrello e0a629db58 improve error formatting for jsonschema failures on Credential.inputs
this provides error messages keyed by input fields, so that instead of
e.g.,

{
    'inputs': ['Invalid certificate or key: u'XYZ']
}

...you get:

{
    'inputs': {
        'ssh_key_data': ['Invalid certificate or key: u'XYZ']
    }
}

Includes /api/v1/ compatability for error message format.  Requests to
/api/v1/ will get:

{'ssh_key_data': ['Invalid certificate or key: u'XYZ']}
2017-06-01 09:48:42 -04:00
Ryan Petrello 98fa654be2 enforce dict for credential API input payloads
see: #6363
2017-06-01 09:19:21 -04:00
Ryan Petrello 462b2c6696 improve a few permission-related errors for CredentialType update/delete
see: #6385
2017-05-31 15:48:34 -04:00
Ryan Petrello 23a1c94d38 OpenStack CredentialType needs a domain input field
see: #6380
2017-05-31 13:44:38 -04:00
Ryan Petrello 2d0ae56d94 fix incorrect AWS Credential Type labels
see: #6372
2017-05-31 13:29:22 -04:00
Ryan Petrello 34b4270b1c add more context for default extra_credentials to the JT launch endpoint
see: #6394
2017-05-31 13:22:46 -04:00
Ryan Petrello 5a34f3a1a4 remove a hook QE no longer uses for testing task execution 2017-05-18 15:59:36 -04:00
Ryan Petrello 1ba71acf07 fix a typo causing tests to fail 2017-05-12 15:34:23 -04:00
Ryan Petrello b9007997eb squash together a few credential-related migrations 2017-05-12 14:42:12 -04:00
Ryan Petrello 0ac4f71e5b refactor and fix ssh_private_key and ssh_key_unlock validation
`clean_ssh_key_data` and `clean_ssh_key_unlock` no longer work because
they're not actual fields on `model.Credential` anymore.  This change
refactors/moves their validation to a place that works (and makes more
sense).
2017-05-12 09:42:29 -04:00
Ryan Petrello 53c3ece32b add prompting for JT.extra_credentials
see: #2913
2017-05-11 07:29:53 -04:00
Ryan Petrello 3126bfa1a2 add tooling to aid in remote debugging sessions 2017-05-09 15:36:13 -04:00
Ryan Petrello 9a6dc4d0ef add a view for /api/v2/credential_types/N/activity_stream/ 2017-05-09 15:01:04 -04:00
Ryan Petrello f7660e1778 properly filter User.password from ActivityStream entries
see: #6160
2017-05-09 14:56:27 -04:00
Ryan Petrello 2feb7d5a69 add a new read-only view for /api/v2/credential_types/N/credentials/ 2017-05-08 17:11:09 -04:00
Ryan Petrello 1568fddde1 Make CredentialType conditionally editable/readonly.
* CredentialTypes should not be editable *or* deletable if they're
  "managed_by_tower".
* CredentialTypes should not be deletable if they're in use by one or
  more Credentials.
* CredentialType.inputs should not be editable if they're in use by one
  or more Credentials.

see: #6077
2017-05-08 16:13:26 -04:00
Ryan Petrello b7b9fb531e properly support (cloud|network)_credential for JT update *and* create
fix a bug which caused `POST /api/v1/job_templates/` to not properly set
`JobTemplate.extra_credentials`.

see: #5807
2017-05-08 13:50:28 -04:00
Ryan Petrello 422a62fb34 fix broken credential type migration upgrade 2017-05-05 13:19:59 -04:00
Ryan Petrello ef09744b75 enforce variable name syntax and uniqueness for Credential Types
see: #6158
2017-05-04 16:18:56 -04:00
Ryan Petrello f3591b81a7 more multicredential JobTemplate changes
* allow for filtering Jobs and JobTemplates by v1 `cloud_credential` and
 `network_credential` fields
* properly validate uniqueness of `extra_credentials` types

see: #5807
2017-05-04 16:18:48 -04:00
Ryan Petrello 4a438e044e acceptance docs for [AUDIC] Arbitrary User-Defined Inventory & Credential 2017-05-03 14:09:46 -04:00
Ryan Petrello d0a848d49a Add a new extra_credentials endpoint for Jobs and JobTemplates
additionally, add backwards compatible support for `cloud_credential`
and `network_credential` in /api/v1/job_templates/ and /api/v1/jobs/.

see: #5807
2017-05-03 13:38:10 -04:00
Ryan Petrello accf7cdea2 Replace Job/JT cloud/network credentials with a single M2M relation.
The following fields:

    * (Job | JobTemplate).cloud_credential
    * (Job | JobTemplate).network_credential

...are replaced by M2M relationships:

    * Job.extra_credentials
    * JobTemplate.extra_credentials

Includes support for task execution with multiple cloud credentials.

see: #5807
2017-05-03 13:36:59 -04:00
Ryan Petrello 61b3f7afb7 disallow ask_at_runtime fields for custom credential types 2017-05-02 10:45:43 -04:00
Ryan Petrello a1fa9243bc split machine CredentialType into two distinct (ssh and vault) kinds 2017-05-02 10:26:37 -04:00
Ryan Petrello f6046d47c9 work around a change in ansible 2.3 to Task.get_name()
Properly detect setup tasks so that we can filter out sensitive
variables from the `ansible_facts` portion of the event payload.
2017-05-01 10:38:05 -04:00
Ryan Petrello def2034883 add a new vault_credential relationship to Job/JobTemplate
additionally, properly assign vault credentials to Jobs and JobTemplates
as they're migrated to the new split model

see: #5807
see: #5878
2017-04-28 15:00:01 -04:00
Ryan Petrello 03460b311e treat tower as a reserved field name for custom CredentialTypes 2017-04-28 11:26:18 -04:00
Ryan Petrello 66e004a9db fix a few more migration-related issues for credentials 2017-04-26 09:24:33 -04:00
Ryan Petrello c0add33212 properly migrate vault credentials to the new credentialtype model 2017-04-25 15:03:16 -04:00
Ryan Petrello 80df1523b0 refactor API v2 version detection and feature inclusion 2017-04-25 09:35:03 -04:00
Ryan Petrello 83dc4f6757 for /api/v1/ requests, filter out v2 (custom) credentials
see: #5877
2017-04-24 15:21:30 -04:00
Ryan Petrello aff25c914e blacklist special env vars from being used in CredentialType injectors
see: #5877
2017-04-24 15:03:50 -04:00
Ryan Petrello a36a53fe40 implement CredentialType env, file, and extra_vars injectors
see: #5877
2017-04-24 15:03:33 -04:00
Ryan Petrello 97b2524156 migrate v1 credentials to the new v2 credentialtype + credential model
see: #5878
2017-04-24 13:00:13 -04:00
Ryan Petrello 3067435799 add additional test coverage for tasks.py credential usage 2017-04-21 15:42:27 -04:00
Ryan Petrello 7e64a6fd60 add "added_in_api_version" metadata to OPTIONS requests 2017-04-21 15:42:27 -04:00
Ryan Petrello ba259e0ad4 Introduce a new CredentialTemplate model
Credentials now have a required CredentialType, which defines inputs
(i.e., username, password) and injectors (i.e., assign the username to
SOME_ENV_VARIABLE at job runtime)

This commit only implements the model changes necessary to support the
new inputs model, and includes code for the credential serializer that
allows backwards-compatible support for /api/v1/credentials/; tasks.py
still needs to be updated to actually respect CredentialType injectors.

This change *will* break the UI for credentials (because it needs to be
updated to use the new v2 endpoint).

see: #5877
see: #5876
see: #5805
2017-04-21 15:42:26 -04:00
Ryan Petrello 9f46b3c704 add tacacs_plus package as a dependency 2017-04-12 15:39:11 -04:00
Ryan Petrello d1eb33bca0 Be more restrictive in catching LDAP exceptions.
django-auth-ldap can throw exceptions, like TypeError if it's
misconfigured.  If we encounter an exception attempting to log into an
LDAP server, log it and move on (otherwise, an uncaught exception will
cause local login to fail, too).

see: #5933
2017-04-11 16:43:26 -04:00
Ryan Petrello 5623f16249 fix a bug in the v1/v2 API switch 2017-04-07 13:58:17 -04:00
Ryan Petrello 95ea370e5e add API versioning for /api/v2/ 2017-03-30 15:11:12 -04:00
Ryan Petrello f9b5b9aa91 property filter no_log for item event loops
see: #5691
2017-03-22 11:15:33 -04:00
Ryan Petrello a69dfced74 Merge remote-tracking branch 'origin/release_3.1.2' into devel 2017-03-21 10:39:16 -04:00
Ryan Petrello ba0e9ab775 copy extra_vars when relaunching ad-hoc commands
see: #1744
2017-03-21 09:45:30 -04:00
Ryan Petrello d0ae818685 show extra variables (if any) in the adhoc command results UI
see: #1744
2017-03-21 09:45:25 -04:00
Ryan Petrello 6a7743b274 fix a callback bug that causes a task_args leak between job events
see: #5802
2017-03-20 12:05:41 -04:00
Ryan Petrello 999a304c3b add no_log filtering for runner_item_on_xyz events
see: #5691
2017-03-17 09:54:46 -04:00
Ryan Petrello 75ba0208ad output logs to /logstash.conf in our default compose file 2017-03-16 13:20:33 -04:00
Ryan Petrello 98ad2684d3 when async log shipping fails, log an explanation 2017-03-15 17:23:44 -04:00
Ryan Petrello b3733d233e always use async http logging - even in celery workers
additionally, add a timeout to requests to avoid thread starvation

see: #5718
see: #5729
2017-03-15 13:17:01 -04:00
Ryan Petrello ba9ce56c35 add callback plugin tests to CI 2017-03-14 14:07:56 -04:00
Ryan Petrello 66c786b971 simplify memcache patching in our callback plugin tests 2017-03-14 14:07:53 -04:00
Ryan Petrello a5ea468bfe add a test to ensure that job events properly strip out environment vars
see: #5601
2017-03-14 12:56:44 -04:00
Ryan Petrello 5bbb824865 add tests for no_log filtering
see: #5691
2017-03-14 12:56:44 -04:00
Ryan Petrello 1005782ee9 add tests for our custom ansible callback plugin 2017-03-14 12:56:40 -04:00
Ryan Petrello 6af9638d81 job event data marked with _ansible_no_log should be censored
see: #5691
2017-03-10 15:47:39 -05:00
Ryan Petrello d4cd9d355f add ui support for specifying extra vars for ad-hoc commands
see: #1744
2017-03-10 15:29:42 -05:00
Ryan Petrello bad9670a0b Merge remote-tracking branch 'origin/release_3.1.2' into devel 2017-03-10 10:57:03 -05:00
Ryan Petrello f4d4c43d94 prohibit order_by= for sensitive fields
see: #5526
2017-03-10 08:49:59 -05:00
Ryan Petrello 9bc162cada start a CHANGELOG 2017-03-09 16:29:40 -05:00
Ryan Petrello 8c1de7f109 add an API endpoint for testing external log aggregrator connectivity
see: #5164
2017-03-09 13:19:56 -05:00
Ryan Petrello 9a6913f184 don't toggle the dropdown when removing items from multiselect fields
see: #5466
2017-03-08 17:37:16 -05:00
Ryan Petrello 255d6afb49 fix a traceback if tower doesn't have LOG_AGGREGATOR_TOWER_UUID set
see: #5527
2017-03-07 16:48:51 -05:00
Ryan Petrello ec52e18be8 allow is_system_auditor with AUTH_LDAP_USER_FLAGS_BY_GROUP
see: #4959
2017-03-07 16:48:51 -05:00
Ryan Petrello ae321536cd fix a regex that doesn't properly strip certain environment variables
see: #5601
2017-03-07 16:48:48 -05:00
Ryan Petrello ed5a01c0cf strip ansible_env from job event callback data
this often contains sensitive details, and the data is already filtered
and stored on the job itself

see: #5601
2017-03-07 16:48:48 -05:00
Ryan Petrello d97ff57cda prohibit API payloads that represent something other than a JSON object
The JSON serializer for our API uses ``json.loads``, which permits *any*
valid JSON (including bare integers, boolean values, etc).  Lots of our
code, however, assumes that inbound JSON content will be a dict.

see: #4756
2017-02-28 13:23:03 -05:00
Ryan Petrello 45819f6b9a requeue websocket messages that don't (yet) have an established user
There's a race between our `ws_connect` and `ws_receive` methods;
it's possible to fall into a scenario where we're handling a legitimate
message *before* django-channels is able to persist the `user_id` into
the channel session.  This results in a scenario where a user can open
a browser tab and never receive new websocket messages.  In this
scenario, we should just toss the message back into the queue and try
again later (up to a reasonable limit of retries).
2017-02-28 11:12:59 -05:00
Ryan Petrello e8ecc4226b use Unicode apostrophes - not single quotes - for French i18n strings
see: #5536
2017-02-27 12:23:07 -05:00
Ryan Petrello 30a3439aa9 pin appdirs==1.4.2
setuptools requires appdirs >= 1.4.0
https://github.com/pypa/setuptools/commit/ff371f18f0076bc63da05334f7e551c1cc29e10d#diff-2eeaed663bd0d25b7e608891384b7298
2017-02-27 12:06:35 -05:00
Ryan Petrello 5dfdef6e6b retain the default interstitial value for TOWER_URL_BASE
to support #5470, check for the presence of this special interstitial
value and replace it with the request hostname for settings OPTIONS
requests
2017-02-23 16:38:16 -05:00
Ryan Petrello b7e48c437d always refetch license config
see: #4931
2017-02-23 12:22:44 -05:00
Ryan Petrello 0c0f3b236f allow idempotent save when editing an inventory script
see: #5094
2017-02-23 11:34:04 -05:00
Ryan Petrello fc583ead21 allow AD_HOC_COMMANDS to be set to [] in the settings ui
see: #4975
2017-02-23 10:51:54 -05:00
Ryan Petrello 6b75ed3dcb move a tall tooltip's placement on the credentials UI to avoid cutoff
when browser text size is increased, this tooltip gets cut off at the
top of the browser window

see: #4979
2017-02-23 09:48:52 -05:00