Commit Graph
100 Commits
Author SHA1 Message Date
Ryan Petrello 1ac5bc5e2b remove angular-md5 license 2018-11-28 10:43:53 -05:00
Ryan Petrello a72f3d2f2f generate host_config_key using random UUIDs, not a time-based md5 hash 2018-11-28 10:43:45 -05:00
Ryan Petrello a810aaf319 fix isolated nodes in the dev environment 2018-11-28 09:54:39 -05:00
Ryan Petrello 4e45c3a66c mock an HTTP call to fix busted unit tests 2018-11-28 09:17:50 -05:00
Ryan Petrello b1bd87bcd2 raise contrast on a few key page elements to pass WCAG contrast checks 2018-11-27 10:20:24 -05:00
Ryan Petrello 32e7ddd43a only override django for FIPS in environments where Django is installed
isolated awx installs don't have this tooling, and so they don't need
this specific monkey-patch
2018-11-26 09:17:48 -05:00
Ryan Petrello b4f906ceb1 add dispatcher status to the sosreport 2018-11-20 12:12:02 -05:00
Ryan Petrello 9e2f4cff08 remove the x icon on the session timeout modal (it doesn't work) 2018-11-19 14:37:49 -05:00
Ryan Petrello e36335f68c only send activity stream create for registered unified jobs
see https://github.com/ansible/awx/issues/2733
2018-11-19 09:44:12 -05:00
Ryan Petrello 661cc896a9 add a user agent for requests to Insights 2018-11-16 16:25:08 -05:00
Ryan Petrello 37234ca66e prevent the dispatcher from using a nonsensical max_workers value 2018-11-16 10:16:39 -05:00
Ryan Petrello d3dc126d45 truncate user first/last name if it exceeds 30 chars on LDAP auth 2018-11-14 15:51:43 -05:00
Ryan Petrello 49f4ed10ca include M2M labels and credentials in Job creation activity stream 2018-11-14 10:36:01 -05:00
Ryan Petrello a3f9c0b012 warn about FIPS mode if the Django version changes 2018-11-13 15:04:36 -05:00
Ryan Petrello cceac8d907 support PKCS8-formatted keys to enable FIPS compliance
see: https://access.redhat.com/solutions/1519083
2018-11-12 16:21:57 -05:00
Ryan Petrello 10667fc855 fix an unbound variable
see: https://github.com/ansible/awx/issues/2642j
2018-11-08 15:14:47 -05:00
Ryan Petrello 001bd4ca59 resolve a few token revocation issues, and add tests 2018-11-08 08:15:24 -05:00
Ryan Petrello 28cd762dd7 properly validate cert data that happens to contain an END substring 2018-11-06 15:57:35 -05:00
Ryan Petrello 1523feee91 send activity stream changes as raw JSON, not a JSON-ified string
see: https://github.com/ansible/awx/issues/2005
2018-11-06 14:28:57 -05:00
Ryan Petrello 05156a5991 move from GEC_PEM_FILE_PATH to GCE_CREDENTIALS_FILE_PATH 2018-11-05 15:44:31 -05:00
Ryan Petrello a748a272fb Merge remote-tracking branch 'tower/release_3.3.1' into devel 2018-11-01 12:07:02 -04:00
Ryan Petrello 479448ff09 remove an old, unused migration file 2018-10-31 11:15:09 -04:00
Ryan Petrello a1012b365c pin docker-compose to a working version 2018-10-31 10:47:45 -04:00
Ryan Petrello 28512e042b remove DeprecatedAuthTokenMiddleware 2018-10-26 10:11:53 -04:00
Ryan Petrello c695ba2e10 fix flake8 2018-10-24 10:11:53 -04:00
Ryan Petrello 3b7336c570 drop old celery/djcelery tables we no longer need 2018-10-22 09:20:10 -04:00
Ryan Petrello 9316c9ea3e implement simple retries for wayward inventory deletes 2018-10-19 10:10:52 -04:00
Ryan Petrello 3be9113d6b fix a bug that breaks job cancel on single node jobs
1.  Install awx w/ a single node.
2.  Start a long-running job.
3.  Forcibly kill the `awx-manage run_dispatcher` process (e.g.,
    SIGKILL) and do not start it again.
4.  The job remains in running - without a second cluster to discover
    the job, it is never reaped.
5.  This PR allows you to cancel the job from the UI+API.
2018-10-19 09:10:33 -04:00
Ryan Petrello 0d29bbfdc6 make the dispatcher more fault-tolerant to prolonged database outages 2018-10-18 20:00:07 -04:00
Ryan Petrello 53ae05094e use the proper logger for the callback receiver 2018-10-17 10:56:29 -04:00
Ryan Petrello 79002ae563 upgrade to the latest kombu + celery 2018-10-16 16:14:58 -04:00
Ryan Petrello 6e4f3efc4b silence the noisy error that's printed w/ awx-manage check_migrations 2018-10-16 13:48:03 -04:00
Ryan Petrello 6ff1fe8548 allow users to specify BROKER_URL with passwords that contain : and @ 2018-10-16 11:56:57 -04:00
Ryan Petrello f3e73bbed8 don't call rabbitmqctl forget_cluster_node for isolated instances 2018-10-16 09:47:53 -04:00
Ryan Petrello 00cae104b3 remove over-eager survey choices validation
it looks like choices can also be a list and _maybe_ comma delimited;
clearly there's a lot of history here; let's verify and test what's _really_ supported and _then_ add any necessary validation
2018-10-15 16:40:17 -04:00
Ryan Petrello 720a634702 don't attempt to recover special QUIT messages in the worker pool
when `--reload` is sent to the dispatcher, it sends a special QUIT
message to each worker in the pool so that it will exit gracefully at
the next opportunity

when a worker process exits unexpectedly, the dispatcher attempts to
recover its queued messages and sends them to another worker in the
pool; in this scenario, we should _never_ re-enqueue these special
QUIT messages (because the process doesn't need to quit, it's already
gone)

To reproduce this race condition:

1.  Launch an adhoc that does `sleep 60`
2.  Run `awx-manage run_dispatcher --reload` to enqueue a `QUIT` message
    into the worker's queue
3.  Find the pid of the worker running the `sleep 60` and `SIGKILL` it.
4.  Observe that dispatcher attempts to requeue the `QUIT` message and
    logs a confusing error.
2018-10-15 12:17:52 -04:00
Ryan Petrello 9d81727d16 fix a typo on the JT add page that breaks the custom venv field 2018-10-15 11:19:58 -04:00
Ryan Petrello 011c8ae822 fix a typo on the JT add page that breaks the custom venv field 2018-10-15 11:04:31 -04:00
Ryan Petrello 867ff5da71 minor nit for https://github.com/ansible/tower/pull/3060 2018-10-12 16:17:14 -04:00
Ryan Petrello d4e3127fb4 properly handle AnsibleVaultEncryptedUnicode objects in the callback 2018-10-12 12:29:46 -04:00
Ryan Petrello ac80bc874a more shippable -> zuul cleanup 2018-10-12 11:50:29 -04:00
Ryan Petrello 517043e209 update to the latest stable 1.1 django-oauth-toolkit
see: https://github.com/jazzband/django-oauth-toolkit/pull/629
2018-10-12 10:21:57 -04:00
Ryan Petrello 1167361128 some minor README updates 2018-10-11 16:05:29 -04:00
Ryan Petrello 10c76e2337 update Azure inventory script to latest from Ansible
rebased version of https://github.com/ansible/awx/pull/2234
2018-10-11 11:47:55 -04:00
Ryan Petrello ff1e8cc356 replace celery task decorators with a kombu-based publisher
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:

- a new decorator, `awx.main.dispatch.task`, which can be used to
  decorate functions or classes so that they can be designated as
  "Tasks"
- support for fanout/broadcast tasks (at this point in time, only
  `conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
  `handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
  on demand
- minimal support for RPC, such as status checks and pool recycle/reload
2018-10-11 10:53:30 -04:00
Ryan Petrello da74f1d01f refactor and test the callback receiver as a base for a task dispatcher 2018-10-11 10:53:26 -04:00
Ryan Petrello f87a09c46a build swagger docs as part of CI 2018-10-10 10:27:54 -04:00
Ryan Petrello 29b90b700e minor docker-compose fix 2018-10-05 13:40:10 -04:00
Ryan Petrello 7752446067 fix busted notification tests 2018-10-05 10:18:27 -04:00
Ryan Petrello 16e89ed081 send test notifications after the transaction closes to avoid a race 2018-09-28 09:43:10 -04:00
Ryan Petrello 9de63832ce properly support deprecated Authorization: Token xyz 2018-09-24 15:16:09 -04:00
Ryan Petrello 23d4122574 properly support deprecated Authorization: Token xyz 2018-09-24 14:50:33 -04:00
Ryan Petrello 43aa0fc741 consolidate celery init signals to avoid an instance registration race 2018-09-07 09:29:53 -04:00
Ryan Petrello ec873dd28c fix LicenseForbids 401/402 precendence for other features 2018-08-31 11:23:11 -04:00
Ryan Petrello 1f8736ce1d workflow endpoints should return 401 on invalid credentials
if you have a license that doesn't allow use of workflows, invalid
credentials yielded an HTTP 402; this commit changes the precedence

see: https://github.com/ansible/tower/issues/2950
2018-08-31 10:57:27 -04:00
Ryan Petrello 34ceaf4551 fix a subtle bug in awx.main.access.OAuth2ApplicationAccess.can_read
see: https://github.com/ansible/tower/issues/2952
2018-08-30 14:21:03 -04:00
Ryan Petrello 24f8cb49b5 don't access the database in our custom route_for_task
If database connectivity is lost/interrupted in this block of celery
internals, beat is *not* smart enough to recover, and it gets stuck in
an endless fail loop.  We don't _need_ to talk to the database here
anyways; just use settings.CLUSTER_HOST_ID to get what we need.

see: https://github.com/ansible/tower/issues/2957
2018-08-30 11:40:43 -04:00
Ryan Petrello 9ae41dc3ba write custom inventory scripts to AWX_PRIVATE_DATA_DIR
this makes it so that custom inventory scripts can access credential
files laid down in `/tmp/awx_N_<xyz>`

see: https://github.com/ansible/tower/issues/2951
2018-08-30 08:32:27 -04:00
Ryan Petrello 6eb406ac39 make awx-manage instance_list easier to read and more useful 2018-08-29 15:00:05 -04:00
Ryan Petrello c69e41b261 more UI i18n
see: https://github.com/ansible/tower/issues/1383
2018-08-28 12:47:18 -04:00
Ryan Petrello 2acc488adf properly sanitize long event keys
see: https://github.com/ansible/tower/issues/1775
2018-08-27 13:40:26 -04:00
Ryan Petrello 4561fd7270 fix an auth-related typo in a docstring 2018-08-24 11:56:11 -04:00
Ryan Petrello 5561eb30f7 emulate /api/v2/authtoken/ to help customers transition to OAuth2.0 2018-08-24 11:05:41 -04:00
Ryan Petrello d564a268fd sprinkle in more i18n translation for the workflow editor
see: https://github.com/ansible/tower/issues/775
2018-08-23 14:07:15 -04:00
Ryan Petrello a271837007 set the session cookie expiry *properly* on each request
see: https://github.com/ansible/tower/issues/2907
2018-08-22 15:26:03 -04:00
Ryan Petrello 4bc63cc37e fix a bug that broke bot avatars for Slack notifications
when a *color* is specified for a Slack notification template, we use
the *web* not the RTM API; when you use a bot with the web API, you have
to specify the `as_user=True` argument to have the message use the bot's
name and avatar

see: https://github.com/ansible/tower/issues/2883
see: https://github.com/ansible/awx/issues/2187
2018-08-22 10:57:13 -04:00
Ryan Petrello 67d1267d98 enforce 0 <= Instance.capacity_adjustment
see: https://github.com/ansible/tower/issues/2839
2018-08-21 15:34:19 -04:00
Ryan Petrello 4eeb62766e apply sensitive field filtering to /api/v2/hosts/?host_filter
see: https://github.com/ansible/tower/issues/2874
see: https://github.com/ansible/tower/issues/2889
2018-08-21 08:17:14 -04:00
Ryan Petrello a36b0061fa fix failing unit tests 2018-08-20 19:57:28 -04:00
Ryan Petrello 30fbeb43bb fail CI if the change includes model changes that are missing migrations 2018-08-16 17:43:32 -04:00
Ryan Petrello 14043f792a fix up remaining Django migrations
these don't really change anything in the schema; they just look like
Django ORM idiosyncrancies that `makemigrations` needs to be happy

see: https://github.com/ansible/tower/issues/2203
2018-08-16 13:59:06 -04:00
Ryan Petrello 4c7c8b6db3 make inventory updates considerably less verbose by default
see: https://github.com/ansible/tower/issues/2858
2018-08-15 16:04:15 -04:00
Ryan Petrello 87adfe5889 close DB and cache sockets _immediately_ before we fork callback workers 2018-08-15 15:10:08 -04:00
Ryan Petrello 6ba1b170d2 provide friendlier help messages if you set up custom venvs wrong 2018-08-15 14:11:48 -04:00
Ryan Petrello 5f6907ba83 show a better error when a custom venv doesn't exist on an isolated node
see: https://github.com/ansible/tower/issues/2852
2018-08-15 13:31:25 -04:00
Ryan Petrello cffa324762 show custom_virtualenvs at /api/v2/config if you have Project/Org access
see: https://github.com/ansible/tower/issues/2843
2018-08-15 13:12:12 -04:00
Ryan Petrello 3b997cdd3a remove extraneous OPTIONS content re: the new named URL feature 2018-08-15 10:22:42 -04:00
Ryan Petrello 79a29ebcc8 add indexes for new event types
not sure why this didn't happen in the original migration that was
generated - may be related to differences in behavior across Django
versions?
2018-08-14 16:57:13 -04:00
Ryan Petrello 8723508225 fix failing tests related to 2c4f7911a6 2018-08-10 12:45:35 -04:00
Ryan Petrello 18f6f68540 Merge remote-tracking branch 'tower/release_3.3.0' into devel 2018-08-10 11:54:34 -04:00
Ryan Petrello d95f1283b5 redact sensitive URLs from stdout at /api/v2/project_updates/N/events/
see: https://github.com/ansible/tower/issues/2805
2018-08-09 17:31:09 -04:00
Ryan Petrello 91c92d57e3 fix a bug in isolated job launches when there are no controllers online
see: https://github.com/ansible/tower/issues/2782
2018-08-07 17:27:23 -04:00
Ryan Petrello 2d4fbffb91 set the correct X-Forwarded-Port header to fix SAML auth
see: https://github.com/ansible/tower/issues/2314
2018-08-07 10:07:06 -04:00
Ryan Petrello 87d6940158 pin django-oauth-toolkit to our fork w/ a security fix
see: https://github.com/jazzband/django-oauth-toolkit/issues/625
2018-08-06 14:54:46 -04:00
Ryan Petrello ec735b7b47 check oauth_scopes in _every_ view
see: https://github.com/ansible/tower/issues/2759
2018-08-06 11:05:59 -04:00
Ryan Petrello 7df027f826 fix to handle nonexistent system_id 2018-08-03 16:08:40 -04:00
Ryan Petrello a3566b4270 fix up a minor OAuth2 token serializer nit
see: https://github.com/ansible/tower/issues/2751
2018-08-03 14:50:37 -04:00
Ryan Petrello 27f4e487ec simpilfy LDAP GROUP_SEARCH misconfiguration failure handling 2018-08-02 10:11:18 -04:00
Ryan Petrello 2102c1ef0a fix a bug in multi-LDAP authentication
see: https://github.com/ansible/tower/issues/2738
2018-08-01 17:55:13 -04:00
Ryan Petrello 0a400ca339 sprinkle some more i18n love for the credential form
see: https://github.com/ansible/tower/issues/2731
2018-07-31 22:13:17 -04:00
Ryan Petrello 5610cbe735 prevent host deletion when the related inventory is pending deletion
see: https://github.com/ansible/tower/issues/2690
2018-07-31 15:54:46 -04:00
Ryan Petrello 0aaa3807a9 allow access to JT labels if you have read access to the JT
see: https://github.com/ansible/tower/issues/2180
2018-07-31 15:13:24 -04:00
Ryan Petrello 8e3b72e1e5 more network UI cleanup (whoops) 2018-07-31 14:53:11 -04:00
Ryan Petrello 184db58fa3 set workflow status to failed if we find a node w/ a missing UJT
see: https://github.com/ansible/tower/issues/2576
2018-07-30 16:37:24 -04:00
Ryan Petrello b99f211c7e add some additional validation to JT.credentials
see: https://github.com/ansible/tower/issues/2612
2018-07-30 14:08:27 -04:00
Ryan Petrello aee986d53c fail immediately if an isolated host is unreachable at launch time
see: https://github.com/ansible/tower/issues/2643
2018-07-30 12:40:26 -04:00
Ryan Petrello 1a1b9e9138 redact the license key from settings.LICENSE activity stream entries
see: https://github.com/ansible/tower/issues/2671
2018-07-30 11:43:42 -04:00
Ryan Petrello 6f5259d017 remove the network UI 2018-07-30 11:03:53 -04:00
Ryan Petrello 78eb0444ae fix a bug that causes orphaned auth_user rows when LDAP is misconfigured
see: https://github.com/ansible/tower/issues/2465
2018-07-30 09:59:20 -04:00
Ryan Petrello 3cdd0a94bd simplify dynamic queue binding
we recently made a change so that instances no longer bind to
instance-group specific queues, but now instead they each bind to
a direct queue for their specific hostname
(https://github.com/ansible/tower/pull/1922)

Because of this, we shouldn't *need* to reconfigure the queue binds at
runtime anymore when group membership changes. Under our new model,
every celeryd listens on a queue named after its hostname; when the
scheduler finds a task to run, it picks an Instance in the target
Instance Group and sends the task to the queue for that Instance's
hostname.
2018-07-28 00:48:37 -04:00