Commit Graph
100 Commits
Author SHA1 Message Date
chris meyers c3100afd0e fixed isolated instance query
* Was considering an isolated instance: any instance that has at least 1
group with no controller. This is technically correct since an iso node
can not be a part of a non-iso group.
* The query is now more robust and considers a node an iso node if ALL
groups that a node belong to ALL have a controller.
* Also added better debugging for the special tower instance group
* Added a check for the existance of the special tower group so that
logs are less "messy" during the install process.
2018-04-03 13:50:57 -04:00
chris meyers e5dcfda1fe append registered hostname to policy list
* The Instance Group list of instances was getting over-written with
every call to the register_instance management command.
* This changeset appends --hostnames to the Instance Group policy list.
2018-04-02 16:26:59 -04:00
chris meyers 557637afcb better unicode handling 2018-04-02 12:05:16 -04:00
chris meyers 7e7ff8137d more gracefully account for undefined stdout
* It's possible to have an exception raised in BaseTask.run() before the
stdout handler gets defined. This is problematic when the exception
handler tries to access that undefined var .. causing another exception.
Note that the second exception is caught also but it's not desirable to
lose the first exception.
* This fix checks to see if the stdout handler var is defined before
calling it's methods. Thus, we retain the original error message.
2018-04-02 11:41:30 -04:00
chris meyers 838b723c73 add all instances to special tower instance group
* All instances except isolated instances
* Also, prevent any tower attributes from being modified via the API
2018-03-29 16:47:52 -04:00
chris meyers bf6412ea06 fix api browser endpoint name
* Endpoint exposes all jobs associated with an Instance. This is what we
want. Align the endpoint description with this behavior by removing the
word running.
2018-03-28 16:24:19 -04:00
chris meyers 8438331563 make jobs_running more rich in OPTIONS
* Expose jobs_running as an IntegerField
2018-03-28 16:01:24 -04:00
chris meyers eef6f7ecb0 delay looking up settings SYSTEM_UUID 2018-03-28 09:54:51 -04:00
chris meyers dc46a732bc fix ldap group type params label
* copy pasted ldap group type label and did no previously update. This
updates.
2018-03-28 09:12:06 -04:00
chris meyers 7ce8907b7b reregister node when they come back online
* Nodes are marked offline, then deleted; given enough time. Nodes can
come back for various reasions (i.e. netsplit). When they come back,
have them recreate the node Instance if AWX_AUTO_DEPROVISION_INSTANCES
is True. Otherwise, do nothing. The do nothing case will show up in the
logs as celery job tracebacks as they fail to be self aware.
2018-03-27 14:30:47 -04:00
chris meyers 1503e0505e call celery method with celery context
* Tracebacks are more informative this way.
2018-03-26 12:02:42 -04:00
chris meyers b9b8502738 introspect ldap group types for param validation
* Instead of keeping a hard-coded mapping of valid args for each ldap
group type; introspect the subclass to determine valid/invalid fields
2018-03-26 11:40:49 -04:00
chris meyers cb7e17885f remove uneeded auth ldap settings
* I had thought that setting the settings was required. But carefully
selected defaults for the settings is the correct way to deal with
errors I was seeing early in developing this feature.
2018-03-23 15:41:08 -04:00
chris meyers 305ef6fa7e do not allow tower group delete or name change
* DO allow policy changes and other attribute changes
2018-03-22 08:05:06 -04:00
chris meyers e0803b9f08 use non-unicode queue names
* Use unicode InstanceGroup and queue names up until the point we
actually create the queue
* kombu add_consumers returns a dict with a value that contians the
passed in queue name. Trouble is, the returned dict value is a string
and not a unicode string and this results in an error.
2018-03-21 16:50:07 -04:00
chris meyers 91bfed3d50 handle instance group names unicode 2018-03-21 13:41:48 -04:00
chris meyers 1c578cdd74 validate group type params 2018-03-21 09:13:08 -04:00
chris meyers 17795f82e8 more parameters 2018-03-21 09:13:08 -04:00
chris meyers e3c362956d add ldap group type like posixGroupType
* Adds pattern to easy add django-auth-ldap group types classes and to
pass parameters via AUTH_LDAP_GROUP_TYPE_PARAMS
* Adds new group type PosixUIDGroupType that accepts the attribute,
ldap_group_user_attr, on which to search for the user(s) in the group.
2018-03-21 09:13:08 -04:00
chris meyers 1f7506e982 prevent tower group delete and update
* related to https://github.com/ansible/ansible-tower/issues/7931
* The Tower Instance group is special. It should always exist, so
prevent any delete to it.
* Only allow super users to associate/disassociate instances the 'tower'
instance group.
* Do not allow fields of tower instance group to be changed.
2018-03-15 15:23:06 -04:00
chris meyers 5d5d8152c5 prevent instance group delete if running jobs
* related to https://github.com/ansible/ansible-tower/issues/7936
2018-03-15 14:25:49 -04:00
chris meyers 2ea0b31e2b handle unicode things in task logger
Related to https://github.com/ansible/ansible-tower/issues/7957

* Problem presented itself as Instances falling out of Instance Groups.
This was due to the cluster membership policy decider erroring out on a
logger message with unicode.
* Fixed up potential other unicode logger unicode issues in tasks.py
2018-03-15 14:04:39 -04:00
chris meyers a4859a929c autoscale celery up to 50 workers 2018-03-12 15:36:15 -04:00
chris meyers e2ed1542e6 more celery rollback
* Setting reload code calls a celery 4.x method signature. This changes
it back to a 3.x safe call.
2018-03-09 09:27:09 -05:00
chris meyers ccfb6d64bf handle 404 returned by resolve()
* related to https://github.com/ansible/ansible-tower/issues/7926
* if 404 on url in migration loading middelware, do NOT short circuit
middleware. Simply call the normal middlware code path in this case.
2018-03-05 14:34:53 -05:00
chris meyers 36d59651af inherit rather than monkey patch
* Enable migration in progress page in ALL environments
2018-03-02 12:37:48 -05:00
chris meyers 746a2c1eea short-circuit middleware if migration loading url
* Had to monkey patch django middleware logic.
* Left checks to tell coders to use new middleware behavior in favor of
monkey patching.
2018-03-02 11:21:26 -05:00
chris meyers 17de084d04 perform the min needed DB ops to offline a node
* Don't do an extra save to the DB that could conflict with another
heartbeat when it isn't needed since we will be deleting the node
anyway.
2018-03-02 07:57:59 -05:00
chris meyers b69315f2eb fix up the config map watcher script
* invoke main() in config watcher script
* correctly call hash update by passing the filename
2018-03-01 17:06:07 -05:00
chris meyers a3a618d733 call node init procedures as early as possible
* invoke the first heartbeat as early as possible. Results in a much
better user experience where when a user scales up an awx node, the node
appears with capacity earlier.
2018-03-01 17:05:58 -05:00
chris meyers 5c647c2a0d call node init procedures as early as possible
* invoke the first heartbeat as early as possible. Results in a much
better user experience where when a user scales up an awx node, the node
appears with capacity earlier.
2018-03-01 11:24:45 -05:00
chris meyers e94bd128b8 reap all nodes that havn't checked in
* Before this change we would exclude the reaping of new nodes. With
this change, new nodes will be considered for reaping just like old
nodes.
2018-03-01 11:21:54 -05:00
chris meyers 9436e8ae25 dynamically set rabbitmq cookie 2018-03-01 09:23:45 -05:00
chris meyers 6606a29f57 celery 4.x -> 3.x change route config name 2018-02-27 14:13:05 -05:00
chris meyers 148baf7674 add explicit awx_celery container version 2018-02-27 11:37:10 -05:00
chris meyers 5918fa5573 remove () from postgres port value
* awx task container uses postgres port to wait for postgres to become
available before the container init continues. The () are problematic
and are removed.
* () was originally added to fix an openshift issues. That error does
NOT occur with this fix.
2018-02-27 11:36:55 -05:00
chris meyers e4470aa4cf remove uneeded celery configs
* Celery routes and queues are set and defined at runtime. Thus, a
static definition of routes and queues is not needed.
2018-02-27 11:36:55 -05:00
chris meyers fe05b4c0d5 use celery 3.x BROKER_URL
* Celery 4.x specifies the broker via CELERY_BROKER_URL. Since we are
now on 3.x, use 3.x way of specifying the broker via BROKER_URL
2018-02-27 11:36:55 -05:00
Chris Meyers aaf87c0c04 xss test for per-host recent jobs popup 2018-02-06 10:37:00 -05:00
Chris Meyers 0e97dc4b84 Beat and celery clustering fixes
* use embedded beat rather than standalone
* dynamically set celeryd hostname at runtime
* add embeded beat flag to celery startup
* Embedded beat mode routes will piggyback off of celery worker setup
signal
2018-02-01 16:47:33 -05:00
Chris Meyers c9ff3e99b8 celeryd attach to queues dynamically
* Based on the tower topology (Instance and InstanceGroup
relationships), have celery dyamically listen to queues on boot
* Add celery task capable of "refreshing" what queues each celeryd
worker listens to. This will be used to support changes in the topology.
* Cleaned up some celery task definitions.
* Converged wrongly targeted job launch/finish messages to 'tower'
queue, rather than a 1-off queue.
* Dynamically route celery tasks destined for the local node
* separate beat process

add support for separate beat process
2018-02-01 16:37:33 -05:00
Chris Meyers 290a296f9f add xss test for jobs schedules
* Test for tooltip regression on job schedules list entries
2018-02-01 10:55:13 -05:00
Chris Meyers 0402064c0f expose ansible_facts_modified 2018-01-17 10:28:34 -05:00
Chris Meyers e33265e12c add job_id to fact cache log output 2018-01-17 10:19:27 -05:00
Chris Meyers e49dfd6ee2 only run saml pipeline if saml social auth
* Do not trigger saml social auth pipeline methods if the user logging
in was not created by the saml social auth backend.
2018-01-11 16:20:49 -05:00
Chris Meyers 2ed97aeb0c implement multiple ldap servers 2018-01-11 09:03:14 -05:00
Chris Meyers de02138dfd spelling is hard 2018-01-10 09:26:11 -05:00
Chris Meyers 0a9d3d47b9 more efficiently determine saml team mapping 2018-01-09 12:16:07 -05:00
Chris Meyers 664bdec57f add documentation 2018-01-05 14:43:33 -05:00
Chris Meyers 9d58b15135 allow for saml attributes to define team and org
related to https://github.com/ansible/awx/issues/217

* Adds a configure tower in tower setting for users to configure a saml
attribute that tower will use to put users into teams and orgs.
2018-01-04 15:35:11 -05:00
Chris Meyers a0038276a4 do not use a custom task exception
* Celery + json pickling do not handle custom Exceptions (and may never
do so). Mentioning of, if handling custom Exceptions then the code would
be susceptible to same arbitrary code execution that python pickle is
vulnerable to.
* So don't use custom Exceptions.
2018-01-04 15:30:52 -05:00
Chris Meyers f0ff6ecb0a handle_work_error signature to work
* celery error callback signature isn't well defined. Thus, our error
callback signature is made to handle just about any call signature and
depend on only 1 attribute, id, existing.

See https://github.com/celery/celery/issues/3709
2018-01-04 15:23:13 -05:00
Chris Meyers 032318494b added tests for new settings field type 2017-11-29 11:52:00 -05:00
Chris Meyers 383c3cfe3e add more saml fields 2017-11-28 13:49:35 -05:00
Chris Meyers 6c89935521 correctly cascade job cancel
* Check the reason for a dependent project update failure. If it's
because of a cancel, then let the normal cancel mechanisms update the
jobs status and explanation. Do not update the dependent job's status
for a project update that was canceled, in the run code.
2017-11-27 12:34:55 -05:00
Chris Meyers 98f2d936d9 allow support for saml + 2-factor
* python-social-auth has SOCIAL_AUTH_SAML_SECURITY_CONFIG, which is
forwarded to python-saml settings configuration. This commit exposes
SOCIAL_AUTH_SAML_SECURITY_CONFIG to configure tower in tower to allow
users to set requestedAuthnContext, which will disable the requesting of
password type auth from the idp. Thus, it's up to the idp to choose
which auth to use (i.e. 2-factor).
2017-11-17 09:25:50 -05:00
Chris Meyers c4e6fc23fc add docs for job events
* Focus on the ordering of Job Event creation. Important to know when
thinking through different Ansible execution strategies.
2017-11-16 12:48:48 -05:00
Chris Meyers 9288b53015 cascade cancel proj update when job canceled
* Implicit project update, launch_type='sync', get "associated" with a
job via project_update. When a job is canceled, so should this implicit
project update. This change enforces that logic.
2017-11-15 11:17:52 -05:00
Chris Meyers 4e2cf62e89 fixes ansible callback import json warning
[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin
(<awx_display_callback.module.AWXDefaultCallbackModule object at
0x47f6090>):
'module' object has no attribute 'dumps'

The above error is thrown by ansible if callback plugins don't respect
the same import precedence configuration as Ansible. ansible callback/*
dir includes a json.py file. This is imported by ansible
callback/__init__.py when a callback plugin implementation imports from
Ansible callback base without setting the correct import precedence.
2017-10-16 10:29:41 -04:00
Chris Meyers 62987196cb add speedup support to event replay and stats
* add tests
* add verbosity support
2017-10-13 09:25:18 -04:00
Chris Meyers e19a57c50a add job event replay awx-manage command
* awx-manage replay_job_event --job_id <id>
2017-10-12 09:40:30 -04:00
Chris Meyers cad8710ac7 fix scan job migration unicode issue 2017-10-03 11:31:43 -04:00
Chris Meyers a7ec5876ce add awx install type to issue template
* Recreation of user-issues increasing depends on the chosen install
path used. Thus, we need this information when recreating the issue
locally. Ask the user to include this information in the issue template.
2017-09-28 14:37:26 -04:00
Chris Meyers 26d393e5c2 2-level memoize
* Allows for invalidating an entire function from the memoizer
2017-09-21 15:34:51 -04:00
Chris Meyers 062ff7153d resurrect cchurch's license feature caching 2017-09-20 14:57:24 -04:00
Chris Meyers c265ed2722 Revert "use request response cache for license checks"
This reverts commit 93acae9367023fec153aa7980f2edd9f96822ba2.
2017-09-20 13:47:44 -04:00
Chris Meyers 0b40331107 azure migrations
* delete azure credentials
* delete azure inventory sources
2017-09-18 10:35:42 -04:00
Chris Meyers a08a158672 remove azure 2017-09-18 10:35:32 -04:00
Chris Meyers 1ce3c7937b use request response cache for license checks 2017-09-13 13:34:39 -04:00
Chris Meyers 45813bea16 do not re-create django session on every request
* The django middleware call stack behavior is changed by DRF. As a
result, during the process_request in sso/middlware.py request.user
is not set as you would expect it to be set from the middleware
django.contrib.auth.middleware.AuthenticationMiddleware
2017-08-21 11:19:52 -04:00
Chris Meyers 1df47a2ddd account for waiting tasks not having execution_nodes yet
* Reap running tasks on non-netsplit nodes
* Reap running tasks on known to be offline nodes
* Reap waiting tasks with no celery id anywhere if waiting >= 60 seconds
2017-08-16 13:18:25 -04:00
Chris Meyers d615e2e9ff do not include workflow jobs for reaping
* Workflow jobs are virtual jobs that don't actually run. Thus they
won't have a celery id and aren't candidates for the generic reaping.
* Better error logging when Instance not found in reaping code.
2017-08-16 08:51:27 -04:00
Chris Meyers de82707581 fail all jobs on an offline node 2017-08-15 14:46:28 -04:00
Chris Meyers 90ea9b8bde presume 401 from insights means invalid credential
* Tell the user about the invalid credentials
2017-08-15 08:31:52 -04:00
Chris Meyers 9314db646b only reap non-netsplit nodes 2017-08-15 08:30:14 -04:00
Chris Meyers 5ad29c9081 pass through insights rule.ansible 2017-08-14 15:21:13 -04:00
Chris Meyers 80a90beae8 remove dead code 2017-08-14 10:37:57 -04:00
Chris Meyers 7e86ef9a46 remove scan job choice from launch endpoint 2017-08-10 08:52:54 -04:00
Chris Meyers c0b9c76a41 stringify uuid so it can be serialized 2017-08-10 07:32:03 -04:00
Chris Meyers c3f24d878d reap waiting processes if crash 2017-08-09 14:01:33 -04:00
Chris Meyers 4e2f075fa6 better scm inv restriction error message 2017-08-08 10:42:38 -04:00
Chris Meyers 95c12ac1ab enforce muutally exclusivity on update_on_project_update and update_on_launch for scm inventory 2017-08-07 10:50:36 -04:00
Chris Meyers 4b830e1176 fix fact cache timeout env var key name 2017-08-04 09:39:18 -04:00
Chris Meyers 3c89aeaba4 fix single update_on_project_update per-inventory 2017-08-03 18:54:05 -04:00
Chris Meyers 729d86c485 standardize on 502 for insights errors 2017-08-03 16:12:09 -04:00
Chris Meyers 9b12d88e8c lay the foundation for checking license on upgrade 2017-08-02 12:24:58 -04:00
Chris Meyers 6cc5f14e16 remove conditional inventory sources POST
* Move logic from validator down to model
* Allow only 1 inventory source of type scm with
update_on_project_update set to True; for each inventory
2017-08-02 11:49:10 -04:00
Chris Meyers 3dea02ceac use insights v3 api 2017-07-31 11:04:15 -04:00
Chris Meyers ed8db2419e make specific saml fields required 2017-07-31 09:28:52 -04:00
Chris Meyers f20f4f40a0 trim insights content to only what the UI needs 2017-07-27 13:58:54 -04:00
Chris Meyers 1d0e125aad make password survey field behave like text field
related to https://github.com/ansible/ansible-tower/issues/6083
2017-07-27 12:57:31 -04:00
Chris Meyers ef055110f5 handle unicode host names in fact cache 2017-07-26 12:41:13 -04:00
Chris Meyers 89a39f4bed add scm to cloud sources so computed fields works 2017-07-25 14:33:07 -04:00
Chris Meyers ec212db83b fix api docs sample host_filter queries 2017-07-24 13:48:20 -04:00
Chris Meyers eaddafc920 allow smart inventory to be created
related to https://github.com/ansible/ansible-tower/issues/7261
2017-07-24 13:43:15 -04:00
Chris Meyers 525490a9a0 all dependent jobs must finish before starting job
related to https://github.com/ansible/ansible-tower/issues/6570 https://github.com/ansible/ansible-tower/issues/6489

* Ensure that all jobs dependent on a job have finished (i.e. error,
success, failed) before starting the dependent job.
* Fixes a bug where a smaller set of dependent jobs to fail upon a
update_on_launch job failing is chosen.
* This fixes the bug of jobs starting before dependent Project Updates
created via update on launch. This also fixes similar bugs associated
with inventory updates.
2017-07-24 12:31:35 -04:00
Chris Meyers 41ae7b5120 validate schedule extra_data
related to #4779

* Ensure days is a non-negative integer
* Ensure that extra_data is a json payload
2017-07-21 10:46:44 -04:00
Chris Meyers c11b870084 remove job_type scan from options
* Need to keep the scan job in job_type for migrations so just remove it
from the constructed OPTIONS
2017-07-21 10:45:17 -04:00
Chris Meyers 22e4957698 remove duplicate hosts from host_filter 2017-07-19 17:09:39 -04:00
Chris Meyers c2204e5f27 disallow insights creds on smart inventory
related to #7083
2017-07-19 17:09:00 -04:00