Commit Graph
100 Commits
Author SHA1 Message Date
Chris Meyers 1df47a2ddd account for waiting tasks not having execution_nodes yet
* Reap running tasks on non-netsplit nodes
* Reap running tasks on known to be offline nodes
* Reap waiting tasks with no celery id anywhere if waiting >= 60 seconds
2017-08-16 13:18:25 -04:00
Chris Meyers d615e2e9ff do not include workflow jobs for reaping
* Workflow jobs are virtual jobs that don't actually run. Thus they
won't have a celery id and aren't candidates for the generic reaping.
* Better error logging when Instance not found in reaping code.
2017-08-16 08:51:27 -04:00
Chris Meyers de82707581 fail all jobs on an offline node 2017-08-15 14:46:28 -04:00
Chris Meyers 90ea9b8bde presume 401 from insights means invalid credential
* Tell the user about the invalid credentials
2017-08-15 08:31:52 -04:00
Chris Meyers 9314db646b only reap non-netsplit nodes 2017-08-15 08:30:14 -04:00
Chris Meyers 5ad29c9081 pass through insights rule.ansible 2017-08-14 15:21:13 -04:00
Chris Meyers 80a90beae8 remove dead code 2017-08-14 10:37:57 -04:00
Chris Meyers 7e86ef9a46 remove scan job choice from launch endpoint 2017-08-10 08:52:54 -04:00
Chris Meyers c0b9c76a41 stringify uuid so it can be serialized 2017-08-10 07:32:03 -04:00
Chris Meyers c3f24d878d reap waiting processes if crash 2017-08-09 14:01:33 -04:00
Chris Meyers 4e2f075fa6 better scm inv restriction error message 2017-08-08 10:42:38 -04:00
Chris Meyers 95c12ac1ab enforce muutally exclusivity on update_on_project_update and update_on_launch for scm inventory 2017-08-07 10:50:36 -04:00
Chris Meyers 4b830e1176 fix fact cache timeout env var key name 2017-08-04 09:39:18 -04:00
Chris Meyers 3c89aeaba4 fix single update_on_project_update per-inventory 2017-08-03 18:54:05 -04:00
Chris Meyers 729d86c485 standardize on 502 for insights errors 2017-08-03 16:12:09 -04:00
Chris Meyers 9b12d88e8c lay the foundation for checking license on upgrade 2017-08-02 12:24:58 -04:00
Chris Meyers 6cc5f14e16 remove conditional inventory sources POST
* Move logic from validator down to model
* Allow only 1 inventory source of type scm with
update_on_project_update set to True; for each inventory
2017-08-02 11:49:10 -04:00
Chris Meyers 3dea02ceac use insights v3 api 2017-07-31 11:04:15 -04:00
Chris Meyers ed8db2419e make specific saml fields required 2017-07-31 09:28:52 -04:00
Chris Meyers f20f4f40a0 trim insights content to only what the UI needs 2017-07-27 13:58:54 -04:00
Chris Meyers 1d0e125aad make password survey field behave like text field
related to https://github.com/ansible/ansible-tower/issues/6083
2017-07-27 12:57:31 -04:00
Chris Meyers ef055110f5 handle unicode host names in fact cache 2017-07-26 12:41:13 -04:00
Chris Meyers 89a39f4bed add scm to cloud sources so computed fields works 2017-07-25 14:33:07 -04:00
Chris Meyers ec212db83b fix api docs sample host_filter queries 2017-07-24 13:48:20 -04:00
Chris Meyers eaddafc920 allow smart inventory to be created
related to https://github.com/ansible/ansible-tower/issues/7261
2017-07-24 13:43:15 -04:00
Chris Meyers 525490a9a0 all dependent jobs must finish before starting job
related to https://github.com/ansible/ansible-tower/issues/6570 https://github.com/ansible/ansible-tower/issues/6489

* Ensure that all jobs dependent on a job have finished (i.e. error,
success, failed) before starting the dependent job.
* Fixes a bug where a smaller set of dependent jobs to fail upon a
update_on_launch job failing is chosen.
* This fixes the bug of jobs starting before dependent Project Updates
created via update on launch. This also fixes similar bugs associated
with inventory updates.
2017-07-24 12:31:35 -04:00
Chris Meyers 41ae7b5120 validate schedule extra_data
related to #4779

* Ensure days is a non-negative integer
* Ensure that extra_data is a json payload
2017-07-21 10:46:44 -04:00
Chris Meyers c11b870084 remove job_type scan from options
* Need to keep the scan job in job_type for migrations so just remove it
from the constructed OPTIONS
2017-07-21 10:45:17 -04:00
Chris Meyers 22e4957698 remove duplicate hosts from host_filter 2017-07-19 17:09:39 -04:00
Chris Meyers c2204e5f27 disallow insights creds on smart inventory
related to #7083
2017-07-19 17:09:00 -04:00
Chris Meyers 802f722e95 do not accept array as extra_vars
related to #5645

* Since we pass extra_vars to ansible as -e; we require that the top
most data-structure be a key-value. NOT an array.
2017-07-19 17:07:50 -04:00
Chris Meyers 815732bcba add API docs for host_filter 2017-07-19 10:18:13 -04:00
Chris Meyers 352e83aa84 fix diff_mode migration error 2017-07-13 07:54:58 -04:00
Chris Meyers 6511d2968e use newest aws env var names for creds 2017-07-12 15:27:18 -04:00
Chris Meyers d3e4be66dd disallow new survey spec with password default $encrypted$ 2017-07-12 15:21:55 -04:00
Chris Meyers 376170124e require that insights projects have insights cred 2017-07-12 12:52:26 -04:00
Chris Meyers 6652a960b7 make insights playbook fetching an Ansible module
* Gone are the days of playbook abuse.
* Our project update can now call a single Insight Ansible module to
loop over the set of maintenance plans to download and save playbooks
for use by job templates.
2017-07-11 16:28:01 -04:00
Chris Meyers c34a89714c rework hosts without insights creds error message 2017-07-11 14:25:12 -04:00
Chris Meyers e1a24d9849 structure scan modules as dicts not arrays 2017-07-11 14:14:48 -04:00
Chris Meyers f119b94134 correctly create jobs in migrations
* Do not import Project. Instead, use get_model() and ensure the
polymorphic ctype is set correctly.
* Point migrated job templates at the new fact scan playbook
2017-07-11 13:44:08 -04:00
Chris Meyers 7fda3c0658 back out thought to cause deadlock scenario 2017-07-10 10:43:29 -04:00
Chris Meyers aeb7119796 fix 2 data source inconcistency with failing tasks
* Do not "trust" the list of celery ids for database entries that were
modified after the list of celery ids was gotten.
* err on the side of caution and just let the next heartbeat celery
killer try killing the task if it needs to be reaped.
2017-07-10 10:43:01 -04:00
Chris Meyers 5b9a0b504a celery task fail check now uses pglock
* Align locking used by celery task cleaner upper with regular task manager.
* Uses pglock/advisory lock instead of abusing Instance table lock.
2017-07-10 10:41:54 -04:00
Chris Meyers 7a795b8681 fix job launch deadlock
* This both fixes the deadlock problem and a logic problem. We shouldn't
set the job's job_template current_job to pending jobs.
2017-07-10 10:34:36 -04:00
Chris Meyers df572d1477 migrations do not call overwitten save() methods
related to #6941

* We don't want our special logic to trigger anyway.
2017-07-07 16:47:07 -04:00
Chris Meyers e04c4e30d8 update tests to fix the removal of the word Tower 2017-07-06 11:22:16 -04:00
Chris Meyers 0a25cc2d19 back out parent jt status update short circuit 2017-07-06 10:48:28 -04:00
Chris Meyers 1331865749 take into account memcached key restrictions
* Keys can't contain spaces or control characters
2017-07-05 15:39:04 -04:00
Chris Meyers 0d3fc3389d flake8 fix 2017-07-05 13:37:32 -04:00
Chris Meyers e226b0ab37 noop pglock for unit tests 2017-07-05 13:34:07 -04:00
Chris Meyers f3f9782c0b fix 2 data source inconcistency with failing tasks
* Do not "trust" the list of celery ids for database entries that were
modified after the list of celery ids was gotten.
* err on the side of caution and just let the next heartbeat celery
killer try killing the task if it needs to be reaped.
2017-07-05 13:33:29 -04:00
Chris Meyers 15aee1f8ac celery task fail check now uses pglock
* Align locking used by celery task cleaner upper with regular task manager.
* Uses pglock/advisory lock instead of abusing Instance table lock.
2017-07-05 13:27:08 -04:00
Chris Meyers 3d4f8b0074 support clear_facts 2017-07-05 10:33:36 -04:00
Chris Meyers 4b84cec1e8 do not launch proj update on migrate 2017-07-05 10:32:45 -04:00
Chris Meyers 88b250b2f1 migrate scan jobs to use fact caching instead 2017-07-05 10:32:45 -04:00
Chris Meyers 318b1aebd1 remove special scan job logic 2017-07-05 10:32:45 -04:00
Chris Meyers eda0679eb3 Revert "support clear_facts"
This reverts commit e5432c231d1ab38807c8be10133d35ede237a519.
2017-07-05 10:32:13 -04:00
Chris Meyers 06edb6774a support clear_facts 2017-07-05 10:26:49 -04:00
Chris Meyers f1b1c4ee97 avoid instance record deadlock by using pglock 2017-07-05 09:38:19 -04:00
Chris Meyers 29adb39cae add insights_cred_id to host->inventory summary fields 2017-06-30 13:52:41 -04:00
Chris Meyers 668bce8212 fix job launch deadlock
* This both fixes the deadlock problem and a logic problem. We shouldn't
set the job's job_template current_job to pending jobs.
2017-06-30 09:54:08 -04:00
Chris Meyers d6082a976b retain support for insights system_id discovery 2017-06-22 09:49:45 -04:00
Chris Meyers d679dc7407 do not inject facts for isolated node runs 2017-06-21 13:45:28 -04:00
Chris Meyers 6c67834694 beautify code 2017-06-21 13:45:28 -04:00
Chris Meyers 45421c84a6 fact cache docs 2017-06-21 13:45:27 -04:00
Chris Meyers 965159a1fb rename store_facts to use_fact_cache 2017-06-21 13:45:27 -04:00
Chris Meyers ec2e537f63 remove fache cache receiver 2017-06-21 13:45:27 -04:00
Chris Meyers 4c118159ed system_tracking logging for fact cache 2017-06-21 13:45:27 -04:00
Chris Meyers 12cdbcf8b5 adds per-host timeout 2017-06-21 13:45:27 -04:00
Chris Meyers 817dbe8d33 add fact modified time 2017-06-21 13:45:27 -04:00
Chris Meyers 626e2d1c9b tower fact cache implementation
* Tower now injects facts into jobs via memcached for use by Ansible
playbooks. On the Ansible side, this is accomplished by the existing
mechanism, an Ansible Fact Cache Plugin + memcached. On the Tower side,
memcached is leveraged heavily.
2017-06-21 13:44:08 -04:00
Chris Meyers e437ddb3a5 insights integration docs 2017-06-12 12:01:30 -04:00
Chris Meyers c8ae48f1ac add host filter docs 2017-06-12 10:12:40 -04:00
Chris Meyers d0dc41daac turn single element array into string 2017-06-08 08:24:42 -04:00
Chris Meyers 2317687468 allow changing source_project
* scm project backed inventory now allows source_project to be mutated.
We reset the tracked revision when this happens.
2017-06-07 15:37:51 -04:00
Chris Meyers 3dc329bb0f copy paste fix 2017-06-07 15:04:44 -04:00
Chris Meyers aeb68f02f6 change settings comments 2017-06-07 15:03:53 -04:00
Chris Meyers ba5f553f0c fix up scm inventory updates
* implicit project updates inherit inv update instance group
* added needed settings for SCM group filter and others
2017-06-07 14:51:56 -04:00
Chris Meyers 0d56f42c83 better insights_credential reverse relationship
* From an insights cred, access all related inventories via
cred.insights_inventories rather than cred.insights_credential
* This fixes auto-attaching insights_credential to credential summary
fields. Fixes by NOT attaching them. That is the desired behavior.
2017-06-05 13:48:41 -04:00
Chris Meyers eb6e598f6c add insights_credential to summary fields on inventory 2017-06-01 08:47:21 -04:00
Chris Meyers 0c034f29af use insights etag as a cache
* During insights project updates, if the etag from insights matches the
last etag (scm_revision) then don't get all the maintenance playbooks.
Otherwise, get all the maintenance playbooks and update scm_revision
with the etag.
2017-05-30 18:30:13 -04:00
Chris Meyers 8b97db06b0 abuse group_by and instance_filters 2017-05-30 18:29:02 -04:00
Chris Meyers 9ede92a426 Revert "support vmware via source_vars"
This reverts commit cc64104f4bad6c360dcbd2f0291006956180a416.
2017-05-30 16:16:12 -04:00
Chris Meyers f7f4560de1 support vmware via source_vars
* support vmware host_filters and groupby_patterns by just doing nothing
2017-05-30 15:58:30 -04:00
Chris Meyers 441946d332 add v1 vs. v2 inclusion/exclusion 2017-05-26 14:04:15 -04:00
Chris Meyers b25b397e14 remove insights logic from backwards compat logic
* spelling fixes
2017-05-26 09:58:20 -04:00
Chris Meyers 2f43f0fe2b flake8 2017-05-26 08:52:13 -04:00
Chris Meyers 7c0513d5ee migrate insights credentials
* A credential is considered an insights credential if a project of
scm_type 'insights' is associated.
* all non-insights Projects that point at an insights credential are
nulled
2017-05-26 08:50:53 -04:00
Chris Meyers 87eea59845 add per-inventory insights credential 2017-05-26 07:25:36 -04:00
Chris Meyers ca1eb28d1f don't assume insights id is a uuid 2017-05-25 17:52:46 -04:00
Chris Meyers 4461c8fe91 more robust insights error handling 2017-05-22 17:32:17 -04:00
Chris Meyers 3793f1ec09 insights proxy robust error condition handling 2017-05-22 15:02:16 -04:00
Chris Meyers 9ba0ba07ec nest insights api response data 2017-05-22 10:11:31 -04:00
Chris Meyers fb53ca8455 insights proxy
* Issue request to Red Hat Insights API from Tower
/hosts/<id>/insights endpoint. User the first found Satellite 6
Credential as Basic Auth requests to Insights API.
2017-05-22 10:11:31 -04:00
Chris Meyers cf26678cea remove not operator code instead of commenting 2017-05-19 16:38:41 -04:00
Chris Meyers fceb09dc8c remove not operator
* We've switched from Q() to Host.objects.filter() and you can't do
~(Host.objects.filter()). We'd need to create an AST to track the
operations in order to perform the boolean logic ~ and we don't have
time for that now.
2017-05-19 14:40:02 -04:00
Chris Meyers 136bdc06e9 remove insights_system_id uniqueness constraint 2017-05-18 12:51:25 -04:00
Chris Meyers e81a7e1c25 allow insights_system_id to be set to null via api 2017-05-18 11:31:58 -04:00
Chris Meyers 067fdac8b1 prefetch optimizations for task manager
* Prefetch all Jobs Types related instance group
* Prefetch inventory updates inventory source. The attribute
inventory_source.inventory_id is accessed when building the hash
tables of the running tasks.
2017-05-16 15:05:20 -04:00
Chris Meyers 9b771ae907 only consider update_on_launch inventory sources
* When generating dependencies (i.e. dynamically launching Project
Update and Inventory Update) only create the dynamic dependencies if
update_on_launch is True.
2017-05-16 13:18:15 -04:00