Commit Graph
100 Commits
Author SHA1 Message Date
Chris Meyers 0d56f42c83 better insights_credential reverse relationship
* From an insights cred, access all related inventories via
cred.insights_inventories rather than cred.insights_credential
* This fixes auto-attaching insights_credential to credential summary
fields. Fixes by NOT attaching them. That is the desired behavior.
2017-06-05 13:48:41 -04:00
Chris Meyers eb6e598f6c add insights_credential to summary fields on inventory 2017-06-01 08:47:21 -04:00
Chris Meyers 441946d332 add v1 vs. v2 inclusion/exclusion 2017-05-26 14:04:15 -04:00
Chris Meyers b25b397e14 remove insights logic from backwards compat logic
* spelling fixes
2017-05-26 09:58:20 -04:00
Chris Meyers 2f43f0fe2b flake8 2017-05-26 08:52:13 -04:00
Chris Meyers 7c0513d5ee migrate insights credentials
* A credential is considered an insights credential if a project of
scm_type 'insights' is associated.
* all non-insights Projects that point at an insights credential are
nulled
2017-05-26 08:50:53 -04:00
Chris Meyers 87eea59845 add per-inventory insights credential 2017-05-26 07:25:36 -04:00
Chris Meyers ca1eb28d1f don't assume insights id is a uuid 2017-05-25 17:52:46 -04:00
Chris Meyers 4461c8fe91 more robust insights error handling 2017-05-22 17:32:17 -04:00
Chris Meyers 3793f1ec09 insights proxy robust error condition handling 2017-05-22 15:02:16 -04:00
Chris Meyers 9ba0ba07ec nest insights api response data 2017-05-22 10:11:31 -04:00
Chris Meyers fb53ca8455 insights proxy
* Issue request to Red Hat Insights API from Tower
/hosts/<id>/insights endpoint. User the first found Satellite 6
Credential as Basic Auth requests to Insights API.
2017-05-22 10:11:31 -04:00
Chris Meyers cf26678cea remove not operator code instead of commenting 2017-05-19 16:38:41 -04:00
Chris Meyers fceb09dc8c remove not operator
* We've switched from Q() to Host.objects.filter() and you can't do
~(Host.objects.filter()). We'd need to create an AST to track the
operations in order to perform the boolean logic ~ and we don't have
time for that now.
2017-05-19 14:40:02 -04:00
Chris Meyers 136bdc06e9 remove insights_system_id uniqueness constraint 2017-05-18 12:51:25 -04:00
Chris Meyers e81a7e1c25 allow insights_system_id to be set to null via api 2017-05-18 11:31:58 -04:00
Chris Meyers 067fdac8b1 prefetch optimizations for task manager
* Prefetch all Jobs Types related instance group
* Prefetch inventory updates inventory source. The attribute
inventory_source.inventory_id is accessed when building the hash
tables of the running tasks.
2017-05-16 15:05:20 -04:00
Chris Meyers 9b771ae907 only consider update_on_launch inventory sources
* When generating dependencies (i.e. dynamically launching Project
Update and Inventory Update) only create the dynamic dependencies if
update_on_launch is True.
2017-05-16 13:18:15 -04:00
Chris Meyers cc4692932a update Host manager to handle Query Sets
* filter_host_string now returns a query set instead of Q(). This change
updates the Host manager to remove the .filter() call since the results
returned from filter_host_string() has already turned Q() -> Query Set
2017-05-12 12:14:32 -04:00
Chris Meyers ac534abe24 expose insights_system_id via api
* insights_system_id is read-only. Only modified via the fact gathering
code path.
2017-05-12 12:09:20 -04:00
Chris Meyers eb27233e5c clear insights system id if not found
* Host previously found to have an insights system id that then later
are found to not have an insights system id, have their system id
cleared.
2017-05-12 12:00:30 -04:00
Chris Meyers 3cdc332497 rename insight tracking field; handle duplicate
* rename from insights_machine_id to insights_system_id
* gracefully handle duplicate insights machine id's
2017-05-12 10:47:49 -04:00
Chris Meyers b55af095ef boolean logic performed on filter results
* Before, the boolean logic operators were performed against the Q()
objects, iteratively. Now, boolean logic is done after
Host.objects.filter(Q()). This results in the wanted and expected
results.
2017-05-12 09:26:11 -04:00
Chris Meyers b9b0b29d97 associate insights machine id w/ host on fact scan
* Add inisghts fact scan module
* Update fact scan playbook to call new insight fact scan module
* JT run w/ store_facts=True will save scanned facts. We "skim" the
machine_id fact from Insights fact scans and associate it with the host
that the fact scan came from.
2017-05-11 11:20:52 -04:00
Chris Meyers 0be384af18 rename field to store_facts and fix up tests 2017-05-01 16:39:48 -04:00
Chris Meyers 6dec9d1014 per-jt most recent fact scan gathering
* Remove system tracking piggybacked most-recent-fact-gathering
* Add explicit per-jobtemplate configurable fact gathering
* Continue to support system tracking
* Fixed a bug where we weren't ack()ing system tracking rabbit msgs
2017-05-01 16:03:16 -04:00
Chris Meyers 4795fd28bf always gather facts
* bolt on the fact scan gather to all jobs
2017-05-01 16:03:16 -04:00
Chris Meyers 633da940a8 create .lock file without needing the proj dir to exist
* We create <project_name>.lock file in the tower project root
directory. This should not require that the project directory exists,
only that the project directory name is generatable.
2017-05-01 11:15:18 -04:00
Chris Meyers d14a55911b ack fact scan messages
* Fixes a bug where fact rabbit messages would build up because we never
explicitly ack them.
2017-04-28 13:46:54 -04:00
Chris Meyers d69ae2cc92 coarse json queries to use gin index 2017-04-25 10:21:22 -04:00
Chris Meyers 5a7b78220d pin twisted and asgiref for asgi-amqp
* asgi-amqp translation layer needs a specific version of channels which
requires a specific version of daphne and all of the fall out.
2017-04-19 13:56:36 -04:00
Chris Meyers 8c8ccba5a5 add mock to ansible dev env requirements 2017-04-18 12:31:05 -04:00
Chris Meyers 70a8ccef0f return count, results, next, prev
* This returns /api/vx/hosts/ to having all the nice response items like
count, results, next, prev. I had "chopped" that off by explicitly only
returning result.
2017-04-17 16:43:29 -04:00
Chris Meyers 55268a8ff0 remove facts_latest in favor of ansible_facts
* Closer align our facts data structure with ansible facts data
structure for purposes of ... wait for it ... 2-way fact caching
2017-04-17 14:27:33 -04:00
Chris Meyers 775093294e run host listing through v1 vs. v2 logic
* api/v1/hosts related fields were getting v2 in the related urls. The
context is now included in the serializer instantiation.
2017-04-17 11:43:44 -04:00
Chris Meyers 5ff4966526 add json a=null support and still support a="null"
related to #6016
2017-04-13 16:26:35 -04:00
Chris Meyers 040f9dd545 support > 2 sequential and, also or
* Fixed a bug in the fact search language parser where it would not loop
over all the <and, or> operations. The result, we can support (a=b or
a=c or a=d) also support (a=b and b=b and c=b)
2017-04-13 13:36:38 -04:00
Chris Meyers 23ce5ab9de fix up test 2017-04-13 10:44:28 -04:00
Chris Meyers bd1f8b088f 400 on invalid host filter query instead of 500 2017-04-13 09:40:11 -04:00
Chris Meyers 1a2cbaf5e1 flake8 fix 2017-04-12 15:35:42 -04:00
Chris Meyers a2b37aa7e6 add not logic to json search
* Also fix up single contains [] logic. Values with a single contains
should be wrapped in [] and not, like I thought before, \"\"
2017-04-12 15:11:36 -04:00
Chris Meyers 70fafe75d2 clean up commented code 2017-04-12 11:42:31 -04:00
Chris Meyers d1eba4b607 log lock errors and test it 2017-04-12 11:41:11 -04:00
Chris Meyers 7919c47370 put lock file in project root dir 2017-04-12 09:36:15 -04:00
Chris Meyers 648aa470d7 lock projects on project sync
* Use filesystem, blocking, locks to prevent two project syncs for the
same project from running the project update playbook at the same time.
2017-04-11 15:47:46 -04:00
Chris Meyers a4fe04cbfa handle unicode and types 2017-04-10 14:57:37 -04:00
Chris Meyers 5ce4c80821 support json/python types 2017-04-07 12:24:20 -04:00
Chris Meyers ee09bca558 change facts_recent to facts_latest 2017-04-05 16:50:38 -04:00
Chris Meyers 02795e526c inherit from jsonbfield package not jsonfield
* jsonbfield supports json querying. jsonfield package is still a jsonb
postgres data type, but doesn't support jsonb style querying.
* add undo migration support to GIN index
2017-04-05 13:49:51 -04:00
Chris Meyers f7fb541fe2 add facts recent endpoints
* Hang most recent listing of facts per-host off of /hosts/n/
2017-04-04 17:54:02 -04:00
Chris Meyers f5d7d0bce5 add model fact recent
* Copy of the most recent system tracking fact for each module type.
Utimately, this allows us to GIN index the jsonb object to support
fact searching.
2017-04-04 16:32:30 -04:00
Chris Meyers a94c8ef1e3 add functional tests for host_filter 2017-04-04 09:13:56 -04:00
Chris Meyers 67633b2387 add pyparsing a top level dep and fix other deps
* bump shade
* remove pyrax as a dep
* pin daphne so channels doens't pick up the wrong, too new, version
2017-04-03 16:17:05 -04:00
Chris Meyers 443f14ed85 flake8 is hard 2017-04-03 16:14:43 -04:00
Chris Meyers 80e89b1116 spelling is hard 2017-04-03 15:53:54 -04:00
Chris Meyers 6cbdb8d0e6 faster first time parse generator
* Generating the set of valid unicode characters is expensive in terms
of memory storage. Instead, we define the grammer by the negation of
allowed unicode characters. Much faster.
2017-04-03 15:32:21 -04:00
Chris Meyers a8213661fd basic fact search grammar
* Establish a base grammar for handling json path specification and
value matching. With boolean logic support and parenthesis grouping
i.e. (a.b.c="value") and ((a.b="foo") or (a="bar"))
* generate Q() results for passing to Host.objects.filter()
* Hooked up via /api/v1/hosts?host_filter=...
* DynamicFilterField added to store host filter as string with grammar
parser attached as static methods for later use by DynamicInventory &
continued use by host_filter=...
2017-04-03 15:32:21 -04:00
Chris Meyers 33dca18119 more tests 2017-03-13 15:05:33 -04:00
Chris Meyers a051b41773 add test 2017-03-13 14:45:20 -04:00
Chris Meyers 6a0916371b add job cancel dependency information
* Inventory updates that are canceled and are dependencies of jobs
result in the dependent job being canceled. This code adds a better
description to job_explanation so that a job marked as canceled can be
traced back to the inventory update that triggered this case.
2017-03-13 12:52:33 -04:00
Chris Meyers c255c07ea5 pass ansible the "raw" git repo url
* I explored the |urlencode() filter instead. Ansible and git didn't
like this.
* Quoting doesn't seemed to be required. Ansible and the git module are
smart enough to do the right thing. I tested with a space in the repo
url path and it works.
2017-03-13 09:34:52 -04:00
Chris Meyers 4005cf927e copy existing env before making a safe env
* We should not modify the original env because it's what is going to be
passed to the Ansible process. This will contain all of the passwords,
unobscured. The safe env is what will get saved into the model.
2017-03-07 16:48:48 -05:00
Chris Meyers 1fc40d38c7 jobs/<id>/job_events?search= searches stdout field 2017-03-07 16:48:48 -05:00
Chris Meyers 54bcdccdab give a default other than None
1. If the setting isn't in the database, then the specified default is
returned.
2. If the setting IS in the database, then the 'value' field, from the
database, is returned; Unless, the value is null in the database, then
the defined default is returned.
* Take case 1. If the default=None then an AttributeError is raised
* Thus, by setting default='', in case 1. We don't get an AttributeError
because '' will be returned.
* Also note that we don't allow_null=True because then the database
entry could be None. Accessing the setting would raise an AttributeError
in this case. Further, since we set default='' it doesn't matter if null
is in the database.

TL;DR set default=<something_other_than_None>
2017-03-02 13:36:48 -05:00
Chris Meyers c4fb88c0d9 remove uneeded post commit wrapper
* Since we changed the lower level method to always use post commit
message emit
2017-02-28 13:10:00 -05:00
Chris Meyers e09497108e all job status change websockets events should happen after the job hits
the database
2017-02-28 12:56:55 -05:00
Chris Meyers b85361732b Revert "emit job status over socket after database commit"
This reverts commit edefeeacdaf11cf484a4b4893ee9acde5ef85390.
2017-02-28 12:53:59 -05:00
Chris Meyers 7da52c8bef emit job status over socket after database commit
* Wait until the newly created job record hits the database before
telling the websocket clients that the job's status is "pending"
2017-02-28 12:39:55 -05:00
Chris Meyers c1bace87ab only cancel deps if we can cancel the inv update 2017-02-27 11:37:29 -05:00
Chris Meyers f3e514246e cancel jobs dependent on inv update 2017-02-27 11:28:29 -05:00
Chris Meyers 9f8df08e1d update tests 2017-02-27 10:56:34 -05:00
Chris Meyers a1c76d3adc like inventory updates, check if project update deps already processed 2017-02-27 10:34:44 -05:00
Chris Meyers dd513621f0 Revert "Merge pull request #5553 from chrismeyersfsu/fix-waiting_blocked"
This reverts commit 9ba2122f4f85eecaeb6fa53ac92ea2811b05e83f, reversing
changes made to c3a5f2c96fd85dd1405a8f5c875ffc988dee16a4.
2017-02-27 09:38:45 -05:00
Chris Meyers 903d0472f0 just like we fail running tasks fail waiting tasks
* Associate the celery_id with the job at the earliest point possible.
This ensures that a waiting job has a celery id. Thus, we are free to
fail waiting jobs that don't have a celery id.
2017-02-24 12:07:04 -05:00
Chris Meyers 2f604f30d9 host may be None, account for that 2017-02-24 09:59:52 -05:00
Chris Meyers fe3d79b5ec Revert "Merge pull request #5527 from chrismeyersfsu/fix-log_settings"
This reverts commit 3db0242ef9a8b1ce67e159b8a19e0c1e70330b7c, reversing
changes made to e6893bb7f3cf696c202c39103b432c20afa6ec08.
2017-02-24 09:59:34 -05:00
Chris Meyers 50a8083984 flake8 2017-02-23 15:41:41 -05:00
Chris Meyers 08825a1f49 fix check running status 2017-02-23 15:09:50 -05:00
Chris Meyers 2953479dc8 correct behavior of tower_uuid logging setting
* Register a default=function() with LOG_AGGREGATOR_TOWER_UUID. We know
this function will ONLY be called when there is no database backed
settings. Therefore, we set a one-time uuid at this time.
* The user is free to change LOG_AGGREGATOR_TOWER_UUID.
* The user may not set this field to null
* The user may not set this field to ''
2017-02-23 14:31:47 -05:00
Chris Meyers 0b04001b15 host may be None, account for that 2017-02-23 14:31:24 -05:00
Chris Meyers 427b484075 be forgiving in log api log message fields list 2017-02-21 19:24:27 -05:00
Chris Meyers e78377dcfc remove formatters dependency on settings 2017-02-21 14:06:20 -05:00
Chris Meyers 60aa22b01b flake8 2017-02-21 11:05:30 -05:00
Chris Meyers 98c8ff3475 rename uuid field to trigger restart on update
* Since we trigger a restart on update, we can also cache the tower uuid
value in the formatter init. So do that.
2017-02-21 10:15:02 -05:00
Chris Meyers 31a79c331d move logger logic to formatter 2017-02-20 14:30:17 -05:00
Chris Meyers 0dada82b4b more general name for performance logger 2017-02-18 09:28:38 -05:00
Chris Meyers af9975c6af add a new logger, awx.analytics.performance.api
* Used to log API timing information
2017-02-17 15:35:03 -05:00
Chris Meyers 6e9488a59b ensure job deps are created only once 2017-02-15 15:54:30 -05:00
Chris Meyers 2432a3d3c3 Revert "remove partial dependency job id logic"
This reverts commit 4bb3a4909e616209fc291b2b3cee46469bc58f9e.
2017-02-13 16:16:43 -05:00
Chris Meyers b693b06706 remove partial dependency job id logic
* Late in the release we added job dependency tracking to the DB. We
decided not to use this information in the scheduler. However, I
half-ass added code to the scheduler to use it. Note that we still
determine inv and job update dependency by using a hack of the related
creation time, job.created-2 and job.created-1 respectively.

This removes any use of job dependent id expect for purposes of chain
failing.
2017-02-13 12:48:00 -05:00
Chris Meyers dfb7b08380 check job status before marking as failed
* When rectifying celery <-> db job status, re-check the job status to
ensure it did not finish between the time that we pulled the list of
celery tasks from celery and the time in which we rectify.
2017-02-09 15:27:08 -05:00
Chris Meyers d375560f18 take into account inv org when getting custom inv src list 2017-02-06 14:08:18 -05:00
Chris Meyers 0056699691 no tooltip for user roles, only team roles 2017-02-06 11:40:52 -05:00
Chris Meyers 730cbb0afd name org permissions tab; users / teams 2017-02-06 10:30:04 -05:00
Chris Meyers 335a78d1e1 fixes job elapsed timer
* now with tests
2017-02-03 11:40:25 -05:00
Chris Meyers 51fe00ffac update tests to NOT use json 2017-02-03 11:11:30 -05:00
Chris Meyers a2903199ae logger username/pass not required 2017-02-03 10:19:54 -05:00
Chris Meyers 195a36aef9 embed json into js module 2017-02-02 18:00:41 -05:00
Chris Meyers 76e5cc1fb1 update ui packaging 2017-02-02 14:06:08 -05:00
Chris Meyers 8b40cb14ec add job finished test 2017-02-01 16:42:17 -05:00