Commit Graph

5700 Commits

Author SHA1 Message Date
Ryan Petrello
b1a33869dc convey OpenStack verify_ssl defaults in the CredentialType schema 2019-02-20 09:02:48 -05:00
Hideki Saito
9f04fbe4a4 Add verify_ssl to OpenStack credential type
To avoid verification failures when using a self-signed certificate file,
 Added "Verify SSL" check box to the openstack credential type edit page.

Signed-off-by: Hideki Saito <saito@fgrep.org>
2019-02-19 12:53:13 -05:00
Ryan Petrello
43ca4526b1 define native CredentialType inputs/injectors in code, not in the DB
This has a few benefits:

1.  It makes adding new fields to built-in CredentialTypes _much_
    simpler.  In the past, we've had to write a migration every time we
    want to modify an existing type (changing a label/help text,
    changing options like the recent become_method changes) or
    when adding a new field entirely

2.  It paves the way for third party credential plugins support, where
    importable libraries will define their own source code-based schema
2019-02-19 10:22:26 -05:00
softwarefactory-project-zuul[bot]
889dae357b Merge pull request #3235 from ryanpetrello/sql-profiling
add a custom DB backend that provides system-level SQL profiling

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-15 21:56:28 +00:00
Ryan Petrello
6f43875e80 record profile data in /var/log/tower, not /var/lib/awx 2019-02-15 14:34:55 -05:00
softwarefactory-project-zuul[bot]
003ec64413 Merge pull request #3241 from ryanpetrello/workflow-convergence-i18n
mark a workflow convergence error message for i18n

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-14 21:16:54 +00:00
Ryan Petrello
4f83d44142 mark a workflow convergence error message for i18n 2019-02-14 15:55:20 -05:00
Bill Nottingham
7452eb2fa1 Remove broken user_info command.
This has not worked in a long time, and does not serve much purpose.
2019-02-14 15:34:24 -05:00
Ryan Petrello
eed94b641e add a custom DB backend that provides system-level SQL profiling
run this command on _any_ node in an awx cluster:

$ awx-manage profile_sql --threshold=2.0 --minutes=1

...and for 1 minute, the timing for _every_ SQL query in _every_ awx
Python process that uses the Django ORM will be measured

queries that run longer than (in this example) 2 seconds will be
written to a per-process sqlite database in /var/lib/awx/profile, and
the file will contain an EXPLAIN VERBOSE for the query and the full
Python stack that led to that SQL query's execution (this includes not
just WSGI requests, but background processes like the runworker and
dispatcher)

$ awx-manage profile_sql --threshold=0

...can be used to disable profiling again (if you don't want to wait for
the minute to expire)
2019-02-14 15:04:46 -05:00
softwarefactory-project-zuul[bot]
ebd9d3dc67 Merge pull request #3234 from wenottingham/the-only-good-code-is-a-dead-code
Delete some unused functions.

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-14 14:10:31 +00:00
softwarefactory-project-zuul[bot]
80cf154fb7 Merge pull request #3233 from ryanpetrello/F405
remove usage of import * and enforce F405 in our linter

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-14 13:32:18 +00:00
Ryan Petrello
9bebf3217e remove usage of import * and enforce F405 in our linter
import * is a scourge upon the earth
2019-02-13 17:10:33 -05:00
softwarefactory-project-zuul[bot]
ae7d26fab0 Merge pull request #3186 from vismay-golwala/update_schedule_constraint
Update standalone schedule name uniqueness combining it with unified …

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-13 22:03:54 +00:00
Bill Nottingham
0f54d30f2c Remove some unused functions. 2019-02-13 16:29:59 -05:00
softwarefactory-project-zuul[bot]
551218fd44 Merge pull request #3206 from AlanCoding/learn_to_share
Do not remove edges from other inventory sources

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-13 18:09:18 +00:00
Vismay Golwala
4af54517d2 Update standalone schedule name uniqueness combining it with unified job template.
Signed-off-by: Vismay Golwala <vgolwala@redhat.com>
2019-02-13 12:46:28 -05:00
softwarefactory-project-zuul[bot]
295afa805c Merge pull request #3212 from AlanCoding/model_star_imports
Remove star imports in tasks and non-base models

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-13 15:05:01 +00:00
Ryan Petrello
e4eda3ef0d remove field size limit on adhoc limit
related: 4b669fb16d
2019-02-13 08:34:10 -05:00
AlanCoding
fbf6315a8c remove star imports in tasks and non-base models 2019-02-12 19:50:30 -05:00
softwarefactory-project-zuul[bot]
8a3c10686e Merge pull request #3191 from chrismeyersfsu/fix-job_event_smart_inv_slow_take_two_devel
do not observe queries when constructing them

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-12 22:27:11 +00:00
chris meyers
c121565209 add tests for host_filter
* Ensure that building the smart inventory query string doesn't invoke
the database.
2019-02-12 16:11:54 -05:00
Ryan Petrello
e245e50ee4 optimize a slow query in inventory script generation
if we don't preload this column, Django needs it, and so it generates
one query per-host (!!!) to get it.  For large (10k+ host) inventories,
this is incredibly slow.

see: https://github.com/ansible/awx/issues/3214
2019-02-12 12:55:38 -05:00
AlanCoding
1bf2a455c6 Do not remove edges from other inventory sources 2019-02-11 13:08:19 -05:00
Yanis Guenane
f61b6f9615 awx.main.tasks: Remove reference to unimport six
d4c3c08 re:introduced the use of six that has been removed by daeeaf4.
This lead to ""NameError: name 'six' is not defined"". This commit fixes
the issue.

Signed-off-by: Yanis Guenane <yguenane@redhat.com>
2019-02-11 09:41:09 +01:00
chris meyers
ed78978b5f do not observe queries when constructing them
* While parsing host_filter in the smart inventory code it was
triggering sql queries. This changset avoids executing the query that is
being constructed.
2019-02-08 12:30:51 -05:00
chris meyers
d4c3c089df fix scheduled jobs race condition
* The periodic scheduler that runs and spawns jobs from Schedule()'s can
end up spawning more jobs than intended, for a single Schedule.
Specifically, when tower clustering is involed. This change adds a
"global" database lock around this critical code. If another process is
already doing the scheduling, short circuit.
2019-02-08 08:43:11 -05:00
Ryan Petrello
229e997e7e don't update parent event changed|failed in bulk (it's expensive) 2019-02-06 20:02:52 -05:00
softwarefactory-project-zuul[bot]
902fb83493 Merge pull request #3172 from bverschueren/fix_inventory_sync_virtualenv
use source_project custom_virtualenv if configured

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-06 16:55:49 +00:00
Ryan Petrello
a6b362e455 fix a subtle bug in ansible_facts lookup filtering 2019-02-06 11:02:47 -05:00
Bram Verschueren
016fc7f6bf use source_project custom_virtualenv if configured
Signed-off-by: Bram Verschueren <verschueren.bram@gmail.com>
2019-02-06 10:51:52 +01:00
softwarefactory-project-zuul[bot]
e8eda28ce5 Merge pull request #3162 from wenottingham/cady-heron-was-right
Remove limit on `limit` field.

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-06 01:22:34 +00:00
Ryan Petrello
5e84782b9c provide a better ansible-inventory fallback error message old ansibles
see: https://github.com/ansible/awx/issues/3139
2019-02-05 17:12:48 -05:00
Bill Nottingham
4b669fb16d Remove limit on limit field.
This allows 'relaunch-on-failed' on an arbitrary number of failed hosts.
2019-02-05 13:30:51 -05:00
softwarefactory-project-zuul[bot]
b53621e74c Merge pull request #3152 from ryanpetrello/jsonb_prevent_field_lookups
prevent field lookups on Host.ansible_facts keys (it doesn't work)

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-05 17:24:06 +00:00
Ryan Petrello
bb5312f4fc prevent field lookups on Host.ansible_facts keys (it doesn't work)
under the hood, Host.ansible_facts is a postgres jsonb field which
performs match operations using the JSON containment operator (@>)

this operator _only_ works on exact matches on containment (i.e.,
"does the `ansible_distribution` jsonb value contain _this exact_ JSON
structure"):

SELECT ...
FROM main_host
WHERE ansible_facts @> '{"ansible_distribution": "centos"}'

SELECT ...
FROM main_host
WHERE ansible_facts @> '{"packages": {"dnsmasq": [{"version": 2}]}}'

postgres does _not_ expose any operator for fuzzy or lookup-based
matches with this operator, so host filter values like these don't
really make sense (postgres can't _filter_ in the way intended in these
examples):

ansible_distribution__startswith=\"Cent\"
ansible_distribution__icontains=\"CentOS\"
ansible_facts__packages__dnsmasq[]__version__startswith=\"2\"
2019-02-05 10:43:51 -05:00
Yanis Guenane
5e20dcb6ca jt, wfjt: Ensure SCHEDULE_MAX_JOBS is accurately respect
Currently SCHEDULE_MAX_JOBS+1 can be scheduled rather than
SCHEDULE_MAX_JOBS. This is due to the fact that we using strictly
greater rather than greater or equal.

Imagine we set SCHEDULE_MAX_JOBS=1, current logic:

  * First time (count = 0), count < 1 -> proceed
  * Second time (count = 1), count =< 1 -> proceed
  * Third time (count = 2), count > 1 -> prevented

Imagine we set SCHEDULE_MAX_JOBS=1, new logic:

  * First time (count = 0), count < 1 -> proceed
  * Second time (count = 1), count =< 1 -> prevented

Signed-off-by: Yanis Guenane <yguenane@redhat.com>
2019-02-05 13:44:39 +01:00
AlanCoding
7676ccdbac use custom venv in inventory prooting 2019-02-01 13:29:45 -05:00
softwarefactory-project-zuul[bot]
4626aa0144 Merge pull request #3093 from jbradberry/become_plugins
Support become plugins

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-01 17:48:09 +00:00
Robert Zahradníček
6f1cbac324 Add SCHEDULE_MAX_JOBS implementation for WFJTs for #2975 2019-01-31 21:52:36 +01:00
Ryan Petrello
10945faba1 properly detect ansible-playbook vs ansible runs in bwrap arg building
a recent change (65641c7) made it so that we call
ansible-playbook using its _absolute path_ (i.e.,
/usr/bin/ansible-playbook, /var/lib/venv/xyz/bin/ansible-playbook), so
this logic is no longer correct
2019-01-31 11:06:50 -05:00
Ryan Petrello
2927803a82 fix overindent lint failures 2019-01-30 12:12:39 -05:00
Jeff Bradberry
6560ab0fab Migrated the inputs schema on existing CredentialTypes
to convert the custom become_method into a plain string.

related #2630

Signed-off-by: Jeff Bradberry <jeff.bradberry@gmail.com>
2019-01-29 15:04:35 -05:00
Jeff Bradberry
6e1deed79e Removed the special-case logic for maintaining the schema of the become_method field
related #2630

Signed-off-by: Jeff Bradberry <jeff.bradberry@gmail.com>
2019-01-29 14:06:26 -05:00
Jeff Bradberry
0ecd6542bf Changed the become_method field into one that takes arbitrary input
related #2630

Signed-off-by: Jeff Bradberry <jeff.bradberry@gmail.com>
2019-01-28 16:53:54 -05:00
Molnar, Shawn
31b3bad658 Fixed role download typo in help text 2019-01-25 16:37:51 -08:00
Ryan Petrello
88eaf1154a move awx.main.utils.ansible tests data into the correct location 2019-01-25 11:11:12 -05:00
softwarefactory-project-zuul[bot]
5cdab1b57a Merge pull request #3070 from ryanpetrello/bye-bye-six
clean up unnecessary usage of the six library (awx only supports py3)

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-25 15:20:38 +00:00
softwarefactory-project-zuul[bot]
2a86c5b944 Merge pull request #3061 from jakemcdermott/inventory-scm-job-linkage
add linked status indicator for scm inventory project updates

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-25 13:05:36 +00:00
Ryan Petrello
daeeaf413a clean up unnecessary usage of the six library (awx only supports py3) 2019-01-25 00:19:48 -05:00
softwarefactory-project-zuul[bot]
68950d56ca Merge pull request #3065 from ryanpetrello/dispatcher-reap-refactor
clean up some unnecessary dispatcher reaping code

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-24 22:30:45 +00:00