Commit Graph

5676 Commits

Author SHA1 Message Date
AlanCoding
1bf2a455c6 Do not remove edges from other inventory sources 2019-02-11 13:08:19 -05:00
chris meyers
d4c3c089df fix scheduled jobs race condition
* The periodic scheduler that runs and spawns jobs from Schedule()'s can
end up spawning more jobs than intended, for a single Schedule.
Specifically, when tower clustering is involed. This change adds a
"global" database lock around this critical code. If another process is
already doing the scheduling, short circuit.
2019-02-08 08:43:11 -05:00
Ryan Petrello
229e997e7e don't update parent event changed|failed in bulk (it's expensive) 2019-02-06 20:02:52 -05:00
softwarefactory-project-zuul[bot]
902fb83493 Merge pull request #3172 from bverschueren/fix_inventory_sync_virtualenv
use source_project custom_virtualenv if configured

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-06 16:55:49 +00:00
Ryan Petrello
a6b362e455 fix a subtle bug in ansible_facts lookup filtering 2019-02-06 11:02:47 -05:00
Bram Verschueren
016fc7f6bf use source_project custom_virtualenv if configured
Signed-off-by: Bram Verschueren <verschueren.bram@gmail.com>
2019-02-06 10:51:52 +01:00
softwarefactory-project-zuul[bot]
e8eda28ce5 Merge pull request #3162 from wenottingham/cady-heron-was-right
Remove limit on `limit` field.

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-06 01:22:34 +00:00
Ryan Petrello
5e84782b9c provide a better ansible-inventory fallback error message old ansibles
see: https://github.com/ansible/awx/issues/3139
2019-02-05 17:12:48 -05:00
Bill Nottingham
4b669fb16d Remove limit on limit field.
This allows 'relaunch-on-failed' on an arbitrary number of failed hosts.
2019-02-05 13:30:51 -05:00
softwarefactory-project-zuul[bot]
b53621e74c Merge pull request #3152 from ryanpetrello/jsonb_prevent_field_lookups
prevent field lookups on Host.ansible_facts keys (it doesn't work)

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-05 17:24:06 +00:00
Ryan Petrello
bb5312f4fc prevent field lookups on Host.ansible_facts keys (it doesn't work)
under the hood, Host.ansible_facts is a postgres jsonb field which
performs match operations using the JSON containment operator (@>)

this operator _only_ works on exact matches on containment (i.e.,
"does the `ansible_distribution` jsonb value contain _this exact_ JSON
structure"):

SELECT ...
FROM main_host
WHERE ansible_facts @> '{"ansible_distribution": "centos"}'

SELECT ...
FROM main_host
WHERE ansible_facts @> '{"packages": {"dnsmasq": [{"version": 2}]}}'

postgres does _not_ expose any operator for fuzzy or lookup-based
matches with this operator, so host filter values like these don't
really make sense (postgres can't _filter_ in the way intended in these
examples):

ansible_distribution__startswith=\"Cent\"
ansible_distribution__icontains=\"CentOS\"
ansible_facts__packages__dnsmasq[]__version__startswith=\"2\"
2019-02-05 10:43:51 -05:00
Yanis Guenane
5e20dcb6ca jt, wfjt: Ensure SCHEDULE_MAX_JOBS is accurately respect
Currently SCHEDULE_MAX_JOBS+1 can be scheduled rather than
SCHEDULE_MAX_JOBS. This is due to the fact that we using strictly
greater rather than greater or equal.

Imagine we set SCHEDULE_MAX_JOBS=1, current logic:

  * First time (count = 0), count < 1 -> proceed
  * Second time (count = 1), count =< 1 -> proceed
  * Third time (count = 2), count > 1 -> prevented

Imagine we set SCHEDULE_MAX_JOBS=1, new logic:

  * First time (count = 0), count < 1 -> proceed
  * Second time (count = 1), count =< 1 -> prevented

Signed-off-by: Yanis Guenane <yguenane@redhat.com>
2019-02-05 13:44:39 +01:00
AlanCoding
7676ccdbac use custom venv in inventory prooting 2019-02-01 13:29:45 -05:00
softwarefactory-project-zuul[bot]
4626aa0144 Merge pull request #3093 from jbradberry/become_plugins
Support become plugins

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-01 17:48:09 +00:00
Robert Zahradníček
6f1cbac324 Add SCHEDULE_MAX_JOBS implementation for WFJTs for #2975 2019-01-31 21:52:36 +01:00
Ryan Petrello
10945faba1 properly detect ansible-playbook vs ansible runs in bwrap arg building
a recent change (65641c7) made it so that we call
ansible-playbook using its _absolute path_ (i.e.,
/usr/bin/ansible-playbook, /var/lib/venv/xyz/bin/ansible-playbook), so
this logic is no longer correct
2019-01-31 11:06:50 -05:00
Ryan Petrello
2927803a82 fix overindent lint failures 2019-01-30 12:12:39 -05:00
Jeff Bradberry
6560ab0fab Migrated the inputs schema on existing CredentialTypes
to convert the custom become_method into a plain string.

related #2630

Signed-off-by: Jeff Bradberry <jeff.bradberry@gmail.com>
2019-01-29 15:04:35 -05:00
Jeff Bradberry
6e1deed79e Removed the special-case logic for maintaining the schema of the become_method field
related #2630

Signed-off-by: Jeff Bradberry <jeff.bradberry@gmail.com>
2019-01-29 14:06:26 -05:00
Jeff Bradberry
0ecd6542bf Changed the become_method field into one that takes arbitrary input
related #2630

Signed-off-by: Jeff Bradberry <jeff.bradberry@gmail.com>
2019-01-28 16:53:54 -05:00
Molnar, Shawn
31b3bad658 Fixed role download typo in help text 2019-01-25 16:37:51 -08:00
Ryan Petrello
88eaf1154a move awx.main.utils.ansible tests data into the correct location 2019-01-25 11:11:12 -05:00
softwarefactory-project-zuul[bot]
5cdab1b57a Merge pull request #3070 from ryanpetrello/bye-bye-six
clean up unnecessary usage of the six library (awx only supports py3)

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-25 15:20:38 +00:00
softwarefactory-project-zuul[bot]
2a86c5b944 Merge pull request #3061 from jakemcdermott/inventory-scm-job-linkage
add linked status indicator for scm inventory project updates

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-25 13:05:36 +00:00
Ryan Petrello
daeeaf413a clean up unnecessary usage of the six library (awx only supports py3) 2019-01-25 00:19:48 -05:00
softwarefactory-project-zuul[bot]
68950d56ca Merge pull request #3065 from ryanpetrello/dispatcher-reap-refactor
clean up some unnecessary dispatcher reaping code

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-24 22:30:45 +00:00
Jake McDermott
477c5df022 add linked status indicator for scm inventory project updates
Signed-off-by: Jake McDermott <yo@jakemcdermott.me>
2019-01-24 16:24:42 -05:00
softwarefactory-project-zuul[bot]
be4b3c75b4 Merge pull request #3052 from jakemcdermott/inputs_enc
explicitly handle .inputs in decrypt_field, encrypt_field

Reviewed-by: Ryan Petrello
             https://github.com/ryanpetrello
2019-01-24 17:04:30 +00:00
Ryan Petrello
4707dc2a05 clean up some unnecessary dispatcher reaping code 2019-01-24 11:11:05 -05:00
Ryan Petrello
9817ab14d0 clean up some minor linting issues 2019-01-24 10:23:56 -05:00
softwarefactory-project-zuul[bot]
51b51a9bf7 Merge pull request #3051 from ryanpetrello/record-venv-usage
record the virtualenv used when a job or inventory sync runs

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-23 21:33:31 +00:00
Ryan Petrello
0e5e23372d use the correct ansible-inventory executable for custom venvs 2019-01-23 14:07:48 -05:00
softwarefactory-project-zuul[bot]
1e44d5c833 Merge pull request #2899 from AlanCoding/copy_created_by
Copy WFJT created_by to jobs it spawns

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-23 16:53:49 +00:00
Ryan Petrello
f7bc8fb662 record the virtualenv used when a job or inventory sync runs
see: https://github.com/ansible/awx/issues/2264
2019-01-23 11:48:06 -05:00
softwarefactory-project-zuul[bot]
416dcc83c9 Merge pull request #3020 from beeankha/fix_unicode_error
Fix unicode error

Reviewed-by: Ryan Petrello
             https://github.com/ryanpetrello
2019-01-23 15:26:07 +00:00
AlanCoding
13ed656506 Copy WFJT created_by to jobs it spawns 2019-01-23 09:33:14 -05:00
Jake McDermott
a8728670e1 handle credential.inputs in decryption utils 2019-01-22 22:56:24 -05:00
Matthew Jones
058e2c0d81 Make schema generator handle alternative iso 8601 datetimes 2019-01-22 11:44:56 -05:00
Ryan Petrello
b2442d42a3 detect dead DB connections in the dispatcher when reaping jobs 2019-01-22 08:40:26 -05:00
softwarefactory-project-zuul[bot]
88d5fb0420 Merge pull request #3039 from AlanCoding/inventory_venv
Use custom virtual environment in inventory updates

Reviewed-by: Ryan Petrello
             https://github.com/ryanpetrello
2019-01-22 13:12:37 +00:00
softwarefactory-project-zuul[bot]
c0ff4dad59 Merge pull request #3021 from jakemcdermott/credential_input_access_methods
add input access methods to credentials

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-21 21:10:12 +00:00
softwarefactory-project-zuul[bot]
d021c253aa Merge pull request #2959 from crab86/devel
Add Grafana notification type

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-21 17:21:14 +00:00
AlanCoding
99288a5e18 Use custom virtual environment in inventory updates 2019-01-21 06:45:55 -05:00
Sebastian
ebe0ded9c2 Add grafana notification type unit tests 2019-01-20 22:42:03 +01:00
Jake McDermott
2dadfbcc14 use credential input access methods in injectors.py 2019-01-20 14:02:01 -05:00
Jake McDermott
5010e98b8f use credential input access methods in projects.py 2019-01-20 13:08:38 -05:00
Jake McDermott
c01c671642 use credential input access methods in tasks.py 2019-01-20 13:08:30 -05:00
Jake McDermott
a86e270905 add credential input access methods 2019-01-20 13:08:23 -05:00
Sebastian
4058d18593 Add grafana notification type 2019-01-20 13:51:23 +01:00
Ryan Petrello
caa55f112f add robust handling of non-UTF8 when detecting inventory/playbooks 2019-01-19 13:25:41 -05:00