Commit Graph

615 Commits

Author SHA1 Message Date
Ryan Petrello
dfa8d44eb8 don't ship external logs from the main thread of the dispatcher
this is a fairly esoteric change that attempts to work around a bug
we've discovered in cpython itself

context: https://github.com/ansible/awx/issues/4181
2019-06-27 16:24:36 -04:00
Gabe Muniz
bd93d97a60 fix so enabled value is case insensitive 2019-06-13 15:36:54 -04:00
Ryan Petrello
6da445f7c0 remove /api/v1 and deprecated credential fields 2019-06-06 12:23:00 -04:00
Ryan Petrello
40b1e89b67 add the ability to disable RabbitMQ queue durability 2019-05-28 15:49:32 -04:00
Christian Adams
45458b3265 fix analytics logging msg 2019-05-23 14:26:03 -04:00
Christian Adams
8ca0c1b992 Add clearsessions and cleartokens system jobs
* add system job for gathering insights analytics
* enforce schedule enablement with analytics setting
* remove celery beat analytics job
* keep analytics schedule & setting enablement in sync in API
* handles updating schedules for multiple sys job schedules
* add analytics setting & schedule test
* rm ui modal from collection sys job
2019-05-22 13:11:48 -04:00
Christian Adams
0c4925afe8 rm extra collection error line 2019-05-16 14:48:41 -04:00
Christian Adams
21105b836e more verbose list_instance messaging 2019-05-09 15:08:56 -04:00
Ryan Petrello
9d000a76de properly set ansible_python_interpreter for local task execution
this works a limitation in ansible for connection: local and makes it so
that you can run connection: local playbooks with a /usr/bin/python that
is Python3 and an Ansible virtualenv that is Python2

see: https://github.com/ansible/awx/issues/3267
2019-04-16 21:24:23 -04:00
softwarefactory-project-zuul[bot]
c2daccade7 Merge pull request #3667 from chrismeyersfsu/delete-system-tracking
remove system tracking

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-16 17:24:03 +00:00
softwarefactory-project-zuul[bot]
97e2137d07 Merge pull request #3694 from ryanpetrello/gbye-callback-plugins
remove the old callback plugin import paths

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-12 21:01:34 +00:00
Ryan Petrello
17a803f49c remove the old callback plugin import paths and callback-specific tests 2019-04-12 16:11:23 -04:00
chris meyers
0c6a522813 remove system tracking
* Leave the artisanal handcrafted migration from vendored fact scan to
external fact gathering.
2019-04-12 14:30:29 -04:00
beeankha
de34a64115 Basic License feature gating changes 2019-04-12 11:28:27 -04:00
softwarefactory-project-zuul[bot]
ef1a5c09b6 Merge pull request #3510 from jbradberry/errors-on-change-password
Use Django's own logic to invalidate sessions of users when changing passwords

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-05 14:54:20 +00:00
softwarefactory-project-zuul[bot]
487473f0d1 Merge pull request #3586 from AlanCoding/update_verbosity
Set ansible-inventory verbosity for Ansible 2.8

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-03 18:51:09 +00:00
Jake McDermott
c209955400 add credential plugin system and minimal working hashivault 2019-04-02 11:19:26 -04:00
Vismay Golwala
9710015a2f Revert "Add scm_branch while preloading data" 2019-04-01 16:56:19 -04:00
softwarefactory-project-zuul[bot]
b70c354dfc Merge pull request #3575 from vismay-golwala/update_preload_data
Add scm_branch while preloading data

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-01 20:39:46 +00:00
softwarefactory-project-zuul[bot]
1bcda0a4cb Merge pull request #3529 from chrismeyersfsu/fix-inventory_update_deadlock
fixes inventory update deadlock

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-01 18:05:36 +00:00
AlanCoding
750c8d8a6d Set ansible-inventory verbosity for Ansible 2.8 2019-04-01 13:39:20 -04:00
Vismay Golwala
74e264a6a5 Add scm_branch while preloading data
Currently, the SCM url for default project is not able to
fetch playbooks as branch is not specified. So, adding the
scm_branch value resolves this issue.

Signed-off-by: Vismay Golwala <vgolwala@redhat.com>
2019-04-01 11:53:34 -04:00
Ryan Petrello
563a0cc2a4 move awx.main.expect to awx.main.isolated 2019-03-29 12:14:40 -04:00
softwarefactory-project-zuul[bot]
2ab290ff2d Merge pull request #3534 from ryanpetrello/iso-pexpect-cleanup
replace our usage of pexpect in IsolatedManager with ansible-runner

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-03-29 13:17:05 +00:00
Ryan Petrello
dd81f59d9a set PYTHONPATH properly for isolated usage of runner 2019-03-28 12:40:21 -04:00
Ryan Petrello
99478f5d25 replace our usage of pexpect in IsolatedManager with ansible-runner 2019-03-28 12:40:21 -04:00
Ryan Petrello
c586fa9821 add a minimal framework for generating analytics/metrics
annotate queries & add license analytics
2019-03-27 19:53:00 -04:00
Jeff Bradberry
a803e86a95 Log errors directly from inventory_import.py only if running by hand 2019-03-27 18:02:46 -04:00
chris meyers
cb4d55b47a fixes inventory update deadlock
* all inventory updates continue to occur in parallel up to the point
that they update the database with their results.
* the "funnel" is achieved by using a global per-inventory postgres
named lock
2019-03-27 16:52:07 -04:00
Jeff Bradberry
efb4fb6fd0 Remove some no longer used imports 2019-03-27 10:06:13 -04:00
Jeff Bradberry
f2be4de544 Use Django's own logic to invalidate sessions of users when changing passwords
The key is django.contrib.auth.update_session_auth_hash(), which knows
how to inject a recalculated session hash back into the session if the
requesting user is changing their own password, in order to keep that
user logged in.
2019-03-27 10:06:13 -04:00
AlanCoding
e097f5a021 implement playbook-dir option in ansible-inventory calls 2019-03-26 14:09:08 -04:00
AlanCoding
cd7e358b73 Inventory plugins transition dev finishing work
Bump keystone auth to resolve problem with openstack script

Clarify code path, routing to template vs. managed injector
  behavior is also now reflected in test data files

Refactor test data layout for inventory injector logic

Add developer docs for inventory plugins transition

Memoize only get_ansible_version with no parameters

Make inventory plugin injector enablement a separate
  concept from the initial_version
  switch tests to look for plugin_name as well

Add plugin injectors for tower and foreman.

Add jinja2 native types compat feature

move tower source license compare logic to management command

introduce inventory source compat mode

pin jinja2 for native Ansible types

Add parent group keys, and additional translations

manual dash sanitization for un-region-like ec2 groups

nest zones under regions using Ansible core feature just merged
  implement conditionally only with BOTH group_by options

Make compat mode default be true
  in API models, UI add and edit controllers

Add several additional hostvars to translation
Add Azure tags null case translation

Make Azure group_by key off source_vars
  to be consistent with the script

support top-level ec2 boto_profile setting
2019-03-26 10:29:39 -04:00
AlanCoding
bc5881ad21 Primary development of inventory plugins, partial compat layer
Initialize some inventory plugin test data files
Implement openstack inventory plugin

This may be removed later:
- port non-JSON line strip method from core

Dupliate effort with AWX mainline devel
- Produce ansible_version related to venv

Refactor some of injector management, moving more
  of this overhead into tasks.py, when it comes to
  managing injector kwargs

Upgrade and move openstack inventory script
  sync up parameters

Add extremely detailed logic to inventory file creation
for ec2, Azure, and gce so that they are closer to a
genuine superset of what the contrib script used to give.
2019-03-26 10:29:39 -04:00
AlanCoding
6c130fa6c3 Build-in inventory plugin code structure with gce working
supporting and related changes
 - Fix inconsistency between can_update / can_start
 - Avoid creating inventory file twice unnecessarily
 - Non-functional consolidation in Azure injection logic
 - Inject GCE creds as indented JSON for readability
 - Create new injector class structure, add gce
 - Reduce management command overrides of runtime environment
2019-03-26 10:29:35 -04:00
Ryan Petrello
602ef9750f update isolated task execution for ansible-runner 2019-03-18 14:21:47 -04:00
Jeff Bradberry
046385d72e Make the inventory_import command log an error message on exception
when it happens in the big try/except block in the middle of handle().
Previously it wasn't doing anything with it, except exiting with a
code of 1.
2019-02-28 15:54:09 -05:00
Jeff Bradberry
6d70651611 Update the inventory_import management command
to respect the new Organization.max_hosts limit.
2019-02-28 15:51:50 -05:00
softwarefactory-project-zuul[bot]
889dae357b Merge pull request #3235 from ryanpetrello/sql-profiling
add a custom DB backend that provides system-level SQL profiling

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-15 21:56:28 +00:00
Ryan Petrello
6f43875e80 record profile data in /var/log/tower, not /var/lib/awx 2019-02-15 14:34:55 -05:00
Bill Nottingham
7452eb2fa1 Remove broken user_info command.
This has not worked in a long time, and does not serve much purpose.
2019-02-14 15:34:24 -05:00
Ryan Petrello
eed94b641e add a custom DB backend that provides system-level SQL profiling
run this command on _any_ node in an awx cluster:

$ awx-manage profile_sql --threshold=2.0 --minutes=1

...and for 1 minute, the timing for _every_ SQL query in _every_ awx
Python process that uses the Django ORM will be measured

queries that run longer than (in this example) 2 seconds will be
written to a per-process sqlite database in /var/lib/awx/profile, and
the file will contain an EXPLAIN VERBOSE for the query and the full
Python stack that led to that SQL query's execution (this includes not
just WSGI requests, but background processes like the runworker and
dispatcher)

$ awx-manage profile_sql --threshold=0

...can be used to disable profiling again (if you don't want to wait for
the minute to expire)
2019-02-14 15:04:46 -05:00
AlanCoding
1bf2a455c6 Do not remove edges from other inventory sources 2019-02-11 13:08:19 -05:00
Ryan Petrello
5e84782b9c provide a better ansible-inventory fallback error message old ansibles
see: https://github.com/ansible/awx/issues/3139
2019-02-05 17:12:48 -05:00
AlanCoding
7676ccdbac use custom venv in inventory prooting 2019-02-01 13:29:45 -05:00
Ryan Petrello
daeeaf413a clean up unnecessary usage of the six library (awx only supports py3) 2019-01-25 00:19:48 -05:00
Ryan Petrello
0e5e23372d use the correct ansible-inventory executable for custom venvs 2019-01-23 14:07:48 -05:00
softwarefactory-project-zuul[bot]
416dcc83c9 Merge pull request #3020 from beeankha/fix_unicode_error
Fix unicode error

Reviewed-by: Ryan Petrello
             https://github.com/ryanpetrello
2019-01-23 15:26:07 +00:00
AlanCoding
99288a5e18 Use custom virtual environment in inventory updates 2019-01-21 06:45:55 -05:00
Bianca
bf5657a06a Update register_queue.py file to enable registration of instance groups with unicode 2019-01-17 14:36:46 -05:00