Commit Graph
100 Commits
Author SHA1 Message Date
Ryan Petrello 82e41b40bb enforce strings for secret password inputs on Credentials
see: https://github.com/ansible/ansible-tower/issues/7898
2018-02-12 17:03:32 -05:00
Ryan Petrello 72715df751 fix a bug for "users should be able to change type of unused credential"
see: https://github.com/ansible/ansible-tower/issues/7516
related: https://github.com/ansible/tower/pull/441
2018-02-08 15:44:14 -05:00
Ryan Petrello 982539f444 fix a bug when testing UDP-based logging configuration
see: https://github.com/ansible/ansible-tower/issues/7868
2018-01-29 12:05:51 -05:00
Ryan Petrello d57470ce49 don't process artifacts from custom set_stat calls asynchronously
previously, we persisted custom artifacts to the database on
`Job.artifacts` via the callback receiver.  when the callback receiver
is backed up processing events, this can result in race conditions for
workflows where a playbook calls `set_stat()`, but the artifact data is
not persisted in the database before the next job in the workflow starts

see: https://github.com/ansible/ansible-tower/issues/7831
2018-01-23 17:09:23 -05:00
Ryan Petrello 2955842c44 don't overwrite env['ANSIBLE_LIBRARY'] when fact caching is enabled
see: https://github.com/ansible/awx/issues/815
see: https://github.com/ansible/ansible-tower/issues/7830
2018-01-15 13:39:46 -05:00
Ryan Petrello e1d50a43fd only allow facts to cache in the proper file system location 2018-01-15 11:45:49 -05:00
Ryan Petrello 983b192a45 replace our memcached-based fact cache implementation with local files
see: https://github.com/ansible/ansible-tower/issues/7840
2018-01-15 09:16:44 -05:00
Ryan Petrello 563f730268 add support for new "BECOME" prompt in Ansible 2.5+
see: https://github.com/ansible/ansible-tower/issues/7850
2018-01-12 10:40:40 -05:00
Ryan Petrello b8758044e0 fix a bug in inventory generation for isolated nodes
see: https://github.com/ansible/ansible-tower/issues/7849
related: https://github.com/ansible/awx/pull/551
2018-01-11 10:41:58 -05:00
Ryan Petrello 4c40791d06 properly handle unicode for isolated job buffers
from: https://docs.python.org/2/library/stringio.html#module-cStringIO
"Unlike the StringIO module, this module is not able to accept Unicode
strings that cannot be encoded as plain ASCII strings."

see: https://github.com/ansible/ansible-tower/issues/7846
2018-01-10 10:56:59 -05:00
Ryan Petrello ce94ba4c83 change how we detect the current user to avoid a nuanced recursion error
see: https://github.com/ansible/ansible-tower/issues/7802
2017-12-08 15:35:47 -05:00
Ryan Petrello 1fb890f4eb fix a race condition in "host.last_job" when jobs are deleted
see: https://github.com/ansible/ansible-tower/issues/7815
2017-12-06 11:30:19 -05:00
Ryan Petrello fe93ef5488 more unit tests for survey default handling 2017-12-05 09:04:57 -05:00
Ryan Petrello 33dedc88c8 don't fetch stdout when purging jobs - it's slow and causes OOMs
see: https://github.com/ansible/ansible-tower/issues/7751
2017-12-04 15:14:55 -05:00
Ryan Petrello 112757e202 properly handle JT launch POST for required survey fields w/ no default
see: ansible/ansible-tower#7805
2017-12-04 09:45:21 -05:00
Ryan Petrello 12380fe1b1 add more tests for survey default encryption
see: https://github.com/ansible/ansible-tower/issues/7805
see: https://github.com/ansible/ansible-tower/issues/7806
see: https://github.com/ansible/ansible-tower/issues/7807
2017-12-04 09:45:14 -05:00
Ryan Petrello 6c7851b51f fix another survey encryption-related bug 2017-12-01 17:11:00 -05:00
Ryan Petrello 1057b93570 refactor survey spec validation into a separate testable function 2017-12-01 15:34:07 -05:00
Ryan Petrello 8c6a1e348d upgrade survey encryption migration to work around an old survey bug
see: https://github.com/ansible/ansible-tower/issues/7800
2017-12-01 11:34:47 -05:00
Ryan Petrello a3dd9eb4b7 removing some cruft we thought would help us catch bugs (it didn't) 2017-11-29 16:23:07 -05:00
Ryan Petrello 68ada92f3b more survey password encryption bug squashing
the nature of this latest bug is that the WorkflowJob has a *different*
implementation of _accept_or_ignore_job_kwargs, and it wasn't performing
encryption for extra vars provided at launch time; this change places the
encryption mechanism in UJT.create_unified_job so that it works the same
for _all_ UJTs

see: https://github.com/ansible/ansible-tower/issues/7798
see: https://github.com/ansible/ansible-tower/issues/7046
2017-11-29 14:40:41 -05:00
Ryan Petrello 1425021106 fix another encrypted survey password bug
properly encrypt extra_vars that overlap with survey passwords when
`ask_variables_on_launch=true`

see: https://github.com/ansible/ansible-tower/issues/7796
2017-11-28 16:52:47 -05:00
Ryan Petrello 26845642f0 fix a bug which caused v1 cred backwards-compat to apply to v2 requests
see: https://github.com/ansible/ansible-tower/issues/7793
2017-11-28 13:05:13 -05:00
Ryan Petrello 6fa0d9d4ed rename oVirt4 to Red Hat Virtualization
see: https://github.com/ansible/ansible-tower/issues/7790
2017-11-28 11:02:42 -05:00
Ryan Petrello 044c047ac6 fix a bug in survey password default validation
see: https://github.com/ansible/ansible-tower/issues/7046
see: https://github.com/ansible/ansible-tower/issues/7764
see: https://github.com/ansible/ansible-tower/issues/7784
2017-11-27 17:25:45 -05:00
Ryan Petrello a80d5b1b39 reencrypt WFJT.survey_spec too
https://github.com/ansible/ansible-tower/issues/7046
2017-11-16 23:00:22 -05:00
Ryan Petrello 4187d02b8a make settings.AWX_ISOLATED_KEY_GENERATION readonly
see: https://github.com/ansible/ansible-tower/issues/7380
2017-11-16 13:35:37 -05:00
Ryan Petrello 35c374fc79 fix a bug in the survey reencryption migration
see: https://github.com/ansible/ansible-tower/issues/7046
2017-11-15 10:26:46 -05:00
Ryan Petrello be00b1ca96 render survey_spec for display purposes in a safe manner
survey_spec is a nested dict, so if we don't `deepcopy()` it, updates
to the individual fields could corrupt the original data structure;
this was causing a bug whereby activity stream updates converted
encrypted survey password defaults -> `$encrypted$`, but inadvertently
modified the originating model due to shared references

see: https://github.com/ansible/ansible-tower/issues/7769
2017-11-13 13:01:56 -05:00
Ryan Petrello 78961c8037 properly perform validation on encrypted survey defaults
see: https://github.com/ansible/ansible-tower/issues/7764
2017-11-10 10:52:09 -05:00
Ryan Petrello 91ad0a9f89 add a useful tool for isolated connectivity debugging 2017-11-08 09:27:33 -05:00
Ryan Petrello 8d2ab3de42 improve the callback worker's ability to deal with idle/disconnected DB
if database connectivity is lost, callback workers currently raise an
uncaught exception and hang; this can cause the entire process to stop
handling callback events

see: https://github.com/ansible/ansible-tower/issues/7660
2017-10-31 09:51:13 -04:00
Ryan Petrello c95d7d465a work around an ansible bug that can cause project syncs to fail
https://github.com/ansible/ansible-tower/issues/7746
https://github.com/ansible/ansible/issues/30064
2017-10-25 11:09:39 -04:00
Ryan Petrello b60a30cbd4 fix a unicode handling bug in inventory source name migration
see: https://github.com/ansible/ansible-tower/issues/7740
2017-10-20 12:13:44 -04:00
Ryan Petrello c1e20fe7a0 properly support authentication for ovirt4 ansible modules
see: https://github.com/ansible/ansible-tower/issues/6522
see: https://github.com/ansible/ansible-tower/issues/6522#issuecomment-337909863
2017-10-19 09:47:25 -04:00
Ryan Petrello 10a7544d68 properly follow symlinks for bwrap'd working directories
see: https://github.com/ansible/ansible-tower/issues/7741
2017-10-18 17:03:10 -04:00
Ryan Petrello 5895654538 store cloudforms inventory cache files in the proper location on disk
with process isolation enabled (which is the awx default), cloudforms
caches inventory script results on disk; awx should direct cloudforms to
store these cache files in a location that's exposed to the isolated
environment

see: ansible/ansible#31760
2017-10-17 17:07:21 -04:00
Ryan Petrello 82d05e0a10 properly sanitize encrypted default passwords in JT.survey_spec
see: https://github.com/ansible/ansible-tower/issues/7259
2017-10-10 14:38:47 -04:00
Ryan Petrello b372cebf8d fix a bug when Tower is integrated with ipsilon SAML server
https://github.com/ansible/ansible-tower/issues/6683
2017-10-10 14:38:46 -04:00
Ryan Petrello f26bdb3e96 migrate existing survey passwords to be encrypted
see: https://github.com/ansible/ansible-tower/issues/7046
2017-10-10 14:38:46 -04:00
Ryan Petrello 4be4e3db7f encrypt job survey data
see: https://github.com/ansible/ansible-tower/issues/7046
2017-10-10 14:38:46 -04:00
Ryan Petrello a0cfbb93e9 fix busted 3.2.2 activity stream migration
see: ansible/ansible-tower#7704
2017-10-10 14:38:45 -04:00
Ryan Petrello f4a252a331 add new credential types in a more stable way in migrations
instead of writing individual migrations for new built-in credential
types, this change makes the "setup_tower_managed_defaults" function
idempotent so that it only adds the credential types you're missing
2017-10-10 14:38:45 -04:00
Ryan Petrello e06d4d7734 don't install pycurl from pypi; use a system package instead
the ovirt4 sdk relies on pycurl, which is complicated to install w/ pip;
rely on pycurl to be provided by a system package instead
2017-10-10 14:38:44 -04:00
Ryan Petrello 7438062b97 add ovirt sdk dependency for ovirt4 support 2017-10-10 14:38:44 -04:00
Ryan Petrello 74f2509482 support ovirt4 as a built-in inventory source
see: https://github.com/ansible/ansible-tower/issues/6522
2017-10-10 14:38:44 -04:00
Ryan Petrello 94b4dabee2 disable GCE inventory caching w/ a .ini file
see: https://github.com/ansible/ansible-tower/issues/7609
see: https://github.com/ansible/tower/pull/451#pullrequestreview-64454393
2017-10-10 14:38:44 -04:00
Ryan Petrello 94d44e8791 disable GCE inventory source cache
by default, the GCE inventory script caches results on disk for
5 minutes; disable this behavior

see: https://github.com/ansible/ansible-tower/issues/7609
2017-10-10 14:38:44 -04:00
Ryan Petrello 7a21a45781 properly encode LDAP DN values on validation
see: https://github.com/ansible/ansible-tower/issues/7554
2017-10-10 14:38:43 -04:00
Ryan Petrello c8f4320b58 allow the credential type to be changed for unused credentials
see: https://github.com/ansible/ansible-tower/issues/7607
2017-10-10 14:38:43 -04:00
Ryan Petrello 96572fe3d4 don't show polymorphic_ctype in unique validation error messaging
see: https://github.com/ansible/ansible-tower/issues/7620
2017-10-10 14:38:43 -04:00
Ryan Petrello 554a9586c6 add awx meta variables to adhoc command extra_vars
see: https://github.com/ansible/ansible-tower/issues/7513
2017-10-10 14:38:37 -04:00
Ryan Petrello f2f42c2c8a don't append to the activity stream on LDAP group disassociate
for organizations w/ a large number of ldap orgs/teams, this results in
a _huge_ number of extraneous activity stream entries

see: https://github.com/ansible/ansible-tower/issues/7655
2017-10-10 14:38:18 -04:00
Ryan Petrello 4c5ec2fb3a work around an ansible 2.4 inventory caching bug
see: https://github.com/ansible/awx/issues/246
2017-10-03 15:45:11 -04:00
Ryan Petrello 328b70ceb8 fix a bug that causes azure inventory syncs w/ region='all' to fail
see: https://github.com/ansible/ansible-tower/issues/7668
2017-09-21 15:08:54 -04:00
Ryan Petrello 1af9c43b5b Fix an issue not populating the version metadata for isolated sdist
see: https://github.com/ansible/ansible-tower/issues/6810
2017-09-19 10:25:21 -04:00
Ryan Petrello cfe1f1e8e4 more legacy azure deprecation cleanup 2017-09-18 10:43:16 -04:00
Ryan Petrello 14b0f9aa24 remove reference to legacy rax credentials 2017-09-18 10:43:02 -04:00
Ryan Petrello 4dd265633e remove legacy azure inventory script
see: ansible/ansible-tower#7629
2017-09-18 10:41:49 -04:00
Ryan Petrello 68b924efe5 flake8 fixup 2017-09-18 09:44:39 -04:00
Ryan Petrello 6a4b4edea3 properly detect deferred ORM objects 2017-09-18 09:08:53 -04:00
Ryan Petrello 7a958a1af1 more result_stdout_text defer optimization for inventory updates
see: https://github.com/ansible/ansible-tower/issues/7568
2017-09-15 09:51:53 -04:00
Ryan Petrello 1bb6c17fe2 trick django-polymorphic into allowing defer() on polymorphic objects
django-polymorphic itself generates queries for polymorphic object
lookups, and these queries for UnifiedJob are *not* properly defering the
`result_stdout_text` column, resulting in more very slow queries.  This
solution is _very_ hacky, and very specific to this specific
version of Django and django-polymorphic, but it works until we can
solve this problem the proper way in 3.3 (by removing large stdout blobs
from the database).

see: https://github.com/ansible/ansible-tower/issues/7568
2017-09-15 09:51:43 -04:00
Ryan Petrello d3df5de0ce build extra_cred related urls for jobs and JTs a less volatile way
see: https://github.com/ansible/ansible-tower/issues/7635
2017-09-13 13:28:26 -04:00
Ryan Petrello bd42dfe474 defer UnifiedJob.result_stdout_text for improved performance
result_stdout_text can be _very_ large - some customers have 5MB+ per
job; querying for this in list contexts results in _very_ large datasets
being read from the database which is very slow.  It's very uncommon to
actually need this column outside of the context of job details, so
defer it.

see: https://github.com/ansible/ansible-tower/issues/7568
2017-09-12 16:35:49 -04:00
Ryan Petrello 4213960ec3 write the scm_revision_output to the project path instead of /tmp
see: https://github.com/ansible/ansible-tower/issues/7558
2017-09-11 17:44:53 -04:00
Ryan Petrello a9c9ecb5ea bind ansible and awx virtualenvs readonly so that jobs can't modify them
see: https://github.com/ansible/ansible-tower/issues/7558
2017-09-11 15:57:35 -04:00
Ryan Petrello a2ca0e6012 add process isolation to project updates
see: https://github.com/ansible/ansible-tower/issues/7506
2017-09-11 15:57:28 -04:00
Ryan Petrello 4cc58a221b fix busted conf unit tests 2017-09-11 11:28:43 -04:00
Ryan Petrello 768c7ba3dc bump azurerm dependencies to support Ansible 2.4
see: https://github.com/ansible/ansible-tower/issues/7470
2017-09-01 15:15:53 -04:00
Ryan Petrello 7192d5c4bb include extra_credentials data in summary fields for JT/Job detail view
see: https://github.com/ansible/ansible-tower/issues/7521
2017-08-30 11:56:24 -04:00
Ryan Petrello beae7ffa20 update the license check command to return license type, not validity
we care about preventing upgradability from AWX installs w/ the open
source license, not general license validity

see: https://github.com/ansible/ansible-tower/issues/6555
2017-08-29 15:25:32 -04:00
Ryan Petrello 0b68ad9b10 properly sanitize conf.settings debug logs
cache.set() and cache.get() arguments are logged when the log level is
DEBUG; this _may_ include plaintext secrets; strip sensitive values
before logging them

see: https://github.com/ansible/ansible-tower/issues/7476
2017-08-21 10:23:39 -04:00
Ryan Petrello b0a1988c29 add required fields for cloudforms credentials
see: https://github.com/ansible/ansible-tower/issues/7462
2017-08-18 16:18:06 -04:00
Ryan Petrello bcd8e13c24 add required fields for gce credentials
see: https://github.com/ansible/ansible-tower/issues/7463
2017-08-18 16:18:06 -04:00
Ryan Petrello fc73bdcc18 add required fields for azure classic credentials
https://github.com/ansible/ansible-tower/issues/7464
2017-08-18 16:18:05 -04:00
Ryan Petrello 5ba76f28ce add required fields for azure credentials
see: https://github.com/ansible/ansible-tower/issues/7465
2017-08-18 16:18:05 -04:00
Ryan Petrello 90b5d98e5c add required fields for network credentials
see: https://github.com/ansible/ansible-tower/issues/7466
2017-08-18 16:18:04 -04:00
Ryan Petrello 50782b9465 add required fields for RHSatellite6 credentials
see: https://github.com/ansible/ansible-tower/issues/7467
2017-08-18 16:18:04 -04:00
Ryan Petrello 438d41c986 make vault_password required for Vault credentials
see: https://github.com/ansible/ansible-tower/issues/7468
2017-08-18 14:10:19 -04:00
Ryan Petrello 9d931e7f7e don't run debug_tree for production inventory imports
the output of the `debug_tree` function is *very* verbose, and is most
useful for people who are debugging inventory import code

pexpect reads large stdout/stderr streams *very slowly*;
when verbosity is set to `DEBUG` for inventory imports, it's not
uncommon for this function to write 50MB+ of data into stderr, causing
pexpect to read over a pseudoterminal for 30+ minutes

see: https://github.com/ansible/ansible-tower/issues/7414#issuecomment-321615104
2017-08-10 16:25:37 -04:00
Ryan Petrello 61aa7c1ad7 return proper stdout size for DB-backed stdout (like system jobs)
see: https://github.com/ansible/ansible-tower/issues/7398
2017-08-07 17:08:25 -04:00
Ryan Petrello 4bc5858ab4 replace PyCrypto usage w/ crytography; ansible no longer bundles it
see: https://github.com/ansible/ansible-tower/issues/7395
2017-08-07 12:19:51 -04:00
Ryan Petrello b4d69cb5c7 don't delete settings that are marked as read_only 2017-08-04 10:06:35 -04:00
Ryan Petrello 7db9b48e9c add a configurable for disabling the auto-generated isolated RSA key
some users won't want to utilize the RSA key we auto-generate for
isolated node SSH access, but will instead want to manage SSH
authentication by hand outside of Tower

see: https://github.com/ansible/ansible-tower/issues/7380
2017-08-03 17:16:28 -04:00
Ryan Petrello b34adcb10d add special case read-only support for isolated public/private keys
We recently made AWX_ISOLATED_PRIVATE_KEY and AWX_ISOLATED_PUBLIC_KEY
read-only so that they're not inadvertently modified and/or deleted
(which would cause isolated task execution to break).  Tower's notion of
a read-only setting, though, is really more like "hard-coded in a .py
file".  What we really need is support for settings that are not
user configurable/changeable, but that we still want to display to users
from the DB.

In leiu of a complicated change to `awx.conf.settings`, this change
provides special support to these two settings so they behave in the
manner we expect.

see: https://github.com/ansible/ansible-tower/issues/7375
2017-08-03 16:41:23 -04:00
Ryan Petrello d09872d13a fix a bug which breaks inventory update stdout downloads
see: https://github.com/ansible/ansible-tower/issues/7363
see: https://github.com/ansible/ansible-tower/issues/7337
2017-08-01 10:48:40 -04:00
Ryan Petrello 63273d8ffd filter Python unicode string prefix from jsonschema validation errors.
see: https://github.com/ansible/ansible-tower/issues/6417
2017-07-31 15:22:19 -04:00
Ryan Petrello 147d795fa0 force python-ldap to create a new SSL context for OPT_X_TLS_REQUIRE_CERT
according to the internet, this is a thing that is necessary for
python-ldap:

https://github.com/rbarrois/python-ldap/commit/71024518dbc9c539a5bab834080e3792ac4dbfe9#diff-1a5a32c6cc1ae9767cb2008703e4534fR41
https://stackoverflow.com/a/29722445
https://stackoverflow.com/a/38136255

see: https://github.com/ansible/ansible-tower/issues/7335
2017-07-31 13:59:36 -04:00
Ryan Petrello 5e15f9e04e add validation errors for certain dependent credential fields
see: https://github.com/ansible/ansible-tower/issues/7323
see: https://github.com/ansible/ansible-tower/issues/7293
see: https://github.com/ansible/ansible-tower/issues/7289
see: https://github.com/ansible/ansible-tower/issues/7292
2017-07-28 16:07:37 -04:00
Ryan Petrello 568986b39f force isolated timeout settings to be positive integers
see: https://github.com/ansible/ansible-tower/issues/7328
2017-07-28 12:24:21 -04:00
Ryan Petrello ad66ff9938 pin kombu to a known working version (from tower 3.1.4)
we're encountering some *really* gnarly bugs with newer version of
celery and kombu; pin to *exactly* what works in 3.1.4 to try to avoid
them, because we can't upgrade celery to 4.0 (where this bug *may* be
fixed)

see: https://github.com/ansible/ansible-tower/issues/6534
see: https://github.com/celery/celery/issues/3712
see: https://github.com/celery/kombu/pull/760
2017-07-28 10:59:06 -04:00
Ryan Petrello 60da24d82f fix a few activity stream bugs related to setting creation/update
* when a setting is created, only create *one* activity stream record
  for the creation, not one for create and another for update (similar
  to https://github.com/ansible/tower/pull/53)
* add code to hide `$encrypted$` activity stream content

see: https://github.com/ansible/ansible-tower/issues/7320
2017-07-27 11:51:10 -04:00
Ryan Petrello cb85038976 filter credential_type__search from related search fields in API v1
see: https://github.com/ansible/ansible-tower/issues/6116
2017-07-26 16:42:16 -04:00
Ryan Petrello c71e8d38b7 improve private passphrase error when it's provided unnecessarily
see: https://github.com/ansible/ansible-tower/issues/7293
2017-07-26 15:04:10 -04:00
Ryan Petrello 2c2e5cadbf don't require a credential for job launch if vault is specified
see: https://github.com/ansible/ansible-tower/issues/7310
2017-07-26 14:35:06 -04:00
Ryan Petrello d3796e81ba disable extraneous activity stream messages for credential creation
see: https://github.com/ansible/ansible-tower/issues/7257
2017-07-25 15:47:55 -04:00
Ryan Petrello 0ce3152e6f fix busted test runs 2017-07-25 12:47:46 -04:00
Ryan Petrello e29492a259 more tower -> awx for task execution and isolated tooling 2017-07-25 10:36:06 -04:00
Ryan Petrello 8ce1421c6a fix tower-expect -> awx-expect for isolated tower builds 2017-07-24 16:03:58 -04:00