this approach totally removes the process of reading and writing stdout
files on the local file system at settings.JOBOUTPUT_ROOT when jobs are
run; now stdout content is only written on-demand as it's fetched for
the deprecated `stdout` endpoint
see: https://github.com/ansible/awx/issues/200
* introduces three new models: `ProjectUpdateEvent`,
`InventoryUpdateEvent`, and `SystemJobEvent`
* simplifies the stdout callback management in `tasks.py` - now _all_
job run types capture and emit events to the callback receiver
* supports stdout reconstruction from events for stdout downloads for
_all_ job types
* configures `ProjectUpdate` runs to configure the awx display callback
(so we can capture real playbook events for `project_update.yml`)
* ProjectUpdate, InventoryUpdate, and SystemJob runs no longer write
text blobs to the deprecated `main_unifiedjob.result_stdout_text` column
see: https://github.com/ansible/awx/issues/200
This data often (in the case of inventory updates) represents large data
blobs (5+MB per job run). Storing it on the polymorphic base class
table, `main_unifiedjob`, causes it to be automatically fetched on every
query (and every polymorphic join) against that table, which can result
in _very_ poor performance for awx across the board. Django offers
`defer()`, but it's quite complicated to sprinkle this everywhere (and
easy to get wrong/introduce side effects related to our RBAC and usage
of polymorphism).
This change moves the field definition to a separate unmanaged model
(which references the same underlying `main_unifiedjob` table) and adds
a proxy for fetching the data as needed
see https://github.com/ansible/awx/issues/200
the nature of this latest bug is that the WorkflowJob has a *different*
implementation of _accept_or_ignore_job_kwargs, and it wasn't performing
encryption for extra vars provided at launch time; this change places the
encryption mechanism in UJT.create_unified_job so that it works the same
for _all_ UJTs
see: https://github.com/ansible/ansible-tower/issues/7798
see: https://github.com/ansible/ansible-tower/issues/7046
survey_spec is a nested dict, so if we don't `deepcopy()` it, updates
to the individual fields could corrupt the original data structure;
this was causing a bug whereby activity stream updates converted
encrypted survey password defaults -> `$encrypted$`, but inadvertently
modified the originating model due to shared references
see: https://github.com/ansible/ansible-tower/issues/7769
if database connectivity is lost, callback workers currently raise an
uncaught exception and hang; this can cause the entire process to stop
handling callback events
see: https://github.com/ansible/ansible-tower/issues/7660
When Jobs and Adhoc Commands are launched, awx uses a job-scoped auth
token to dynamically fetch inventory via the awx REST API; this process
is complicated, hard to debug, and likely won't work going forward with
oauth2-based tokens in awx
see: https://github.com/ansible/awx/issues/21
with process isolation enabled (which is the awx default), cloudforms
caches inventory script results on disk; awx should direct cloudforms to
store these cache files in a location that's exposed to the isolated
environment
see: ansible/ansible#31760
with process isolation enabled (which is the awx default), cloudforms
caches inventory script results on disk; awx should direct cloudforms to
store these cache files in a location that's exposed to the isolated
environment
see: ansible/ansible#31760
instead of writing individual migrations for new built-in credential
types, this change makes the "setup_tower_managed_defaults" function
idempotent so that it only adds the credential types you're missing
django-polymorphic itself generates queries for polymorphic object
lookups, and these queries for UnifiedJob are *not* properly defering the
`result_stdout_text` column, resulting in more very slow queries. This
solution is _very_ hacky, and very specific to this specific
version of Django and django-polymorphic, but it works until we can
solve this problem the proper way in 3.3 (by removing large stdout blobs
from the database).
see: https://github.com/ansible/ansible-tower/issues/7568
result_stdout_text can be _very_ large - some customers have 5MB+ per
job; querying for this in list contexts results in _very_ large datasets
being read from the database which is very slow. It's very uncommon to
actually need this column outside of the context of job details, so
defer it.
see: https://github.com/ansible/ansible-tower/issues/7568
cache.set() and cache.get() arguments are logged when the log level is
DEBUG; this _may_ include plaintext secrets; strip sensitive values
before logging them
see: https://github.com/ansible/ansible-tower/issues/7476
the output of the `debug_tree` function is *very* verbose, and is most
useful for people who are debugging inventory import code
pexpect reads large stdout/stderr streams *very slowly*;
when verbosity is set to `DEBUG` for inventory imports, it's not
uncommon for this function to write 50MB+ of data into stderr, causing
pexpect to read over a pseudoterminal for 30+ minutes
see: https://github.com/ansible/ansible-tower/issues/7414#issuecomment-321615104
some users won't want to utilize the RSA key we auto-generate for
isolated node SSH access, but will instead want to manage SSH
authentication by hand outside of Tower
see: https://github.com/ansible/ansible-tower/issues/7380
We recently made AWX_ISOLATED_PRIVATE_KEY and AWX_ISOLATED_PUBLIC_KEY
read-only so that they're not inadvertently modified and/or deleted
(which would cause isolated task execution to break). Tower's notion of
a read-only setting, though, is really more like "hard-coded in a .py
file". What we really need is support for settings that are not
user configurable/changeable, but that we still want to display to users
from the DB.
In leiu of a complicated change to `awx.conf.settings`, this change
provides special support to these two settings so they behave in the
manner we expect.
see: https://github.com/ansible/ansible-tower/issues/7375
Ryan Petrello