External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-09-16 00:40:11 -02:30
Code Issues Packages Projects Releases Wiki Activity
25,267 Commits 63 Branches 102 Tags
8cdd42307cf520ee6a29a3bd3c16e03f77f3bd1a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ryan Petrello 8cdd42307c clarify that logging username/password is only valid for HTTP/s 2020-04-13 11:44:00 -04:00
Ryan Petrello 269558876e only use a basic auth password for external logging if username is set 2020-04-13 11:44:00 -04:00
Ryan Petrello bba680671b when writing the rsyslog config, do it post-commit 
there's a race condition if we do this pre-commit where the correct
value isn't actually *persisted* to the database yet, and we end up
saving the *prior* setting values
 2020-04-13 11:44:00 -04:00
Ryan Petrello f70a76109c make rsyslog fall back to no-op if logging is disabled 2020-04-13 11:44:00 -04:00
Ryan Petrello f7dac8e68d more external logging unit test fixups 2020-04-13 11:44:00 -04:00
Ryan Petrello 39648b4f0b fix up a few test and lint errors related to external logging 2020-04-13 11:44:00 -04:00
Ryan Petrello ce82b87d9f rsyslog hardening (fixing a few weird things we noticed) 2020-04-13 11:44:00 -04:00
Ryan Petrello 589d27c88c POC: replace our external log aggregation feature with rsyslog 
- this change adds rsyslog (https://github.com/rsyslog/rsyslog) as
  a new service that runs on every AWX node (managed by supervisord)
  in particular, this feature requires a recent version (v8.38+) of
  rsyslog that supports the omhttp module
  (https://github.com/rsyslog/rsyslog-doc/pull/750)
- the "external_logger" handler in AWX is now a SysLogHandler that ships
  logs to the local UDP port where rsyslog is configured to listen (by
  default, 51414)
- every time a LOG_AGGREGATOR_* setting is changed, every AWX node
  reconfigures and restarts its local instance of rsyslog so that its
  fowarding settings match what has been configured in AWX
- unlike the prior implementation, if the external logging aggregator
  (splunk/logstash) goes temporarily offline, rsyslog will retain the
  messages and ship them when the log aggregator is back online
- 4xx or 5xx level errors are recorded at /var/log/tower/external.err
 2020-04-13 11:43:59 -04:00
Ryan Petrello 9843e21632 skip non-files when consuming events synced from isolated hosts 
see: https://github.com/ansible/awx/issues/6675
 2020-04-13 10:14:10 -04:00
Ryan Petrello 80147acc1c work around redis connection failures in the callback receiver 
if redis stops/starts, sometimes the callback receiver doesn't recover
without a restart; this fixes that
 2020-04-09 15:38:03 -04:00
Ryan Petrello 6b4219badb more ansible runner isolated cleanup 
follow-up to https://github.com/ansible/awx/pull/6296
 2020-04-08 01:18:05 -04:00
Ryan Petrello 0bf1116ef8 properly respect REFRESH_TOKEN_EXPIRE_SECONDS when generating new tokens 
see: https://github.com/ansible/awx/issues/6630
see: https://github.com/jazzband/django-oauth-toolkit/issues/746
 2020-04-07 11:34:01 -04:00
Ryan Petrello 771ca2400a don't send tags to the Grafana annotations API if none are specified 
see: https://github.com/ansible/awx/issues/6580
 2020-04-06 15:47:48 -04:00
Ryan Petrello 2cefba6f96 properly serialize external logs that contain decimal.Decimal objects 2020-04-06 14:24:24 -04:00
Ryan Petrello 014c995a8f fix a typo in the credentials UI 
this is causing 400 level errors for some users
 2020-04-06 10:45:33 -04:00
Ryan Petrello 6bd5053ae8 remove the limitation on (very) old DTSTART values for schedules 2020-04-03 10:59:35 -04:00
Ryan Petrello 8b00b8c9c2 remove deprecated legacy manual inventory source support 
see: https://github.com/ansible/awx/issues/6309
 2020-04-03 10:54:43 -04:00
Ryan Petrello 9fe2211f82 get rid of the activity stream middleware 
it has bugs and is very confusing

see: https://github.com/ansible/tower/issues/4037
 2020-04-01 16:02:42 -04:00
Ryan Petrello f9b8a69f7b remove some unused code from the redis rewrite 2020-04-01 08:03:59 -04:00
Ryan Petrello 4423e6edae update to the latest twisted to address two open CVEs 2020-03-31 13:47:56 -04:00
Ryan Petrello 42336355bb bump django-extensions version to address a bug in shell_plus 
see: https://github.com/ansible/awx/pull/6441
see: https://github.com/django-extensions/django-extensions/commit/e8d5daa06e5b6419e54d6c925f86a0adbe58ce8b
 2020-03-31 13:39:13 -04:00
Ryan Petrello d19a9db523 detect event migration tables in a less noisy way 
see: https://github.com/ansible/awx/issues/6493
 2020-03-31 00:05:30 -04:00
Ryan Petrello 140dbbaa7d disable host key checking when installing galaxy roles/collections 
see: https://github.com/ansible/awx/issues/5947
 2020-03-30 17:03:14 -04:00
Ryan Petrello 831bf9124f prevent manual updates at POST /api/v2/inventory_sources/N/update/ 
see: https://github.com/ansible/awx/issues/6309
 2020-03-30 15:35:04 -04:00
Ryan Petrello 32627ce51a promote AWX CLI installation instructions to the global INSTALL.md 
a few users have had trouble finding these instructions, so let's move
them into the top level installation docs
 2020-03-30 11:46:10 -04:00
Ryan Petrello 04dd1352c9 prevent ws group subscription if not specified in the valid format 2020-03-27 17:13:21 -04:00
Ryan Petrello ea54815e6b fix a bug that broke awx config 
see: https://github.com/ansible/tower/issues/4206
 2020-03-27 17:07:48 -04:00
Ryan Petrello 6914213aa0 bump version 10.0.0 2020-03-27 12:51:18 -04:00
Ryan Petrello 301d6ff616 make the job event bigint migration chunk size configurable 2020-03-27 09:28:10 -04:00
Ryan Petrello c8044b4755 migrate event table primary keys from integer to bigint 
see: https://github.com/ansible/awx/issues/6010
 2020-03-26 15:54:38 -04:00
Ryan Petrello 90d35f07f3 clarify some documentation on summary_fields 2020-03-26 14:54:28 -04:00
Ryan Petrello 464db28be5 update to the latest version of ansible-runner 2020-03-26 14:49:45 -04:00
Ryan Petrello 61a0d1f77b fix busted shell_plus in the development environment 
for some reason (unsure why), django-extensions has begun noticing
ipython importability and treating "shell_plus" as "start an IPython
notebook by default

it could be that this is a bug in django-extensions that will be fixed
soon, but for now, this fixes the issue
 2020-03-26 13:37:13 -04:00
Ryan Petrello b755fa6777 update social-auth-core to address a GitHub API deprecation 2020-03-25 12:17:36 -04:00
Ryan Petrello 8f1db173c1 remove a bunch of RabbitMQ references 2020-03-24 18:46:58 -04:00
Ryan Petrello 68b0b40e91 update Django to address CVE-2020-9402 
we don't use Oracle GIS, so this isn't really applicable, but it'll make
security scanners happy <shrug>

see: https://docs.djangoproject.com/en/3.0/releases/2.2.11/
 2020-03-24 16:41:53 -04:00
Ryan Petrello 65cafa37c7 pin a minimum pyyaml version to address (CVE-2017-18342) 
see: https://github.com/ansible/awx/issues/6393
 2020-03-24 15:59:31 -04:00
Ryan Petrello 51ef57188c update to the latest version of ansible-runner 2020-03-24 10:01:17 -04:00
Ryan Petrello d40a5dec8f change when we send job notifications to avoid a race condition 
success/failure notifications for *playbooks* include summary data about
the hosts in based on the contents of the playbook_on_stats event

the current implementation suffers from a number of race conditions that
sometimes can cause that data to be missing or incomplete; this change
makes it so that for *playbooks* we build (and send) the notification in
response to the playbook_on_stats event, not the EOF event
 2020-03-19 10:01:52 -04:00
Ryan Petrello 06b3e54fb1 remove python2 support from awxkit 2020-03-19 09:02:39 -04:00
Ryan Petrello db7f0f9421 Merge pull request #6034 from chrismeyersfsu/pg2_no_pubsub 
Replace rabbitmq with redis
 2020-03-18 17:19:51 -04:00
Ryan Petrello f1ee963bd0 fix up rebased migrations 2020-03-18 16:19:04 -04:00
Ryan Petrello 7c3cbe6e58 add a license for redis-cli 2020-03-18 16:10:20 -04:00
Ryan Petrello 1caa2e0287 work around a limitation in postgres notify to properly support copying 
postgres has a limitation on its notify message size (8k), and the
messages we generate for deep copying functionality easily go over this
limit; instead of passing a giant nested data structure across the
message bus, this change makes it so that we temporarily store the JSON
structure in memcached, and look it up from *within* the task

see: https://github.com/ansible/tower/issues/4162
 2020-03-18 16:10:20 -04:00
Ryan Petrello b73e8d8a56 fix a bug in isolated event handling 
see: https://github.com/ansible/awx/issues/6280
 2020-03-16 13:15:10 -04:00
Ryan Petrello 27b48fe55b make User.last_login read_only=True in its serializer 2020-03-13 12:53:40 -04:00
Ryan Petrello f8818730d4 consolidate isolated event handling code into one function 
make the non-isolated *and* isolated event handling share the same
function so we don't regress on behavior between the two
 2020-03-13 10:05:48 -04:00
Ryan Petrello 88571f6dcb save approval node start time *before* sending "started" notifications 
see: https://github.com/ansible/awx/issues/6267
 2020-03-12 14:14:56 -04:00
Ryan Petrello 36078651d3 bump version to 9.3.0 2020-03-11 14:34:27 -04:00
Ryan Petrello c95624e27f optimize the SCM URL sanitizer regex 
\w+ is too greedy for large strings that don't contain URLs
 2020-03-11 10:10:35 -04:00
Ryan Petrello ad1a7fc9c9 add RBAC tests for schedules on inventory and project updates 2020-03-10 13:45:57 -04:00
Ryan Petrello 35f414ccf2 clarify how AWX stores cerdentials 2020-03-06 16:00:41 -05:00
Ryan Petrello 862de0b6f3 record a start time for WorkflowApproval jobs 
see: https://github.com/ansible/awx/issues/6202
 2020-03-06 14:16:27 -05:00
Ryan Petrello acc34c1393 fix global schedule creation for project and inventory updates too 
related: https://github.com/ansible/awx/pull/6193
 2020-03-06 08:55:13 -05:00
Ryan Petrello cd1ff6b16a fix a bug in OPTIONS /api/v2/schedules/ 
a side effect of this bug is that `awx schedules create` doesn't work
properly for non-admin users (i.e., users who have execute access for
a JT)

see: https://github.com/ansible/awx/issues/5717
 2020-03-05 14:43:54 -05:00
Ryan Petrello b5e5fea117 respect home directory symlinks for BASE_VENV_PATH 2020-03-03 16:47:38 -05:00
Ryan Petrello e232cd392c make fact saving code more robust to unexpected fact data 
see: https://github.com/ansible/awx/issues/5935
 2020-03-03 16:38:33 -05:00
Ryan Petrello e7f36eb2ea remove a few custom notification fields that don't work 
these aren't top-level serializer fields; they're summary fields
if we want to support these properly, we should treat them as
enhancements, and write support, tests, and documentation
 2020-03-03 15:20:41 -05:00
Ryan Petrello 32ef805e23 properly support job host summary data in custom notification templates 
see: https://github.com/ansible/tower/issues/4148
 2020-03-03 15:00:41 -05:00
Ryan Petrello 6207dad226 fix broken grafana notifications 
since the custom notification template refactor, grafana notification
support has been broken; this is largely because grafana functions more
like the webhooks, and needs to send JSON in its notification body

see: https://github.com/ansible/awx/issues/6137
 2020-03-03 12:44:28 -05:00
Ryan Petrello 3e0e4b6c8f cli: add the ability to specify a token description w/ awx login 
see: https://github.com/ansible/awx/issues/6122
 2020-02-28 11:12:26 -05:00
Ryan Petrello 5364e78397 switch the periodic scheduler to a child process (instead of a thread) 
I have a hunch that our usage of a daemon thread is causing import lock
contention related to https://github.com/ansible/awx/issues/5617
We've encountered similar issues before with threads across dispatcher
processes at fork time, and cpython has had bugs like this in recent
history:

https://bugs.python.org/issue38884

My gut tells me this might be related.

The prior implementation - based on celerybeat - ran its code in
a process (not a thread), and the timing of that merge matches the
period of time we started noticing issues.

Currently testing it to see if it resolves some of the issues we're
seeing.
 2020-02-27 12:15:15 -05:00
Ryan Petrello 946d643795 cli: make launch with --monitor return code respect the final job status 
see: https://github.com/ansible/awx/issues/5920
see: https://github.com/ansible/awx/issues/6079
 2020-02-26 12:27:06 -05:00
Ryan Petrello eaa766df77 update Django to address a few open CVEs 
https://github.com/advisories/GHSA-hmr4-m2h5-33qx
https://github.com/advisories/GHSA-vfq6-hq5r-27r6
 2020-02-26 08:18:14 -05:00
Ryan Petrello 8b1806d4ca add code for detecting (and killing) a hung task manager task 2020-02-26 07:53:04 -05:00
Ryan Petrello 544d4cd3b0 add translations from memsource 2020-02-25 12:24:24 -05:00
Ryan Petrello 2e3547d5cf translate a missing string in the license UI 
see: https://github.com/ansible/tower/issues/3949
 2020-02-24 09:00:07 -05:00
Ryan Petrello 46fceb03a5 scope counter/start/end line updates to the current job for firehose.py 2020-02-19 16:15:33 -05:00
Ryan Petrello 3a95114c3a properly configure AWX loggers for openshift installs 
see: https://github.com/ansible/tower/issues/3793
 2020-02-18 12:50:46 -05:00
Ryan Petrello 90cb02e0bf fix start/end line incrementing behavior 2020-02-18 11:31:05 -05:00
Ryan Petrello 717698b659 properly inherit JT fields when creating many jobs with firehose.py 2020-02-18 10:34:51 -05:00
Ryan Petrello 4d06c812e6 add the ability to load lots of jobs with firehose.py 
$ awx-python tools/scripts/firehose.py --jobs 5000000 --events 100000000
 2020-02-18 08:55:06 -05:00
Ryan Petrello 92cc597e84 set actual counter/start/end values in the event generation script 2020-02-12 10:22:31 -05:00
Ryan Petrello b8ec3104a9 bump version to 9.2.0 2020-02-11 14:07:02 -05:00
Ryan Petrello 487343a022 include credential type details in the webhook credential summary field 
see: https://github.com/ansible/awx/issues/5882
 2020-02-11 10:51:07 -05:00
Ryan Petrello 38a08d163c get rid of celery/celerybeat 
alternative to https://github.com/ansible/awx/pull/2530 which makes use
of https://pypi.org/project/schedule/

this doesn't have support for any persistence (like how celery beat uses
a shelve file), because all of our periodic jobs run at most every few
minutes
 2020-02-10 17:32:02 -05:00
Ryan Petrello ad5d0b92db pin virtualenv < 20 for ansible venv builds 
virtualenv version 20 just got released and broken a bunch of stuff
(like the --system-site-packages flag)
 2020-02-10 16:31:34 -05:00
Ryan Petrello b08e5db267 update the changelog in anticipation of a forthcoming AWX release 2020-02-07 17:12:38 -05:00
Ryan Petrello 543a87ac88 fix error message on initial awx-manage migrate 2020-02-06 06:37:05 -05:00
Ryan Petrello 1800b49822 fix a py2/py3 compat bug in the settings CLI 2020-02-03 11:37:39 -05:00
Ryan Petrello 7055460c4c fix broken project update secret filtering for external logging 2020-02-03 10:27:31 -05:00
Ryan Petrello 326ed22efe properly handle import errors in the isolated capacity healthcheck 
if the awx_capacity module runs on an isolated node with missing
libraries (i.e., psutil) or bad permissions, then the runner status will
be "failed"

in this scenario, we *still* want to react by recording a capacity=0
 2020-01-31 10:17:20 -05:00
Ryan Petrello c7c899375b when a license is installed, only set TOWER_URL_BASE if necessary 
it's possible for users to set this manually in /etc/tower/conf.d
prior to license application
 2020-01-30 16:23:25 -05:00
Ryan Petrello 3c31e0ed16 some more minor callback cleanup and development tweaks 2020-01-27 17:18:09 -05:00
Ryan Petrello 78b00652bd add the ability to enable profiling for the callback receiver workers 2020-01-27 12:03:53 -05:00
Ryan Petrello 8f33f1a6c2 remove another expensive logging lookup in the parent callback process 2020-01-24 16:46:32 -05:00
Ryan Petrello 256404ba03 fix a bug that causes __search filters to not properly call .distinct() 2020-01-23 13:40:22 -05:00
Ryan Petrello f9af5e8959 optimize awx-manage callback_stats for larger datasets 
to monitor this historically, we'd probably need to introduce a new
index on the modified column of all our event types
 2020-01-22 16:52:38 -05:00
Ryan Petrello e18639b26b further optimize conf.settings access when logging is enabled 
the callback receiver is still fairly slow when logging is enabled due
to constant setting lookups; this speeds things up considerably

related: https://github.com/ansible/awx/pull/5618
 2020-01-22 16:17:33 -05:00
Ryan Petrello d8d1ccf810 add a script for quickly inserting lots of events 2020-01-22 11:57:47 -05:00
Ryan Petrello dfc769b8fe drastically speed up /api/v2/jobs/N/job_events/ with large counts 2020-01-20 13:24:39 -05:00
Ryan Petrello cc2ba09d3a remove an unnecessary settings optimzation 2020-01-17 14:59:56 -05:00
Ryan Petrello f79b6d3708 remove JobTemplate.job_type = 'scan' 
see: https://github.com/ansible/awx/issues/5603
 2020-01-15 15:05:01 -05:00
Ryan Petrello 0f0d9ba00d send inv computed tasks *after* commit to avoid a race condition 2020-01-15 14:14:26 -05:00
Ryan Petrello 6e5028587a update to latest vmware_inventory.py 
https://github.com/jladdjr/ansible/blob/63737ec0f8a37da17278cc257dde39e89eec6df2/contrib/inventory/vmware_inventory.py

see: https://github.com/ansible/awx/issues/5648
 2020-01-15 10:40:01 -05:00
Ryan Petrello 8c8713885b fix a bug that breaks inv sync stdout within workflows 
see: https://github.com/ansible/tower/issues/4068
 2020-01-15 10:12:27 -05:00
Ryan Petrello 829e9054d6 update data migration instructions 2020-01-15 09:03:36 -05:00
Ryan Petrello be68a199ec reorder migrations after a rebase 2020-01-14 16:58:33 -05:00
Ryan Petrello 568606d2c8 remove computed inventory fields from Host and Group 2020-01-14 16:37:16 -05:00
Ryan Petrello 79b8e6b6f0 renumber migrations correctly 2020-01-14 16:00:33 -05:00