125 Commits

Author SHA1 Message Date
AlanCoding
b393a60ece prevent overwriting of playbook data when censoring 2017-07-11 08:45:55 -04:00
Ryan Petrello
405c01a847 more isolated production tinkering
see: #5903
see: #6507
2017-06-29 09:35:26 -04:00
AlanCoding
40287d8e78 multi-host isolated heartbeat w tower-isolated check
* use tower-expect command to determine job status when running
  the isolated heartbeat playbook
* grok JSON output of playbook to obtain result information
* run playbook against multiple isolated hosts at the same time
  (addresses scalability concerns)
2017-06-20 14:36:18 -04:00
AlanCoding
7eb8fdd0f1 Change use of subprocess lib for calculating system capacity
Suggested by Ryan Petrello as a part of the PR feedback
for the isolated instance setup work.
2017-06-19 12:13:36 -04:00
AlanCoding
1d9f2be0e4 add task in isolated heartbeat to delete rogue files 2017-06-19 12:13:36 -04:00
AlanCoding
dd1a261bc3 setup playbook and heartbeat for isolated deployments
* Allow isolated_group_ use in setup playbook
* Tweaks to host/queue registration commands complementing setup
* Create isolated heartbeat task and check capacity
* Add content about isolated instances to acceptance docs
2017-06-19 12:13:36 -04:00
Ryan Petrello
44e0c8621a isolated ramparts: replace systemd unit with a tower-expect binary
instead of launching isolated tasks via `systemctl`, treat
`awx.main.isolated.run` as an executable that knows how to daemonize

additionally, add `setup.py isolated_build` for isolated Tower source
distribution
2017-06-16 09:59:21 -04:00
Ryan Petrello
422950f45d Support for executing job and adhoc commands on isolated Tower nodes (#6524) 2017-06-14 11:47:30 -04:00
Ryan Petrello
f6046d47c9 work around a change in ansible 2.3 to Task.get_name()
Properly detect setup tasks so that we can filter out sensitive
variables from the `ansible_facts` portion of the event payload.
2017-05-01 10:38:05 -04:00
Aaron Tan
6884e44bb3 Unit test added. 2017-04-11 11:40:47 -04:00
Aaron Tan
9b1107c054 Prevent unexpected callback module attribute warning. 2017-04-10 14:40:57 -04:00
Ryan Petrello
f9b5b9aa91 property filter no_log for item event loops
see: #5691
2017-03-22 11:15:33 -04:00
Ryan Petrello
6a7743b274 fix a callback bug that causes a task_args leak between job events
see: #5802
2017-03-20 12:05:41 -04:00
Ryan Petrello
999a304c3b add no_log filtering for runner_item_on_xyz events
see: #5691
2017-03-17 09:54:46 -04:00
Ryan Petrello
66c786b971 simplify memcache patching in our callback plugin tests 2017-03-14 14:07:53 -04:00
Ryan Petrello
a5ea468bfe add a test to ensure that job events properly strip out environment vars
see: #5601
2017-03-14 12:56:44 -04:00
Ryan Petrello
5bbb824865 add tests for no_log filtering
see: #5691
2017-03-14 12:56:44 -04:00
Ryan Petrello
1005782ee9 add tests for our custom ansible callback plugin 2017-03-14 12:56:40 -04:00
Ryan Petrello
6af9638d81 job event data marked with _ansible_no_log should be censored
see: #5691
2017-03-10 15:47:39 -05:00
Ryan Petrello
ed5a01c0cf strip ansible_env from job event callback data
this often contains sensitive details, and the data is already filtered
and stored on the job itself

see: #5601
2017-03-07 16:48:48 -05:00
Matthew Jones
2621f48189 We specifically need to convert the ansible included file type
for include events
2017-02-01 15:30:22 -05:00
Matthew Jones
d9a7f2fadb Silently ignore psutil errors when the process is already gone
During ssh CP cleanuip
2017-01-30 13:39:05 -05:00
Matthew Jones
7ca496b6ef Merge branch 'release_3.1.0' into multi_worker_callbacks 2017-01-18 20:39:15 -05:00
Matthew Jones
1cdeb4d2af Refactor event emitter to not use event partials
event emitter now caches to memcache where it is eventually picked up
by the stdout event emitter. This obviates event reassembly in the
callback receiver.
2017-01-17 15:59:43 -05:00
Wayne Witzel III
9592c67c31 fixing flake8 errors 2017-01-16 14:16:24 -05:00
Wayne Witzel III
cd5eed9828 remove extra censoring code from tower, not needed since Ansible 2.1+ 2017-01-16 13:54:07 -05:00
Matthew Jones
974fe1d244 Fix an issue with set_stats for unsupported Ansible versions 2017-01-11 10:24:17 -05:00
Chris Meyers
86a9a921eb remove deepcopy, not sure how that slipped in 2017-01-06 11:47:45 -05:00
Chris Meyers
a1267a3dee update artifacts to work with ansible set_stats 2017-01-06 10:04:57 -05:00
Chris Meyers
03734015c5 set_stats support 2017-01-05 13:42:29 -05:00
Matthew Jones
a61e729ebb Purge event res dict if it is over a certain size
Also purge/update some old settings values
2016-12-14 15:05:28 -05:00
Chris Church
b85c98afd2 Split job event data between callback queue and stdout. Send most of event data directly over queue and capture only stdout/counter/start_line/end_line in celery task; recombine into single event in callback receiver. 2016-12-12 16:34:36 -05:00
Chris Church
ae16cadcc2 Ignore exception in sitecustomize. 2016-12-07 23:50:58 -05:00
Matthew Jones
24b858c6fb Switch away from deepcopy to improve performance 2016-12-06 13:33:14 -05:00
Chris Church
d6d5d4263a Only run ssh control persist cleanup from main process, flush after writing end marker for job event data. 2016-12-01 14:47:24 -05:00
Chris Church
1db697e4eb Add multiprocessing RLock around event data output to ensure only one process is writing to stdout at a time. 2016-11-17 09:09:19 -05:00
Aaron Tan
9e4655419e Fix flake8 E302 errors. 2016-11-15 20:59:39 -05:00
Aaron Tan
473ed016de Fix flake8 E305 errors. 2016-11-15 16:32:27 -05:00
Chris Church
5760654671 Fix warning from v2_playbook_on_task_start with ad hoc commands. 2016-11-07 21:42:39 -05:00
Chris Church
c43334f8f4 Update job events based on how they are used in Ansible 2.x. 2016-10-28 21:58:24 -04:00
Chris Church
440f0539b0 Flake8 fixes. 2016-10-28 21:58:24 -04:00
Chris Church
c18b6c1352 Add support for capturing stdout associated with job events and ad hoc command events.
* New event types for stdout lines not associated with a callback event.
* New stdout, start_line, end_line and verbosity fields for job/ahc events.
* Callback plugins to wrap Ansible default/minimal stdout callbacks and embed callback event data using ANSI escape sequences.
* Callback plugin library to wrap ansible.display.Display class methods.
* Output filter to extract event data from stdout and create job/ahc events.
* Update stdout formats to strip new ANSI escape sequences.
2016-10-28 21:58:24 -04:00
AlanCoding
1fec29dcb3 remove awx.compat, since it looks like it is not necessary 2016-10-26 17:22:03 -04:00
Chris Meyers
358bc31450 remove metric 2016-05-10 14:18:05 -04:00
Matthew Jones
5d21ad5ae6 Fix up statsd work to support python 2.6
Format specifiers must include field specifier
2016-02-02 09:58:19 -05:00
Matthew Jones
c7fd21ae52 Fix up flake8 and unit test failures
* Make sure we default to localhost to not fail at the GAI check
* Cleanup some flake8 issues in the metrics module
2016-01-29 13:59:37 -05:00
Joshua "jag" Ginsberg
f25b76aee9 Integrate statsd metrics into ansible playbook execution.
* Add dependencies for pystatsd and django-statsd-mozilla
* Default turned off except for development environment
* Modify docker-compose to install statsd/graphite host
2016-01-26 16:54:24 -05:00
Chris Meyers
fa1643e330 all vendor dependencies expressed in requirements.txt 2015-08-05 08:10:16 -04:00
Chris Meyers
9ee9d9031a fixes stoud formats unicode problem 2015-07-24 14:50:36 -04:00
Bill Nottingham
1d951fa9cc Further normalization of names. 2015-07-10 01:25:26 -04:00