604 Commits

Author SHA1 Message Date
Christian Adams
c8ceb62269 Rename awx rsyslog socket and PID dir 2020-04-15 14:11:15 -04:00
Ryan Petrello
0f74a05fea
rsyslogd: ignore /dev/log when we load imuxsock 2020-04-14 11:34:58 -04:00
Christian Adams
a0e31b9c01 Map logging timeout value to healthchecktimeout for http in rsyslog config 2020-04-13 15:22:16 -04:00
Christian Adams
5d54877183 Add action to default rsyslog.conf so supervisor starts correctly the first time 2020-04-13 11:44:00 -04:00
Christian Adams
b942fde59a Ensure log messages have valid json
- Fix messages getting contatenated at 8k
 - Fix rsyslog cutting off the opening brace of log messages
 - Make valid default conf and emit logs based on prescence of .sock and
 settings
2020-04-13 11:44:00 -04:00
Christian Adams
70391f96ae Revert rsyslog valid config to one that fails intentionally 2020-04-13 11:43:59 -04:00
Christian Adams
2329c1b797 Add rsyslog config to container from file for consistency 2020-04-13 11:43:59 -04:00
Christian Adams
470159b4d7 Enable innocuous but valid config for rsyslog if disabled 2020-04-13 11:43:59 -04:00
Christian Adams
4d5507d344 Add default rsyslog.conf without including /etc/rsyslog.conf 2020-04-13 11:43:59 -04:00
Christian Adams
d350551547 Tweaks to Test Button logic and cleans up flake8 and test failures 2020-04-13 11:43:59 -04:00
Christian Adams
fb047b1267 Add unit tests for reconfiguring rsyslog & for test endpoint 2020-04-13 11:43:59 -04:00
Christian Adams
d31c528257 Fix Logging settings "Test" button functionality 2020-04-13 11:43:59 -04:00
Christian Adams
996d7ce054 Move supervisor and rsyslog sock files to their own dirs under /var/run 2020-04-13 11:43:59 -04:00
Christian Adams
7040fcfd88 Fix container rsyslog dir permissions 2020-04-13 11:43:59 -04:00
Shane McDonald
c0af3c537b Configure rsyslog to listen over a unix domain socket instead of a port
- Add a placeholder rsyslog.conf so it doesn't fail on start
 - Create access restricted directory for unix socket to be created in
 - Create RSyslogHandler to exit early when logging socket doesn't exist
 - Write updated logging settings when dispatcher comes up and restart rsyslog so they  take effect
 - Move rsyslogd to the web container and create rpc supervisor.sock
 - Add env var for supervisor.conf path
2020-04-13 11:43:59 -04:00
Christian Adams
f8afae308a Add rsyslog to supervisor for the task container
- Add proper paths for rsyslog's supervisor logs
 - Do not enable debug mode for rsyslogd
 - Include system rsyslog.conf, and specify tower logging conf when
   starting rsyslog.
2020-04-13 11:43:59 -04:00
Ryan Petrello
589d27c88c POC: replace our external log aggregation feature with rsyslog
- this change adds rsyslog (https://github.com/rsyslog/rsyslog) as
  a new service that runs on every AWX node (managed by supervisord)
  in particular, this feature requires a recent version (v8.38+) of
  rsyslog that supports the omhttp module
  (https://github.com/rsyslog/rsyslog-doc/pull/750)
- the "external_logger" handler in AWX is now a SysLogHandler that ships
  logs to the local UDP port where rsyslog is configured to listen (by
  default, 51414)
- every time a LOG_AGGREGATOR_* setting is changed, every AWX node
  reconfigures and restarts its local instance of rsyslog so that its
  fowarding settings match what has been configured in AWX
- unlike the prior implementation, if the external logging aggregator
  (splunk/logstash) goes temporarily offline, rsyslog will retain the
  messages and ship them when the log aggregator is back online
- 4xx or 5xx level errors are recorded at /var/log/tower/external.err
2020-04-13 11:43:59 -04:00
softwarefactory-project-zuul[bot]
2da22ccd8a
Merge pull request #6659 from shanemcd/pre-tty
Enable tty in dev container

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-08 16:35:37 +00:00
Shane McDonald
677a8dae7b
Enable tty in dev container
Pretty colors and real-time migration logs
2020-04-08 11:43:30 -04:00
softwarefactory-project-zuul[bot]
e57991d498
Merge pull request #6652 from matburt/update_zome_docz
Update some contributing docs

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-08 14:58:40 +00:00
Matthew Jones
6abeaf2c55 Update some contributing docs
* Update the tools called in the dev environment
* More RMQ purges from architecture docs
* Remove the old clusterdev target
2020-04-08 10:03:22 -04:00
softwarefactory-project-zuul[bot]
91f2e0c32b
Merge pull request #6605 from ansible/firehose_pkey
update firehose script for bigint migration

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-08 13:33:19 +00:00
Shane McDonald
7e52f4682c
Use a docker volume for the dev env db 2020-04-07 13:14:19 -04:00
softwarefactory-project-zuul[bot]
05b28efd9c
Merge pull request #6617 from chrismeyersfsu/fix-memcached
fix memcached in dev env

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-06 23:49:04 +00:00
softwarefactory-project-zuul[bot]
606c3c3595
Merge pull request #6338 from rooftopcellist/update_logstash_docs
Update logstash docs

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-06 21:10:44 +00:00
chris meyers
c4d704bee1 fix memcached in dev env
* create memcached dir via git so that the current user owns it.
Otherwise, docker will create the dir as root at runtime
2020-04-06 16:35:52 -04:00
Mat Wilson
59477aa221 update firehose script for bigint migration 2020-04-06 10:54:08 -07:00
Shane McDonald
bb319136e4
Merge pull request #6585 from shanemcd/cleanup-cleanup
Tidy up the dev environment a bit
2020-04-06 13:09:39 -04:00
Shane McDonald
6fc815937b
Tidy up the dev environment a bit 2020-04-06 11:13:51 -04:00
chris meyers
7433aab258 switch memcached from tcp to unix domain socket 2020-04-06 08:35:12 -04:00
chris meyers
d7f0642f48 add ws broadcast status to sos report 2020-04-02 21:46:12 -04:00
softwarefactory-project-zuul[bot]
67d48a87f8
Merge pull request #6408 from ryanpetrello/rabbitmq-cleanup
remove a bunch of RabbitMQ references

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-25 17:02:54 +00:00
Christian Adams
6335004c94
Add common debugging tools to the dev container
- nmap-ncat
 - sdb
 - tcpdump
 - strace
 - vim
2020-03-25 08:03:32 -04:00
Ryan Petrello
8f1db173c1
remove a bunch of RabbitMQ references 2020-03-24 18:46:58 -04:00
chris meyers
5ba90b629e fixup dev cluster bringup
* change from bootstrap script to launch_awx.sh script
2020-03-23 07:33:35 -04:00
Christian Adams
9a38971d47 Update ELK Stack container files 2020-03-19 09:35:08 -04:00
Christian Adams
c4e697879d Improve docs for using the logstash container 2020-03-18 18:32:45 -04:00
chris meyers
87de0cf0b3
flake8, pytest, license fixes 2020-03-18 16:10:20 -04:00
chris meyers
770b457430
redis socket support 2020-03-18 16:10:19 -04:00
chris meyers
0da94ada2b
add missing service name to dev env
* Dev env was bringing the wsbroadcast service up but not under the
tower-process dependency. This is cleaner.
2020-03-18 16:10:18 -04:00
chris meyers
3c5c9c6fde
move broadcast websocket out into its own process 2020-03-18 16:10:18 -04:00
chris meyers
2a2c34f567
combine all the broker replacement pieces
* local redis for event processing
* postgres for message broker
* redis for websockets
2020-03-18 16:10:15 -04:00
chris meyers
c8eeacacca
POC channels 2 2020-03-18 16:10:12 -04:00
softwarefactory-project-zuul[bot]
7120e92078
Merge pull request #6262 from rooftopcellist/mv_bootstrap_script
Update dev container to be consistent with other installation methods

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-12 19:51:18 +00:00
Christian Adams
3755151cc5 Update dev container to be consistent with other installation methods
- rename start_development.sh script to `launch_awx.sh`, like it is in k8 installations
2020-03-11 16:23:31 -04:00
Bill Nottingham
871695ea5e Remove SCL python from awx-python
This really should be created at build time.
2020-03-09 15:51:56 -04:00
Yanis Guenane
07232f3694 awx-api-lint: Fix setup.cfg syntax for linter test
Signed-off-by: Yanis Guenane <yguenane@redhat.com>
2020-02-26 11:07:46 +01:00
Philip Douglass
7ae1c7c3d2 Set setuid bit on bwrap in development Dockerfile
Related: #5224

Signed-off-by: Philip Douglass <philip.douglass@amadeus.com>
2020-02-25 15:20:01 -05:00
AlanCoding
866dd6b259
Make job batch size configurable, fix _by fields 2020-02-21 14:49:45 -05:00
AlanCoding
06eb1b6683
Randomize JT for each job batch
Populate some more fields just to have them populated

Include some missing ForeignKey links for data integrity

Add some more to fields from JT set
2020-02-20 22:49:11 -05:00