42 Commits

Author SHA1 Message Date
Shane McDonald
0184a7c267 Create receptor mesh in cluster development environment 2021-03-03 18:52:54 -05:00
Jeff Bradberry
3cbf384ad1 Run a receptor node in the dev environment 2021-03-03 18:52:54 -05:00
Christian M. Adams
36466b9694 Upgrade PostgreSQL from 10 to 12 2021-01-08 10:54:00 -05:00
Ryan Petrello
0ad78874ce
remove TCP ports for redis (it only listens on a unix domain socket) 2020-06-09 12:29:33 -04:00
Rebeccah
d7f9e66710
added changelog entry 2020-06-09 11:09:04 -04:00
Rebeccah
d2bbe7aa1a
remove memcache from everywhere and add djagno-redis to cover it 2020-06-09 10:24:23 -04:00
Bill Nottingham
131f5ff018 Remove dev env futzing of supervisord.conf permissions
If we just link it into the dev env, we don't need to copy it at
startup, and we don't need write permissions on it.
2020-05-28 13:12:57 -04:00
chris meyers
72248db76d fix copy paste error in docker compose cluster 2020-04-16 14:12:30 -04:00
chris meyers
c4d704bee1 fix memcached in dev env
* create memcached dir via git so that the current user owns it.
Otherwise, docker will create the dir as root at runtime
2020-04-06 16:35:52 -04:00
chris meyers
7433aab258 switch memcached from tcp to unix domain socket 2020-04-06 08:35:12 -04:00
Ryan Petrello
8f1db173c1
remove a bunch of RabbitMQ references 2020-03-24 18:46:58 -04:00
chris meyers
5ba90b629e fixup dev cluster bringup
* change from bootstrap script to launch_awx.sh script
2020-03-23 07:33:35 -04:00
chris meyers
87de0cf0b3
flake8, pytest, license fixes 2020-03-18 16:10:20 -04:00
chris meyers
770b457430
redis socket support 2020-03-18 16:10:19 -04:00
chris meyers
2a2c34f567
combine all the broker replacement pieces
* local redis for event processing
* postgres for message broker
* redis for websockets
2020-03-18 16:10:15 -04:00
chris meyers
c8eeacacca
POC channels 2 2020-03-18 16:10:12 -04:00
Yanis Guenane
ca247182df yamllint: Make all files in awx pass yamllint
This commit updates all files that weren't passing yamllint for them to
pass.

A new yamllint target has been added. One can run `tox -e yamllint` or
`yamllint -s .` locally to ensure yaml files are still passing.

This check will be enabled in the CI so it can get on every new
contributions, and prevent merging non-compliant code.

Signed-off-by: Yanis Guenane <yguenane@redhat.com>
2019-12-02 15:12:51 +01:00
Ryan Petrello
4b62f4845a
fix broken docker-compose-cluster config 2019-09-27 00:37:29 -04:00
Shane McDonald
22441d280e Fix pg password in cluster dev env 2019-09-17 08:45:19 -04:00
Ryan Petrello
5ed97e0f65
change the default port range for the sdb debugging tool
the current range conflicts w/ a port used by the pycharm editor
2019-09-13 11:55:43 -04:00
Christian Adams
ec1e93cc69 Upgrade to postgres 10.6
- use awx-python in shebang in dev env
  - scl enable where needed for rhel7 & container installs
  - use scram-sha-256 pg user hashing by default
  - ensure psycopg2 is using the correct PG_CONFIG at build time for the right libpq version
2019-09-12 12:52:43 -04:00
Jose OrPa
04ab736f09 #3778 Upgrading postgresql to v10 2019-09-12 12:52:42 -04:00
Shane McDonald
b29a9cd86e Fix dev environment when running as root on the host
Without this, CURRENT_UID isnt actually passed in from the host, and wipes out /etc/passwd even when we’re actually running as root.

I tested this as a non-root user on Linux, and on Docker for Mac
2019-03-06 17:08:56 -05:00
AlanCoding
0b32733dc8
set fixed container names 2018-11-26 08:26:57 -05:00
AlanCoding
19030b9d5f
apply docker-compose fix to cluster target too 2018-10-26 09:36:11 -04:00
Ryan Petrello
ff1e8cc356
replace celery task decorators with a kombu-based publisher
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:

- a new decorator, `awx.main.dispatch.task`, which can be used to
  decorate functions or classes so that they can be designated as
  "Tasks"
- support for fanout/broadcast tasks (at this point in time, only
  `conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
  `handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
  on demand
- minimal support for RPC, such as status checks and pool recycle/reload
2018-10-11 10:53:30 -04:00
Ryan Petrello
1733a20094
make sdb-listen work for docker-compose-cluster
use a different port range for each container, because docker can't map
them all to the same port range
2018-06-12 13:57:28 -04:00
Matthew Jones
22dd6ddfea
Remove the logstash container from the base dev docker compose
Now with less java running while you code!
2018-06-05 09:42:38 -04:00
Ryan Petrello
d743b77353 replace our rdb tooling w/ the sdb PyPI package 2018-02-26 19:05:50 -05:00
Alan Rominger
dcfcfb6c7b Merge pull request #245 from AlanCoding/fix_supervisor
specify all group queues, get supervisor working
2017-08-14 11:31:00 -04:00
AlanCoding
2385f62311 specify all group queues, get supervisor working 2017-08-10 14:57:25 -04:00
Matthew Jones
67474c8de1 Updating development tooling to be more generic
* Not assuming GCR is being used for image hosting
* Breaking out the dev environment bootstrapping from service starting
2017-08-09 15:53:25 -04:00
Matthew Jones
eb64d6e808 Update cluster compose for repo migration 2017-07-24 21:03:55 -04:00
AlanCoding
3cedcf22a9 add instance groups and queues to cluster tooling 2017-06-22 10:40:58 -04:00
adamscmRH
e0581af9ff Missing Procfile docker-compose fix 2017-06-21 14:29:49 -04:00
Matthew Jones
eb2e7c38b0 Bring up logstash as part of the development cluster 2017-03-21 10:42:32 -04:00
Wayne Witzel III
50f187b78c run clustered as privleged to avoid namespace error 2017-02-24 11:12:39 -05:00
Matthew Jones
9f777f6153 Add some provisions for https into haproxy and the cluster 2017-01-03 13:59:29 -05:00
Matthew Jones
2872409271 Integrate rabbitmqctl interface to cluster and standalone containers 2016-09-15 12:07:16 -04:00
Matthew Jones
807cced571 Implement a more dynamic celery queue system
* Meant to be a starting point to more efficiently manage work routing
  and to balance work across all tower nodes
* Integrate flower as a dev tool that starts alongside other nodes.
  Helpful for observing and monitoring the queues/exchanges
* For the moment, force the task manager to only run on one node (not
  sure if this is needed)
* Define queues and routes for all task work
* Bump celery version to 3.1.23
* Expose flower through haproxy
2016-09-09 15:18:18 -04:00
Matthew Jones
eafb6c92b5 Docker compose improvements
* Make sure we explicitly set a hostname for tower nodes
* Switch rabbit vhost to use the root
2016-09-08 11:02:51 -04:00
Matthew Jones
ebf103f345 Initial Docker Compose workflow for Tower cluster
The goal is to share a common pattern with the existing development work
2016-09-08 10:18:14 -04:00