Move visualization containers into docker-compose

Makefile

@@ -470,7 +470,9 @@ docker-compose-sources: .git/hooks/pre-commit
 	    -e minikube_container_group=$(MINIKUBE_CONTAINER_GROUP) \
 	    -e enable_keycloak=$(KEYCLOAK) \
 	    -e enable_ldap=$(LDAP) \
-	    -e enable_splunk=$(SPLUNK)
+	    -e enable_splunk=$(SPLUNK) \
+	    -e enable_prometheus=$(PROMETHEUS) \
+	    -e enable_grafana=$(GRAFANA)
 
 
 docker-compose: awx/projects docker-compose-sources

tools/docker-compose/README.md

@@ -404,7 +404,7 @@ Anytime you want to run an OpenLDAP instance alongside AWX we can start docker-c
 LDAP=true make docker-compose
 ```
 
-Once the containers come up two new ports (389, 636) should be exposed and the LDAP server should be running on those ports. The first port (389) is non-SSL and the second port (636) is SSL enabled. 
+Once the containers come up two new ports (389, 636) should be exposed and the LDAP server should be running on those ports. The first port (389) is non-SSL and the second port (636) is SSL enabled.
 
 Now we are ready to configure and plumb OpenLDAP with AWX. To do this we have provided a playbook which will:
 * Backup and configure the LDAP adapter in AWX. NOTE: this will back up your existing settings but the password fields can not be backed up through the API, you need a DB backup to recover this.
@@ -427,7 +427,7 @@ Once the playbook is done running LDAP should now be setup in your development e
 3. awx_ldap_auditor:audit123
 4. awx_ldap_org_admin:orgadmin123
 
-The first account is a normal user. The second account will be a super user in AWX. The third account will be a system auditor in AWX. The fourth account is an org admin. All users belong to an org called "LDAP Organization". To log in with one of these users go to the AWX login screen enter the username/password. 
+The first account is a normal user. The second account will be a super user in AWX. The third account will be a system auditor in AWX. The fourth account is an org admin. All users belong to an org called "LDAP Organization". To log in with one of these users go to the AWX login screen enter the username/password.
 
 
 ### Splunk Integration
@@ -449,7 +449,7 @@ Once the containers are up we are ready to configure and plumb Splunk with AWX.
 * Backup and configure the External Logging adapter in AWX. NOTE: this will back up your existing settings but the password fields can not be backed up through the API, you need a DB backup to recover this.
 * Create a TCP port in Splunk for log forwarding
 
-For routing traffic between AWX and Splunk we will use the internal docker compose network. The `Logging Aggregator` will be configured using the internal network machine name of `splunk`. 
+For routing traffic between AWX and Splunk we will use the internal docker compose network. The `Logging Aggregator` will be configured using the internal network machine name of `splunk`.
 
 Once you have have the collections installed (from above) you can run the playbook like:
 ```bash
@@ -464,6 +464,16 @@ Once the playbook is done running Splunk should now be setup in your development
 ### Prometheus and Grafana integration
 
 Prometheus is a metrics collecting tool, and we support prometheus formatted data at the `api/v2/metrics` endpoint.
+You can use this as part of the docker-compose target:
+
+```
+PROMETHEUS=true GRAFANA=true make docker-compose
+```
+
+TODO, internal, delete
+PROMETHEUS=true GRAFANA=true MAIN_NODE_TYPE=hybrid EXECUTION_NODE_COUNT=0 COMPOSE_TAG=devel make docker-compose
+
+Alternatively, you can run as separate commands (deprecated, may be deleted in future).
 
 1. Change the `username` and `password` in `tools/prometheus/prometheus.yml`. You can also change the scrape interval.
 2. (optional) if you are in a clustered environment, you can change the target to `haproxy:8043` so that the incoming prometheus requests go through the load balancer. Leaving it set to `awx1` also works.

tools/docker-compose/ansible/roles/sources/defaults/main.yml

@@ -29,3 +29,5 @@ ldap_private_key_file: '{{ ldap_cert_dir }}/{{ ldap_private_key_file_name }}'
 ldap_cert_subject: "/C=US/ST=NC/L=Durham/O=awx/CN="
 
 enable_splunk: false
+enable_grafana: false
+enable_prometheus: false

tools/docker-compose/ansible/roles/sources/templates/docker-compose.yml.j2

@@ -135,6 +135,35 @@ services:
     environment:
       SPLUNK_START_ARGS: --accept-license
       SPLUNK_PASSWORD: splunk_admin
+{% endif %}
+{% if enable_prometheus|bool %}
+  prometheus:
+    image: prom/prometheus:latest
+    container_name: tools_prometheus_1
+    hostname: splunk
+    ports:
+      - "9090:9090"
+    volumes:
+      - "../../prometheus:/etc/prometheus"
+      - "prometheus_storage:/prometheus:rw"
+    links:
+      - awx_1:awx1
+{% endif %}
+{% if enable_grafana|bool %}
+  grafana:
+    image: grafana/grafana-enterprise:latest
+    container_name: tools_grafana_1
+    hostname: splunk
+    ports:
+      - "3001:3000"
+    volumes:
+      - "../../grafana:/etc/grafana/provisioning"
+      - "grafana_storage:/var/lib/grafana:rw"
+    environment:
+      SPLUNK_START_ARGS: --accept-license
+      SPLUNK_PASSWORD: splunk_admin
+    links:
+      - prometheus
 {% endif %}
   # A useful container that simply passes through log messages to the console
   # helpful for testing awx/tower logging
@@ -198,6 +227,14 @@ volumes:
     name: tools_ldap_1
     driver: local
 {% endif %}
+{% if enable_prometheus|bool %}
+  prometheus_storage:
+    name: tools_prometheus_storage
+{% endif %}
+{% if enable_grafana|bool %}
+  grafana_storage:
+    name: tools_grafana_storage
+{% endif %}
 {% if minikube_container_group|bool %}
 networks:
   default: