External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-02-02 01:58:09 -03:30
Code Issues Packages Projects Releases Wiki Activity

Consolidate the Local Docker installer and the dev env

Browse Source 
- removes local_docker installer and points community users to our development environment (make docker-compose)
  - provides a migration path from Local Docker Compose installations --> the dev environment
  - the dev env can now be configured to use an external database
  - consolidated the Local Docker and dev env docker-compose.yml files into one template file, used by the dockerfile role
  - added a 'sources' role to template out config files
  - the postgres data dir is no longer a bind-mount, it is a docker volume
  - the redis socket is not longer a bind-mount, it is a docker volume
  - the local_settings.py.docker-compose file no longer needs to be copied over in the dev env
  - Create tmp rsyslog.conf in rsyslog volume to avoid cross-linking. Previously, the tmp code-generated rsyslog.conf was being written to /tmp (by default).  As a result, we were attempting to shutil.move() across volumes.
  - move k8s image build and push roles under tools/ansible
  - See tools/docker-compose/README.md for usage of these changes
This commit is contained in:
Christian M. Adams
2021-01-27 11:01:17 -05:00
parent 0f6d2c36a0
commit 9672e72834
52 changed files with 1325 additions and 808 deletions
Download Patch File Download Diff File Expand all files Collapse all files

207
tools/docker-compose/README.md
View File

@@ -1,86 +1,171 @@
# Docker Compose for Dev container
# Docker Compose for Development
## How to start the Dev container
## Overview
In the root directory of your awx clone, run the following to build your docker image.
This step takes a while, but once your image is built, you don't need to do this again
unless you make changes to the Dockerfile or any of the files used by the Dockerfile.
Here are the main make targets:
```
make docker-compose-build
* `docker-compose-build` - used for building the development image, which is used by both `docker-compose`
* `docker-compose` - Make target for development, passes awx_devel image and tag.
Notable files:
* `tools/docker-compose/inventory` file - used to configure the local AWX development deployment.
* `migrate.yml` - playbook for migrating data from Local Docker to the Development Environment.
### Prerequisites
- [Docker](https://docs.docker.com/engine/installation/) on the host where AWX will be deployed. After installing Docker, the Docker service must be started (depending on your OS, you may have to add the local user that uses Docker to the ``docker`` group, refer to the documentation for details)
- [docker-compose](https://pypi.org/project/docker-compose/) Python module.
+ This also installs the `docker` Python module, which is incompatible with `docker-py`. If you have previously installed `docker-py`, please uninstall it.
- [Docker Compose](https://docs.docker.com/compose/install/).
## Configuration
In the `inventory` file, set your `pg_password`, `broadcast_websocket_secret`, `secret_key`, and any other settings you need for your deployment.
AWX requires access to a PostgreSQL database, and by default, one will be created and deployed in a container, and data will be persisted to a docker volume. When the container is stopped, the database files will still exist in the docker volume. An external database can be used by setting the `pg_host`, `pg_hostname`, and `pg_username`.
> If you are coming from a Local Docker installation of AWX, consider migrating your data first, see the [data migration section](#migrating-data-from-local-docker) below.
## Starting the Development Environment
### Build the image
The AWX base container image (defined in the Dockerfile templated from [Dockerfile.j2](./../ansible/roles/dockerfile/templates/Dockerfile.j2)) contains basic OS dependencies and symbolic links into the development environment that make running the services easy.
Run the following to build the image:
```bash
$ make docker-compose-build
```
> Note: By default, this image will be tagged with your branch name.
> The image will need to be rebuilt if there are any changes to Dockerfile.j2 or any of the files used by the templated Dockerfile.
Copy over your local settings
Once the build completes, you will have a `ansible/awx_devel` image in your local image cache. Use the `docker images` command to view it, as follows:
```
cp awx/settings/local_settings.py.docker_compose awx/settings/local_settings.py
```bash
(host)$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_devel latest ba9ec3e8df74 26 minutes ago 1.42GB
```
Build the UI
> By default, this image will be tagged with your branch name. You can specify a custom tag by setting an environment variable, for example: `DEVEL_IMAGE_NAME=quay.io/your_user/awx_devel:17.0.1`
```
make ui-devel
### Run AWX
##### Start the containers
Run the awx, postgres and redis containers. This utilizes the image built in the previous step, and will automatically start all required services and dependent containers. Once the containers launch, your session will be attached to the awx container, and you'll be able to watch log messages and events in real time. You will see messages from Django and the front end build process.
```bash
$ make docker-compose
```
Run the container
> The make target assumes that the image you built is tagged with your current branch. This allows you to build images for different contexts or branches. When starting the containers, you can choose a specific branch by setting `COMPOSE_TAG=<branch name> `in your environment. For example, you might be working in a feature branch, but you want to run the containers using the devel image you built previously. To do that, start the containers using the following command: `$ COMPOSE_TAG=devel make docker-compose`
```
make docker-compose
```
##### Wait for migrations to complete
> Note: You may need to add `COMPOSE_TAG=<yourbranch>` at the end of this to use the correct image for your branch.
The first time you start the environment, database migrations need to run in order to build the PostgreSQL database. It will take few moments, but eventually you will see output in your terminal session that looks like the following:
The app should now be accessible in your browser at `https://localhost:8043/#/home`
## How to use the logstash container
#### Modify the docker-compose.yml
Uncomment the following lines in the `docker-compose.yml`
```
#- logstash
```bash
awx_1 | Operations to perform:
awx_1 | Synchronize unmigrated apps: solo, api, staticfiles, debug_toolbar, messages, channels, django_extensions, ui, rest_framework, polymorphic
awx_1 | Apply all migrations: sso, taggit, sessions, sites, kombu_transport_django, social_auth, contenttypes, auth, conf, main
awx_1 | Synchronizing apps without migrations:
awx_1 | Creating tables...
awx_1 | Running deferred SQL...
awx_1 | Installing custom SQL...
awx_1 | Running migrations:
awx_1 | Rendering model states... DONE
awx_1 | Applying contenttypes.0001_initial... OK
awx_1 | Applying contenttypes.0002_remove_content_type_name... OK
awx_1 | Applying auth.0001_initial... OK
...
#logstash:
# build:
# context: ./docker-compose
# dockerfile: Dockerfile-logstash
```
POST the following content to `/api/v2/settings/logging/` (this uses
authentication set up inside of the logstash configuration file).
```
{
"LOG_AGGREGATOR_HOST": "http://logstash",
"LOG_AGGREGATOR_PORT": 8085,
"LOG_AGGREGATOR_TYPE": "logstash",
"LOG_AGGREGATOR_USERNAME": "awx_logger",
"LOG_AGGREGATOR_PASSWORD": "workflows",
"LOG_AGGREGATOR_LOGGERS": [
"awx",
"activity_stream",
"job_events",
"system_tracking"
],
"LOG_AGGREGATOR_INDIVIDUAL_FACTS": false,
"LOG_AGGREGATOR_TOWER_UUID": "991ac7e9-6d68-48c8-bbde-7ca1096653c6",
"LOG_AGGREGATOR_ENABLED": true
}
##### Clean and Build UI
```bash
$ docker exec tools_awx_1 make clean-ui ui-devel
```
> Note: HTTP must be specified in the `LOG_AGGREGATOR_HOST` if you are using the docker development environment.
See [the ui development documentation](../../awx/ui_next/README.md) for more information on using the frontend development, build, and test tooling.
An example of how to view the most recent logs from the container:
Once migrations are completed and the UI is built, you can begin using AWX. The UI can be reached in your browser at `https://localhost:8043/#/home`, and the API can be found at `https://localhost:8043/api/v2`.
```
docker exec -i -t $(docker ps -aqf "name=tools_logstash_1") tail -n 50 /logstash.log
##### Create an admin user
Before you can log into AWX, you need to create an admin user. With this user you will be able to create more users, and begin configuring the server. From within the container shell, run the following command:
```bash
$ docker exec -ti tools_awx_1 awx-manage createsuperuser
```
#### How to add logstash plugins
> Remember the username and password, as you will use them to log into the web interface for the first time.
Add any plugins you need in `tools/elastic/logstash/Dockerfile` before running the container.
##### Load Demo Data
Optionally, you may also want to load some demo data. This will create a demo project, inventory, and job template.
```bash
$ docker exec tools_awx_1 awx-manage create_preload_data
```
> This information will persist in the database running in the `tools_postgres_1` container, until the container is removed. You may periodically need to recreate
this container, and thus the database, if the database schema changes in an upstream commit.
## Migrating Data from Local Docker
If you are migrating data from a Local Docker installation (17.0.1 and prior), you can
migrate your data to the development environment via the migrate.yml playbook using the steps described [here](./docs/data_migration.md).
## Upgrading the Development Environment
Upgrading AWX involves checking out the new source code and re-running the make target. Download a newer release from [https://github.com/ansible/awx/releases](https://github.com/ansible/awx/releases) and re-populate the inventory file with your customized variables.
After updating the inventory file with any custom values, run the make target from the root of your AWX clone.
```bash
$ make docker-compose
```
## Extras
* [Start a shell](#start-a-shell)
* [Start AWX from the container shell](#start-awx-from-the-container-shell)
* [Using Logstash](./docs/logstash.md)
### Start a Shell
To run `awx-manage` commands and modify things inside the container, you will want to start a shell session on the *awx* container. In a new terminal session, use the `docker exec` command to start the shell session:
```bash
(host)$ docker exec -it tools_awx_1 bash
```
This creates a session in the *awx* containers, just as if you were using `ssh`, and allows you execute commands within the running container.
### Start AWX from the container shell
Often times you'll want to start the development environment without immediately starting all of the services in the *awx* container, and instead be taken directly to a shell. You can do this with the following:
```bash
(host)$ make docker-compose-test
```
Using `docker exec`, this will create a session in the running *awx* container, and place you at a command prompt, where you can run shell commands inside the container.
If you want to start and use the development environment, you'll first need to bootstrap it by running the following command:
```bash
(container)# /usr/bin/bootstrap_development.sh
```
The above will do all the setup tasks, including running database migrations, so it may take a couple minutes. Once it's done it
will drop you back to the shell.
In order to launch all developer services:
```bash
(container)# /usr/bin/launch_awx.sh
```
`launch_awx.sh` also calls `bootstrap_development.sh` so if all you are doing is launching the supervisor to start all services, you don't
need to call `bootstrap_development.sh` first.

7
tools/docker-compose/ansible/migrate.yml Normal file
View File

@@ -0,0 +1,7 @@
---
- name: Migrate data for upgrades and from Local Docker installs
hosts: localhost
gather_facts: true
roles:
- {role: sources}
- {role: migrate}

5
tools/docker-compose/ansible/roles/migrate/defaults/main.yml Normal file
View File

@@ -0,0 +1,5 @@
---
postgres_data_dir: "~/.awx/pgdocker"
migrate_local_docker: false
old_docker_compose_dir: "~/.awx/awxcompose"
pg_volume_name: "postgres_data"

5
tools/docker-compose/ansible/roles/migrate/tasks/main.yml Normal file
View File

@@ -0,0 +1,5 @@
---
# Migrate data from a bind-mount to a volume
- import_tasks: migrate-from-local-docker.yml
when: migrate_local_docker

46
tools/docker-compose/ansible/roles/migrate/tasks/migrate-from-local-docker.yml Normal file
View File

@@ -0,0 +1,46 @@
---
# Migrate data from old Local Docker to a fresh development environment
- name: Remove awx_postgres to ensure consistent start state
shell: |
docker rm -f awx_postgres
- name: Start Local Docker database container
docker_compose:
project_src: "{{ old_docker_compose_dir }}"
services:
- postgres
state: present
recreate: always
- name: Database dump to local filesystem
shell: |
docker-compose -f {{ old_docker_compose_dir }}/docker-compose.yml exec -T postgres pg_dumpall -U {{ pg_username }} > awx_dump.sql
- name: Stop AWX containers so the old postgres container does not get used
docker_compose:
project_src: "{{ old_docker_compose_dir }}"
state: absent
ignore_errors: true
- name: Start dev env database container
docker_compose:
project_src: "{{ playbook_dir }}/../_sources"
files: "docker-compose.yml"
services:
- postgres
state: present
recreate: always
- name: Wait for postgres to initialize
wait_for:
timeout: 3
- name: Restore to new postgres container
shell: |
docker-compose -f {{ playbook_dir }}/../_sources/docker-compose.yml exec -T postgres psql -U {{ pg_username }} -d {{ pg_database }} -p {{ pg_port }} < awx_dump.sql
- name: Clean up temporary awx db dump
file:
path: awx_dump.sql
state: absent

7
tools/docker-compose/ansible/roles/sources/defaults/main.yml Normal file
View File

@@ -0,0 +1,7 @@
---
sources_dest: '../_sources'
compose_name: 'docker-compose.yml'
awx_image: 'quay.io/ansible/awx_devel'
pg_port: 5432
pg_username: 'awx'
pg_database: 'awx'

68
tools/docker-compose/ansible/roles/sources/files/local_settings.py Normal file
View File

@@ -0,0 +1,68 @@
# Copyright (c) 2015 Ansible, Inc. (formerly AnsibleWorks, Inc.)
# All Rights Reserved.
# Local Django settings for AWX project. Rename to "local_settings.py" and
# edit as needed for your development environment.
# All variables defined in awx/settings/development.py will already be loaded
# into the global namespace before this file is loaded, to allow for reading
# and updating the default settings as needed.
###############################################################################
# MISC PROJECT SETTINGS
###############################################################################
import os
import sys
# Enable the following lines and install the browser extension to use Django debug toolbar
# if your deployment method is not VMWare of Docker-for-Mac you may
# need a different IP address from request.META['REMOTE_ADDR']
# INTERNAL_IPS = ('172.19.0.1', '172.18.0.1', '192.168.100.1')
# ALLOWED_HOSTS = ['*']
# Use SQLite for unit tests instead of PostgreSQL. If the lines below are
# commented out, Django will create the test_awx-dev database in PostgreSQL to
# run unit tests.
if "pytest" in sys.modules:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'awx.sqlite3'),
'TEST': {
# Test database cannot be :memory: for inventory tests.
'NAME': os.path.join(BASE_DIR, 'awx_test.sqlite3'),
},
}
}
# Location for cross-development of inventory plugins
AWX_ANSIBLE_COLLECTIONS_PATHS = '/var/lib/awx/vendor/awx_ansible_collections'
# The UUID of the system, for HA.
SYSTEM_UUID = '00000000-0000-0000-0000-000000000000'
# If set, use -vvv for project updates instead of -v for more output.
# PROJECT_UPDATE_VVV=True
###############################################################################
# LOGGING SETTINGS
###############################################################################
# Enable logging to syslog. Setting level to ERROR captures 500 errors,
# WARNING also logs 4xx responses.
# Enable the following lines to turn on lots of permissions-related logging.
#LOGGING['loggers']['awx.main.access']['level'] = 'DEBUG'
#LOGGING['loggers']['awx.main.signals']['level'] = 'DEBUG'
#LOGGING['loggers']['awx.main.permissions']['level'] = 'DEBUG'
# Enable the following line to turn on database settings logging.
#LOGGING['loggers']['awx.conf']['level'] = 'DEBUG'
# Enable the following lines to turn on LDAP auth logging.
#LOGGING['loggers']['django_auth_ldap']['handlers'] = ['console']
#LOGGING['loggers']['django_auth_ldap']['level'] = 'DEBUG'
BROADCAST_WEBSOCKET_PORT = 8013
BROADCAST_WEBSOCKET_VERIFY_CERT = False
BROADCAST_WEBSOCKET_PROTOCOL = 'http'

76
tools/docker-compose/ansible/roles/sources/tasks/main.yml Normal file
View File

@@ -0,0 +1,76 @@
---
- name: Create _sources directory
file:
path: "{{ sources_dest }}/secrets"
state: 'directory'
mode: '0700'
- name: Detect secrets
stat:
path: "{{ sources_dest }}/secrets/{{ item }}.yml"
register: secrets
when: not lookup('vars', item, default='')
loop:
- pg_password
- secret_key
- broadcast_websocket_secret
- name: Generate secrets if needed
template:
src: 'secrets.yml.j2'
dest: '{{ sources_dest }}/secrets/{{ item.item }}.yml'
mode: '0600'
when: not lookup('vars', item.item, default='') and not item.stat.exists
loop: "{{ secrets.results }}"
- name: Include generated secrets unless they are explicitly passed in
include_vars: "{{ sources_dest }}/secrets/{{ item.item }}.yml"
no_log: true
when: not lookup('vars', item.item, default='')
loop: "{{ secrets.results }}"
- name: Render configuration templates
template:
src: "{{ item }}.j2"
dest: "{{ sources_dest }}/{{ item }}"
mode: '0600'
with_items:
- "database.py"
- "websocket_secret.py"
- name: Delete old local_settings.py
file:
path: "{{ playbook_dir }}/../../../awx/settings/local_settings.py"
state: absent
- name: Copy local_settings.py
copy:
src: "local_settings.py"
dest: "{{ sources_dest }}/local_settings.py"
- name: Get OS info for sdb
shell: |
docker info | grep 'Operating System'
register: os_info
changed_when: false
- name: Get user UID
shell: id -u
register: current_user
changed_when: false
- name: Set fact with user UID
set_fact:
user_id: "'{{ current_user.stdout }}'"
- name: Set global version if not provided
set_fact:
awx_image_tag: "{{ lookup('file', playbook_dir + '/../../../VERSION') }}"
when: awx_image_tag is not defined
- name: Render Docker-Compose
template:
src: docker-compose.yml.j2
dest: "{{ sources_dest }}/{{ compose_name }}"
mode: '0600'

11
tools/docker-compose/ansible/roles/sources/templates/database.py.j2 Normal file
View File

@@ -0,0 +1,11 @@
DATABASES = {
'default': {
'ATOMIC_REQUESTS': True,
'ENGINE': 'awx.main.db.profiled_pg',
'NAME': "{{ pg_database }}",
'USER': "{{ pg_username }}",
'PASSWORD': "{{ pg_password }}",
'HOST': "{{ pg_hostname | default('postgres') }}",
'PORT': "{{ pg_port }}",
}
}

65
tools/docker-compose/ansible/roles/sources/templates/docker-compose.yml.j2 Normal file
View File

@@ -0,0 +1,65 @@
---
version: '2'
services:
# Primary AWX Development Container
awx:
user: "{{ ansible_user_uid }}"
image: "{{ awx_image }}:{{ awx_image_tag }}"
container_name: tools_awx_1
hostname: awx
command: launch_awx.sh
environment:
OS: "{{ os_info.stdout }}"
SDB_HOST: 0.0.0.0
SDB_PORT: 7899
AWX_GROUP_QUEUES: tower
ports:
- "8888:8888"
- "8080:8080"
- "8013:8013"
- "8043:8043"
- "6899:6899" # default port range for sdb-listen
- "7899-7999:7899-7999" # default port range for sdb-listen
links:
- postgres
- redis
working_dir: "/awx_devel"
volumes:
- "../../../:/awx_devel"
- "../../docker-compose/supervisor.conf:/etc/supervisord.conf"
- "../../docker-compose/_sources/database.py:/etc/tower/conf.d/database.py"
- "../../docker-compose/_sources/websocket_secret.py:/etc/tower/conf.d/websocket_secret.py"
- "../../docker-compose/_sources/local_settings.py:/etc/tower/conf.d/local_settings.py"
- "redis_socket:/var/run/redis/:rw"
privileged: true
tty: true
# A useful container that simply passes through log messages to the console
# helpful for testing awx/tower logging
# logstash:
# build:
# context: ./docker-compose
# dockerfile: Dockerfile-logstash
# Postgres Database Container
postgres:
image: postgres:12
container_name: tools_postgres_1
environment:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_USER: {{ pg_username }}
POSTGRES_DB: {{ pg_database }}
POSTGRES_PASSWORD: {{ pg_password }}
volumes:
- "awx_db:/var/lib/postgresql/data"
redis:
image: redis:latest
container_name: tools_redis_1
volumes:
- "../../redis/redis.conf:/usr/local/etc/redis/redis.conf"
- "redis_socket:/var/run/redis/:rw"
entrypoint: ["redis-server"]
command: ["/usr/local/etc/redis/redis.conf"]
volumes:
awx_db:
redis_socket:

31
tools/docker-compose/ansible/roles/sources/templates/docker-credential-plugins-override.yml Normal file
View File

@@ -0,0 +1,31 @@
---
version: '2'
services:
# Primary Tower Development Container link
awx:
links:
- hashivault
- conjur
hashivault:
image: vault
container_name: tools_hashivault_1
ports:
- '8200:8200'
cap_add:
- IPC_LOCK
environment:
VAULT_DEV_ROOT_TOKEN_ID: 'vaultdev'
conjur:
image: cyberark/conjur
container_name: tools_conjur_1
command: server -p 8300
environment:
DATABASE_URL: postgres://awx@postgres/postgres
CONJUR_DATA_KEY: 'dveUwOI/71x9BPJkIgvQRRBF3SdASc+HP4CUGL7TKvM='
depends_on:
- postgres
links:
- postgres
ports:
- "8300:8300"

10
tools/docker-compose/ansible/roles/sources/templates/environment.sh.j2 Normal file
View File

@@ -0,0 +1,10 @@
DATABASE_USER={{ pg_username|quote }}
DATABASE_NAME={{ pg_database|quote }}
DATABASE_HOST={{ pg_hostname|default('postgres')|quote }}
DATABASE_PORT={{ pg_port|default('5432')|quote }}
DATABASE_PASSWORD={{ pg_password|default('awxpass')|quote }}
{% if pg_admin_password is defined %}
DATABASE_ADMIN_PASSWORD={{ pg_admin_password|quote }}
{% endif %}
AWX_ADMIN_USER={{ admin_user|quote }}
AWX_ADMIN_PASSWORD={{ admin_password|quote }}

122
tools/docker-compose/ansible/roles/sources/templates/nginx.conf.j2 Normal file
View File

@@ -0,0 +1,122 @@
#user awx;
worker_processes 1;
pid /tmp/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
server_tokens off;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /dev/stdout main;
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
sendfile on;
#tcp_nopush on;
#gzip on;
upstream uwsgi {
server 127.0.0.1:8050;
}
upstream daphne {
server 127.0.0.1:8051;
}
{% if ssl_certificate is defined %}
server {
listen 8052 default_server;
server_name _;
# Redirect all HTTP links to the matching HTTPS page
return 301 https://$host$request_uri;
}
{%endif %}
server {
{% if (ssl_certificate is defined) and (ssl_certificate_key is defined) %}
listen 8053 ssl;
ssl_certificate /etc/nginx/awxweb.pem;
ssl_certificate_key /etc/nginx/awxweb_key.pem;
{% elif (ssl_certificate is defined) and (ssl_certificate_key is not defined) %}
listen 8053 ssl;
ssl_certificate /etc/nginx/awxweb.pem;
ssl_certificate_key /etc/nginx/awxweb.pem;
{% else %}
listen 8052 default_server;
{% endif %}
# If you have a domain name, this is where to add it
server_name _;
keepalive_timeout 65;
# HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
add_header Strict-Transport-Security max-age=15768000;
# Protect against click-jacking https://www.owasp.org/index.php/Testing_for_Clickjacking_(OTG-CLIENT-009)
add_header X-Frame-Options "DENY";
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
location /static/ {
alias /var/lib/awx/public/static/;
}
location /favicon.ico { alias /var/lib/awx/public/static/favicon.ico; }
location /websocket {
# Pass request to the upstream alias
proxy_pass http://daphne;
# Require http version 1.1 to allow for upgrade requests
proxy_http_version 1.1;
# We want proxy_buffering off for proxying to websockets.
proxy_buffering off;
# http://en.wikipedia.org/wiki/X-Forwarded-For
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if you use HTTPS:
proxy_set_header X-Forwarded-Proto https;
# pass the Host: header from the client for the sake of redirects
proxy_set_header Host $http_host;
# We've set the Host header, so we don't need Nginx to muddle
# about with redirects
proxy_redirect off;
# Depending on the request value, set the Upgrade and
# connection headers
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location / {
# Add trailing / if missing
rewrite ^(.*)$http_host(.*[^/])$ $1$http_host$2/ permanent;
uwsgi_read_timeout 120s;
uwsgi_pass uwsgi;
include /etc/nginx/uwsgi_params;
{%- if extra_nginx_include is defined %}
include {{ extra_nginx_include }};
{%- endif %}
proxy_set_header X-Forwarded-Port 443;
uwsgi_param HTTP_X_FORWARDED_PORT 443;
}
}
}

1
tools/docker-compose/ansible/roles/sources/templates/secrets.yml.j2 Normal file
View File

@@ -0,0 +1 @@
{{ item.item }}: '{{ lookup('vars', item.item, default='') or lookup('password', '/dev/null chars=ascii_letters') }}'

1
tools/docker-compose/ansible/roles/sources/templates/websocket_secret.py.j2 Normal file
View File

@@ -0,0 +1 @@
BROADCAST_WEBSOCKET_SECRET = "{{ broadcast_websocket_secret | b64encode }}"

6
tools/docker-compose/ansible/sources.yml Normal file
View File

@@ -0,0 +1,6 @@
---
- name: Render AWX Dockerfile and sources
hosts: localhost
gather_facts: true
roles:
- {role: sources}

3
tools/docker-compose/bootstrap_development.sh
View File

@@ -9,9 +9,6 @@ ansible -i "127.0.0.1," -c local -v -m wait_for -a "path=/var/run/redis/redis.so
# TODO: FIX
#/etc/init.d/ssh start
ansible -i "127.0.0.1," -c local -v -m postgresql_user --become-user postgres -a "name=awx-dev password=AWXsome1 encrypted=yes login_user=postgres login_password=postgrespass login_host=postgres" all
ansible -i "127.0.0.1," -c local -v -m postgresql_db --become-user postgres -a "name=awx-dev owner=awx-dev login_user=postgres login_password=postgrespass login_host=postgres" all
# Move to the source directory so we can bootstrap
if [ -f "/awx_devel/manage.py" ]; then
cd /awx_devel

60
tools/docker-compose/docs/data_migration.md Normal file
View File

@@ -0,0 +1,60 @@
# Migrating Data from Local Docker
If you are migrating data from a Local Docker installation (17.0.1 and prior), you can
migrate your data to the development environment via the migrate.yml playbook, or by using the manual steps described below.
> Note: This will also convert your postgresql bind-mount into a docker volume.
### Migrate data with migrate.yml
If you had a custom pgdocker or awxcompose location, you will need to set the `postgres_data_dir` and `old_docker_compose_dir` variables.
1. Run the [migrate playbook](./ansible/migrate.yml) to migrate your data to the new postgresql container and convert the data directory to a volume mount.
```bash
$ ansible-playbook migrate.yml -e "migrate_local_docker=true" -e "postgres_data_dir=~/.awx/pgdocker" -e "old_docker_compose_dir=~/.awx/awxcompose"
```
2. Change directory to the top of your awx checkout and start your containers
```bash
$ make docker-compose
```
3. After ensuring your data has been successfully migrated, you may delete your old data directory (typically stored at `~/.awx/pgdocker`).
### Migrating data manually
1. With Local Docker still running, perform a pg_dumpall:
> Note: If Local Docker is no longer running
`docker-compose -f ~/.awx/awxcompose/docker-compose.yml up postgres`
```bash
$ docker-compose -f ~/.awx/awxcompose/docker-compose.yml exec postgres pg_dumpall -U awx > awx_dump.sql
```
2. Remove all local docker containers (specifically awx_postgres)
```bash
$ docker -f rm awx_postgres
```
3. Template the new docker-compose.yml
```bash
$ ansible-playbook -i tools/ansible/inventory tools/ansible/sources.yml
```
4. Start a container with a volume (using the new tools/docker-compose/_sources/docker-compose.yml)
```bash
$ docker-compose -f ../docker-compose/_sources/docker-compose.yml up postgres
```
5. Restore to new `awx_postgres`
```bash
$ docker-compose -f ../docker-compose/_sources/docker-compose.yml exec -T postgres psql -U awx -d awx -p 5432 < awx_dump.sql
```
6. Run the docker-compose.yml to start the containers
```bash
$ docker-compose -f ../docker-compose/_sources/docker-compose.yml up task
```
7. Check to ensure your data migration was successful, then you can delete your the `awx_dump.sql` backup and your old data directory.

49
tools/docker-compose/docs/logstash.md Normal file
View File

@@ -0,0 +1,49 @@
# How to use the logstash container
#### Modify the docker-compose.yml
Uncomment the following lines in the `docker-compose.yml`
```
#- logstash
...
#logstash:
# build:
# context: ./docker-compose
# dockerfile: Dockerfile-logstash
```
POST the following content to `/api/v2/settings/logging/` (this uses
authentication set up inside of the logstash configuration file).
```
{
"LOG_AGGREGATOR_HOST": "http://logstash",
"LOG_AGGREGATOR_PORT": 8085,
"LOG_AGGREGATOR_TYPE": "logstash",
"LOG_AGGREGATOR_USERNAME": "awx_logger",
"LOG_AGGREGATOR_PASSWORD": "workflows",
"LOG_AGGREGATOR_LOGGERS": [
"awx",
"activity_stream",
"job_events",
"system_tracking"
],
"LOG_AGGREGATOR_INDIVIDUAL_FACTS": false,
"LOG_AGGREGATOR_TOWER_UUID": "991ac7e9-6d68-48c8-bbde-7ca1096653c6",
"LOG_AGGREGATOR_ENABLED": true
}
```
> Note: HTTP must be specified in the `LOG_AGGREGATOR_HOST` if you are using the docker development environment.
An example of how to view the most recent logs from the container:
```
docker exec -i -t $(docker ps -aqf "name=tools_logstash_1") tail -n 50 /logstash.log
```
#### How to add logstash plugins
Add any plugins you need in `tools/elastic/logstash/Dockerfile` before running the container.

19
tools/docker-compose/inventory Normal file
View File

@@ -0,0 +1,19 @@
localhost ansible_connection=local ansible_python_interpreter="/usr/bin/env python3"
[all:vars]
# AWX-Managed Database Settings
# If left blank, these will be generated upon install.
# Values are written out to tools/docker-compose/_sources/secrets/
# pg_password=""
# broadcast_websocket_secret=""
# secret_key=""
# External Database Settings
# pg_host=""
# pg_password=""
# pg_username=""
# pg_hostname=""
# awx_image="ansible/awx"
# migrate_local_docker=false

2
tools/docker-compose/start_tests.sh
View File

@@ -5,5 +5,5 @@ cd /awx_devel
make clean
make awx-link
cp awx/settings/local_settings.py.docker_compose awx/settings/local_settings.py
cp tools/docker-compose/ansible/roles/sources/files/local_settings.py awx/settings/local_settings.py
make "${1:-test}"