External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-01-11 01:57:35 -03:30
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #9289 from rooftopcellist/docker-community

Browse Source 
Replace Local Docker Install with Community Docker-Compose

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
This commit is contained in:
softwarefactory-project-zuul[bot] 2021-02-22 23:25:24 +00:00 committed by GitHub
commit 5e6c978a47
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
55 changed files with 545 additions and 1042 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.env
View File

@ -1,3 +1,3 @@
PYTHONUNBUFFERED=true
SELENIUM_DOCKER_TAG=latest
COMPOSE_PROJECT_NAME=tools

5
.gitignore vendored
View File

@ -36,7 +36,11 @@ awx/ui_next/.env.local
awx/ui_next/instrumented
rsyslog.pid
tools/prometheus/data
tools/docker-compose/ansible/awx_dump.sql
tools/docker-compose/Dockerfile
tools/docker-compose/_build
tools/docker-compose/_sources
tools/docker-compose/overrides/
# Tower setup playbook testing
setup/test/roles/postgresql
@ -146,7 +150,6 @@ use_dev_supervisor.txt
.idea/*
*.unison.tmp
*.#
/tools/docker-compose/overrides/
/awx/ui_next/.ui-built
/Dockerfile
/_build/

224
CONTRIBUTING.md
View File

@ -11,24 +11,15 @@ Have questions about this document or anything not covered here? Come chat with
* [Prerequisites](#prerequisites)
* [Docker](#docker)
* [Docker compose](#docker-compose)
* [Node and npm](#node-and-npm)
* [Build the environment](#build-the-environment)
* [Frontend Development](#frontend-development)
* [Build and Run the Development Environment](#build-and-run-the-development-environment)
* [Fork and clone the AWX repo](#fork-and-clone-the-awx-repo)
* [Create local settings](#create-local-settings)
* [Build the base image](#build-the-base-image)
* [Build the user interface](#build-the-user-interface)
* [Running the environment](#running-the-environment)
* [Start the containers](#start-the-containers)
* [Start from the container shell](#start-from-the-container-shell)
* [Post Build Steps](#post-build-steps)
* [Start a shell](#start-a-shell)
* [Create a superuser](#create-a-superuser)
* [Load the data](#load-the-data)
* [Building API Documentation](#build-api-documentation)
* [Building API Documentation](#building-api-documentation)
* [Accessing the AWX web interface](#accessing-the-awx-web-interface)
* [Purging containers and images](#purging-containers-and-images)
* [What should I work on?](#what-should-i-work-on)
* [Submitting Pull Requests](#submitting-pull-requests)
* [PR Checks run by Zuul](#pr-checks-run-by-zuul)
* [Reporting Issues](#reporting-issues)
## Things to know prior to submitting code
@ -42,7 +33,7 @@ Have questions about this document or anything not covered here? Come chat with
## Setting up your development environment
The AWX development environment workflow and toolchain is based on Docker, and the docker-compose tool, to provide dependencies, services, and databases necessary to run all of the components. It also binds the local source tree into the development container, making it possible to observe and test changes in real time.
The AWX development environment workflow and toolchain uses Docker and the docker-compose tool, to provide dependencies, services, and databases necessary to run all of the components. It also bind-mounts the local source tree into the development container, making it possible to observe and test changes in real time.
### Prerequisites
@ -55,29 +46,19 @@ respectively.
For Linux platforms, refer to the following from Docker:
**Fedora**
* **Fedora** - https://docs.docker.com/engine/installation/linux/docker-ce/fedora/
> https://docs.docker.com/engine/installation/linux/docker-ce/fedora/
* **CentOS** - https://docs.docker.com/engine/installation/linux/docker-ce/centos/
**CentOS**
* **Ubuntu** - https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/
> https://docs.docker.com/engine/installation/linux/docker-ce/centos/
* **Debian** - https://docs.docker.com/engine/installation/linux/docker-ce/debian/
**Ubuntu**
* **Arch** - https://wiki.archlinux.org/index.php/Docker
> https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/
#### Docker Compose
**Debian**
> https://docs.docker.com/engine/installation/linux/docker-ce/debian/
**Arch**
> https://wiki.archlinux.org/index.php/Docker
#### Docker compose
If you're not using Docker for Mac, or Docker for Windows, you may need, or choose to, install the Docker compose Python module separately, in which case you'll need to run the following:
If you're not using Docker for Mac, or Docker for Windows, you may need, or choose to, install the `docker-compose` Python module separately.
```bash
(host)$ pip3 install docker-compose
@ -87,186 +68,15 @@ If you're not using Docker for Mac, or Docker for Windows, you may need, or choo
See [the ui development documentation](awx/ui_next/CONTRIBUTING.md).
### Build the environment
#### Fork and clone the AWX repo
If you have not done so already, you'll need to fork the AWX repo on GitHub. For more on how to do this, see [Fork a Repo](https://help.github.com/articles/fork-a-repo/).
#### Create local settings
### Build and Run the Development Environment
AWX will import the file `awx/settings/local_settings.py` and combine it with defaults in `awx/settings/defaults.py`. This file is required for starting the development environment and startup will fail if it's not provided.
See the [README.md](./tools/docker-compose/README.md) for docs on how to build the awx_devel image and run the development environment.
An example is provided. Make a copy of it, and edit as needed (the defaults are usually fine):
```bash
(host)$ cp awx/settings/local_settings.py.docker_compose awx/settings/local_settings.py
```
#### Build the base image
The AWX base container image (defined in `tools/docker-compose/Dockerfile`) contains basic OS dependencies and symbolic links into the development environment that make running the services easy.
Run the following to build the image:
```bash
(host)$ make docker-compose-build
```
**NOTE**
> The image will need to be rebuilt, if the Python requirements or OS dependencies change.
Once the build completes, you will have a `ansible/awx_devel` image in your local image cache. Use the `docker images` command to view it, as follows:
```bash
(host)$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_devel latest ba9ec3e8df74 26 minutes ago 1.42GB
```
#### Build the user interface
Run the following to build the AWX UI:
```bash
(host) $ make ui-devel
```
See [the ui development documentation](awx/ui/README.md) for more information on using the frontend development, build, and test tooling.
### Running the environment
#### Start the containers
Start the development containers by running the following:
```bash
(host)$ make docker-compose
```
The above utilizes the image built in the previous step, and will automatically start all required services and dependent containers. Once the containers launch, your session will be attached to the *awx* container, and you'll be able to watch log messages and events in real time. You will see messages from Django and the front end build process.
If you start a second terminal session, you can take a look at the running containers using the `docker ps` command. For example:
```bash
# List running containers
(host)$ docker ps
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
44251b476f98 gcr.io/ansible-tower-engineering/awx_devel:devel "/entrypoint.sh /bin…" 27 seconds ago Up 23 seconds 0.0.0.0:6899->6899/tcp, 0.0.0.0:7899-7999->7899-7999/tcp, 0.0.0.0:8013->8013/tcp, 0.0.0.0:8043->8043/tcp, 0.0.0.0:8080->8080/tcp, 22/tcp, 0.0.0.0:8888->8888/tcp tools_awx_run_9e820694d57e
40de380e3c2e redis:latest "docker-entrypoint.s…" 28 seconds ago Up 26 seconds
b66a506d3007 postgres:12 "docker-entrypoint.s…" 28 seconds ago Up 26 seconds 0.0.0.0:5432->5432/tcp tools_postgres_1
```
**NOTE**
> The Makefile assumes that the image you built is tagged with your current branch. This allows you to build images for different contexts or branches. When starting the containers, you can choose a specific branch by setting `COMPOSE_TAG=<branch name>` in your environment.
> For example, you might be working in a feature branch, but you want to run the containers using the `devel` image you built previously. To do that, start the containers using the following command: `$ COMPOSE_TAG=devel make docker-compose`
##### Wait for migrations to complete
The first time you start the environment, database migrations need to run in order to build the PostgreSQL database. It will take few moments, but eventually you will see output in your terminal session that looks like the following:
```bash
awx_1 | Operations to perform:
awx_1 | Synchronize unmigrated apps: solo, api, staticfiles, debug_toolbar, messages, channels, django_extensions, ui, rest_framework, polymorphic
awx_1 | Apply all migrations: sso, taggit, sessions, sites, kombu_transport_django, social_auth, contenttypes, auth, conf, main
awx_1 | Synchronizing apps without migrations:
awx_1 | Creating tables...
awx_1 | Running deferred SQL...
awx_1 | Installing custom SQL...
awx_1 | Running migrations:
awx_1 | Rendering model states... DONE
awx_1 | Applying contenttypes.0001_initial... OK
awx_1 | Applying contenttypes.0002_remove_content_type_name... OK
awx_1 | Applying auth.0001_initial... OK
awx_1 | Applying auth.0002_alter_permission_name_max_length... OK
awx_1 | Applying auth.0003_alter_user_email_max_length... OK
awx_1 | Applying auth.0004_alter_user_username_opts... OK
awx_1 | Applying auth.0005_alter_user_last_login_null... OK
awx_1 | Applying auth.0006_require_contenttypes_0002... OK
awx_1 | Applying taggit.0001_initial... OK
awx_1 | Applying taggit.0002_auto_20150616_2121... OK
awx_1 | Applying main.0001_initial... OK
awx_1 | Applying main.0002_squashed_v300_release... OK
awx_1 | Applying main.0003_squashed_v300_v303_updates... OK
awx_1 | Applying main.0004_squashed_v310_release... OK
awx_1 | Applying conf.0001_initial... OK
awx_1 | Applying conf.0002_v310_copy_tower_settings... OK
...
```
Once migrations are completed, you can begin using AWX.
#### Start from the container shell
Often times you'll want to start the development environment without immediately starting all of the services in the *awx* container, and instead be taken directly to a shell. You can do this with the following:
```bash
(host)$ make docker-compose-test
```
Using `docker exec`, this will create a session in the running *awx* container, and place you at a command prompt, where you can run shell commands inside the container.
If you want to start and use the development environment, you'll first need to bootstrap it by running the following command:
```bash
(container)# /usr/bin/bootstrap_development.sh
```
The above will do all the setup tasks, including running database migrations, so it may take a couple minutes. Once it's done it
will drop you back to the shell.
In order to launch all developer services:
```bash
(container)# /usr/bin/launch_awx.sh
```
`launch_awx.sh` also calls `bootstrap_development.sh` so if all you are doing is launching the supervisor to start all services, you don't
need to call `bootstrap_development.sh` first.
### Post Build Steps
Before you can log in and use the system, you will need to create an admin user. Optionally, you may also want to load some demo data.
##### Start a shell
To create the admin user, and load demo data, you first need to start a shell session on the *awx* container. In a new terminal session, use the `docker exec` command as follows to start the shell session:
```bash
(host)$ docker exec -it tools_awx_1 bash
```
This creates a session in the *awx* containers, just as if you were using `ssh`, and allows you execute commands within the running container.
##### Create an admin user
Before you can log into AWX, you need to create an admin user. With this user you will be able to create more users, and begin configuring the server. From within the container shell, run the following command:
```bash
(container)# awx-manage createsuperuser
```
You will be prompted for a username, an email address, and a password, and you will be asked to confirm the password. The email address is not important, so just enter something that looks like an email address. Remember the username and password, as you will use them to log into the web interface for the first time.
##### Load demo data
You can optionally load some demo data. This will create a demo project, inventory, and job template. From within the container shell, run the following to load the data:
```bash
(container)# awx-manage create_preload_data
```
**NOTE**
> This information will persist in the database running in the `tools_postgres_1` container, until the container is removed. You may periodically need to recreate
this container, and thus the database, if the database schema changes in an upstream commit.
##### Building API Documentation
### Building API Documentation
AWX includes support for building [Swagger/OpenAPI
documentation](https://swagger.io). To build the documentation locally, run:
@ -284,7 +94,7 @@ is an example of one such service.
You can now log into the AWX web interface at [https://localhost:8043](https://localhost:8043), and access the API directly at [https://localhost:8043/api/](https://localhost:8043/api/).
To log in use the admin user and password you created above in [Create an admin user](#create-an-admin-user).
[Create an admin user](./tools/docker-compose/README.md#create-an-admin-user) if needed.
### Purging containers and images
@ -335,7 +145,7 @@ Sometimes it might take us a while to fully review your PR. We try to keep the `
All submitted PRs will have the linter and unit tests run against them via Zuul, and the status reported in the PR.
## PR Checks ran by Zuul
## PR Checks run by Zuul
Zuul jobs for awx are defined in the [zuul-jobs](https://github.com/ansible/zuul-jobs) repo.
Zuul runs the following checks that must pass:

247
INSTALL.md
View File

@ -12,7 +12,6 @@ This document provides a guide for installing AWX.
+ [System Requirements](#system-requirements)
+ [Choose a deployment platform](#choose-a-deployment-platform)
+ [Official vs Building Images](#official-vs-building-images)
* [Upgrading from previous versions](#upgrading-from-previous-versions)
* [OpenShift](#openshift)
+ [Prerequisites](#prerequisites-1)
+ [Pre-install steps](#pre-install-steps)
@ -29,17 +28,6 @@ This document provides a guide for installing AWX.
+ [Post-install](#post-install-1)
+ [Accessing AWX](#accessing-awx-1)
+ [SSL Termination](#ssl-termination)
* [Docker-Compose](#docker-compose)
+ [Prerequisites](#prerequisites-3)
+ [Pre-install steps](#pre-install-steps-2)
- [Deploying to a remote host](#deploying-to-a-remote-host)
- [Inventory variables](#inventory-variables)
- [Docker registry](#docker-registry)
- [Proxy settings](#proxy-settings)
- [PostgreSQL](#postgresql-1)
+ [Run the installer](#run-the-installer-2)
+ [Post-install](#post-install-2)
+ [Accessing AWX](#accessing-awx-2)
- [Installing the AWX CLI](#installing-the-awx-cli)
* [Building the CLI Documentation](#building-the-cli-documentation)
@ -83,10 +71,6 @@ Before you can run a deployment, you'll need the following installed in your loc
- [GNU Make](https://www.gnu.org/software/make/)
- [Git](https://git-scm.com/) Requires Version 1.8.4+
- Python 3.6+
- [Node 14.x LTS version](https://nodejs.org/en/download/)
+ This is only required if you're [building your own container images](#official-vs-building-images) with `use_container_for_build=false`
- [NPM 6.x LTS](https://docs.npmjs.com/)
+ This is only required if you're [building your own container images](#official-vs-building-images) with `use_container_for_build=false`
### System Requirements
@ -100,14 +84,13 @@ The system that runs the AWX service will need to satisfy the following requirem
### Choose a deployment platform
We currently support running AWX as a containerized application using Docker images deployed to either an OpenShift cluster, a Kubernetes cluster, or docker-compose. The remainder of this document will walk you through the process of building the images, and deploying them to either platform.
We currently support running AWX as a containerized application using Docker images deployed to either an OpenShift cluster or a Kubernetes cluster. The remainder of this document will walk you through the process of building the images, and deploying them to either platform.
The [installer](./installer) directory contains an [inventory](./installer/inventory) file, and a playbook, [install.yml](./installer/install.yml). You'll begin by setting variables in the inventory file according to the platform you wish to use, and then you'll start the image build and deployment process by running the playbook.
In the sections below, you'll find deployment details and instructions for each platform:
- [OpenShift](#openshift)
- [Kubernetes](#kubernetes)
- [Docker Compose](#docker-compose).
### Official vs Building Images
@ -130,28 +113,15 @@ If these variables are present then all deployments will use these hosted images
> Multiple versions are provided. `latest` always pulls the most recent. You may also select version numbers at different granularities: 1, 1.0, 1.0.1, 1.0.0.123
*use_container_for_build*
> Use a local distribution build container image for building the AWX package. This is helpful if you don't want to bother installing the build-time dependencies as it is taken care of already.
## Upgrading from previous versions
Upgrading AWX involves rerunning the install playbook. Download a newer release from [https://github.com/ansible/awx/releases](https://github.com/ansible/awx/releases) and re-populate the inventory file with your customized variables.
For convenience, you can create a file called `vars.yml`:
To build your own container use the `build.yml` playbook:
```
admin_password: 'adminpass'
pg_password: 'pgpass'
secret_key: 'mysupersecret'
ansible-playbook tools/ansible/build.yml -e awx_version=test-build
```
And pass it to the installer:
The resulting image will automatically be pushed to a registry if `docker_registry` is defined.
```
$ ansible-playbook -i inventory install.yml -e @vars.yml
```
## OpenShift
@ -437,213 +407,6 @@ If your provider is able to allocate an IP Address from the Ingress controller t
Unlike Openshift's `Route` the Kubernetes `Ingress` doesn't yet handle SSL termination. As such the default configuration will only expose AWX through HTTP on port 80. You are responsible for configuring SSL support until support is added (either to Kubernetes or AWX itself).
## Docker-Compose
### Prerequisites
- [Docker](https://docs.docker.com/engine/installation/) on the host where AWX will be deployed. After installing Docker, the Docker service must be started (depending on your OS, you may have to add the local user that uses Docker to the ``docker`` group, refer to the documentation for details)
- [docker-compose](https://pypi.org/project/docker-compose/) Python module.
+ This also installs the `docker` Python module, which is incompatible with `docker-py`. If you have previously installed `docker-py`, please uninstall it.
- [Docker Compose](https://docs.docker.com/compose/install/).
### Pre-install steps
#### Deploying to a remote host
By default, the delivered [installer/inventory](./installer/inventory) file will deploy AWX to the local host. It is possible, however, to deploy to a remote host. The [installer/install.yml](./installer/install.yml) playbook can be used to build images on the local host, and ship the built images to, and run deployment tasks on, a remote host. To do this, modify the [installer/inventory](./installer/inventory) file, by commenting out `localhost`, and adding the remote host.
For example, suppose you wish to build images locally on your CI/CD host, and deploy them to a remote host named *awx-server*. To do this, add *awx-server* to the [installer/inventory](./installer/inventory) file, and comment out or remove `localhost`, as demonstrated by the following:
```yaml
# localhost ansible_connection=local
awx-server
[all:vars]
...
```
In the above example, image build tasks will be delegated to `localhost`, which is typically where the clone of the AWX project exists. Built images will be archived, copied to remote host, and imported into the remote Docker image cache. Tasks to start the AWX containers will then execute on the remote host.
If you choose to use the official images then the remote host will be the one to pull those images.
**Note**
> You may also want to set additional variables to control how Ansible connects to the host. For more information about this, view [Behavioral Inventory Parameters](http://docs.ansible.com/ansible/latest/intro_inventory.html#id12).
> As mentioned above, in [Prerequisites](#prerequisites-1), the prerequisites are required on the remote host.
> When deploying to a remote host, the playbook does not execute tasks with the `become` option. For this reason, make sure the user that connects to the remote host has privileges to run the `docker` command. This typically means that non-privileged users need to be part of the `docker` group.
#### Inventory variables
Before starting the install process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section:
*admin_password*
> Provide a strong password to prevent malicious logins after the installation.
*postgres_data_dir*
> If you're using the default PostgreSQL container (see [PostgreSQL](#postgresql-1) below), provide a path that can be mounted to the container, and where the database can be persisted.
*host_port*
> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container. If undefined no port will be exposed. Defaults to *80*.
*host_port_ssl*
> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container for SSL support. If undefined no port will be exposed. Defaults to *443*, only works if you also set `ssl_certificate` (see below).
*ssl_certificate*
> Optionally, provide the path to a file that contains a certificate and its private key. This needs to be a .pem-file
*docker_compose_dir*
> When using docker-compose, the `docker-compose.yml` file will be created there (default `~/.awx/awxcompose`).
*custom_venv_dir*
> Adds the custom venv environments from the local host to be passed into the containers at install.
*ca_trust_dir*
> If you're using a non trusted CA, provide a path where the untrusted Certs are stored on your Host.
#### Docker registry
If you wish to tag and push built images to a Docker registry, set the following variables in the inventory file:
*docker_registry*
> IP address and port, or URL, for accessing a registry.
*docker_registry_repository*
> Namespace to use when pushing and pulling images to and from the registry. Defaults to *awx*.
*docker_registry_username*
> Username of the user that will push images to the registry. Defaults to *developer*.
**Note**
> These settings are ignored if using official images
#### Proxy settings
*http_proxy*
> IP address and port, or URL, for using an http_proxy.
*https_proxy*
> IP address and port, or URL, for using an https_proxy.
*no_proxy*
> Exclude IP address or URL from the proxy.
#### PostgreSQL
AWX requires access to a PostgreSQL database, and by default, one will be created and deployed in a container, and data will be persisted to a host volume. In this scenario, you must set the value of `postgres_data_dir` to a path that can be mounted to the container. When the container is stopped, the database files will still exist in the specified path.
If you wish to use an external database, in the inventory file, set the value of `pg_hostname`, and update `pg_username`, `pg_password`, `pg_admin_password`, `pg_database`, and `pg_port` with the connection information.
### Run the installer
If you are not pushing images to a Docker registry, start the install by running the following:
```bash
# Set the working directory to installer
$ cd installer
# Run the Ansible playbook
$ ansible-playbook -i inventory install.yml
```
If you're pushing built images to a repository, then use the `-e` option to pass the registry password as follows, replacing *password* with the password of the username assigned to `docker_registry_username` (note that you will also need to remove `dockerhub_base` and `dockerhub_version` from the inventory file):
```bash
# Set the working directory to installer
$ cd installer
# Run the Ansible playbook
$ ansible-playbook -i inventory -e docker_registry_password=password install.yml
```
### Post-install
After the playbook run completes, Docker starts a series of containers that provide the services that make up AWX. You can view the running containers using the `docker ps` command.
If you're deploying using Docker Compose, container names will be prefixed by the name of the folder where the docker-compose.yml file is created (by default, `awx`).
Immediately after the containers start, the *awx_task* container will perform required setup tasks, including database migrations. These tasks need to complete before the web interface can be accessed. To monitor the progress, you can follow the container's STDOUT by running the following:
```bash
# Tail the awx_task log
$ docker logs -f awx_task
```
You will see output similar to the following:
```bash
Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | SUCCESS => {
"changed": false,
"db": "awx"
}
Operations to perform:
Synchronize unmigrated apps: solo, api, staticfiles, messages, channels, django_extensions, ui, rest_framework, polymorphic
Apply all migrations: sso, taggit, sessions, sites, kombu_transport_django, social_auth, contenttypes, auth, conf, main
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Installing custom SQL...
Running migrations:
Rendering model states... DONE
Applying contenttypes.0001_initial... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0001_initial... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying taggit.0001_initial... OK
Applying taggit.0002_auto_20150616_2121... OK
Applying main.0001_initial... OK
...
```
Once migrations complete, you will see the following log output, indicating that migrations have completed:
```bash
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> <User: admin>
>>> Default organization added.
Demo Credential, Inventory, and Job Template added.
Successfully registered instance awx
(changed: True)
Creating instance group tower
Added instance awx to tower
(changed: True)
...
```
### Accessing AWX
The AWX web server is accessible on the deployment host, using the *host_port* value set in the *inventory* file. The default URL is [http://localhost](http://localhost).
You will prompted with a login dialog. The default administrator username is `admin`, and the password is `password`.
# Installing the AWX CLI
`awx` is the official command-line client for AWX. It:

75
Makefile
View File

@ -25,6 +25,8 @@ SCL_PREFIX ?=
CELERY_SCHEDULE_FILE ?= /var/lib/awx/beat.db
DEV_DOCKER_TAG_BASE ?= gcr.io/ansible-tower-engineering
DEVEL_IMAGE_NAME ?= $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
# Python packages to install only from source (not from binary wheels)
# Comma separated list
SRC_ONLY_PKGS ?= cffi,pycparser,psycopg2,twilio,pycurl
@ -64,7 +66,7 @@ I18N_FLAG_FILE = .i18n_built
receiver test test_unit test_coverage coverage_html \
dev_build release_build release_clean sdist \
ui-docker-machine ui-docker ui-release ui-devel \
ui-test ui-deps ui-test-ci VERSION
ui-test ui-deps ui-test-ci VERSION docker-compose-sources
clean-tmp:
rm -rf tmp/
@ -511,6 +513,12 @@ ui-release: awx/ui_next/node_modules $(UI_BUILD_FLAG_FILE)
ui-devel: awx/ui_next/node_modules
@$(MAKE) -B $(UI_BUILD_FLAG_FILE)
ui-devel-instrumented: awx/ui_next/node_modules
$(NPM_BIN) --prefix awx/ui_next --loglevel warn run start-instrumented
ui-devel-test: awx/ui_next/node_modules
$(NPM_BIN) --prefix awx/ui_next --loglevel warn run start
ui-zuul-lint-and-test:
$(NPM_BIN) --prefix awx/ui_next install
$(NPM_BIN) run --prefix awx/ui_next lint
@ -559,31 +567,31 @@ docker-auth:
awx/projects:
@mkdir -p $@
# Docker isolated rampart
docker-compose-isolated: awx/projects
CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/docker-isolated-override.yml up
COMPOSE_UP_OPTS ?=
# Docker Compose Development environment
docker-compose: docker-auth awx/projects
CURRENT_UID=$(shell id -u) OS="$(shell docker info | grep 'Operating System')" TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml $(COMPOSE_UP_OPTS) up --no-recreate awx
docker-compose-sources:
ansible-playbook -i tools/docker-compose/inventory tools/docker-compose/ansible/sources.yml \
-e awx_image=$(DEV_DOCKER_TAG_BASE)/awx_devel \
-e awx_image_tag=$(COMPOSE_TAG)
docker-compose: docker-auth awx/projects docker-compose-sources
docker-compose -f tools/docker-compose/_sources/docker-compose.yml $(COMPOSE_UP_OPTS) up --no-recreate awx
docker-compose-cluster: docker-auth awx/projects
CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose-cluster.yml up
docker-compose -f tools/docker-compose-cluster.yml up
docker-compose-credential-plugins: docker-auth awx/projects
docker-compose-credential-plugins: docker-auth awx/projects docker-compose-sources
echo -e "\033[0;31mTo generate a CyberArk Conjur API key: docker exec -it tools_conjur_1 conjurctl account create quick-start\033[0m"
CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/docker-credential-plugins-override.yml up --no-recreate awx
docker-compose -f tools/docker-compose/_sources/docker-compose.yml -f tools/docker-credential-plugins-override.yml up --no-recreate awx
docker-compose-test: docker-auth awx/projects
cd tools && CURRENT_UID=$(shell id -u) OS="$(shell docker info | grep 'Operating System')" TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /bin/bash
docker-compose-test: docker-auth awx/projects docker-compose-sources
docker-compose -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports awx /bin/bash
docker-compose-runtest: awx/projects
cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /start_tests.sh
docker-compose-runtest: awx/projects docker-compose-sources
docker-compose -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports awx /start_tests.sh
docker-compose-build-swagger: awx/projects
cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports --no-deps awx /start_tests.sh swagger
docker-compose-build-swagger: awx/projects docker-compose-sources
docker-compose -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports --no-deps awx /start_tests.sh swagger
detect-schema-change: genschema
curl https://s3.amazonaws.com/awx-public-ci-files/schema.json -o reference-schema.json
@ -591,24 +599,14 @@ detect-schema-change: genschema
diff -u -b reference-schema.json schema.json
docker-compose-clean: awx/projects
cd tools && TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose rm -sf
docker-compose -f tools/docker-compose/_sources/docker-compose.yml rm -sf
# Base development image build
docker-compose-build:
ansible-playbook installer/dockerfile.yml -e build_dev=True
docker build -t ansible/awx_devel \
ansible-playbook tools/ansible/dockerfile.yml -e build_dev=True
docker build -t $(DEVEL_IMAGE_NAME) \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) .
docker tag ansible/awx_devel $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
#docker push $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
# For use when developing on "isolated" AWX deployments
docker-compose-isolated-build: docker-compose-build
docker build -t ansible/awx_isolated \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f tools/docker-isolated/Dockerfile .
docker tag ansible/awx_isolated $(DEV_DOCKER_TAG_BASE)/awx_isolated:$(COMPOSE_TAG)
#docker push $(DEV_DOCKER_TAG_BASE)/awx_isolated:$(COMPOSE_TAG)
docker-clean:
$(foreach container_id,$(shell docker ps -f name=tools_awx -aq),docker stop $(container_id); docker rm -f $(container_id);)
@ -620,11 +618,11 @@ docker-clean-volumes: docker-compose-clean
docker-refresh: docker-clean docker-compose
# Docker Development Environment with Elastic Stack Connected
docker-compose-elk: docker-auth awx/projects
CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/elastic/docker-compose.logstash-link.yml -f tools/elastic/docker-compose.elastic-override.yml up --no-recreate
docker-compose-elk: docker-auth awx/projects docker-compose-sources
docker-compose -f tools/docker-compose/_sources/docker-compose.yml -f tools/elastic/docker-compose.logstash-link.yml -f tools/elastic/docker-compose.elastic-override.yml up --no-recreate
docker-compose-cluster-elk: docker-auth awx/projects
TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose-cluster.yml -f tools/elastic/docker-compose.logstash-link-cluster.yml -f tools/elastic/docker-compose.elastic-override.yml up --no-recreate
docker-compose-cluster-elk: docker-auth awx/projects docker-compose-sources
docker-compose -f tools/docker-compose/_sources/docker-compose.yml -f tools/elastic/docker-compose.logstash-link-cluster.yml -f tools/elastic/docker-compose.elastic-override.yml up --no-recreate
prometheus:
docker run -u0 --net=tools_default --link=`docker ps | egrep -o "tools_awx(_run)?_([^ ]+)?"`:awxweb --volume `pwd`/tools/prometheus:/prometheus --name prometheus -d -p 0.0.0.0:9090:9090 prom/prometheus --web.enable-lifecycle --config.file=/prometheus/prometheus.yml
@ -643,11 +641,12 @@ psql-container:
VERSION:
@echo "awx: $(VERSION)"
Dockerfile: installer/roles/dockerfile/templates/Dockerfile.j2
ansible-playbook installer/dockerfile.yml
Dockerfile: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
ansible-playbook tools/ansible/dockerfile.yml
Dockerfile.kube-dev: installer/roles/dockerfile/templates/Dockerfile.j2
ansible-playbook installer/dockerfile.yml \
# TODO: test kube-dev target after changing dockerfile.yml location - refer to https://github.com/ansible/awx/commit/7c8bd471980d26083d4c4e11067bb53730175496
Dockerfile.kube-dev: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
ansible-playbook tools/ansible/dockerfile.yml \
-e dockerfile_name=Dockerfile.kube-dev \
-e kube_dev=True \
-e template_dest=_build_kube_dev

2
awx/main/utils/external_logging.py
View File

@ -116,7 +116,7 @@ def construct_rsyslog_conf_template(settings=settings):
def reconfigure_rsyslog():
tmpl = construct_rsyslog_conf_template()
# Write config to a temp file then move it to preserve atomicity
with tempfile.TemporaryDirectory(prefix='rsyslog-conf-') as temp_dir:
with tempfile.TemporaryDirectory(dir='/var/lib/awx/rsyslog/', prefix='rsyslog-conf-') as temp_dir:
path = temp_dir + '/rsyslog.conf.temp'
with open(path, 'w') as f:
os.chmod(path, 0o640)

3
installer/install.yml
View File

@ -3,7 +3,4 @@
hosts: all
roles:
- {role: check_vars}
- {role: image_build, when: "dockerhub_base is not defined"}
- {role: image_push, when: "docker_registry is defined and dockerhub_base is not defined"}
- {role: kubernetes, when: "openshift_host is defined or kubernetes_context is defined"}
- {role: local_docker, when: "openshift_host is not defined and kubernetes_context is not defined"}

8
installer/roles/check_vars/tasks/check_docker.yml
View File

@ -1,8 +0,0 @@
# check_docker.yml
---
- name: postgres_data_dir should be defined
assert:
that:
- postgres_data_dir is defined and postgres_data_dir != ''
msg: "Set the value of 'postgres_data_dir' in the inventory file."
when: pg_hostname is not defined or pg_hostname == ''

3
installer/roles/check_vars/tasks/main.yml
View File

@ -8,6 +8,3 @@
- include_tasks: check_openshift.yml
when: openshift_host is defined and openshift_host != ''
- include_tasks: check_docker.yml
when: openshift_host is not defined or openshift_host == ''

11
installer/roles/local_docker/defaults/main.yml
View File

@ -1,11 +0,0 @@
---
dockerhub_version: "{{ lookup('file', playbook_dir + '/../VERSION') }}"
awx_image: "awx"
redis_image: "redis"
postgresql_version: "12"
postgresql_image: "postgres:{{postgresql_version}}"
compose_start_containers: true
upgrade_postgres: false

74
installer/roles/local_docker/tasks/compose.yml
View File

@ -1,74 +0,0 @@
---
- name: Create {{ docker_compose_dir }} directory
file:
path: "{{ docker_compose_dir }}"
state: directory
- name: Create Redis socket directory
file:
path: "{{ docker_compose_dir }}/redis_socket"
state: directory
mode: 0777
- name: Create Docker Compose Configuration
template:
src: "{{ item.file }}.j2"
dest: "{{ docker_compose_dir }}/{{ item.file }}"
mode: "{{ item.mode }}"
loop:
- file: environment.sh
mode: "0600"
- file: credentials.py
mode: "0600"
- file: docker-compose.yml
mode: "0600"
- file: nginx.conf
mode: "0600"
- file: redis.conf
mode: "0664"
register: awx_compose_config
- name: Render SECRET_KEY file
copy:
content: "{{ secret_key }}"
dest: "{{ docker_compose_dir }}/SECRET_KEY"
mode: 0600
register: awx_secret_key
- block:
- name: Remove AWX containers before migrating postgres so that the old postgres container does not get used
docker_compose:
project_src: "{{ docker_compose_dir }}"
state: absent
ignore_errors: true
- name: Run migrations in task container
shell: docker-compose run --rm --service-ports task awx-manage migrate --no-input
args:
chdir: "{{ docker_compose_dir }}"
- name: Start the containers
docker_compose:
project_src: "{{ docker_compose_dir }}"
restarted: "{{ awx_compose_config is changed or awx_secret_key is changed }}"
register: awx_compose_start
- name: Update CA trust in awx_web container
command: docker exec awx_web '/usr/bin/update-ca-trust'
when: awx_compose_config.changed or awx_compose_start.changed
- name: Update CA trust in awx_task container
command: docker exec awx_task '/usr/bin/update-ca-trust'
when: awx_compose_config.changed or awx_compose_start.changed
- name: Wait for launch script to create user
wait_for:
timeout: 10
delegate_to: localhost
- name: Create Preload data
command: docker exec awx_task bash -c "/usr/bin/awx-manage create_preload_data"
when: create_preload_data|bool
register: cdo
changed_when: "'added' in cdo.stdout"
when: compose_start_containers|bool

15
installer/roles/local_docker/tasks/main.yml
View File

@ -1,15 +0,0 @@
---
- name: Generate broadcast websocket secret
set_fact:
broadcast_websocket_secret: "{{ lookup('password', '/dev/null length=128') }}"
run_once: true
no_log: true
when: broadcast_websocket_secret is not defined
- import_tasks: upgrade_postgres.yml
when:
- postgres_data_dir is defined
- pg_hostname is not defined
- import_tasks: set_image.yml
- import_tasks: compose.yml

46
installer/roles/local_docker/tasks/set_image.yml
View File

@ -1,46 +0,0 @@
---
- name: Manage AWX Container Images
block:
- name: Export Docker awx image if it isnt local and there isnt a registry defined
docker_image:
name: "{{ awx_image }}"
tag: "{{ awx_version }}"
archive_path: "{{ awx_local_base_config_path|default('/tmp') }}/{{ awx_image }}_{{ awx_version }}.tar"
when: inventory_hostname != "localhost" and docker_registry is not defined
delegate_to: localhost
- name: Set docker base path
set_fact:
docker_deploy_base_path: "{{ awx_base_path|default('/tmp') }}/docker_deploy"
when: ansible_connection != "local" and docker_registry is not defined
- name: Ensure directory exists
file:
path: "{{ docker_deploy_base_path }}"
state: directory
when: ansible_connection != "local" and docker_registry is not defined
- name: Copy awx image to docker execution
copy:
src: "{{ awx_local_base_config_path|default('/tmp') }}/{{ awx_image }}_{{ awx_version }}.tar"
dest: "{{ docker_deploy_base_path }}/{{ awx_image }}_{{ awx_version }}.tar"
when: ansible_connection != "local" and docker_registry is not defined
- name: Load awx image
docker_image:
name: "{{ awx_image }}"
tag: "{{ awx_version }}"
load_path: "{{ docker_deploy_base_path }}/{{ awx_image }}_{{ awx_version }}.tar"
timeout: 300
when: ansible_connection != "local" and docker_registry is not defined
- name: Set full image path for local install
set_fact:
awx_docker_actual_image: "{{ awx_image }}:{{ awx_version }}"
when: docker_registry is not defined
when: dockerhub_base is not defined
- name: Set DockerHub Image Paths
set_fact:
awx_docker_actual_image: "{{ dockerhub_base }}/awx:{{ dockerhub_version }}"
when: dockerhub_base is defined

64
installer/roles/local_docker/tasks/upgrade_postgres.yml
View File

@ -1,64 +0,0 @@
---
- name: Create {{ postgres_data_dir }} directory
file:
path: "{{ postgres_data_dir }}"
state: directory
- name: Get full path of postgres data dir
shell: "echo {{ postgres_data_dir }}"
register: fq_postgres_data_dir
- name: Register temporary docker container
set_fact:
container_command: "docker run --rm -v '{{ fq_postgres_data_dir.stdout }}:/var/lib/postgresql' centos:8 bash -c "
- name: Check for existing Postgres data (run from inside the container for access to file)
shell:
cmd: |
{{ container_command }} "[[ -f /var/lib/postgresql/10/data/PG_VERSION ]] && echo 'exists'"
register: pg_version_file
ignore_errors: true
- name: Record Postgres version
shell: |
{{ container_command }} "cat /var/lib/postgresql/10/data/PG_VERSION"
register: old_pg_version
when: pg_version_file is defined and pg_version_file.stdout == 'exists'
- name: Determine whether to upgrade postgres
set_fact:
upgrade_postgres: "{{ old_pg_version.stdout == '10' }}"
when: old_pg_version.changed
- name: Set up new postgres paths pre-upgrade
shell: |
{{ container_command }} "mkdir -p /var/lib/postgresql/12/data/"
when: upgrade_postgres | bool
- name: Stop AWX before upgrading postgres
docker_compose:
project_src: "{{ docker_compose_dir }}"
stopped: true
when: upgrade_postgres | bool
- name: Upgrade Postgres
shell: |
docker run --rm \
-v {{ postgres_data_dir }}/10/data:/var/lib/postgresql/10/data \
-v {{ postgres_data_dir }}/12/data:/var/lib/postgresql/12/data \
-e PGUSER={{ pg_username }} -e POSTGRES_INITDB_ARGS="-U {{ pg_username }}" \
tianon/postgres-upgrade:10-to-12 --username={{ pg_username }}
when: upgrade_postgres | bool
- name: Copy old pg_hba.conf
shell: |
{{ container_command }} "cp /var/lib/postgresql/10/data/pg_hba.conf /var/lib/postgresql/12/data/pg_hba.conf"
when: upgrade_postgres | bool
- name: Remove old data directory
shell: |
{{ container_command }} "rm -rf /var/lib/postgresql/10/data"
when:
- upgrade_postgres | bool
- compose_start_containers|bool

208
installer/roles/local_docker/templates/docker-compose.yml.j2
View File

@ -1,208 +0,0 @@
#jinja2: lstrip_blocks: True
version: '2'
services:
web:
image: {{ awx_docker_actual_image }}
container_name: awx_web
depends_on:
- redis
{% if pg_hostname is not defined %}
- postgres
{% endif %}
{% if (host_port is defined) or (host_port_ssl is defined) %}
ports:
{% if (host_port_ssl is defined) and (ssl_certificate is defined) %}
- "{{ host_port_ssl }}:8053"
{% endif %}
{% if host_port is defined %}
- "{{ host_port }}:8052"
{% endif %}
{% endif %}
hostname: {{ awx_web_hostname }}
user: root
restart: unless-stopped
{% if (awx_web_container_labels is defined) and (',' in awx_web_container_labels) %}
{% set awx_web_container_labels_list = awx_web_container_labels.split(',') %}
labels:
{% for awx_web_container_label in awx_web_container_labels_list %}
- {{ awx_web_container_label }}
{% endfor %}
{% elif awx_web_container_labels is defined %}
labels:
- {{ awx_web_container_labels }}
{% endif %}
volumes:
- supervisor-socket:/var/run/supervisor
- rsyslog-socket:/var/run/awx-rsyslog/
- rsyslog-config:/var/lib/awx/rsyslog/
- "{{ docker_compose_dir }}/SECRET_KEY:/etc/tower/SECRET_KEY"
- "{{ docker_compose_dir }}/environment.sh:/etc/tower/conf.d/environment.sh"
- "{{ docker_compose_dir }}/credentials.py:/etc/tower/conf.d/credentials.py"
- "{{ docker_compose_dir }}/nginx.conf:/etc/nginx/nginx.conf:ro"
- "{{ docker_compose_dir }}/redis_socket:/var/run/redis/:rw"
{% if project_data_dir is defined %}
- "{{ project_data_dir +':/var/lib/awx/projects:rw' }}"
{% endif %}
{% if custom_venv_dir is defined %}
- "{{ custom_venv_dir +':'+ custom_venv_dir +':rw' }}"
{% endif %}
{% if ca_trust_dir is defined %}
- "{{ ca_trust_dir +':/etc/pki/ca-trust/source/anchors:ro' }}"
{% endif %}
{% if (ssl_certificate is defined) and (ssl_certificate_key is defined) %}
- "{{ ssl_certificate +':/etc/nginx/awxweb.pem:ro' }}"
- "{{ ssl_certificate_key +':/etc/nginx/awxweb_key.pem:ro' }}"
{% elif (ssl_certificate is defined) and (ssl_certificate_key is not defined) %}
- "{{ ssl_certificate +':/etc/nginx/awxweb.pem:ro' }}"
{% endif %}
{% if (awx_container_search_domains is defined) and (',' in awx_container_search_domains) %}
{% set awx_container_search_domains_list = awx_container_search_domains.split(',') %}
dns_search:
{% for awx_container_search_domain in awx_container_search_domains_list %}
- {{ awx_container_search_domain }}
{% endfor %}
{% elif awx_container_search_domains is defined %}
dns_search: "{{ awx_container_search_domains }}"
{% endif %}
{% if (awx_alternate_dns_servers is defined) and (',' in awx_alternate_dns_servers) %}
{% set awx_alternate_dns_servers_list = awx_alternate_dns_servers.split(',') %}
dns:
{% for awx_alternate_dns_server in awx_alternate_dns_servers_list %}
- {{ awx_alternate_dns_server }}
{% endfor %}
{% elif awx_alternate_dns_servers is defined %}
dns: "{{ awx_alternate_dns_servers }}"
{% endif %}
{% if (docker_compose_extra_hosts is defined) and (':' in docker_compose_extra_hosts) %}
{% set docker_compose_extra_hosts_list = docker_compose_extra_hosts.split(',') %}
extra_hosts:
{% for docker_compose_extra_host in docker_compose_extra_hosts_list %}
- "{{ docker_compose_extra_host }}"
{% endfor %}
{% endif %}
environment:
http_proxy: {{ http_proxy | default('') }}
https_proxy: {{ https_proxy | default('') }}
no_proxy: {{ no_proxy | default('') }}
{% if docker_logger is defined %}
logging:
driver: {{ docker_logger }}
{% endif %}
task:
image: {{ awx_docker_actual_image }}
container_name: awx_task
depends_on:
- redis
- web
{% if pg_hostname is not defined %}
- postgres
{% endif %}
command: /usr/bin/launch_awx_task.sh
hostname: {{ awx_task_hostname }}
user: root
restart: unless-stopped
volumes:
- supervisor-socket:/var/run/supervisor
- rsyslog-socket:/var/run/awx-rsyslog/
- rsyslog-config:/var/lib/awx/rsyslog/
- "{{ docker_compose_dir }}/SECRET_KEY:/etc/tower/SECRET_KEY"
- "{{ docker_compose_dir }}/environment.sh:/etc/tower/conf.d/environment.sh"
- "{{ docker_compose_dir }}/credentials.py:/etc/tower/conf.d/credentials.py"
- "{{ docker_compose_dir }}/redis_socket:/var/run/redis/:rw"
{% if project_data_dir is defined %}
- "{{ project_data_dir +':/var/lib/awx/projects:rw' }}"
{% endif %}
{% if custom_venv_dir is defined %}
- "{{ custom_venv_dir +':'+ custom_venv_dir +':rw' }}"
{% endif %}
{% if ca_trust_dir is defined %}
- "{{ ca_trust_dir +':/etc/pki/ca-trust/source/anchors:ro' }}"
{% endif %}
{% if ssl_certificate is defined %}
- "{{ ssl_certificate +':/etc/nginx/awxweb.pem:ro' }}"
{% endif %}
{% if (awx_container_search_domains is defined) and (',' in awx_container_search_domains) %}
{% set awx_container_search_domains_list = awx_container_search_domains.split(',') %}
dns_search:
{% for awx_container_search_domain in awx_container_search_domains_list %}
- {{ awx_container_search_domain }}
{% endfor %}
{% elif awx_container_search_domains is defined %}
dns_search: "{{ awx_container_search_domains }}"
{% endif %}
{% if (awx_alternate_dns_servers is defined) and (',' in awx_alternate_dns_servers) %}
{% set awx_alternate_dns_servers_list = awx_alternate_dns_servers.split(',') %}
dns:
{% for awx_alternate_dns_server in awx_alternate_dns_servers_list %}
- {{ awx_alternate_dns_server }}
{% endfor %}
{% elif awx_alternate_dns_servers is defined %}
dns: "{{ awx_alternate_dns_servers }}"
{% endif %}
{% if (docker_compose_extra_hosts is defined) and (':' in docker_compose_extra_hosts) %}
{% set docker_compose_extra_hosts_list = docker_compose_extra_hosts.split(',') %}
extra_hosts:
{% for docker_compose_extra_host in docker_compose_extra_hosts_list %}
- "{{ docker_compose_extra_host }}"
{% endfor %}
{% endif %}
environment:
AWX_SKIP_MIGRATIONS: "1"
http_proxy: {{ http_proxy | default('') }}
https_proxy: {{ https_proxy | default('') }}
no_proxy: {{ no_proxy | default('') }}
SUPERVISOR_WEB_CONFIG_PATH: '/etc/supervisord.conf'
redis:
image: {{ redis_image }}
container_name: awx_redis
restart: unless-stopped
environment:
http_proxy: {{ http_proxy | default('') }}
https_proxy: {{ https_proxy | default('') }}
no_proxy: {{ no_proxy | default('') }}
command: ["/usr/local/etc/redis/redis.conf"]
volumes:
- "{{ docker_compose_dir }}/redis.conf:/usr/local/etc/redis/redis.conf:ro"
- "{{ docker_compose_dir }}/redis_socket:/var/run/redis/:rw"
{% if docker_logger is defined %}
logging:
driver: {{ docker_logger }}
{% endif %}
{% if pg_hostname is not defined %}
postgres:
image: {{ postgresql_image }}
container_name: awx_postgres
restart: unless-stopped
volumes:
- "{{ postgres_data_dir }}/12/data/:/var/lib/postgresql/data:Z"
environment:
POSTGRES_USER: {{ pg_username }}
POSTGRES_PASSWORD: {{ pg_password }}
POSTGRES_DB: {{ pg_database }}
http_proxy: {{ http_proxy | default('') }}
https_proxy: {{ https_proxy | default('') }}
no_proxy: {{ no_proxy | default('') }}
{% if docker_logger is defined %}
logging:
driver: {{ docker_logger }}
{% endif %}
{% endif %}
{% if docker_compose_subnet is defined %}
networks:
default:
driver: bridge
ipam:
driver: default
config:
- subnet: {{ docker_compose_subnet }}
{% endif %}
volumes:
supervisor-socket:
rsyslog-socket:
rsyslog-config:

4
installer/roles/local_docker/templates/redis.conf.j2
View File

@ -1,4 +0,0 @@
unixsocket /var/run/redis/redis.sock
unixsocketperm 660
port 0
bind 127.0.0.1

4
setup.cfg
View File

@ -13,9 +13,9 @@
# W391 - Blank line at end of file
# W293 - Blank line contains whitespace
ignore=E201,E203,E221,E225,E231,E241,E251,E261,E265,E303,W291,W391,W293
exclude=.tox,venv,awx/lib/site-packages,awx/plugins/inventory/ec2.py,awx/plugins/inventory/gce.py,awx/plugins/inventory/vmware.py,awx/plugins/inventory/openstack.py,awx/ui,awx/api/urls.py,awx/main/migrations,awx/main/tests/data,installer/openshift/settings.py
exclude=.tox,venv,awx/lib/site-packages,awx/plugins/inventory/ec2.py,awx/plugins/inventory/gce.py,awx/plugins/inventory/vmware.py,awx/plugins/inventory/openstack.py,awx/ui,awx/api/urls.py,awx/main/migrations,awx/main/tests/data,installer/openshift/settings.py,tools/docker-compose/ansible/roles/sources/files/local_settings.py,tools/ansible/roles/dockerfile/files/settings.py
[flake8]
max-line-length=160
ignore=E201,E203,E221,E225,E231,E241,E251,E261,E265,E303,W291,W391,W293,E731,W504
exclude=.tox,venv,awx/lib/site-packages,awx/plugins/inventory,awx/ui,awx/api/urls.py,awx/main/migrations,awx/main/tests/data,node_modules/,awx/projects/,tools/docker,awx/settings/local_*.py,installer/openshift/settings.py,build/,installer/,awxkit/test,awx_collection/
exclude=.tox,venv,awx/lib/site-packages,awx/plugins/inventory,awx/ui,awx/api/urls.py,awx/main/migrations,awx/main/tests/data,node_modules/,awx/projects/,tools/docker,awx/settings/local_*.py,installer/openshift/settings.py,build/,installer/,awxkit/test,awx_collection/,tools/docker-compose/ansible/roles/sources/files/local_settings.py,tools/ansible/roles/dockerfile/files/settings.py

0
installer/build.yml → tools/ansible/build.yml
View File

6
tools/ansible/dockerfile.yml Normal file
View File

@ -0,0 +1,6 @@
---
- name: Render AWX Dockerfile and sources
hosts: localhost
gather_facts: true
roles:
- {role: dockerfile}

2
installer/roles/dockerfile/defaults/main.yml → tools/ansible/roles/dockerfile/defaults/main.yml
View File

@ -1,6 +1,6 @@
---
build_dev: false
kube_dev: false
dockerfile_dest: '..'
dockerfile_dest: '../..'
dockerfile_name: 'Dockerfile'
template_dest: '_build'

0
installer/roles/dockerfile/files/RPM-GPG-KEY-ansible-release → tools/ansible/roles/dockerfile/files/RPM-GPG-KEY-ansible-release
View File

0
installer/roles/dockerfile/files/launch_awx.sh → tools/ansible/roles/dockerfile/files/launch_awx.sh Executable file → Normal file
View File

1
installer/roles/dockerfile/files/launch_awx_task.sh → tools/ansible/roles/dockerfile/files/launch_awx_task.sh Executable file → Normal file
View File

@ -19,6 +19,7 @@ ANSIBLE_REMOTE_TEMP=/tmp ANSIBLE_LOCAL_TEMP=/tmp ansible -i "127.0.0.1," -c loca
ANSIBLE_REMOTE_TEMP=/tmp ANSIBLE_LOCAL_TEMP=/tmp ansible -i "127.0.0.1," -c local -v -m postgresql_db --become-user $DATABASE_USER -a "name=$DATABASE_NAME owner=$DATABASE_USER login_user=$DATABASE_USER login_host=$DATABASE_HOST login_password=$DATABASE_PASSWORD port=$DATABASE_PORT" all
if [ -z "$AWX_SKIP_MIGRATIONS" ]; then
echo "Running migrations..."
awx-manage migrate --noinput
fi

0
installer/roles/dockerfile/files/rsyslog.conf → tools/ansible/roles/dockerfile/files/rsyslog.conf
View File

0
installer/roles/dockerfile/files/settings.py → tools/ansible/roles/dockerfile/files/settings.py
View File

2
installer/roles/dockerfile/tasks/main.yml → tools/ansible/roles/dockerfile/tasks/main.yml
View File

@ -1,6 +1,6 @@
---
- name: Create .build directory
- name: Create _build directory
file:
path: "{{ dockerfile_dest }}/{{ template_dest }}"
state: directory

13
installer/roles/dockerfile/templates/Dockerfile.j2 → tools/ansible/roles/dockerfile/templates/Dockerfile.j2
View File

@ -1,5 +1,5 @@
### This file is generated from
### installer/roles/dockerfile/templates/Dockerfile.j2
### tools/ansible/roles/dockerfile/templates/Dockerfile.j2
###
### DO NOT EDIT
###
@ -182,7 +182,7 @@ RUN openssl req -nodes -newkey rsa:2048 -keyout /etc/nginx/nginx.key -out /etc/n
{% endif %}
# Create default awx rsyslog config
ADD installer/roles/dockerfile/files/rsyslog.conf /var/lib/awx/rsyslog/rsyslog.conf
ADD tools/ansible/roles/dockerfile/files/rsyslog.conf /var/lib/awx/rsyslog/rsyslog.conf
## File mappings
{% if build_dev|bool %}
@ -193,9 +193,9 @@ ADD tools/docker-compose/start_tests.sh /start_tests.sh
ADD tools/docker-compose/bootstrap_development.sh /usr/bin/bootstrap_development.sh
ADD tools/docker-compose/entrypoint.sh /entrypoint.sh
{% else %}
ADD installer/roles/dockerfile/files/launch_awx.sh /usr/bin/launch_awx.sh
ADD installer/roles/dockerfile/files/launch_awx_task.sh /usr/bin/launch_awx_task.sh
ADD installer/roles/dockerfile/files/settings.py /etc/tower/settings.py
ADD tools/ansible/roles/dockerfile/files/launch_awx.sh /usr/bin/launch_awx.sh
ADD tools/ansible/roles/dockerfile/files/launch_awx_task.sh /usr/bin/launch_awx_task.sh
ADD tools/ansible/roles/dockerfile/files/settings.py /etc/tower/settings.py
ADD {{ template_dest }}/supervisor.conf /etc/supervisord.conf
ADD {{ template_dest }}/supervisor_task.conf /etc/supervisord_task.conf
ADD tools/scripts/config-watcher /usr/bin/config-watcher
@ -219,7 +219,8 @@ RUN for dir in \
/var/lib/nginx ; \
do mkdir -m 0775 -p $dir ; chmod g+rw $dir ; chgrp root $dir ; done && \
for file in \
/etc/passwd ; \
/etc/passwd \
/var/lib/awx/rsyslog/rsyslog.conf ; \
do touch $file ; chmod g+rw $file ; chgrp root $file ; done
# Adjust any remaining permissions

0
installer/roles/dockerfile/templates/supervisor.conf.j2 → tools/ansible/roles/dockerfile/templates/supervisor.conf.j2
View File

0
installer/roles/dockerfile/templates/supervisor_task.conf.j2 → tools/ansible/roles/dockerfile/templates/supervisor_task.conf.j2
View File

0
installer/roles/image_build/defaults/main.yml → tools/ansible/roles/image_build/defaults/main.yml
View File

4
installer/roles/image_build/tasks/main.yml → tools/ansible/roles/image_build/tasks/main.yml
View File

@ -1,7 +1,7 @@
---
- name: Set global version if not provided
set_fact:
awx_version: "{{ lookup('file', playbook_dir + '/../VERSION') }}"
awx_version: "{{ lookup('file', playbook_dir + '/../../VERSION') }}"
when: awx_version is not defined
- name: Verify awx-logos directory exists for official install
@ -23,7 +23,7 @@
# Calling Docker directly because docker-py doesnt support BuildKit
- name: Build AWX image
command: docker build -t {{ awx_image }}:{{ awx_version }} ..
command: docker build -t {{ awx_image }}:{{ awx_version }} -f ../../{{ dockerfile_name }} ../..
- name: Tag awx images as latest
command: "docker tag {{ item }}:{{ awx_version }} {{ item }}:latest"

0
installer/roles/image_push/tasks/main.yml → tools/ansible/roles/image_push/tasks/main.yml
View File

207
tools/docker-compose/README.md
View File

@ -1,86 +1,171 @@
# Docker Compose for Dev container
# Docker Compose for Development
## How to start the Dev container
## Overview
In the root directory of your awx clone, run the following to build your docker image.
This step takes a while, but once your image is built, you don't need to do this again
unless you make changes to the Dockerfile or any of the files used by the Dockerfile.
Here are the main make targets:
```
make docker-compose-build
* `docker-compose-build` - used for building the development image, which is used by the `docker-compose` target
* `docker-compose` - make target for development, passes awx_devel image and tag
Notable files:
* `tools/docker-compose/inventory` file - used to configure the local AWX development deploymen
* `migrate.yml` - playbook for migrating data from Local Docker to the Development Environment
### Prerequisites
- [Docker](https://docs.docker.com/engine/installation/) on the host where AWX will be deployed. After installing Docker, the Docker service must be started (depending on your OS, you may have to add the local user that uses Docker to the ``docker`` group, refer to the documentation for details)
- [docker-compose](https://pypi.org/project/docker-compose/) Python module.
+ This also installs the `docker` Python module, which is incompatible with [`docker-py`](https://pypi.org/project/docker-py/). If you have previously installed `docker-py`, please uninstall it.
- [Docker Compose](https://docs.docker.com/compose/install/).
## Configuration
In the `inventory` file, set your `pg_password`, `broadcast_websocket_secret`, `secret_key`, and any other settings you need for your deployment.
AWX requires access to a PostgreSQL database, and by default, one will be created and deployed in a container, and data will be persisted to a docker volume. When the container is stopped, the database files will still exist in the docker volume. An external database can be used by setting the `pg_host`, `pg_hostname`, and `pg_username`.
> If you are coming from a Local Docker installation of AWX, consider migrating your data first, see the [data migration section](#migrating-data-from-local-docker) below.
## Starting the Development Environment
### Build the image
The AWX base container image (defined in the Dockerfile templated from [Dockerfile.j2](./../ansible/roles/dockerfile/templates/Dockerfile.j2)) contains basic OS dependencies and symbolic links into the development environment that make running the services easy.
Run the following to build the image:
```bash
$ make docker-compose-build
```
> Note: By default, this image will be tagged with your branch name.
> The image will need to be rebuilt if there are any changes to Dockerfile.j2 or any of the files used by the templated Dockerfile.
Copy over your local settings
Once the build completes, you will have a `ansible/awx_devel` image in your local image cache. Use the `docker images` command to view it, as follows:
```
cp awx/settings/local_settings.py.docker_compose awx/settings/local_settings.py
```bash
(host)$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_devel latest ba9ec3e8df74 26 minutes ago 1.42GB
```
Build the UI
> By default, this image will be tagged with your branch name. You can specify a custom tag by setting an environment variable, for example: `DEVEL_IMAGE_NAME=quay.io/your_user/awx_devel:17.0.1`
```
make ui-devel
### Run AWX
##### Start the containers
Run the awx, postgres and redis containers. This utilizes the image built in the previous step, and will automatically start all required services and dependent containers. Once the containers launch, your session will be attached to the awx container, and you'll be able to watch log messages and events in real time. You will see messages from Django and the front end build process.
```bash
$ make docker-compose
```
Run the container
> The make target assumes that the image you built is tagged with your current branch. This allows you to build images for different contexts or branches. When starting the containers, you can choose a specific branch by setting `COMPOSE_TAG=<branch name> `in your environment. For example, you might be working in a feature branch, but you want to run the containers using the devel image you built previously. To do that, start the containers using the following command: `$ COMPOSE_TAG=devel make docker-compose`
```
make docker-compose
```
##### Wait for migrations to complete
> Note: You may need to add `COMPOSE_TAG=<yourbranch>` at the end of this to use the correct image for your branch.
The first time you start the environment, database migrations need to run in order to build the PostgreSQL database. It will take few moments, but eventually you will see output in your terminal session that looks like the following:
The app should now be accessible in your browser at `https://localhost:8043/#/home`
## How to use the logstash container
#### Modify the docker-compose.yml
Uncomment the following lines in the `docker-compose.yml`
```
#- logstash
```bash
awx_1 | Operations to perform:
awx_1 | Synchronize unmigrated apps: solo, api, staticfiles, debug_toolbar, messages, channels, django_extensions, ui, rest_framework, polymorphic
awx_1 | Apply all migrations: sso, taggit, sessions, sites, kombu_transport_django, social_auth, contenttypes, auth, conf, main
awx_1 | Synchronizing apps without migrations:
awx_1 | Creating tables...
awx_1 | Running deferred SQL...
awx_1 | Installing custom SQL...
awx_1 | Running migrations:
awx_1 | Rendering model states... DONE
awx_1 | Applying contenttypes.0001_initial... OK
awx_1 | Applying contenttypes.0002_remove_content_type_name... OK
awx_1 | Applying auth.0001_initial... OK
...
#logstash:
# build:
# context: ./docker-compose
# dockerfile: Dockerfile-logstash
```
POST the following content to `/api/v2/settings/logging/` (this uses
authentication set up inside of the logstash configuration file).
```
{
"LOG_AGGREGATOR_HOST": "http://logstash",
"LOG_AGGREGATOR_PORT": 8085,
"LOG_AGGREGATOR_TYPE": "logstash",
"LOG_AGGREGATOR_USERNAME": "awx_logger",
"LOG_AGGREGATOR_PASSWORD": "workflows",
"LOG_AGGREGATOR_LOGGERS": [
"awx",
"activity_stream",
"job_events",
"system_tracking"
],
"LOG_AGGREGATOR_INDIVIDUAL_FACTS": false,
"LOG_AGGREGATOR_TOWER_UUID": "991ac7e9-6d68-48c8-bbde-7ca1096653c6",
"LOG_AGGREGATOR_ENABLED": true
}
##### Clean and Build UI
```bash
$ docker exec tools_awx_1 make clean-ui ui-devel
```
> Note: HTTP must be specified in the `LOG_AGGREGATOR_HOST` if you are using the docker development environment.
See [the ui development documentation](../../awx/ui_next/README.md) for more information on using the frontend development, build, and test tooling.
An example of how to view the most recent logs from the container:
Once migrations are completed and the UI is built, you can begin using AWX. The UI can be reached in your browser at `https://localhost:8043/#/home`, and the API can be found at `https://localhost:8043/api/v2`.
```
docker exec -i -t $(docker ps -aqf "name=tools_logstash_1") tail -n 50 /logstash.log
##### Create an admin user
Before you can log into AWX, you need to create an admin user. With this user you will be able to create more users, and begin configuring the server. From within the container shell, run the following command:
```bash
$ docker exec -ti tools_awx_1 awx-manage createsuperuser
```
#### How to add logstash plugins
> Remember the username and password, as you will use them to log into the web interface for the first time.
Add any plugins you need in `tools/elastic/logstash/Dockerfile` before running the container.
##### Load Demo Data
Optionally, you may also want to load some demo data. This will create a demo project, inventory, and job template.
```bash
$ docker exec tools_awx_1 awx-manage create_preload_data
```
> This information will persist in the database running in the `tools_postgres_1` container, until the container is removed. You may periodically need to recreate
this container, and thus the database, if the database schema changes in an upstream commit.
## Migrating Data from Local Docker
If you are migrating data from a Local Docker installation (17.0.1 and prior), you can
migrate your data to the development environment via the migrate.yml playbook using the steps described [here](./docs/data_migration.md).
## Upgrading the Development Environment
Upgrading AWX involves checking out the new source code and re-running the make target. Download a newer release from [https://github.com/ansible/awx/releases](https://github.com/ansible/awx/releases) and re-populate the inventory file with your customized variables.
After updating the inventory file with any custom values, run the make target from the root of your AWX clone.
```bash
$ make docker-compose
```
## Extras
* [Start a shell](#start-a-shell)
* [Start AWX from the container shell](#start-awx-from-the-container-shell)
* [Using Logstash](./docs/logstash.md)
### Start a Shell
To run `awx-manage` commands and modify things inside the container, you will want to start a shell session on the *awx* container. In a new terminal session, use the `docker exec` command to start the shell session:
```bash
(host)$ docker exec -it tools_awx_1 bash
```
This creates a session in the *awx* containers, just as if you were using `ssh`, and allows you execute commands within the running container.
### Start AWX from the container shell
Often times you'll want to start the development environment without immediately starting all of the services in the *awx* container, and instead be taken directly to a shell. You can do this with the following:
```bash
(host)$ make docker-compose-test
```
Using `docker exec`, this will create a session in the running *awx* container, and place you at a command prompt, where you can run shell commands inside the container.
If you want to start and use the development environment, you'll first need to bootstrap it by running the following command:
```bash
(container)# /usr/bin/bootstrap_development.sh
```
The above will do all the setup tasks, including running database migrations, so it may take a couple minutes. Once it's done it
will drop you back to the shell.
In order to launch all developer services:
```bash
(container)# /usr/bin/launch_awx.sh
```
`launch_awx.sh` also calls `bootstrap_development.sh` so if all you are doing is launching the supervisor to start all services, you don't
need to call `bootstrap_development.sh` first.

7
tools/docker-compose/ansible/migrate.yml Normal file
View File

@ -0,0 +1,7 @@
---
- name: Migrate data for upgrades and from Local Docker installs
hosts: localhost
gather_facts: true
roles:
- {role: sources}
- {role: migrate}

5
tools/docker-compose/ansible/roles/migrate/defaults/main.yml Normal file
View File

@ -0,0 +1,5 @@
---
postgres_data_dir: "~/.awx/pgdocker"
migrate_local_docker: false
old_docker_compose_dir: "~/.awx/awxcompose"
pg_volume_name: "postgres_data"

5
tools/docker-compose/ansible/roles/migrate/tasks/main.yml Normal file
View File

@ -0,0 +1,5 @@
---
# Migrate data from a bind-mount to a volume
- import_tasks: migrate-from-local-docker.yml
when: migrate_local_docker

53
tools/docker-compose/ansible/roles/migrate/tasks/migrate-from-local-docker.yml Normal file
View File

@ -0,0 +1,53 @@
---
# Migrate data from old Local Docker to a fresh development environment
- name: Remove awx_postgres to ensure consistent start state
shell: |
docker rm -f awx_postgres
ignore_errors: true
- name: Start Local Docker database container
docker_compose:
project_src: "{{ old_docker_compose_dir }}"
services:
- postgres
state: present
recreate: always
- name: Wait for postgres to initialize
wait_for:
timeout: 3
- name: Database dump to local filesystem
shell: |
docker-compose -f {{ old_docker_compose_dir }}/docker-compose.yml exec -T postgres pg_dumpall -U {{ pg_username }} > awx_dump.sql
- name: Stop AWX containers so the old postgres container does not get used
docker_compose:
project_src: "{{ old_docker_compose_dir }}"
state: absent
ignore_errors: true
- name: Start dev env database container
docker_compose:
project_src: "{{ playbook_dir }}/../_sources"
files: "docker-compose.yml"
services:
- postgres
state: present
recreate: always
environment:
COMPOSE_PROJECT_NAME: "tools"
- name: Wait for postgres to initialize
wait_for:
timeout: 3
- name: Restore to new postgres container
shell: |
COMPOSE_PROJECT_NAME=tools docker-compose -f {{ playbook_dir }}/../_sources/docker-compose.yml exec -T postgres psql -U {{ pg_username }} -d {{ pg_database }} -p {{ pg_port }} < awx_dump.sql
- name: Clean up temporary awx db dump
file:
path: awx_dump.sql
state: absent

7
tools/docker-compose/ansible/roles/sources/defaults/main.yml Normal file
View File

@ -0,0 +1,7 @@
---
sources_dest: '../_sources'
compose_name: 'docker-compose.yml'
awx_image: 'quay.io/ansible/awx_devel'
pg_port: 5432
pg_username: 'awx'
pg_database: 'awx'

14
awx/settings/local_settings.py.docker_compose → tools/docker-compose/ansible/roles/sources/files/local_settings.py
View File

@ -20,19 +20,6 @@ import sys
# INTERNAL_IPS = ('172.19.0.1', '172.18.0.1', '192.168.100.1')
# ALLOWED_HOSTS = ['*']
# Database settings to use PostgreSQL for development.
DATABASES = {
'default': {
'ENGINE': 'awx.main.db.profiled_pg',
'NAME': 'awx-dev',
'USER': 'awx-dev',
'PASSWORD': 'AWXsome1',
'ATOMIC_REQUESTS': True,
'HOST': 'postgres',
'PORT': '',
}
}
# Use SQLite for unit tests instead of PostgreSQL. If the lines below are
# commented out, Django will create the test_awx-dev database in PostgreSQL to
# run unit tests.
@ -76,7 +63,6 @@ SYSTEM_UUID = '00000000-0000-0000-0000-000000000000'
#LOGGING['loggers']['django_auth_ldap']['handlers'] = ['console']
#LOGGING['loggers']['django_auth_ldap']['level'] = 'DEBUG'
BROADCAST_WEBSOCKET_SECRET = '🤖starscream🤖'
BROADCAST_WEBSOCKET_PORT = 8013
BROADCAST_WEBSOCKET_VERIFY_CERT = False
BROADCAST_WEBSOCKET_PROTOCOL = 'http'

76
tools/docker-compose/ansible/roles/sources/tasks/main.yml Normal file
View File

@ -0,0 +1,76 @@
---
- name: Create _sources directory
file:
path: "{{ sources_dest }}/secrets"
state: 'directory'
mode: '0700'
- name: Detect secrets
stat:
path: "{{ sources_dest }}/secrets/{{ item }}.yml"
register: secrets
when: not lookup('vars', item, default='')
loop:
- pg_password
- secret_key
- broadcast_websocket_secret
- name: Generate secrets if needed
template:
src: 'secrets.yml.j2'
dest: '{{ sources_dest }}/secrets/{{ item.item }}.yml'
mode: '0600'
when: not lookup('vars', item.item, default='') and not item.stat.exists
loop: "{{ secrets.results }}"
- name: Include generated secrets unless they are explicitly passed in
include_vars: "{{ sources_dest }}/secrets/{{ item.item }}.yml"
no_log: true
when: not lookup('vars', item.item, default='')
loop: "{{ secrets.results }}"
- name: Render configuration templates
template:
src: "{{ item }}.j2"
dest: "{{ sources_dest }}/{{ item }}"
mode: '0600'
with_items:
- "database.py"
- "websocket_secret.py"
- name: Delete old local_settings.py
file:
path: "{{ playbook_dir }}/../../../awx/settings/local_settings.py"
state: absent
- name: Copy local_settings.py
copy:
src: "local_settings.py"
dest: "{{ sources_dest }}/local_settings.py"
- name: Get OS info for sdb
shell: |
docker info | grep 'Operating System'
register: os_info
changed_when: false
- name: Get user UID
shell: id -u
register: current_user
changed_when: false
- name: Set fact with user UID
set_fact:
user_id: "'{{ current_user.stdout }}'"
- name: Set global version if not provided
set_fact:
awx_image_tag: "{{ lookup('file', playbook_dir + '/../../../VERSION') }}"
when: awx_image_tag is not defined
- name: Render Docker-Compose
template:
src: docker-compose.yml.j2
dest: "{{ sources_dest }}/{{ compose_name }}"
mode: '0600'

4
installer/roles/local_docker/templates/credentials.py.j2 → tools/docker-compose/ansible/roles/sources/templates/database.py.j2
View File

@ -1,7 +1,7 @@
DATABASES = {
'default': {
'ATOMIC_REQUESTS': True,
'ENGINE': 'django.db.backends.postgresql',
'ENGINE': 'awx.main.db.profiled_pg',
'NAME': "{{ pg_database }}",
'USER': "{{ pg_username }}",
'PASSWORD': "{{ pg_password }}",
@ -9,5 +9,3 @@ DATABASES = {
'PORT': "{{ pg_port }}",
}
}
BROADCAST_WEBSOCKET_SECRET = "{{ broadcast_websocket_secret | b64encode }}"

30
tools/docker-compose.yml → tools/docker-compose/ansible/roles/sources/templates/docker-compose.yml.j2
View File

@ -3,14 +3,13 @@ version: '2'
services:
# Primary AWX Development Container
awx:
user: ${CURRENT_UID}
image: ${DEV_DOCKER_TAG_BASE}/awx_devel:${TAG}
user: "{{ ansible_user_uid }}"
image: "{{ awx_image }}:{{ awx_image_tag }}"
container_name: tools_awx_1
hostname: awx
command: launch_awx.sh
environment:
CURRENT_UID:
OS:
OS: "{{ os_info.stdout }}"
SDB_HOST: 0.0.0.0
SDB_PORT: 7899
AWX_GROUP_QUEUES: tower
@ -24,15 +23,14 @@ services:
links:
- postgres
- redis
# - sync
# volumes_from:
# - sync
working_dir: "/awx_devel"
volumes:
- "../:/awx_devel"
- "../awx/projects/:/var/lib/awx/projects/"
- "./redis/redis_socket_standalone:/var/run/redis/"
- "./docker-compose/supervisor.conf:/etc/supervisord.conf"
- "../../../:/awx_devel"
- "../../docker-compose/supervisor.conf:/etc/supervisord.conf"
- "../../docker-compose/_sources/database.py:/etc/tower/conf.d/database.py"
- "../../docker-compose/_sources/websocket_secret.py:/etc/tower/conf.d/websocket_secret.py"
- "../../docker-compose/_sources/local_settings.py:/etc/tower/conf.d/local_settings.py"
- "redis_socket:/var/run/redis/:rw"
privileged: true
tty: true
# A useful container that simply passes through log messages to the console
@ -48,16 +46,20 @@ services:
container_name: tools_postgres_1
environment:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_USER: {{ pg_username }}
POSTGRES_DB: {{ pg_database }}
POSTGRES_PASSWORD: {{ pg_password }}
volumes:
- "awx_db:/var/lib/postgresql/data"
redis:
image: redis:latest
container_name: tools_redis_1
user: ${CURRENT_UID}
volumes:
- "./redis/redis.conf:/usr/local/etc/redis/redis.conf"
- "./redis/redis_socket_standalone:/var/run/redis/"
- "../../redis/redis.conf:/usr/local/etc/redis/redis.conf"
- "redis_socket:/var/run/redis/:rw"
entrypoint: ["redis-server"]
command: ["/usr/local/etc/redis/redis.conf"]
volumes:
awx_db:
redis_socket:

0
installer/roles/local_docker/templates/environment.sh.j2 → tools/docker-compose/ansible/roles/sources/templates/environment.sh.j2
View File

0
installer/roles/local_docker/templates/nginx.conf.j2 → tools/docker-compose/ansible/roles/sources/templates/nginx.conf.j2
View File

1
tools/docker-compose/ansible/roles/sources/templates/secrets.yml.j2 Normal file
View File

@ -0,0 +1 @@
{{ item.item }}: '{{ lookup('vars', item.item, default='') or lookup('password', '/dev/null chars=ascii_letters') }}'

1
tools/docker-compose/ansible/roles/sources/templates/websocket_secret.py.j2 Normal file
View File

@ -0,0 +1 @@
BROADCAST_WEBSOCKET_SECRET = "{{ broadcast_websocket_secret | b64encode }}"

6
tools/docker-compose/ansible/sources.yml Normal file
View File

@ -0,0 +1,6 @@
---
- name: Render AWX Dockerfile and sources
hosts: localhost
gather_facts: true
roles:
- {role: sources}

3
tools/docker-compose/bootstrap_development.sh
View File

@ -9,9 +9,6 @@ ansible -i "127.0.0.1," -c local -v -m wait_for -a "path=/var/run/redis/redis.so
# TODO: FIX
#/etc/init.d/ssh start
ansible -i "127.0.0.1," -c local -v -m postgresql_user --become-user postgres -a "name=awx-dev password=AWXsome1 encrypted=yes login_user=postgres login_password=postgrespass login_host=postgres" all
ansible -i "127.0.0.1," -c local -v -m postgresql_db --become-user postgres -a "name=awx-dev owner=awx-dev login_user=postgres login_password=postgrespass login_host=postgres" all
# Move to the source directory so we can bootstrap
if [ -f "/awx_devel/manage.py" ]; then
cd /awx_devel

60
tools/docker-compose/docs/data_migration.md Normal file
View File

@ -0,0 +1,60 @@
# Migrating Data from Local Docker
If you are migrating data from a Local Docker installation (17.0.1 and prior), you can
migrate your data to the development environment via the migrate.yml playbook, or by using the manual steps described below.
> Note: This will also convert your postgresql bind-mount into a docker volume.
### Migrate data with migrate.yml
If you had a custom pgdocker or awxcompose location, you will need to set the `postgres_data_dir` and `old_docker_compose_dir` variables.
1. Run the [migrate playbook](./ansible/migrate.yml) to migrate your data to the new postgresql container and convert the data directory to a volume mount.
```bash
$ ansible-playbook migrate.yml -e "migrate_local_docker=true" -e "postgres_data_dir=~/.awx/pgdocker" -e "old_docker_compose_dir=~/.awx/awxcompose"
```
2. Change directory to the top of your awx checkout and start your containers
```bash
$ make docker-compose
```
3. After ensuring your data has been successfully migrated, you may delete your old data directory (typically stored at `~/.awx/pgdocker`).
### Migrating data manually
1. With Local Docker still running, perform a pg_dumpall:
> Note: If Local Docker is no longer running
`docker-compose -f ~/.awx/awxcompose/docker-compose.yml up postgres`
```bash
$ docker-compose -f ~/.awx/awxcompose/docker-compose.yml exec postgres pg_dumpall -U awx > awx_dump.sql
```
2. Remove all local docker containers (specifically awx_postgres)
```bash
$ docker -f rm awx_postgres
```
3. Template the new docker-compose.yml
```bash
$ ansible-playbook -i tools/ansible/inventory tools/ansible/sources.yml
```
4. Start a container with a volume (using the new tools/docker-compose/_sources/docker-compose.yml)
```bash
$ docker-compose -f ../docker-compose/_sources/docker-compose.yml up postgres
```
5. Restore to new `awx_postgres`
```bash
$ docker-compose -f ../docker-compose/_sources/docker-compose.yml exec -T postgres psql -U awx -d awx -p 5432 < awx_dump.sql
```
6. Run the docker-compose.yml to start the containers
```bash
$ docker-compose -f ../docker-compose/_sources/docker-compose.yml up task
```
7. Check to ensure your data migration was successful, then you can delete your the `awx_dump.sql` backup and your old data directory.

49
tools/docker-compose/docs/logstash.md Normal file
View File

@ -0,0 +1,49 @@
# How to use the logstash container
#### Modify the docker-compose.yml
Uncomment the following lines in the `docker-compose.yml`
```
#- logstash
...
#logstash:
# build:
# context: ./docker-compose
# dockerfile: Dockerfile-logstash
```
POST the following content to `/api/v2/settings/logging/` (this uses
authentication set up inside of the logstash configuration file).
```
{
"LOG_AGGREGATOR_HOST": "http://logstash",
"LOG_AGGREGATOR_PORT": 8085,
"LOG_AGGREGATOR_TYPE": "logstash",
"LOG_AGGREGATOR_USERNAME": "awx_logger",
"LOG_AGGREGATOR_PASSWORD": "workflows",
"LOG_AGGREGATOR_LOGGERS": [
"awx",
"activity_stream",
"job_events",
"system_tracking"
],
"LOG_AGGREGATOR_INDIVIDUAL_FACTS": false,
"LOG_AGGREGATOR_TOWER_UUID": "991ac7e9-6d68-48c8-bbde-7ca1096653c6",
"LOG_AGGREGATOR_ENABLED": true
}
```
> Note: HTTP must be specified in the `LOG_AGGREGATOR_HOST` if you are using the docker development environment.
An example of how to view the most recent logs from the container:
```
docker exec -i -t $(docker ps -aqf "name=tools_logstash_1") tail -n 50 /logstash.log
```
#### How to add logstash plugins
Add any plugins you need in `tools/elastic/logstash/Dockerfile` before running the container.

19
tools/docker-compose/inventory Normal file
View File

@ -0,0 +1,19 @@
localhost ansible_connection=local ansible_python_interpreter="/usr/bin/env python3"
[all:vars]
# AWX-Managed Database Settings
# If left blank, these will be generated upon install.
# Values are written out to tools/docker-compose/_sources/secrets/
# pg_password=""
# broadcast_websocket_secret=""
# secret_key=""
# External Database Settings
# pg_host=""
# pg_password=""
# pg_username=""
# pg_hostname=""
# awx_image="ansible/awx"
# migrate_local_docker=false

2
tools/docker-compose/start_tests.sh
View File

@ -5,5 +5,5 @@ cd /awx_devel
make clean
make awx-link
cp awx/settings/local_settings.py.docker_compose awx/settings/local_settings.py
cp tools/docker-compose/ansible/roles/sources/files/local_settings.py awx/settings/local_settings.py
make "${1:-test}"

3
tools/docker-credential-plugins-override.yml
View File

@ -18,9 +18,10 @@ services:
conjur:
image: cyberark/conjur
container_name: tools_conjur_1
command: server -p 8300
environment:
DATABASE_URL: postgres://postgres@postgres/postgres
DATABASE_URL: postgres://awx@postgres/postgres
CONJUR_DATA_KEY: 'dveUwOI/71x9BPJkIgvQRRBF3SdASc+HP4CUGL7TKvM='
depends_on:
- postgres

11
tools/elastic/docker-compose.elastic-override.yml
View File

@ -10,7 +10,7 @@ services:
# Components of ELK stack for logging
elasticsearch:
build: elastic/elasticsearch/
build: ../../elastic/elasticsearch/
ports:
- "9200:9200"
- "9300:9300"
@ -18,12 +18,11 @@ services:
ES_JAVA_OPTS: "-Xms1g -Xmx1g"
# networks: # add back in when a connection to tower_tools is possible
# - docker_elk
logstash:
build: elastic/logstash/
build: ../../elastic/logstash/
command: -f /etc/logstash/conf.d/
volumes:
- ./elastic/logstash/config:/etc/logstash/conf.d
- ../../elastic/logstash/config:/etc/logstash/conf.d
ports:
- "8085:8085"
links:
@ -33,9 +32,9 @@ services:
depends_on:
- elasticsearch
kibana:
build: elastic/kibana/
build: ../../elastic/kibana/
volumes:
- ./elastic/kibana/config/:/opt/kibana/config/
- ../../elastic/kibana/config/:/opt/kibana/config/
ports:
- "5601:5601"
links:

1
tools/redis/redis_socket_standalone/.dir_placeholder
View File

@ -1 +0,0 @@
This dir must pre-exist and be owned by the user you are launching awx dev env as. If the dir does not exist before launching the awx dev environment then docker will create the dir and it will be owned by root. Since we start our awx dev environment with user: ${CURRENT_UID} the redis container will be unable to create a socket file in a directory owned by root.