diff --git a/docs/docsite/rst/administration/instances.rst b/docs/docsite/rst/administration/instances.rst index f25957f04e..07a2eccf8c 100644 --- a/docs/docsite/rst/administration/instances.rst +++ b/docs/docsite/rst/administration/instances.rst @@ -51,6 +51,213 @@ Prerequisites - To manage instances from the AWX user interface, you must have System Administrator or System Auditor permissions. +Common topologies +------------------ + +Instances make up the network of devices that communicate with one another. They are the building blocks of an automation mesh. These building blocks serve as nodes in a mesh topology. There are several kinds of instances: + ++-----------+-----------------------------------------------------------------------------------------------------------------+ +| Node Type | Description | ++===========+=================================================================================================================+ +| Control | Nodes that run persistent Ansible Automation Platform services, and delegate jobs to hybrid and execution nodes | ++-----------+-----------------------------------------------------------------------------------------------------------------+ +| Hybrid | Nodes that run persistent Ansible Automation Platform services and execute jobs | +| | (not applicable to operator-based installations) | ++-----------+-----------------------------------------------------------------------------------------------------------------+ +| Hop | Used for relaying across the mesh only | ++-----------+-----------------------------------------------------------------------------------------------------------------+ +| Execution | Nodes that run jobs delivered from control nodes (jobs submitted from the user’s Ansible automation) | ++-----------+-----------------------------------------------------------------------------------------------------------------+ + +Simple topology +~~~~~~~~~~~~~~~~ + +One of the ways to expand job capacity is to create a standalone execution node that can be added to run alongside the Kubernetes deployment of AWX. These machines will not be a part of the AWX Kubernetes cluster. The control nodes running in the cluster will connect and submit work to these machines via Receptor. The machines are registered in AWX as type "execution" instances, meaning they will only be used to run AWX jobs, not dispatch work or handle web requests as control nodes do. + +Hop nodes can be added to sit between the control plane of AWX and standalone execution nodes. These machines will not be a part of the AWX Kubernetes cluster and they will be registered in AWX as node type "hop", meaning they will only handle inbound and outbound traffic for otherwise unreachable nodes in a different or more strict network. + +Below is an example of an AWX task pod with two execution nodes. Traffic to execution node 2 flows through a hop node that is setup between it and the control plane. + +.. image:: ../common/images/instances_awx_task_pods_hopnode.png + :alt: AWX task pod with a hop node between the control plane of AWX and standalone execution nodes. + + +An example of a simple topology may look like the following: + +.. list-table:: + :widths: 20 30 10 20 15 + :header-rows: 1 + + * - Instance type + - Hostname + - Listener port + - Peers from control nodes + - Peers + * - Control plane + - awx-task-65d6d96987-mgn9j + - 27199 + - True + - [] + * - Hop node + - awx-hop-node + - 27199 + - True + - [] + * - Execution node + - awx-example.com + - n/a + - False + - ["hop node"] + + + +Mesh topology +~~~~~~~~~~~~~~ + +Mesh ingress is a feature that allows remote nodes to connect inbound to the control plane. This is especially useful when creating remote nodes in restricted networking environments that disallow inbound traffic. + + +.. image:: ../common/images/instances_mesh_ingress_topology.png + :alt: Mesh ingress architecture showing the peering relationship between nodes. + + +An example of a topology that uses mesh ingress may look like the following: + +.. list-table:: + :widths: 20 30 10 20 15 + :header-rows: 1 + + * - Instance type + - Hostname + - Listener port + - Peers from control nodes + - Peers + * - Control plane + - awx-task-xyz + - 27199 + - True + - [] + * - Hop node + - awx-hop-node + - 27199 + - True + - [] + * - Execution node + - awx-example.com + - n/a + - False + - ["hop node"] + +In order to create a mesh ingress for AWX, see the `Mesh Ingress `_ chapter of the AWX Operator Documentation for information on setting up this type of topology. The last step is to create a remote execution node and add the execution node to an instance group in order for it to be used in your job execution. Whatever execution environment image used to run a playbook needs to be accessible for your remote execution node. Everything you are using in your playbook also needs to be accessible from this remote execution node. + +.. image:: ../common/images/instances-job-template-using-remote-execution-ig.png + :alt: Job template using the instance group with the execution node to run jobs. + + + +.. _ag_instances_add: + +Add an instance +---------------- + +To create an instance in AWX: + +1. Click **Instances** from the left side navigation menu of the AWX UI. + +2. In the Instances list view, click the **Add** button and the Create new Instance window opens. + +.. image:: ../common/images/instances_create_new.png + :alt: Create a new instance form. + +An instance has several attributes that may be configured: + +- Enter a fully qualified domain name (ping-able DNS) or IP address for your instance in the **Host Name** field (required). This field is equivalent to ``hostname`` in the API. +- Optionally enter a **Description** for the instance +- The **Instance State** field is auto-populated, indicating that it is being installed, and cannot be modified +- Optionally specify the **Listener Port** for the receptor to listen on for incoming connections. This is an open port on the remote machine used to establish inbound TCP connections. This field is equivalent to ``listener_port`` in the API. +- Select from the options in **Instance Type** field to specify the type you want to create. Only execution and hop nodes can be created as operator-based installations do not support hybrid nodes. This field is equivalent to ``node_type`` in the API. +- In the **Peers** field, select the instance hostnames you want your new instance to connect outbound to. +- In the **Options** fields: + - Check the **Enable Instance** box to make it available for jobs to run on an execution node. + - Check the **Managed by Policy** box to allow policy to dictate how the instance is assigned. + - Check the **Peers from control nodes** box to allow control nodes to peer to this instance automatically. Listener port needs to be set if this is enabled or the instance is a peer. + + + +3. Once the attributes are configured, click **Save** to proceed. + +Upon successful creation, the Details of the one of the created instances opens. + +.. image:: ../common/images/instances_create_details.png + :alt: Details of the newly created instance. + +.. note:: + + The proceeding steps 4-8 are intended to be ran from any computer that has SSH access to the newly created instance. + +4. Click the download button next to the **Install Bundle** field to download the tarball that contain files to allow AWX to make proper TCP connections to the remote machine. + +.. image:: ../common/images/instances_install_bundle.png + :alt: Instance details showing the Download button in the Install Bundle field of the Details tab. + +5. Extract the downloaded ``tar.gz`` file from the location you downloaded it. The install bundle contains TLS certificates and keys, a certificate authority, and a proper Receptor configuration file. To facilitate that these files will be in the right location on the remote machine, the install bundle includes an ``install_receptor.yml`` playbook. The playbook requires the Receptor collection which can be obtained via: + +:: + + ansible-galaxy collection install -r requirements.yml + +6. Before running the ``ansible-playbook`` command, edit the following fields in the ``inventory.yml`` file: + +- ``ansible_user`` with the username running the installation +- ``ansible_ssh_private_key_file`` to contain the filename of the private key used to connect to the instance + +:: + + --- + all: + hosts: + remote-execution: + ansible_host: + ansible_user: # user provided + ansible_ssh_private_key_file: ~/.ssh/id_rsa + +The content of the ``inventory.yml`` file serves as a template and contains variables for roles that are applied during the installation and configuration of a receptor node in a mesh topology. You may modify some of the other fields, or replace the file in its entirety for advanced scenarios. Refer to `Role Variables `_ for more information on each variable. + +7. Save the file to continue. + +8. Run the following command on the machine you want to update your mesh: + +:: + + ansible-playbook -i inventory.yml install_receptor.yml + +Wait a few minutes for the periodic AWX task to do a health check against the new instance. You may run a health check by selecting the node and clicking the **Run health check** button from its Details page at any time. Once the instances endpoint or page reports a "Ready" status for the instance, jobs are now ready to run on this machine! + +9. To view other instances within the same topology or associate peers, click the **Peers** tab. + +.. image:: ../common/images/instances_peers_tab.png + :alt: "Peers" tab showing two peers. + +To associate peers with your node, click the **Associate** button to open a dialog box of instances eligible for peering. + +.. image:: ../common/images/instances_associate_peer.png + :alt: Instances available to peer with the example hop node. + +Execution nodes can peer with either hop nodes or other execution nodes. Hop nodes can only peer with execution nodes unless you check the **Peers from control nodes** check box from the **Options** field. + +.. note:: + + If you associate or disassociate a peer, a notification will inform you to re-run the install bundle from the Peer Detail view (the :ref:`ag_topology_viewer` has the download link). + + .. image:: ../common/images/instances_associate_peer_reinstallmsg.png + :alt: Notification to re-run the installation bundle due to change in the peering. + +You can remove an instance by clicking **Remove** in the Instances page, or by setting the instance ``node_state = deprovisioning`` via the API. Upon deleting, a pop-up message will appear to notify that you may need to re-run the install bundle to make sure things that were removed are no longer connected. + + +10. To view a graphical representation of your updated topology, refer to the :ref:`ag_topology_viewer` section of this guide. + + Manage instances ----------------- @@ -113,153 +320,10 @@ The example health check shows the status updates with an error on node 'one': :alt: Health check showing an error in one of the instances. -Add an instance ----------------- - -One of the ways to expand capacity is to create an instance. Standalone execution nodes can be added to run alongside the Kubernetes deployment of AWX. These machines will not be a part of the AWX Kubernetes cluster. The control nodes running in the cluster will connect and submit work to these machines via Receptor. The machines are registered in AWX as type "execution" instances, meaning they will only be used to run AWX jobs, not dispatch work or handle web requests as control nodes do. - -Hop nodes can be added to sit between the control plane of AWX and standalone execution nodes. These machines will not be a part of the AWX Kubernetes cluster and they will be registered in AWX as node type "hop", meaning they will only handle inbound and outbound traffic for otherwise unreachable nodes in a different or more strict network. - -Below is an example of an AWX task pod with two execution nodes. Traffic to execution node 2 flows through a hop node that is setup between it and the control plane. - -.. image:: ../common/images/instances_awx_task_pods_hopnode.png - :alt: AWX task pod with a hop node between the control plane of AWX and standalone execution nodes. - -To create an instance in AWV: - -1. Click **Instances** from the left side navigation menu of the AWX UI. - -2. In the Instances list view, click the **Add** button and the Create new Instance window opens. - -.. image:: ../common/images/instances_create_new.png - :alt: Create a new instance form. - -An instance has several attributes that may be configured: - -- Enter a fully qualified domain name (ping-able DNS) or IP address for your instance in the **Host Name** field (required). This field is equivalent to ``hostname`` in the API. -- Optionally enter a **Description** for the instance -- The **Instance State** field is auto-populated, indicating that it is being installed, and cannot be modified -- Optionally specify the **Listener Port** for the receptor to listen on for incoming connections. This is an open port on the remote machine used to establish inbound TCP connections. This field is equivalent to ``listener_port`` in the API. -- Select from the options in **Instance Type** field to specify the type you want to create. Only execution and hop nodes can be created as operator-based installations do not support hybrid nodes. This field is equivalent to ``node_type`` in the API. -- In the **Peers** field, select the instance hostnames you want your new instance to connect outbound to. -- In the **Options** fields: - - Check the **Enable Instance** box to make it available for jobs to run on an execution node. - - Check the **Managed by Policy** box to allow policy to dictate how the instance is assigned. - - Check the **Peers from control nodes** box to allow control nodes to peer to this instance automatically. Listener port needs to be set if this is enabled or the instance is a peer. - -In the example diagram above, the configurations are as follows: - -+------------------+---------------+--------------------------+--------------+ -| instance name | listener_port | peers_from_control_nodes | peers | -+==================+===============+==========================+==============+ -| execution node 1 | 27199 | true | [] | -+------------------+---------------+--------------------------+--------------+ -| hop node | 27199 | true | [] | -+------------------+---------------+--------------------------+--------------+ -| execution node 2 | null | false | ["hop node"] | -+------------------+---------------+--------------------------+--------------+ - - -3. Once the attributes are configured, click **Save** to proceed. - -Upon successful creation, the Details of the one of the created instances opens. - -.. image:: ../common/images/instances_create_details.png - :alt: Details of the newly created instance. - -.. note:: - - The proceeding steps 4-8 are intended to be ran from any computer that has SSH access to the newly created instance. - -4. Click the download button next to the **Install Bundle** field to download the tarball that contain files to allow AWX to make proper TCP connections to the remote machine. - -.. image:: ../common/images/instances_install_bundle.png - :alt: Instance details showing the Download button in the Install Bundle field of the Details tab. - -5. Extract the downloaded ``tar.gz`` file from the location you downloaded it. The install bundle contains TLS certificates and keys, a certificate authority, and a proper Receptor configuration file. To facilitate that these files will be in the right location on the remote machine, the install bundle includes an ``install_receptor.yml`` playbook. The playbook requires the Receptor collection which can be obtained via: - -:: - - ansible-galaxy collection install -r requirements.yml - -6. Before running the ``ansible-playbook`` command, edit the following fields in the ``inventory.yml`` file: - -- ``ansible_user`` with the username running the installation -- ``ansible_ssh_private_key_file`` to contain the filename of the private key used to connect to the instance - -:: - - --- - all: - hosts: - remote-execution: - ansible_host: 18.206.206.34 - ansible_user: # user provided - ansible_ssh_private_key_file: ~/.ssh/id_rsa - -The content of the ``inventory.yml`` file serves as a template and contains variables for roles that are applied during the installation and configuration of a receptor node in a mesh topology. You may modify some of the other fields, or replace the file in its entirety for advanced scenarios. Refer to `Role Variables `_ for more information on each variable. - -7. Save the file to continue. - -8. Run the following command on the machine you want to update your mesh: - -:: - - ansible-playbook -i inventory.yml install_receptor.yml - -Wait a few minutes for the periodic AWX task to do a health check against the new instance. You may run a health check by selecting the node and clicking the **Run health check** button from its Details page at any time. Once the instances endpoint or page reports a "Ready" status for the instance, jobs are now ready to run on this machine! - -9. To view other instances within the same topology or associate peers, click the **Peers** tab. - -.. image:: ../common/images/instances_peers_tab.png - :alt: "Peers" tab showing two peers. - -To associate peers with your node, click the **Associate** button to open a dialog box of instances eligible for peering. - -.. image:: ../common/images/instances_associate_peer.png - :alt: Instances available to peer with the example hop node. - -Execution nodes can peer with either hop nodes or other execution nodes. Hop nodes can only peer with execution nodes unless you check the **Peers from control nodes** check box from the **Options** field. - -.. note:: - - If you associate or disassociate a peer, a notification will inform you to re-run the install bundle from the Peer Detail view (the :ref:`ag_topology_viewer` has the download link). - - .. image:: ../common/images/instances_associate_peer_reinstallmsg.png - :alt: Notification to re-run the installation bundle due to change in the peering. - -You can remove an instance by clicking **Remove** in the Instances page, or by setting the instance ``node_state = deprovisioning`` via the API. Upon deleting, a pop-up message will appear to notify that you may need to re-run the install bundle to make sure things that were removed are no longer connected. - - -10. To view a graphical representation of your updated topology, refer to the :ref:`ag_topology_viewer` section of this guide. - - Using a custom Receptor CA --------------------------- -The control nodes on the K8S cluster will communicate with execution nodes via mutual TLS TCP connections, running via Receptor. Execution nodes will verify incoming connections by ensuring the x509 certificate was issued by a trusted Certificate Authority (CA). - -You may choose to provide your own CA for this validation. If no CA is provided, AWX operator will automatically generate one using OpenSSL. - -Given custom ``ca.crt`` and ``ca.key`` stored locally, run the following: - -:: - - kubectl create secret tls awx-demo-receptor-ca \ - --cert=/path/to/ca.crt --key=/path/to/ca.key - -The secret should be named ``{AWX Custom Resource name}-receptor-ca``. In the above, the AWX Custom Resource name is "awx-demo". Replace "awx-demo" with your AWX Custom Resource name. - -If this secret is created after AWX is deployed, run the following to restart the deployment: - -:: - - kubectl rollout restart deployment awx-demo - - -.. note:: - - Changing the receptor CA will sever connections to any existing execution nodes. These nodes will enter an *Unavailable* state, and jobs will not be able to run on them. You will need to download and re-run the install bundle for each execution node. This will replace the TLS certificate files with those signed by the new CA. The execution nodes will then appear in a *Ready* state after a few minutes. +Refer to the AWX Operator Documentation, `Custom Receptor CA `_ for detail. Using a private image for the default EE diff --git a/docs/docsite/rst/common/images/download-icon.png b/docs/docsite/rst/common/images/download-icon.png new file mode 100644 index 0000000000..2529682406 Binary files /dev/null and b/docs/docsite/rst/common/images/download-icon.png differ diff --git a/docs/docsite/rst/common/images/instances-execution-node-download.png b/docs/docsite/rst/common/images/instances-execution-node-download.png new file mode 100644 index 0000000000..b605bb12a7 Binary files /dev/null and b/docs/docsite/rst/common/images/instances-execution-node-download.png differ diff --git a/docs/docsite/rst/common/images/instances-job-template-using-remote-execution-ig.png b/docs/docsite/rst/common/images/instances-job-template-using-remote-execution-ig.png new file mode 100644 index 0000000000..1c9173b5f8 Binary files /dev/null and b/docs/docsite/rst/common/images/instances-job-template-using-remote-execution-ig.png differ diff --git a/docs/docsite/rst/common/images/instances_associate_peer.png b/docs/docsite/rst/common/images/instances_associate_peer.png index 397d7c2916..60cfad6122 100644 Binary files a/docs/docsite/rst/common/images/instances_associate_peer.png and b/docs/docsite/rst/common/images/instances_associate_peer.png differ diff --git a/docs/docsite/rst/common/images/instances_awx_task_pods_hopnode.png b/docs/docsite/rst/common/images/instances_awx_task_pods_hopnode.png index c9b65e64dc..5682c34be3 100644 Binary files a/docs/docsite/rst/common/images/instances_awx_task_pods_hopnode.png and b/docs/docsite/rst/common/images/instances_awx_task_pods_hopnode.png differ diff --git a/docs/docsite/rst/common/images/instances_create_details.png b/docs/docsite/rst/common/images/instances_create_details.png index 0ceefefaf5..2e282d5d19 100644 Binary files a/docs/docsite/rst/common/images/instances_create_details.png and b/docs/docsite/rst/common/images/instances_create_details.png differ diff --git a/docs/docsite/rst/common/images/instances_create_new.png b/docs/docsite/rst/common/images/instances_create_new.png index 5e69c6f7f5..288c2287e5 100644 Binary files a/docs/docsite/rst/common/images/instances_create_new.png and b/docs/docsite/rst/common/images/instances_create_new.png differ diff --git a/docs/docsite/rst/common/images/instances_health_check.png b/docs/docsite/rst/common/images/instances_health_check.png index 0918b5f635..1086237e1b 100644 Binary files a/docs/docsite/rst/common/images/instances_health_check.png and b/docs/docsite/rst/common/images/instances_health_check.png differ diff --git a/docs/docsite/rst/common/images/instances_health_check_pending.png b/docs/docsite/rst/common/images/instances_health_check_pending.png index 3a8cd7a29d..c19fc94fe9 100644 Binary files a/docs/docsite/rst/common/images/instances_health_check_pending.png and b/docs/docsite/rst/common/images/instances_health_check_pending.png differ diff --git a/docs/docsite/rst/common/images/instances_install_bundle.png b/docs/docsite/rst/common/images/instances_install_bundle.png index e45f79da85..3041d6b097 100644 Binary files a/docs/docsite/rst/common/images/instances_install_bundle.png and b/docs/docsite/rst/common/images/instances_install_bundle.png differ diff --git a/docs/docsite/rst/common/images/instances_list_view.png b/docs/docsite/rst/common/images/instances_list_view.png index e66eb45726..5da391986e 100644 Binary files a/docs/docsite/rst/common/images/instances_list_view.png and b/docs/docsite/rst/common/images/instances_list_view.png differ diff --git a/docs/docsite/rst/common/images/instances_mesh_ingress_topology.png b/docs/docsite/rst/common/images/instances_mesh_ingress_topology.png new file mode 100644 index 0000000000..63144158ae Binary files /dev/null and b/docs/docsite/rst/common/images/instances_mesh_ingress_topology.png differ diff --git a/docs/docsite/rst/common/images/instances_peers_tab.png b/docs/docsite/rst/common/images/instances_peers_tab.png index c75c17de8d..ae145bb93a 100644 Binary files a/docs/docsite/rst/common/images/instances_peers_tab.png and b/docs/docsite/rst/common/images/instances_peers_tab.png differ diff --git a/docs/docsite/rst/common/images/topology-viewer-instance-with-errors.png b/docs/docsite/rst/common/images/topology-viewer-instance-with-errors.png index 214e65301f..f1e0163b55 100644 Binary files a/docs/docsite/rst/common/images/topology-viewer-instance-with-errors.png and b/docs/docsite/rst/common/images/topology-viewer-instance-with-errors.png differ diff --git a/docs/docsite/rst/userguide/glossary.rst b/docs/docsite/rst/userguide/glossary.rst index c0cf2749b0..f55659f69c 100644 --- a/docs/docsite/rst/userguide/glossary.rst +++ b/docs/docsite/rst/userguide/glossary.rst @@ -90,19 +90,6 @@ Glossary Node A node corresponds to entries in the instance database model, or the ``/api/v2/instances/`` endpoint, and is a machine participating in the cluster / mesh. The unified jobs API reports ``awx_node`` and ``execution_node`` fields. The execution node is where the job runs, and AWX node interfaces between the job and server functions. - +-----------+-----------------------------------------------------------------------------------------------------------------+ - | Node Type | Description | - +-----------+-----------------------------------------------------------------------------------------------------------------+ - | Control | Nodes that run persistent Ansible Automation Platform services, and delegate jobs to hybrid and execution nodes | - +-----------+-----------------------------------------------------------------------------------------------------------------+ - | Hybrid | Nodes that run persistent Ansible Automation Platform services and execute jobs | - | | (not applicable to operator-based installations) | - +-----------+-----------------------------------------------------------------------------------------------------------------+ - | Hop | Used for relaying across the mesh only | - +-----------+-----------------------------------------------------------------------------------------------------------------+ - | Execution | Nodes that run jobs delivered from control nodes (jobs submitted from the user’s Ansible automation) | - +-----------+-----------------------------------------------------------------------------------------------------------------+ - Notification Template An instance of a notification type (Email, Slack, Webhook, etc.) with a name, description, and a defined configuration.