External-Mirrors/kubespray
2
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kubespray.git synced 2026-02-18 03:30:07 -03:30
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #11748 from VannTen/cleanup/remove_inventory_builder

Browse Source 
Remove inventory_builder and re-organize docs
This commit is contained in:
Kubernetes Prow Robot
2024-11-27 14:52:58 +00:00
committed by GitHub
parent 9d6344aac7 db9852e853
commit e1ab3122c8
30 changed files with 142 additions and 1979 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/_sidebar.md generated
View File

@@ -14,6 +14,7 @@
* Ansible
* [Ansible](/docs/ansible/ansible.md)
* [Ansible Collection](/docs/ansible/ansible_collection.md)
* [Inventory](/docs/ansible/inventory.md)
* [Vars](/docs/ansible/vars.md)
* Cloud Controllers
* [Openstack](/docs/cloud_controllers/openstack.md)

139
docs/ansible/ansible.md
View File

@@ -34,91 +34,22 @@ Based on the table below and the available python version for your ansible host
|-----------------|----------------|
| >= 2.16.4 | 3.10-3.12 |
## Inventory
## Customize Ansible vars
The inventory is composed of 3 groups:
* **kube_node** : list of kubernetes nodes where the pods will run.
* **kube_control_plane** : list of servers where kubernetes control plane components (apiserver, scheduler, controller) will run.
* **etcd**: list of servers to compose the etcd server. You should have at least 3 servers for failover purpose.
When _kube_node_ contains _etcd_, you define your etcd cluster to be as well schedulable for Kubernetes workloads.
If you want it a standalone, make sure those groups do not intersect.
If you want the server to act both as control-plane and node, the server must be defined
on both groups _kube_control_plane_ and _kube_node_. If you want a standalone and
unschedulable control plane, the server must be defined only in the _kube_control_plane_ and
not _kube_node_.
There are also two special groups:
* **calico_rr** : explained for [advanced Calico networking cases](/docs/CNI/calico.md)
* **bastion** : configure a bastion host if your nodes are not directly reachable
Lastly, the **k8s_cluster** is dynamically defined as the union of **kube_node**, **kube_control_plane** and **calico_rr**.
This is used internally and for the purpose of defining whole cluster variables (`<inventory>/group_vars/k8s_cluster/*.yml`)
Below is a complete inventory example:
```ini
## Configure 'ip' variable to bind kubernetes services on a
## different ip than the default iface
node1 ansible_host=95.54.0.12 ip=10.3.0.1
node2 ansible_host=95.54.0.13 ip=10.3.0.2
node3 ansible_host=95.54.0.14 ip=10.3.0.3
node4 ansible_host=95.54.0.15 ip=10.3.0.4
node5 ansible_host=95.54.0.16 ip=10.3.0.5
node6 ansible_host=95.54.0.17 ip=10.3.0.6
[kube_control_plane]
node1
node2
[etcd]
node1
node2
node3
[kube_node]
node2
node3
node4
node5
node6
```
## Group vars and overriding variables precedence
The group variables to control main deployment options are located in the directory ``inventory/sample/group_vars``.
Optional variables are located in the `inventory/sample/group_vars/all.yml`.
Mandatory variables that are common for at least one role (or a node group) can be found in the
`inventory/sample/group_vars/k8s_cluster.yml`.
There are also role vars for docker, kubernetes preinstall and control plane roles.
According to the [ansible docs](https://docs.ansible.com/ansible/latest/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable),
those cannot be overridden from the group vars. In order to override, one should use
the `-e` runtime flags (most simple way) or other layers described in the docs.
Kubespray uses only a few layers to override things (or expect them to
be overridden for roles):
Kubespray expects users to use one of the following variables sources for settings and customization:
| Layer | Comment |
|----------------------------------------|------------------------------------------------------------------------------|
| **role defaults** | provides best UX to override things for Kubespray deployments |
| inventory vars | Unused |
| **inventory group_vars** | Expects users to use ``all.yml``,``k8s_cluster.yml`` etc. to override things |
| inventory host_vars | Unused |
| playbook group_vars | Unused |
| playbook host_vars | Unused |
| **host facts** | Kubespray overrides for internal roles' logic, like state flags |
| play vars | Unused |
| play vars_prompt | Unused |
| play vars_files | Unused |
| registered vars | Unused |
| set_facts | Kubespray overrides those, for some places |
| **role and include vars** | Provides bad UX to override things! Use extra vars to enforce |
| block vars (only for tasks in block) | Kubespray overrides for internal roles' logic |
| task vars (only for the task) | Unused for roles, but only for helper scripts |
| inventory vars | |
| - **inventory group_vars** | most used |
| - inventory host_vars | host specifc vars overrides, group_vars is usually more practical |
| **extra vars** (always win precedence) | override with ``ansible-playbook -e @foo.yml`` |
[!IMPORTANT]
Extra vars are best used to override kubespray internal variables, for instances, roles/vars/.
Those vars are usually **not expected** (by Kubespray developers) to be modified by end users, and not part of Kubespray
interface. Thus they can change, disappear, or break stuff unexpectedly.
## Ansible tags
The following tags are defined in playbooks:
@@ -257,42 +188,32 @@ ansible-playbook -i inventory/sample/hosts.ini cluster.yml \
--tags download --skip-tags upload,upgrade
```
Note: use `--tags` and `--skip-tags` wise and only if you're 100% sure what you're doing.
## Bastion host
If you prefer to not make your nodes publicly accessible (nodes with private IPs only),
you can use a so-called _bastion_ host to connect to your nodes. To specify and use a bastion,
simply add a line to your inventory, where you have to replace x.x.x.x with the public IP of the
bastion host.
```ShellSession
[bastion]
bastion ansible_host=x.x.x.x
```
For more information about Ansible and bastion hosts, read
[Running Ansible Through an SSH Bastion Host](https://blog.scottlowe.org/2015/12/24/running-ansible-through-ssh-bastion-host/)
Note: use `--tags` and `--skip-tags` wisely and only if you're 100% sure what you're doing.
## Mitogen
Mitogen support is deprecated, please see [mitogen related docs](/docs/advanced/mitogen.md) for usage and reasons for deprecation.
## Beyond ansible 2.9
## Troubleshooting Ansible issues
Ansible project has decided, in order to ease their maintenance burden, to split between
two projects which are now joined under the Ansible umbrella.
Ansible-base (2.10.x branch) will contain just the ansible language implementation while
ansible modules that were previously bundled into a single repository will be part of the
ansible 3.x package. Please see [this blog post](https://blog.while-true-do.io/ansible-release-3-0-0/)
that explains in detail the need and the evolution plan.
**Note:** this change means that ansible virtual envs cannot be upgraded with `pip install -U`.
You first need to uninstall your old ansible (pre 2.10) version and install the new one.
Having the wrong version of ansible, ansible collections or python dependencies can cause issue.
In particular, Kubespray ship custom modules which Ansible needs to find, for which you should specify [ANSIBLE_LIBRAY](https://docs.ansible.com/ansible/latest/dev_guide/developing_locally.html#adding-a-module-or-plugin-outside-of-a-collection)
```ShellSession
pip uninstall ansible ansible-base ansible-core
cd kubespray/
pip install -U .
export ANSIBLE_LIBRAY=<kubespray_dir>/library`
```
A simple way to ensure you get all the correct version of Ansible is to use
the [pre-built docker image from Quay](https://quay.io/repository/kubespray/kubespray?tab=tags).
You will then need to use [bind mounts](https://docs.docker.com/storage/bind-mounts/)
to access the inventory and SSH key in the container, like this:
```ShellSession
git checkout v2.26.0
docker pull quay.io/kubespray/kubespray:v2.26.0
docker run --rm -it --mount type=bind,source="$(pwd)"/inventory/sample,dst=/inventory \
--mount type=bind,source="${HOME}"/.ssh/id_rsa,dst=/root/.ssh/id_rsa \
quay.io/kubespray/kubespray:v2.26.0 bash
# Inside the container you may now run the kubespray playbooks:
ansible-playbook -i /inventory/inventory.ini --private-key /root/.ssh/id_rsa cluster.yml
```

71
docs/ansible/inventory.md Normal file
View File

@@ -0,0 +1,71 @@
# Inventory
The inventory is composed of 3 groups:
* **kube_node** : list of kubernetes nodes where the pods will run.
* **kube_control_plane** : list of servers where kubernetes control plane components (apiserver, scheduler, controller) will run.
* **etcd**: list of servers to compose the etcd server. You should have at least 3 servers for failover purpose.
When _kube_node_ contains _etcd_, you define your etcd cluster to be as well schedulable for Kubernetes workloads.
If you want it a standalone, make sure those groups do not intersect.
If you want the server to act both as control-plane and node, the server must be defined
on both groups _kube_control_plane_ and _kube_node_. If you want a standalone and
unschedulable control plane, the server must be defined only in the _kube_control_plane_ and
not _kube_node_.
There are also two special groups:
* **calico_rr** : explained for [advanced Calico networking cases](/docs/CNI/calico.md)
* **bastion** : configure a bastion host if your nodes are not directly reachable
Lastly, the **k8s_cluster** is dynamically defined as the union of **kube_node**, **kube_control_plane** and **calico_rr**.
This is used internally and for the purpose of defining whole cluster variables (`<inventory>/group_vars/k8s_cluster/*.yml`)
Below is a complete inventory example:
```ini
## Configure 'ip' variable to bind kubernetes services on a
## different ip than the default iface
node1 ansible_host=95.54.0.12 ip=10.3.0.1
node2 ansible_host=95.54.0.13 ip=10.3.0.2
node3 ansible_host=95.54.0.14 ip=10.3.0.3
node4 ansible_host=95.54.0.15 ip=10.3.0.4
node5 ansible_host=95.54.0.16 ip=10.3.0.5
node6 ansible_host=95.54.0.17 ip=10.3.0.6
[kube_control_plane]
node1
node2
[etcd]
node1
node2
node3
[kube_node]
node2
node3
node4
node5
node6
```
## Inventory customization
See [Customize Ansible vars](/docs/ansible/ansible.md#customize-ansible-vars)
and [Ansible documentation on group_vars](https://docs.ansible.com/ansible/latest/inventory_guide/intro_inventory.html#assigning-a-variable-to-many-machines-group-variables)
## Bastion host
If you prefer to not make your nodes publicly accessible (nodes with private IPs only),
you can use a so-called _bastion_ host to connect to your nodes. To specify and use a bastion,
simply add a line to your inventory, where you have to replace x.x.x.x with the public IP of the
bastion host.
```ShellSession
[bastion]
bastion ansible_host=x.x.x.x
```
For more information about Ansible and bastion hosts, read
[Running Ansible Through an SSH Bastion Host](https://blog.scottlowe.org/2015/12/24/running-ansible-through-ssh-bastion-host/)

44
docs/getting_started/getting-started.md
View File

@@ -1,43 +1,33 @@
# Getting started
## Install ansible
Install Ansible according to [Ansible installation guide](/docs/ansible/ansible.md#installing-ansible).
## Building your own inventory
Ansible inventory can be stored in 3 formats: YAML, JSON, or INI-like. There is
an example inventory located
[here](https://github.com/kubernetes-sigs/kubespray/blob/master/inventory/sample/inventory.ini).
You can use an
[inventory generator](https://github.com/kubernetes-sigs/kubespray/blob/master/contrib/inventory_builder/inventory.py)
to create or modify an Ansible inventory. Currently, it is limited in
functionality and is only used for configuring a basic Kubespray cluster inventory, but it does
support creating inventory file for large clusters as well. It now supports
separated ETCD and Kubernetes control plane roles from node role if the size exceeds a
certain threshold. Run `python3 contrib/inventory_builder/inventory.py help` for more information.
Example inventory generator usage:
Ansible inventory can be stored in 3 formats: YAML, JSON, or INI-like. See the
[example inventory](/inventory/sample/inventory.ini)
and [Ansible documentation on building your inventory](https://docs.ansible.com/ansible/latest/inventory_guide/intro_inventory.html),
and [details on the inventory structure expected by Kubespray](/docs/ansible/inventory.md).
```ShellSession
cp -r inventory/sample inventory/mycluster
declare -a IPS=(10.10.1.3 10.10.1.4 10.10.1.5)
CONFIG_FILE=inventory/mycluster/hosts.yml python3 contrib/inventory_builder/inventory.py ${IPS[@]}
<your-favorite-editor> inventory/mycluster/inventory.ini
# Review and change parameters under ``inventory/mycluster/group_vars``
<your-favorite-editor> inventory/mycluster/group_vars/all.yml # for every node, including etcd
<your-favorite-editor> inventory/mycluster/group_vars/k8s_cluster.yml # for every node in the cluster (not etcd when it's separate)
<your-favorite-editor> inventory/mycluster/group_vars/kube_control_plane.yml # for the control plane
<your-favorite-editor> inventory/myclsuter/group_vars/kube_node.yml # for worker nodes
```
Then use `inventory/mycluster/hosts.yml` as inventory file.
## Starting custom deployment
Once you have an inventory, you may want to customize deployment data vars
and start the deployment:
**IMPORTANT**: Edit my\_inventory/groups\_vars/\*.yaml to override data vars:
## Installing the cluster
```ShellSession
ansible-playbook -i inventory/mycluster/hosts.yml cluster.yml -b -v \
ansible-playbook -i inventory/mycluster/ cluster.yml -b -v \
--private-key=~/.ssh/private_key
```
See more details in the [ansible guide](/docs/ansible/ansible.md).
### Adding nodes
You may want to add worker, control plane or etcd nodes to your existing cluster. This can be done by re-running the `cluster.yml` playbook, or you can target the bare minimum needed to get kubelet installed on the worker and talking to your control planes. This is especially helpful when doing something like autoscaling your clusters.

16
docs/getting_started/setting-up-your-first-cluster.md
View File

@@ -212,17 +212,15 @@ Copy ``inventory/sample`` as ``inventory/mycluster``:
cp -rfp inventory/sample inventory/mycluster
```
Update Ansible inventory file with inventory builder:
Update the sample Ansible inventory file with ip given by gcloud:
```ShellSession
declare -a IPS=($(gcloud compute instances list --filter="tags.items=kubernetes-the-kubespray-way" --format="value(EXTERNAL_IP)" | tr '\n' ' '))
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}
gcloud compute instances list --filter="tags.items=kubernetes-the-kubespray-way"
```
Open the generated `inventory/mycluster/hosts.yaml` file and adjust it so
that controller-0, controller-1 and controller-2 are control plane nodes and
worker-0, worker-1 and worker-2 are worker nodes. Also update the `ip` to the respective local VPC IP and
remove the `access_ip`.
Open `inventory/mycluster/inventory.ini` file and add it so
that controller-0, controller-1 and controller-2 in the `kube_control_plane` group and
worker-0, worker-1 and worker-2 in the `kube_node` group. Add respective `ip` to the respective local VPC IP for each node.
The main configuration for the cluster is stored in
`inventory/mycluster/group_vars/k8s_cluster/k8s_cluster.yml`. In this file we
@@ -242,7 +240,7 @@ the kubernetes cluster, just change the 'false' to 'true' for
Now we will deploy the configuration:
```ShellSession
ansible-playbook -i inventory/mycluster/hosts.yaml -u $USERNAME -b -v --private-key=~/.ssh/id_rsa cluster.yml
ansible-playbook -i inventory/mycluster/ -u $USERNAME -b -v --private-key=~/.ssh/id_rsa cluster.yml
```
Ansible will now execute the playbook, this can take up to 20 minutes.
@@ -596,7 +594,7 @@ If you want to keep the VMs and just remove the cluster state, you can simply
run another Ansible playbook:
```ShellSession
ansible-playbook -i inventory/mycluster/hosts.yaml -u $USERNAME -b -v --private-key=~/.ssh/id_rsa reset.yml
ansible-playbook -i inventory/mycluster/ -u $USERNAME -b -v --private-key=~/.ssh/id_rsa reset.yml
```
Resetting the cluster to the VMs original state usually takes about a couple

40
docs/operations/mirror.md
View File

@@ -7,6 +7,7 @@ The public mirror is useful to make the public resources download quickly in som
You can follow the [offline](offline-environment.md) to config the image/file download configuration to the public mirror site. If you want to download quickly in China, the configuration can be like:
```shell
# this should be in <your_inventory>/group_vars/k8s_cluster.yml
gcr_image_repo: "gcr.m.daocloud.io"
kube_image_repo: "k8s.m.daocloud.io"
docker_image_repo: "docker.m.daocloud.io"
@@ -19,45 +20,6 @@ files_repo: "https://files.m.daocloud.io"
Use mirror sites only if you trust the provider. The Kubespray team cannot verify their reliability or security.
You can replace the `m.daocloud.io` with any site you want.
## Example Usage Full Steps
You can follow the full steps to use the kubesray with mirror. for example:
Install Ansible according to Ansible installation guide then run the following steps:
```shell
# Copy ``inventory/sample`` as ``inventory/mycluster``
cp -rfp inventory/sample inventory/mycluster
# Update Ansible inventory file with inventory builder
declare -a IPS=(10.10.1.3 10.10.1.4 10.10.1.5)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}
# Use the download mirror
cp inventory/mycluster/group_vars/all/offline.yml inventory/mycluster/group_vars/all/mirror.yml
sed -i -E '/# .*\{\{ files_repo/s/^# //g' inventory/mycluster/group_vars/all/mirror.yml
tee -a inventory/mycluster/group_vars/all/mirror.yml <<EOF
gcr_image_repo: "gcr.m.daocloud.io"
kube_image_repo: "k8s.m.daocloud.io"
docker_image_repo: "docker.m.daocloud.io"
quay_image_repo: "quay.m.daocloud.io"
github_image_repo: "ghcr.m.daocloud.io"
files_repo: "https://files.m.daocloud.io"
EOF
# Review and change parameters under ``inventory/mycluster/group_vars``
cat inventory/mycluster/group_vars/all/all.yml
cat inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml
# Deploy Kubespray with Ansible Playbook - run the playbook as root
# The option `--become` is required, as for example writing SSL keys in /etc/,
# installing packages and interacting with various systemd daemons.
# Without --become the playbook will fail to run!
ansible-playbook -i inventory/mycluster/hosts.yaml --become --become-user=root cluster.yml
```
The above steps are by adding the "Use the download mirror" step to the [README.md](../README.md) steps.
## Community-run mirror sites
DaoCloud(China)