mirror of
https://github.com/kubernetes-sigs/kubespray.git
synced 2026-02-01 01:28:11 -03:30
modify doc structure and update existing doc-links as preparation for new doc generation script
This commit is contained in:
151
docs/CRI/containerd.md
Normal file
151
docs/CRI/containerd.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# containerd
|
||||
|
||||
[containerd] An industry-standard container runtime with an emphasis on simplicity, robustness and portability
|
||||
Kubespray supports basic functionality for using containerd as the default container runtime in a cluster.
|
||||
|
||||
_To use the containerd container runtime set the following variables:_
|
||||
|
||||
## k8s_cluster.yml
|
||||
|
||||
When kube_node contains etcd, you define your etcd cluster to be as well schedulable for Kubernetes workloads. Thus containerd and dockerd can not run at same time, must be set to bellow for running etcd cluster with only containerd.
|
||||
|
||||
```yaml
|
||||
container_manager: containerd
|
||||
```
|
||||
|
||||
## etcd.yml
|
||||
|
||||
```yaml
|
||||
etcd_deployment_type: host
|
||||
```
|
||||
|
||||
## Containerd config
|
||||
|
||||
Example: define registry mirror for docker hub
|
||||
|
||||
```yaml
|
||||
containerd_registries_mirrors:
|
||||
- prefix: docker.io
|
||||
mirrors:
|
||||
- host: https://mirror.gcr.io
|
||||
capabilities: ["pull", "resolve"]
|
||||
skip_verify: false
|
||||
- host: https://registry-1.docker.io
|
||||
capabilities: ["pull", "resolve"]
|
||||
skip_verify: false
|
||||
```
|
||||
|
||||
containerd falls back to `https://{{ prefix }}` when none of the mirrors have the image.
|
||||
This can be changed with the [`server` field](https://github.com/containerd/containerd/blob/main/docs/hosts.md#server-field):
|
||||
|
||||
```yaml
|
||||
containerd_registries_mirrors:
|
||||
- prefix: docker.io
|
||||
mirrors:
|
||||
- host: https://mirror.gcr.io
|
||||
capabilities: ["pull", "resolve"]
|
||||
skip_verify: false
|
||||
- host: https://registry-1.docker.io
|
||||
capabilities: ["pull", "resolve"]
|
||||
skip_verify: false
|
||||
server: https://mirror.example.org
|
||||
```
|
||||
|
||||
The `containerd_registries` and `containerd_insecure_registries` configs are deprecated.
|
||||
|
||||
### Containerd Runtimes
|
||||
|
||||
Containerd supports multiple runtime configurations that can be used with
|
||||
[RuntimeClass] Kubernetes feature. See [runtime classes in containerd] for the
|
||||
details of containerd configuration.
|
||||
|
||||
In kubespray, the default runtime name is "runc", and it can be configured with the `containerd_runc_runtime` dictionary:
|
||||
|
||||
```yaml
|
||||
containerd_runc_runtime:
|
||||
name: runc
|
||||
type: "io.containerd.runc.v2"
|
||||
engine: ""
|
||||
root: ""
|
||||
options:
|
||||
systemdCgroup: "false"
|
||||
binaryName: /usr/local/bin/my-runc
|
||||
base_runtime_spec: cri-base.json
|
||||
```
|
||||
|
||||
Further runtimes can be configured with `containerd_additional_runtimes`, which
|
||||
is a list of such dictionaries.
|
||||
|
||||
Default runtime can be changed by setting `containerd_default_runtime`.
|
||||
|
||||
#### Base runtime specs and limiting number of open files
|
||||
|
||||
`base_runtime_spec` key in a runtime dictionary is used to explicitly
|
||||
specify a runtime spec json file. `runc` runtime has it set to `cri-base.json`,
|
||||
which is generated with `ctr oci spec > /etc/containerd/cri-base.json` and
|
||||
updated to include a custom setting for maximum number of file descriptors per
|
||||
container.
|
||||
|
||||
You can change maximum number of file descriptors per container for the default
|
||||
`runc` runtime by setting the `containerd_base_runtime_spec_rlimit_nofile`
|
||||
variable.
|
||||
|
||||
You can tune many more [settings][runtime-spec] by supplying your own file name and content with `containerd_base_runtime_specs`:
|
||||
|
||||
```yaml
|
||||
containerd_base_runtime_specs:
|
||||
cri-spec-custom.json: |
|
||||
{
|
||||
"ociVersion": "1.0.2-dev",
|
||||
"process": {
|
||||
"user": {
|
||||
"uid": 0,
|
||||
...
|
||||
```
|
||||
|
||||
The files in this dict will be placed in containerd config directory,
|
||||
`/etc/containerd` by default. The files can then be referenced by filename in a
|
||||
runtime:
|
||||
|
||||
```yaml
|
||||
containerd_runc_runtime:
|
||||
name: runc
|
||||
base_runtime_spec: cri-spec-custom.json
|
||||
...
|
||||
```
|
||||
|
||||
Config insecure-registry access to self hosted registries.
|
||||
|
||||
```yaml
|
||||
containerd_registries_mirrors:
|
||||
- prefix: test.registry.io
|
||||
mirrors:
|
||||
- host: http://test.registry.io
|
||||
capabilities: ["pull", "resolve"]
|
||||
skip_verify: true
|
||||
- prefix: 172.19.16.11:5000
|
||||
mirrors:
|
||||
- host: http://172.19.16.11:5000
|
||||
capabilities: ["pull", "resolve"]
|
||||
skip_verify: true
|
||||
- prefix: repo:5000
|
||||
mirrors:
|
||||
- host: http://repo:5000
|
||||
capabilities: ["pull", "resolve"]
|
||||
skip_verify: true
|
||||
```
|
||||
|
||||
[containerd]: https://containerd.io/
|
||||
[RuntimeClass]: https://kubernetes.io/docs/concepts/containers/runtime-class/
|
||||
[runtime classes in containerd]: https://github.com/containerd/containerd/blob/main/docs/cri/config.md#runtime-classes
|
||||
[runtime-spec]: https://github.com/opencontainers/runtime-spec
|
||||
|
||||
### Optional : NRI
|
||||
|
||||
[Node Resource Interface](https://github.com/containerd/nri) (NRI) is disabled by default for the containerd. If you
|
||||
are using contained version v1.7.0 or above, then you can enable it with the
|
||||
following configuration:
|
||||
|
||||
```yaml
|
||||
nri_enabled: true
|
||||
```
|
||||
90
docs/CRI/cri-o.md
Normal file
90
docs/CRI/cri-o.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# CRI-O
|
||||
|
||||
[CRI-O] is a lightweight container runtime for Kubernetes.
|
||||
Kubespray supports basic functionality for using CRI-O as the default container runtime in a cluster.
|
||||
|
||||
* Kubernetes supports CRI-O on v1.11.1 or later.
|
||||
* etcd: configure either kubeadm managed etcd or host deployment
|
||||
|
||||
_To use the CRI-O container runtime set the following variables:_
|
||||
|
||||
## all/all.yml
|
||||
|
||||
```yaml
|
||||
download_container: false
|
||||
skip_downloads: false
|
||||
etcd_deployment_type: host # optionally kubeadm
|
||||
```
|
||||
|
||||
## k8s_cluster/k8s_cluster.yml
|
||||
|
||||
```yaml
|
||||
container_manager: crio
|
||||
```
|
||||
|
||||
## all/crio.yml
|
||||
|
||||
Enable docker hub registry mirrors
|
||||
|
||||
```yaml
|
||||
crio_registries:
|
||||
- prefix: docker.io
|
||||
insecure: false
|
||||
blocked: false
|
||||
location: registry-1.docker.io
|
||||
unqualified: false
|
||||
mirrors:
|
||||
- location: 192.168.100.100:5000
|
||||
insecure: true
|
||||
- location: mirror.gcr.io
|
||||
insecure: false
|
||||
```
|
||||
|
||||
[CRI-O]: https://cri-o.io/
|
||||
|
||||
The following is a method to enable insecure registries.
|
||||
|
||||
```yaml
|
||||
crio_insecure_registries:
|
||||
- 10.0.0.2:5000
|
||||
```
|
||||
|
||||
And you can config authentication for these registries after `crio_insecure_registries`.
|
||||
|
||||
```yaml
|
||||
crio_registry_auth:
|
||||
- registry: 10.0.0.2:5000
|
||||
username: user
|
||||
password: pass
|
||||
```
|
||||
|
||||
## Note about user namespaces
|
||||
|
||||
CRI-O has support for user namespaces. This feature is optional and can be enabled by setting the following two variables.
|
||||
|
||||
```yaml
|
||||
crio_runtimes:
|
||||
- name: runc
|
||||
path: /usr/bin/runc
|
||||
type: oci
|
||||
root: /run/runc
|
||||
allowed_annotations:
|
||||
- "io.kubernetes.cri-o.userns-mode"
|
||||
|
||||
crio_remap_enable: true
|
||||
```
|
||||
|
||||
The `allowed_annotations` configures `crio.conf` accordingly.
|
||||
|
||||
The `crio_remap_enable` configures the `/etc/subuid` and `/etc/subgid` files to add an entry for the **containers** user.
|
||||
By default, 16M uids and gids are reserved for user namespaces (256 pods * 65536 uids/gids) at the end of the uid/gid space.
|
||||
|
||||
## Optional : NRI
|
||||
|
||||
[Node Resource Interface](https://github.com/containerd/nri) (NRI) is disabled by default for the CRI-O. If you
|
||||
are using CRI-O version v1.26.0 or above, then you can enable it with the
|
||||
following configuration:
|
||||
|
||||
```yaml
|
||||
nri_enabled: true
|
||||
```
|
||||
105
docs/CRI/docker.md
Normal file
105
docs/CRI/docker.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# Docker support
|
||||
|
||||
The docker runtime is supported by kubespray and while the `dockershim` is deprecated to be removed in kubernetes 1.24+ there are alternative ways to use docker such as through the [cri-dockerd](https://github.com/Mirantis/cri-dockerd) project supported by Mirantis.
|
||||
|
||||
Using the docker container manager:
|
||||
|
||||
```yaml
|
||||
container_manager: docker
|
||||
```
|
||||
|
||||
*Note:* `cri-dockerd` has replaced `dockershim` across supported kubernetes version in kubespray 2.20.
|
||||
|
||||
Enabling the `overlay2` graph driver:
|
||||
|
||||
```yaml
|
||||
docker_storage_options: -s overlay2
|
||||
```
|
||||
|
||||
Enabling `docker_container_storage_setup`, it will configure devicemapper driver on Centos7 or RedHat7.
|
||||
Deployers must be define a disk path for `docker_container_storage_setup_devs`, otherwise docker-storage-setup will be executed incorrectly.
|
||||
|
||||
```yaml
|
||||
docker_container_storage_setup: true
|
||||
docker_container_storage_setup_devs: /dev/vdb
|
||||
```
|
||||
|
||||
Changing the Docker cgroup driver (native.cgroupdriver); valid options are `systemd` or `cgroupfs`, default is `systemd`:
|
||||
|
||||
```yaml
|
||||
docker_cgroup_driver: systemd
|
||||
```
|
||||
|
||||
If you have more than 3 nameservers kubespray will only use the first 3 else it will fail. Set the `docker_dns_servers_strict` to `false` to prevent deployment failure.
|
||||
|
||||
```yaml
|
||||
docker_dns_servers_strict: false
|
||||
```
|
||||
|
||||
Set the path used to store Docker data:
|
||||
|
||||
```yaml
|
||||
docker_daemon_graph: "/var/lib/docker"
|
||||
```
|
||||
|
||||
Changing the docker daemon iptables support:
|
||||
|
||||
```yaml
|
||||
docker_iptables_enabled: "false"
|
||||
```
|
||||
|
||||
Docker log options:
|
||||
|
||||
```yaml
|
||||
# Rotate container stderr/stdout logs at 50m and keep last 5
|
||||
docker_log_opts: "--log-opt max-size=50m --log-opt max-file=5"
|
||||
```
|
||||
|
||||
Change the docker `bin_dir`, this should not be changed unless you use a custom docker package:
|
||||
|
||||
```yaml
|
||||
docker_bin_dir: "/usr/bin"
|
||||
```
|
||||
|
||||
To keep docker packages after installation; speeds up repeated ansible provisioning runs when '1'.
|
||||
kubespray deletes the docker package on each run, so caching the package makes sense:
|
||||
|
||||
```yaml
|
||||
docker_rpm_keepcache: 1
|
||||
```
|
||||
|
||||
Allowing insecure-registry access to self hosted registries. Can be ipaddress and domain_name.
|
||||
|
||||
```yaml
|
||||
## example define 172.19.16.11 or mirror.registry.io
|
||||
docker_insecure_registries:
|
||||
- mirror.registry.io
|
||||
- 172.19.16.11
|
||||
```
|
||||
|
||||
Adding other registry, i.e. China registry mirror:
|
||||
|
||||
```yaml
|
||||
docker_registry_mirrors:
|
||||
- https://registry.docker-cn.com
|
||||
- https://mirror.aliyuncs.com
|
||||
```
|
||||
|
||||
Overriding default system MountFlags value. This option takes a mount propagation flag: `shared`, `slave` or `private`, which control whether mounts in the file system namespace set up for docker will receive or propagate mounts and unmounts. Leave empty for system default:
|
||||
|
||||
```yaml
|
||||
docker_mount_flags:
|
||||
```
|
||||
|
||||
Adding extra options to pass to the docker daemon:
|
||||
|
||||
```yaml
|
||||
## This string should be exactly as you wish it to appear.
|
||||
docker_options: ""
|
||||
```
|
||||
|
||||
For Debian based distributions, set the path to store the GPG key to avoid using the default one used in `apt_key` module (e.g. /etc/apt/trusted.gpg)
|
||||
|
||||
```yaml
|
||||
docker_repo_key_keyring: /etc/apt/trusted.gpg.d/docker.gpg
|
||||
```
|
||||
16
docs/CRI/gvisor.md
Normal file
16
docs/CRI/gvisor.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# gVisor
|
||||
|
||||
[gVisor](https://gvisor.dev/docs/) is an application kernel, written in Go, that implements a substantial portion of the Linux system call interface. It provides an additional layer of isolation between running applications and the host operating system.
|
||||
|
||||
gVisor includes an Open Container Initiative (OCI) runtime called runsc that makes it easy to work with existing container tooling. The runsc runtime integrates with Docker and Kubernetes, making it simple to run sandboxed containers.
|
||||
|
||||
## Usage
|
||||
|
||||
To enable gVisor you should be using a container manager that is compatible with selecting the [RuntimeClass](https://kubernetes.io/docs/concepts/containers/runtime-class/) such as `containerd`.
|
||||
|
||||
Containerd support:
|
||||
|
||||
```yaml
|
||||
container_manager: containerd
|
||||
gvisor_enabled: true
|
||||
```
|
||||
101
docs/CRI/kata-containers.md
Normal file
101
docs/CRI/kata-containers.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# Kata Containers
|
||||
|
||||
[Kata Containers](https://katacontainers.io) is a secure container runtime with lightweight virtual machines that supports multiple hypervisor solutions.
|
||||
|
||||
## Hypervisors
|
||||
|
||||
_Qemu_ is the only hypervisor supported by Kubespray.
|
||||
|
||||
## Installation
|
||||
|
||||
To enable Kata Containers, set the following variables:
|
||||
|
||||
**k8s-cluster.yml**:
|
||||
|
||||
```yaml
|
||||
container_manager: containerd
|
||||
kata_containers_enabled: true
|
||||
```
|
||||
|
||||
**etcd.yml**:
|
||||
|
||||
```yaml
|
||||
etcd_deployment_type: host
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
By default, runc is used for pods.
|
||||
Kubespray generates the runtimeClass kata-qemu, and it is necessary to specify it as
|
||||
the runtimeClassName of a pod spec to use Kata Containers:
|
||||
|
||||
```shell
|
||||
$ kubectl get runtimeclass
|
||||
NAME HANDLER AGE
|
||||
kata-qemu kata-qemu 3m34s
|
||||
$
|
||||
$ cat nginx.yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: mypod
|
||||
spec:
|
||||
runtimeClassName: kata-qemu
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.14.2
|
||||
$
|
||||
$ kubectl apply -f nginx.yaml
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Recommended : Pod Overhead
|
||||
|
||||
[Pod Overhead](https://kubernetes.io/docs/concepts/configuration/pod-overhead/) is a feature for accounting for the resources consumed by the Runtime Class used by the Pod.
|
||||
|
||||
When this feature is enabled, Kubernetes will count the fixed amount of CPU and memory set in the configuration as used by the virtual machine and not by the containers running in the Pod.
|
||||
|
||||
Pod Overhead is mandatory if you run Pods with Kata Containers that use [resources limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits).
|
||||
|
||||
**Set cgroup driver**:
|
||||
|
||||
To enable Pod Overhead feature you have to configure Kubelet with the appropriate cgroup driver, using the following configuration:
|
||||
|
||||
`cgroupfs` works best:
|
||||
|
||||
```yaml
|
||||
kubelet_cgroup_driver: cgroupfs
|
||||
```
|
||||
|
||||
... but when using `cgroups v2` (see <https://www.redhat.com/en/blog/world-domination-cgroups-rhel-8-welcome-cgroups-v2>) you can use systemd as well:
|
||||
|
||||
```yaml
|
||||
kubelet_cgroup_driver: systemd
|
||||
```
|
||||
|
||||
**Qemu hypervisor configuration**:
|
||||
|
||||
The configuration for the Qemu hypervisor uses the following values:
|
||||
|
||||
```yaml
|
||||
kata_containers_qemu_overhead: true
|
||||
kata_containers_qemu_overhead_fixed_cpu: 10m
|
||||
kata_containers_qemu_overhead_fixed_memory: 290Mi
|
||||
```
|
||||
|
||||
### Optional : Select Kata Containers version
|
||||
|
||||
Optionally you can select the Kata Containers release version to be installed. The available releases are published in [GitHub](https://github.com/kata-containers/kata-containers/releases).
|
||||
|
||||
```yaml
|
||||
kata_containers_version: 2.2.2
|
||||
```
|
||||
|
||||
### Optional : Debug
|
||||
|
||||
Debug is disabled by default for all the components of Kata Containers. You can change this behaviour with the following configuration:
|
||||
|
||||
```yaml
|
||||
kata_containers_qemu_debug: 'false'
|
||||
```
|
||||
Reference in New Issue
Block a user