mirror of
https://github.com/kubernetes-sigs/kubespray.git
synced 2026-10-30 22:16:23 -02:30
refactor: unify inconsistent variables in apiserver endpoint (#13392)
This commit is contained in:
+1
-1
@@ -379,7 +379,7 @@ To clean up any ipvs leftovers:
|
||||
|
||||
Calico node, typha and kube-controllers need to be able to talk to the kubernetes API. Please reference the [Enabling eBPF Calico Docs](https://docs.tigera.io/calico/latest/operations/ebpf/enabling-ebpf) for guidelines on how to do this.
|
||||
|
||||
Kubespray sets up the `kubernetes-services-endpoint` configmap based on the contents of the `loadbalancer_apiserver` inventory variable documented in [HA Mode](/docs/operations/ha-mode.md).
|
||||
Kubespray sets up the `kubernetes-services-endpoint` configmap based on the contents of the `kube_apiserver_cluster_internal_endpoint` inventory variable documented in [HA Mode](/docs/operations/ha-mode.md).
|
||||
|
||||
If no external loadbalancer is used, Calico eBPF can also use the localhost loadbalancer option. We are able to do so only if you use the same port for the localhost apiserver loadbalancer and the kube-apiserver. In this case Calico Automatic Host Endpoints need to be enabled to allow services like `coredns` and `metrics-server` to communicate with the kubernetes host endpoint. See [this blog post](https://www.projectcalico.org/securing-kubernetes-nodes-with-calico-automatic-host-endpoints/) on enabling automatic host endpoints.
|
||||
|
||||
|
||||
@@ -47,13 +47,15 @@ Some variables of note include:
|
||||
* *access_ip6* - similarly ``access_ip`` but IPv6
|
||||
* *ansible_default_ipv6.address* - Not Kubespray-specific, but it is used if ip6
|
||||
and access_ip6 are undefined
|
||||
* *loadbalancer_apiserver* - If defined, all hosts will connect to this
|
||||
address instead of localhost for kube_control_planes and kube_control_plane[0] for
|
||||
kube_nodes. See more details in the
|
||||
[HA guide](/docs/operations/ha-mode.md).
|
||||
* *loadbalancer_apiserver_localhost* - makes all hosts to connect to
|
||||
the apiserver internally load balanced endpoint. Mutual exclusive to the
|
||||
`loadbalancer_apiserver`. See more details in the
|
||||
* *kube_apiserver_endpoint* - Primary API endpoint for external access (e.g., kubectl).
|
||||
If you have a load balancer or VIP, set this in your inventory (e.g. `https://lb.example.com:6443`).
|
||||
See more details in the [HA guide](/docs/operations/ha-mode.md).
|
||||
* *kube_apiserver_cluster_internal_endpoint* - Internal endpoint used for in-cluster
|
||||
traffic, including nodes and pods. Defaults to localhost LB proxy or `kube_apiserver_endpoint`.
|
||||
See more details in the [HA guide](/docs/operations/ha-mode.md).
|
||||
* *loadbalancer_apiserver_localhost* - makes all hosts connect to
|
||||
the apiserver internally via the local load balanced proxy (nginx/haproxy). Defaults to `true`
|
||||
when no external or internal endpoint is defined. See more details in the
|
||||
[HA guide](/docs/operations/ha-mode.md).
|
||||
|
||||
## Special network variables
|
||||
|
||||
@@ -11,6 +11,5 @@
|
||||
# numallowedlocalasnumbers: 0
|
||||
# sourceaddress: "None"
|
||||
|
||||
# loadbalancer_apiserver:
|
||||
# address: "10.99.0.44"
|
||||
# port: "8383"
|
||||
# kube_apiserver_endpoint: "https://10.99.0.44:8383"
|
||||
# kube_apiserver_cluster_internal_endpoint: "https://10.99.0.44:8383"
|
||||
|
||||
@@ -11,6 +11,5 @@
|
||||
# numallowedlocalasnumbers: 0
|
||||
# sourceaddress: "None"
|
||||
|
||||
# loadbalancer_apiserver:
|
||||
# address: "10.99.0.21"
|
||||
# port: "8383"
|
||||
# kube_apiserver_endpoint: "https://10.99.0.21:8383"
|
||||
# kube_apiserver_cluster_internal_endpoint: "https://10.99.0.21:8383"
|
||||
|
||||
@@ -25,9 +25,8 @@ You also need to enable
|
||||
# HA for control-plane, requires a VIP
|
||||
kube_vip_controlplane_enabled: true
|
||||
kube_vip_address: 10.42.42.42
|
||||
loadbalancer_apiserver:
|
||||
address: "{{ kube_vip_address }}"
|
||||
port: 6443
|
||||
kube_apiserver_endpoint: "https://{{ kube_vip_address }}:6443"
|
||||
kube_apiserver_cluster_internal_endpoint: "https://{{ kube_vip_address }}:6443"
|
||||
# kube_vip_interface: ens160
|
||||
|
||||
# LoadBalancer for services
|
||||
|
||||
+27
-14
@@ -21,7 +21,7 @@ is less efficient than a dedicated load balancer because it creates extra
|
||||
health checks on the Kubernetes apiserver, but is more practical for scenarios
|
||||
where an external LB or virtual IP management is inconvenient. This option is
|
||||
configured by the variable `loadbalancer_apiserver_localhost` (defaults to
|
||||
`True`. Or `False`, if there is an external `loadbalancer_apiserver` defined).
|
||||
`True`. Or `False`, if `kube_apiserver_endpoint` or `kube_apiserver_cluster_internal_endpoint` is defined).
|
||||
You may also define the port the local internal loadbalancer uses by changing,
|
||||
`loadbalancer_apiserver_port`. This defaults to the value of
|
||||
`kube_apiserver_port`. It is also important to note that Kubespray will only
|
||||
@@ -29,6 +29,22 @@ configure kubelet and kube-proxy on non-master nodes to use the local internal
|
||||
loadbalancer. If you wish to control the name of the loadbalancer container,
|
||||
you can set the variable `loadbalancer_apiserver_pod_name`.
|
||||
|
||||
## API Server Endpoint Configuration
|
||||
|
||||
Kubespray uses two primary variables to configure API server endpoints:
|
||||
|
||||
* `kube_apiserver_endpoint` - Used to access the API from outside the cluster (kubectl, CI/CD, etc.)
|
||||
* `kube_apiserver_cluster_internal_endpoint` - Used to access the API from inside the cluster (nodes, pods with hostNetwork). Optional, defaults to `kube_apiserver_endpoint` or localhost LB.
|
||||
|
||||
These variables automatically resolve to appropriate values based on your
|
||||
loadbalancer configuration, providing sane defaults for HA, non-HA, and
|
||||
localhost LB scenarios. Legacy variables (`loadbalancer_apiserver`,
|
||||
`apiserver_loadbalancer_domain_name`, `kubeadm_config_api_fqdn`) were removed
|
||||
in v2.33.0 and should be replaced with the endpoint variables above.
|
||||
|
||||
> [!NOTE]
|
||||
> If your load balancer has both a domain name and an IP address (VIP), set `kube_apiserver_endpoint` to the domain name and add the VIP to `supplementary_addresses_in_ssl_keys` so that both are included in the API server certificate SANs.
|
||||
|
||||
If you choose to NOT use the local internal loadbalancer, you will need to
|
||||
use the [kube-vip](/docs/ingress/kube-vip.md) ansible role or configure your own loadbalancer to achieve HA. By default, it only configures a non-HA endpoint, which points to the
|
||||
`access_ip` or IP address of the first server node in the `kube_control_plane` group.
|
||||
@@ -61,10 +77,10 @@ And the corresponding example global vars for such a "cluster-aware"
|
||||
external LB with the cluster API access modes configured in Kubespray:
|
||||
|
||||
```yml
|
||||
apiserver_loadbalancer_domain_name: "my-apiserver-lb.example.com"
|
||||
loadbalancer_apiserver:
|
||||
address: <VIP>
|
||||
port: 8383
|
||||
## Configure endpoints directly
|
||||
kube_apiserver_endpoint: "https://<VIP>:8383"
|
||||
# Optional: if different from external endpoint
|
||||
kube_apiserver_cluster_internal_endpoint: "https://<VIP>:8383"
|
||||
```
|
||||
|
||||
Note: The default kubernetes apiserver configuration binds to all interfaces,
|
||||
@@ -73,12 +89,6 @@ loadbalancer_apiserver:
|
||||
listens on a specific interface (to avoid conflict with haproxy binding the
|
||||
port on the VIP address)
|
||||
|
||||
This domain name, or default "lb-apiserver.kubernetes.local", will be inserted
|
||||
into the `/etc/hosts` file of all servers in the `k8s_cluster` group and wired
|
||||
into the generated self-signed TLS/SSL certificates as well. Note that
|
||||
the HAProxy service should as well be HA and requires a VIP management, which
|
||||
is out of scope of this doc.
|
||||
|
||||
There is a special case for an internal and an externally configured (not with
|
||||
Kubespray) LB used simultaneously. Keep in mind that the cluster is not aware
|
||||
of such an external LB and you need no to specify any configuration variables
|
||||
@@ -90,7 +100,7 @@ for it.
|
||||
`supplementary_addresses_in_ssl_keys` list. Then, kubespray will add them into
|
||||
the generated cluster certificates as well.
|
||||
|
||||
Aside of that specific case, the `loadbalancer_apiserver` considered mutually
|
||||
Aside of that specific case, defining `kube_apiserver_endpoint` or `kube_apiserver_cluster_internal_endpoint` is considered mutually
|
||||
exclusive to `loadbalancer_apiserver_localhost`.
|
||||
|
||||
Access API endpoints are evaluated automatically, as the following:
|
||||
@@ -106,17 +116,20 @@ Access API endpoints are evaluated automatically, as the following:
|
||||
Where:
|
||||
|
||||
* `m[0]` - the first node in the `kube_control_plane` group;
|
||||
* `lb` - LB FQDN, `apiserver_loadbalancer_domain_name`;
|
||||
* `lb` - External loadbalancer address from `kube_apiserver_endpoint`;
|
||||
* `ext` - Externally load balanced VIP:port and FQDN, not managed by Kubespray;
|
||||
* `lc` - localhost;
|
||||
* `cbip` - a custom bind IP, `kube_apiserver_bind_address`;
|
||||
* `dbip` - localhost for the default bind IP '0.0.0.0';
|
||||
* `nsp` - nginx secure port, `loadbalancer_apiserver_port`, defers to `sp`;
|
||||
* `sp` - secure port, `kube_apiserver_port`;
|
||||
* `lp` - LB port, `loadbalancer_apiserver.port`, defers to the secure port;
|
||||
* `lp` - External loadbalancer port from `kube_apiserver_endpoint`;
|
||||
* `ip` - the node IP, defers to the ansible IP;
|
||||
* `aip` - `access_ip`, defers to the ip.
|
||||
|
||||
The `kube_apiserver_endpoint` and `kube_apiserver_cluster_internal_endpoint` variables
|
||||
automatically resolve to the appropriate values based on your loadbalancer configuration.
|
||||
|
||||
A second and a third column represent internal cluster access modes. The last
|
||||
column illustrates an example URI to access the cluster APIs externally.
|
||||
Kubespray has nothing to do with it, this is informational only.
|
||||
|
||||
Reference in New Issue
Block a user