Add etcd proxy support

* Enforce a etcd-proxy role to a k8s-cluster group members. This
provides an HA layout for all of the k8s cluster internal clients.
* Proxies to be run on each node in the group as a separate etcd
instances with a readwrite proxy mode and listen the given endpoint,
which is either the access_ip:2379 or the localhost:2379.
* A notion for the 'kube_etcd_multiaccess' is: ignore endpoints and
loadbalancers and use the etcd members IPs as a comma-separated
list. Otherwise, clients shall use the local endpoint provided by a
etcd-proxy instances on each etcd node. A Netwroking plugins always
use that access mode.
* Fix apiserver's etcd servers args to use the etcd_access_endpoint.
* Fix networking plugins flannel/calico to use the etcd_endpoint.
* Fix name env var for non masters to be set as well.
* Fix etcd_client_url was not used anywhere and other etcd_* facts
evaluation was duplicated in a few places.
* Define proxy modes only in the env file, if not a master. Del
an automatic proxy mode decisions for etcd nodes in init/unit scripts.
* Use Wants= instead of Requires= as "This is the recommended way to
hook start-up of one unit to the start-up of another unit"
* Make apiserver/calico Wants= etcd-proxy to keep it always up

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
Co-authored-by: Matthew Mosesohn <mmosesohn@mirantis.com>

roles/network_plugin/calico/tasks/main.yml

@@ -44,7 +44,7 @@
 
 - name: Calico | Check if calico network pool has already been configured
   uri:
-    url: "http://127.0.0.1:2379/v2/keys/calico/v1/ipam/v4/pool"
+    url: "{{ etcd_endpoint }}/v2/keys/calico/v1/ipam/v4/pool"
     return_content: yes
     status_code: 200,404
   register: calico_conf
@@ -70,7 +70,7 @@
 
 - name: Calico | Get calico configuration from etcd
   uri:
-    url: "http://127.0.0.1:2379/v2/keys/calico/v1/ipam/v4/pool"
+    url: "{{ etcd_endpoint }}/v2/keys/calico/v1/ipam/v4/pool"
     return_content: yes
   register: calico_pools
   run_once: true

roles/network_plugin/calico/templates/calico-node.service.j2

@@ -1,8 +1,8 @@
 [Unit]
 Description=Calico per-node agent
 Documentation=https://github.com/projectcalico/calico-docker
-Requires=docker.service
-After=docker.service etcd.service
+Wants=docker.service etcd-proxy.service
+After=docker.service etcd-proxy.service
 
 [Service]
 User=root

roles/network_plugin/calico/templates/calicoctl-container.j2

@@ -1,6 +1,6 @@
 #!/bin/bash
 /usr/bin/docker run --privileged --rm \
---net=host -e ETCD_AUTHORITY=127.0.0.1:2379 \
+--net=host -e ETCD_AUTHORITY={{ etcd_authority }} \
 -v /usr/bin/docker:/usr/bin/docker \
 -v /var/run/docker.sock:/var/run/docker.sock \
 -v /var/run/calico:/var/run/calico \

roles/network_plugin/calico/templates/network-environment.j2

@@ -6,4 +6,4 @@ DEFAULT_IPV4={{ip | default(ansible_default_ipv4.address) }}
 KUBERNETES_MASTER={{ hostvars[groups['kube-master'][0]]['access_ip'] | default(hostvars[groups['kube-master'][0]]['ip'] | default(hostvars[groups['kube-master'][0]]['ansible_default_ipv4']['address'])) }}
 
 # IP and port of etcd instance used by Calico
-ETCD_AUTHORITY=127.0.0.1:2379
+ETCD_AUTHORITY={{ etcd_authority }}

roles/network_plugin/flannel/templates/flannel-pod.yml

@@ -21,7 +21,7 @@
         args:
           - "--network-config=/etc/flannel-network.json"
           - "--etcd-prefix=/{{ cluster_name }}/network"
-          - "--etcd-server=http://{{ groups['etcd'][0] }}:2379"
+          - "--etcd-server={{ etcd_endpoint }}"
         volumeMounts:
           - name: "networkconfig"
             mountPath: "/etc/flannel-network.json"
@@ -31,7 +31,7 @@
         command:
           - "/bin/sh"
           - "-c"
-          - "/opt/bin/flanneld -etcd-endpoints {% for srv in groups['etcd'] %}http://{{ srv }}:2379{% if not loop.last %},{% endif %}{% endfor %} -etcd-prefix /{{ cluster_name }}/network {% if flannel_interface is defined %}-iface {{ flannel_interface }}{% endif %} {% if flannel_public_ip is defined %}-public-ip {{ flannel_public_ip }}{% endif %}"
+          - "/opt/bin/flanneld -etcd-endpoints {{ etcd_access_endpoint }} -etcd-prefix /{{ cluster_name }}/network {% if flannel_interface is defined %}-iface {{ flannel_interface }}{% endif %} {% if flannel_public_ip is defined %}-public-ip {{ flannel_public_ip }}{% endif %}"
         ports:
           - hostPort: 10253
             containerPort: 10253