Drop linux capabilities and rework users/groups

* Drop linux capabilities for unprivileged containerized
  worlkoads Kargo configures for deployments.
* Configure required securityContext/user/group/groups for kube
  components' static manifests, etcd, calico-rr and k8s apps,
  like dnsmasq daemonset.
* Rework cloud-init (etcd) users creation for CoreOS.
* Fix nologin paths, adjust defaults for addusers role and ensure
  supplementary groups membership added for users.
* Add netplug user for network plugins (yet unused by privileged
  networking containers though).
* Grant the kube and netplug users read access for etcd certs via
  the etcd certs group.
* Grant group read access to kube certs via the kube cert group.
* Remove priveleged mode for calico-rr and run it under its uid/gid
  and supplementary etcd_cert group.
* Adjust docs.
* Align cpu/memory limits and dropped caps with added rkt support
  for control plane.

Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>

roles/kubernetes/master/templates/manifests/kube-apiserver.manifest.j2

@@ -12,6 +12,14 @@ spec:
   - name: kube-apiserver
     image: {{ hyperkube_image_repo }}:{{ hyperkube_image_tag }}
     imagePullPolicy: {{ k8s_image_pull_policy }}
+    securityContext:
+      capabilities:
+        drop:
+{% for c in apps_drop_cap %}
+          - {{ c.upper() }}
+{% endfor %}
+        add:
+          - DAC_OVERRIDE
     resources:
       limits:
         cpu: {{ kube_apiserver_cpu_limit }}

roles/kubernetes/master/templates/manifests/kube-controller-manager.manifest.j2

@@ -11,6 +11,17 @@ spec:
   - name: kube-controller-manager
     image: {{ hyperkube_image_repo }}:{{ hyperkube_image_tag }}
     imagePullPolicy: {{ k8s_image_pull_policy }}
+    securityContext:
+      runAsUser: {{ kubelet_user_id }}
+      fsGroup: {{ kubelet_group_id }}
+      supplementalGroups:
+        - {{ kube_cert_group_id }}
+        - {{ etcd_cert_group_id }}
+      capabilities:
+        drop:
+{% for c in apps_drop_cap %}
+          - {{ c.upper() }}
+{% endfor %}
     resources:
       limits:
         cpu: {{ kube_controller_cpu_limit }}

roles/kubernetes/master/templates/manifests/kube-scheduler.manifest.j2

@@ -11,6 +11,17 @@ spec:
   - name: kube-scheduler
     image: {{ hyperkube_image_repo }}:{{ hyperkube_image_tag }}
     imagePullPolicy: {{ k8s_image_pull_policy }}
+    securityContext:
+      runAsUser: {{ kubelet_user_id }}
+      fsGroup: {{ kubelet_group_id }}
+      supplementalGroups:
+        - {{ kube_cert_group_id }}
+        - {{ etcd_cert_group_id }}
+      capabilities:
+        drop:
+{% for c in apps_drop_cap %}
+          - {{ c.upper() }}
+{% endfor %}
     resources:
       limits:
         cpu: {{ kube_scheduler_cpu_limit }}