fix(container-engine/cri-o): flush Restart crio handler inside the role (#13416)

* fix(container-engine/cri-o): flush Restart crio handler inside the role

During a Kubernetes upgrade (e.g. 1.32 -> 1.33) the cri-o systemd service is
not restarted even though the new cri-o binaries are deployed. This leaves the
node running the old cri-o version (reported as crio://1.32.x after kubelet is
upgraded to 1.33.x); a manual 'systemctl restart crio' fixes it.

The cri-o role notifies 'Restart crio' from several tasks (binary copy,
registries, proxy drop-in, service file, etc.) but never flushes the handler
itself. The handler only fired when the kubernetes/node role ran its own
'meta: flush_handlers' later in the same play. That cross-role dependency is
fragile during upgrades: if kubernetes/node is skipped, tagged out, or the
flush is reordered, the restart is silently dropped.

Route every restart through the single handler: crio.conf and config.json now
notify 'Restart crio' and the ad-hoc 'trigger service restart only when needed'
task is dropped (its restart duplicated what the handler already does, and a
restarted unit also starts a stopped one). Then add 'meta: flush_handlers'
right after the service start task so the pending handlers run as part of this
role regardless of any downstream role's flush timing.

Signed-off-by: Mehrdad Biukian Naeini <mehrdadbiukian@gmail.com>

* test(youki): run youki before cri-o in molecule converge

The converge ran container-engine/cri-o before container-engine/youki, the reverse of the order used by the container-engine role, which installs the runtime roles (kata-containers, gvisor, crun, youki) before the container manager role.

With cri-o flushing its restart handler inside the role, cri-o is now restarted while /usr/local/bin/youki does not exist yet: CRI-O skips the runtime whose binary is missing, so the youki runtime handler is never registered and the verify step fails with 'failed to find runtime handler youki from runtime list map[crun:...]'. Running the roles in the playbook order puts the youki binary in place before cri-o starts.

Signed-off-by: Mehrdad Biukian Naeini <mehrdadbiukian@gmail.com>

---------

Signed-off-by: Mehrdad Biukian Naeini <mehrdadbiukian@gmail.com>

roles/container-engine/cri-o/tasks/main.yaml

@@ -110,14 +110,14 @@
     src: crio.conf.j2
     dest: /etc/crio/crio.conf
     mode: "0644"
-  register: config_install
+  notify: Restart crio
 
 - name: Cri-o | install config.json
   template:
     src: config.json.j2
     dest: /etc/crio/config.json
     mode: "0644"
-  register: reg_auth_install
+  notify: Restart crio
 
 - name: Cri-o | copy binaries
   copy:
@@ -275,15 +275,15 @@
     daemon_reload: true
     enabled: true
     state: started
-  register: service_start
 
-- name: Cri-o | trigger service restart only when needed
-  service:
-    name: crio
-    state: restarted
-  when:
-    - config_install.changed or reg_auth_install.changed
-    - not service_start.changed
+# Every config/binary change above notifies "Restart crio", so this flush runs
+# the pending handlers right away instead of leaving the cri-o restart to the
+# end of the play (where it is lost if kubernetes/node is skipped, tagged out,
+# or runs in a separate play, as happens during cluster upgrades).
+# Note: flush_handlers flushes *all* notified handlers in the play, not only
+# "Restart crio". See kubernetes-sigs/kubespray#13031.
+- name: Cri-o | flush handlers to apply cri-o restart
+  meta: flush_handlers
 
 - name: Cri-o | verify that crio is running
   command: "{{ bin_dir }}/{{ crio_status_command }} info"

roles/container-engine/youki/molecule/default/converge.yml

@@ -7,5 +7,8 @@
     container_manager: crio
   roles:
     - role: kubespray_defaults
-    - role: container-engine/cri-o
+    # Keep the order used by roles/container-engine/tasks/main.yml: runtime
+    # roles run before the container manager, so the youki binary exists when
+    # cri-o is (re)started and the youki runtime handler is registered.
     - role: container-engine/youki
+    - role: container-engine/cri-o