A couple of days ago I managed to get myself into a situation where I had upgraded k3s deployment via Helm chart and deleted the data inside the pod in the process. Luckily, I took a backup from sqlite3 database before starting the upgrade, so not all was lost. But it took me almost a day to get the database running and the main reason was I couldn’t find a way to login to pod with root.

In a standard world, you are running everything with docker. In that case you can save yourself by logging in the node and running docker from there to get root access to pod. I’m running k3s on Raspberry Pi, so there is no Docker on the node. And it doesn’t help to install Docker on the node, since it can’t see the containers that k3s runs on top of containerd.

My next attempt was to use kubectl plugin called kubectl ssh. In theory, it does everything I need, but in practice it didn’t connect to my pod, no matter if I ran it from my own laptop or from the node. It was still worth trying it, because I found the solution to my problem from one of kube ssh’s issues.

Github user called mamiu gave an answer that I had been looking for. His/her answer was meant from k3d, which adds one docker wrapping aroung k3s, but modifying the steps a bit gave me finally my nirvana.

  1. The first step is to find the node on which the pod is running. (Note that in these commands “nginx” is the pod and “nginx-test” is the namespace".
kubectl get pod nginx -n nginx-test -o jsonpath="{.spec.nodeName}"
  1. Next we need to get the container ID of the pod. Note that this was tested on a pod where there was just one container.
kubectl get pod nginx -n nginx-test -o jsonpath="{.status.containerStatuses[].containerID}" | sed 's/.*\/\///'
  1. To get access to the pod and container, we need to access the node runnign the pod first. In my case, this was just SSH’ing in.

  2. On the node, we need to run the following command to get root access to the pod.

runc --root /run/containerd/runc/k8s.io/ exec -t -u 0 <insert_the_container_id_from_previous_command> sh

You can, of course, replace sh with some other shell like bash, if that’s available.