inside containerd: how shim and runc actually work

k8scontainerinfrastructure

kubelet calls containerd, and a container appears. But between that call and the running container, three processes do a relay. One of them is created to die.

This article traces that relay as a forced chain: each process exists because the one before it couldn't finish the job. We follow it from runc to the launcher that's born only to die, and end at the process tree you're left with.

kubectl set image deployment/api api=api:v2
  │
  ① kubectl → API Server → etcd (write new image)
  ② Deployment Controller → creates new ReplicaSet
  ③ ReplicaSet Controller → creates Pod objects
  ④ Scheduler → assigns Pod to a Node
  ⑤ kubelet picks up the Pod
  │
  ▼
containerd
  │ fork
  ▼
containerd-shim-runc-v2  start   (launcher, short-lived)
  │ re-execs itself, then exits (double fork)
  ▼
containerd-shim-runc-v2 (babysitter) ← reparented to PID 1
  │ exec
  ▼
runc
  │ set up namespaces, cgroups, chroot
  │ start container process
  │ exit (job done)
  ▼
container process (running)
  parent = containerd-shim-runc-v2
  grandparent = systemd (PID 1)

Steady state (the start launcher and runc both exited):

systemd (PID 1)
  ├── containerd                         ← manager, talks over ttrpc socket
  └── containerd-shim-runc-v2           ← babysitter, waitpid() sleeping
          └── nginx (container PID 1)    ← your workload

containerd and containerd-shim-runc-v2 are siblings under systemd, not parent and child. They talk over a ttrpc socket, not through the process tree. nginx shows PID 1 inside the container's PID namespace. From the host, it has a normal PID like 50321.

This is containerd's internal machinery. K8s doesn't know any of this exists. For how K8s got to containerd in the first place, see Container Runtime: A Divorce Story.

runc: a builder, not a babysitter

runc is a CLI tool. It builds the container and exits. Sets up Linux namespaces, cgroups, chroot, starts the process, then it's gone.

The container keeps running even though runc has already exited, so which process is its parent now?

Why each layer exists

The container needs a long-lived parent: something that holds its stdio (for kubectl logs), calls waitpid() when it dies and passes the exit code up so kubectl get pod can show Completed or Error, and answers containerd's queries (for kubectl exec, status checks). runc can't do this. It already exited.

Could containerd do it directly? Yes. But containerd gets restarted. Version upgrades, security patches, config changes. Every restart creates process management problems for its children. So the design was forced, one problem at a time (this is the order problems were discovered, not the order processes run):

  1. Container needs a long-lived parent → runc can't stay
  2. Make containerd the parent? → containerd restarts too often
  3. Create a dedicated babysitter → containerd-shim-runc-v2 is born
  4. Babysitter must be independent of containerd's process tree → needs double fork
  5. Double fork needs a short-lived middle process that forks the babysitter then exits → the shim binary runs a first time with the start argument just to do that

Each layer got added because the one before it didn't solve the whole problem.

The orphan problem

Every Linux process has a parent. When a child dies, the parent must call wait() to clean it up. Without wait(), the dead child becomes a zombie. It's no longer running and uses no RAM, but it still occupies a row in the process table (a PID slot). Zombies don't leak memory, they leak PIDs.

A living orphan is worse. When a parent dies, the kernel reparents children to systemd (PID 1). systemd keeps them alive but doesn't know if they should be alive. A runaway orphan process eats CPU and RAM until someone manually kills it or it exits on its own.

containerd-shim-runc-v2 is the container's parent. But who parents containerd-shim-runc-v2? And how do you detach it from containerd's process tree?

The double fork trick

First, the names. containerd-shim and containerd-shim-runc-v2 are not the two actors here, and that is exactly where people get confused. The double fork uses a single binary, containerd-shim-runc-v2, run twice:

containerd-shim-runc-v2 startcontainerd-shim-runc-v2 (no args)
LifespanLess than a secondLives as long as the container
PurposeRe-exec the real shim, report its socket address, then exitGuard the container
AnalogySurrogate mother. Gives birth, leavesLive-in nanny

The first run exists only to fork the second and exit, orphaning it. (containerd-shim without the suffix is a different thing, the old v1 shim binary, covered at the end.)

Now the mechanism. This is a classic Unix trick, double fork: two forks in a row, where the middle process exits.

  1. containerd forks and execs → containerd-shim-runc-v2 start (child)
  2. the start process re-execs the same binary → containerd-shim-runc-v2 daemon (grandchild), in a new process group
  3. the start process reports the daemon's socket address to containerd, then exits
  4. the containerd-shim-runc-v2 daemon is now an orphan
  5. Linux kernel reparents the daemon to PID 1 (systemd)

The reparent in step 5 happens because the launcher exits in step 3, not because of the new process group in step 2. A process reparents to PID 1 only when its own parent dies, and that is the launcher's entire job: it is a throwaway middle process, created to die immediately so the daemon it spawned is orphaned to PID 1 at once, while containerd is still alive. The new process group is a separate effect, covered below.

There's a common belief that a child dies when its parent dies, but Linux doesn't work that way.

Parent dies → child gets adopted by PID 1 → child keeps running.

(The belief comes from shells: bash sends SIGHUP to its children before it exits, and the default response to SIGHUP is to quit. The children die because bash kills them, not because the parent died. That's shell behavior, not the kernel's.)

The launcher process exits without sending any signal. It's not a shell, so the daemon just becomes an orphan and keeps running.

Before (right after fork):

  containerd (PID 500)
    └── containerd-shim-runc-v2 start (PID 501)
          └── containerd-shim-runc-v2 (PID 502)

After (the start process exits):

  containerd (PID 500)              ← no longer parent of anything
  systemd (PID 1)
    └── containerd-shim-runc-v2 (PID 502)   ← adopted by PID 1
          └── container process (PID 503)

containerd and containerd-shim-runc-v2 are no longer in a parent-child relationship. A socket can disconnect and reconnect, but the process tree can't be rewired like that.

Detaching the process tree is only one of three independent ways the shim is kept clear of containerd's fate. Each axis has its own tool:

AxisToolProtects against
process treereparent to PID 1losing the parent / waitpid relationship
process groupsetpgidCtrl+C / kill -<pgid> group signals
cgroupKillMode=processsystemctl stop killing the whole cgroup, not just containerd
Deeper: why not just fork once?

Fork once and the shim is containerd's direct child. Nothing crashes — systemd still reaps orphans — but containerd now carries process-management baggage it shouldn't: SIGCHLD noise in its event loop, child PIDs cluttering its table, and after a restart it can't waitpid() shims it no longer parents (Linux has no call to reassign a parent).

You might think fork-once ends up the same place, since the shim is orphaned to PID 1 when containerd eventually exits anyway. But that's the end state, not the cost. It's a kid who moves out the day they're born versus one who lives at home until the parent dies — both end up out of the house, only one spares the parent the daily upkeep. Double fork makes the shim never containerd's child: not before a restart, not after.

Inferred from the double fork pattern, the iximiuz shim walkthrough, and the Container42 explainer; no containerd doc spells this out.

Deeper: the process-group and cgroup axes

The table's second row is signals. The daemon starts in its own process group, set with setpgid. A process group is what a group signal targets, so if the shim shared containerd's group, a kill -<pgid> (or a Ctrl+C, if containerd ran in a foreground terminal) would reach every process in it, the shim included.

The third row, cgroup, is a separate axis again. setpgid only swaps the shim's process-group label, never its cgroup. Think of two cards a process carries: a new signal-group card doesn't move it out of containerd's department, so a department-wide layoff — a cgroup kill via systemctl stop — still reaches it. That's why containerd.service ships with KillMode=process: stopping containerd then kills only containerd, not the shims sharing its cgroup.

What containerd-shim-runc-v2 actually does all day

After runc exits and the container is running, containerd-shim-runc-v2 sits there and does three things:

JobChannelWhy
Hold stdin/stdout/stderrfile descriptorskubectl logs and kubectl attach need something to read from
Call wait() when container diesprocess treeCollects exit code so kubectl get pod can show Completed or Error
Report to containerd and respond to commandsttrpc socketcontainerd asks "is it still running?" or "kill it"

Only the parent can collect a child's exit code, so the shim's waitpid() runs through the process tree, and that code flows up through containerd to kubelet as the Pod status you see in kubectl get pod. Reporting takes the other channel, a ttrpc socket that can break and reconnect.

If containerd crashes, containerd-shim-runc-v2 still collects the exit code (process tree is intact). It just can't report yet. When the new containerd reconnects the socket, containerd-shim-runc-v2 delivers the result. Reaping and reporting are independent.

Most of the time containerd-shim-runc-v2 is idle — but idle here means blocked, not polling. Each container gets its own goroutine sitting in waitpid(), and the kernel wakes that goroutine the instant the container dies. A Pod with several containers means several waiting goroutines, so the shim watches them all at once and stays responsive on the socket the whole time.

Deeper: why the container reparents to the shim, not PID 1

There's one gap left to close. runc started the container, then exited. By the orphan rule above, the container should now reparent to PID 1 like any other orphan. If it did, the shim could no longer waitpid() it, and the exit code would be lost.

The shim prevents that by marking itself a subreaper when it starts:

prctl(PR_SET_CHILD_SUBREAPER, 1)

That rewrites the rule for its descendants. When runc exits, the container doesn't climb all the way to PID 1. It reparents to the nearest ancestor that set the subreaper flag, which is the shim. So the container stays under the shim, and waitpid() keeps working.

It's the same kernel rule pointed in two directions. The shim isn't caught by any subreaper, so it detaches up to PID 1, away from containerd. The container is caught by the shim, so it stays put.

Deeper: kernel vs systemd, who notifies whom

The kernel and systemd are not the same thing. The kernel is the OS core running in kernel space. systemd is a regular process (PID 1) running in user space.

The notification always comes from the kernel. systemd's only role is being the last-resort parent for orphans:

EventWho handles it
container dieskernel wakes containerd-shim-runc-v2
containerd-shim-runc-v2 dieskernel wakes systemd
systemd dieskernel panic, machine is dead

systemd is a passive adopter. containerd-shim-runc-v2 is an active guardian. systemd doesn't know or care what containerd-shim-runc-v2 is. It just calls wait() when an adopted child dies. containerd-shim-runc-v2 knows exactly what it's watching and reports the exit code back to containerd.

v1 → v2: same job, better packaging

The core mechanism (double fork, reparent, wait, hold stdio) is essentially the same in v1 and v2. What changed is how many shim processes you run:

v1: one containerd-shim per container

  • Pod with 3 containers → 3 containerd-shim processes
  • 30 Pods x 2 containers average → 60 containerd-shim processes on one Node

v2: one containerd-shim-runc-v2 per Pod

  • Pod with 3 containers → 1 containerd-shim-runc-v2 process managing all three
  • 30 Pods → 30 containerd-shim-runc-v2 processes (not 60)

Both versions talk to containerd over ttrpc, a lighter alternative to gRPC that containerd built by dropping the HTTP/2 stack to keep shims small. v2's win is the process count, not the transport. (gRPC for shims only arrived as an experimental option in containerd 1.7, and ttrpc is still recommended.)

The naming convention also made runtimes pluggable:

  • containerd-shim-runc-v2 → uses runc (default)
  • containerd-shim-runsc-v2 → uses gVisor
  • containerd-shim-kata-v2 → uses kata

containerd picks the right binary based on the Pod's RuntimeClass. Adding a new runtime means putting a new binary on the Node.

References