One sh process, two PIDs. Inside the container it is PID 1 with two children and nothing else in the world; on the host it is PID 39023 among ~137 processes — and kill 39037 from outside removes a process the container cannot even name.
A container is not a box. There is no container object in the Linux kernel, no lightweight VM, nothing that "contains" anything. There is one ordinary process tree on one ordinary kernel, and a set of namespaces — per-process filters that change which entries of a kernel table that process is allowed to see. docker run is a clone() with a few extra flags.
This exercise makes that concrete. You'll take one sh process running in a container and look at it from two directions at once. Then you'll kill it from outside using the number it can't see, and finally build a PID namespace by hand with unshare — no Docker, no image, no daemon — to show there was never any magic.
namespaces(7), DESCRIPTION — the sentence the whole exercise tests:
A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource. Changes to the global resource are visible to other processes that are members of the namespace, but are invisible to other processes. One use of namespaces is to implement containers.
Note what that does not say: nothing is moved, copied, or fenced. An abstraction is applied to a global resource. Parts 1–3 are that sentence, run.
Same page, "The /proc/pid/ns/ directory" — the claim Part 2 checks:
If two processes are in the same namespace, then the device IDs and inode numbers of their/proc/pid/ns/xxxsymbolic links will be the same; an application can check this using thestat.st_devandstat.st_inofields returned bystat(2).
pid_namespaces(7), "Nesting PID namespaces" — the asymmetry Part 3 is about, stated by the kernel's own documentation:
A process is visible to other processes in its PID namespace, and to the processes in each direct ancestor PID namespace going back to the root PID namespace. […] Conversely, the processes in a child PID namespace can't see processes in the parent and further removed ancestor namespaces. More succinctly: a process can see (e.g., send signals withkill(2)[…]) only processes contained in its own PID namespace and in descendants of that namespace.
A process has one process ID in each of the layers of the PID namespace hierarchy in which is visible […]
(The "in which is visible" typo is upstream's, quoted as-is.)
proc_pid_status(5) — where NSpid is actually documented. Not in namespaces(7), which is where an earlier draft of this exercise cited it:
NStgid— Thread group ID (i.e., PID) in each of the PID namespaces of which pid is a member. The leftmost entry shows the value with respect to the PID namespace of the process that mounted this procfs (or the root namespace if mounted by the kernel), followed by the value in successively nested inner namespaces. (Since Linux 4.1.)NSpid— Thread ID in each of the PID namespaces of which pid is a member. The fields are ordered as forNStgid.
Docker CLI reference, "PID settings (--pid)" — what Session B does:
'host': use the host's PID namespace inside the container
By default, all containers have the PID namespace enabled. PID namespace provides separation of processes. The PID Namespace removes the view of the system processes, and allows process ids to be reused including PID 1.
Same page, "Escalate container privileges (--privileged)":
The--privilegedflag gives the following capabilities to a container: Enables all Linux kernel capabilities · Disables the default seccomp profile · Disables the default AppArmor profile · Disables the SELinux process label · Grants access to all host devices · Makes/sysread-write · Makes cgroups mounts read-write
In other words, the container can then do almost everything that the host can do.
The exercise's own prose covers namespaces from scratch. Two pieces of background carry the two surprises, and are worth having first:
/proc is a filesystem, not a kernel query API. ps does not ask the kernel "who exists" — it opens /proc and reads directory entries. Which /proc you read is decided by your mount namespace, and which entries appear there is decided by the reader's PID namespace, at the moment /proc was mounted. This is the one that carries steps 15 and 23, both of which look like bugs until you have it. (proc(5))CAP_SYS_ADMIN, so it owns a hostname it cannot set. Used in step 17 and again in step 24. (capabilities(7))user_namespaces(7))6.12.76-linuxkit, 16 GiB allocated to the VMalpine:3.20 — Alpine 3.20.10, digest sha256:d9e853e87e55526f6b2917df91a2115c36dd7c696a35be12163d44e6e2a4b6bcTwo notes on reading the transcripts below. The image tag is pinned deliberately: bare alpine is alpine:latest, a different digest (sha256:28bd5fe8… at time of capture), so every command here says alpine:3.20. And the timestamps inside the Linux VM read Jul 25 16:47 because the VM runs UTC while the Mac was on +08 — same instant, and the Captured date above is the Mac's.
docker ps showed only this exercise's own containers. The host process listings in Session B are therefore the real, complete list, not a filtered one. The one process you will see that this exercise did not start is Docker Desktop's own internal /portainer container — shim id 14fcb7e6…, which never appears in docker ps.
On Linux, "inside the container" and "the host" are the only two places you can stand. On macOS there are three, and knowing which one you're standing on is most of the exercise:
ls /proc/<pid>/ns/" is impossible here — /proc does not exist on macOS. The way in is a container that opts out of the PID namespace: docker run --privileged --pid=host. That's not a workaround for a missing feature; it is the same mechanism this exercise is about, used in reverse. A container is only isolated because of the namespaces it was given, so hand one back and the "container" is a shell on the host.
If you're on native Linux, everything below works identically — just use your own shell wherever the exercise says Session B, and drop the --pid=host container. The /proc paths, inode numbers, NSpid field and unshare behavior are all kernel features, not Docker ones.
ls -l /proc/<pid>/ns/ lists them. Each is a kernel object identified by an inode number; two processes are "in the same namespace" iff they show the same inode. That's the whole ontology.
| Namespace | What it filters | What breaks without it | Flag to disable |
|---|---|---|---|
| pid | The process ID table. Your first process becomes PID 1; you cannot see or signal anything outside. | ps aux in the container lists every host process, and kill reaches them. |
--pid=host, or --pid=container:NAME |
| mnt | The mount table. This is what makes the image's filesystem be /. |
The container sees the host's filesystem tree. | no flag; you'd have to nsenter |
| net | Interfaces, addresses, routes, iptables, socket port space. | Two containers can't both bind :80. | --network host |
| uts | hostname and domainname, nothing else. The oldest and smallest one (UTS = UNIX Time-sharing System). |
hostname foo in a container renames the host. |
--uts=host |
| ipc | SysV shared memory / semaphores / message queues, and POSIX message queues. | Two Postgres containers with the same shm key collide. | --ipc=host, --ipc=container:NAME |
| user | UID/GID mapping. Lets uid 0 inside be uid 100000 outside. | Root in the container is root on the host — which is the default. | on by default; see Part 6 |
| cgroup | The cgroup path you can see, so a container can't read its position in the host's hierarchy. | /proc/self/cgroup leaks the container ID and the host's layout. |
--cgroupns=host |
| time | CLOCK_MONOTONIC and CLOCK_BOOTTIME offsets. Newest (Linux 5.6), and the only one Docker gives you with no offset applied. |
Checkpoint/restore can't rewind a process's idea of uptime. | n/a |
A namespace is a filter, not a wall. It changes what a process can name. It does not move the process anywhere, does not stop the host from acting on it, and does not limit what it can consume — that last one is cgroups' job, a completely separate mechanism.
The isolation is per-namespace-type and individually revocable. There is no "container namespace". --pid=container:ns-a shares exactly one of the eight and leaves the other seven alone, which is precisely how a Kubernetes pod's sidecars work.
Three terminals. A · macOS is your Mac shell, B · Linux VM is a shell on the "host", C · container is a shell inside the container.
A · macOS — start the container under test. It runs a shell with two identifiable children, so there's a small process tree to look at:
docker run -d --name ns-a alpine:3.20 sh -c 'sleep 4242 & sleep 4243 & wait'B · Linux VM — --pid=host opts out of the PID namespace; --privileged keeps the kernel from redacting other processes' /proc entries:
docker run --rm -it --privileged --pid=host --cgroupns=host alpine:3.20 shC · container:
docker exec -it ns-a sh4026531836 and its neighbours in the 402653183x range are the kernel's initial namespaces, created at boot, and are the same on every Linux system.
What this part tests: whether "the container's processes" and "the host's processes" are two sets of processes or one. Per the mental model there is only ever one process tree, so the two views must be the same processes under different names.
ps -o pid,ppid,argsFour processes, and PID 1 is your sh. Note PPID 0 — PID 1's parent isn't merely hidden, it is unrepresentable, because it lives in a namespace this process has no way to name.
docker top ns-adocker top asks the daemon what's running in the container. Same list?The same three commands, with completely different PIDs — and PID 1's phantom parent has a number now, 38999. docker inspect -f '{{.State.Pid}}' ns-a returns 39023 to match. This is already the aha in miniature: 1 and 39023 are the same process, and neither number is more real than the other.
ls /proc/39023/ns/Not "permission denied" — macOS has no /proc at all, so the error is about the very first path component. The PID docker top just printed is not a PID in any table macOS maintains. That's what Session B is for.
What this part tests: the claim from the mental model, and from namespaces(7), that "same namespace" is literally "same inode number". If the container's sh and the VM's PID 39023 are one process, their namespace inodes must be identical — and both must differ from the VM's own.
ls -l /proc/1/ns/Ten entries, not eight. pid_for_children and time_for_children are not extra namespaces — they name the namespace this process's next child will get, which is how unshare(2) can put your children somewhere you aren't yet. Here they match pid and time, as they do in any quiescent process.
ls -l /proc/39023/ns/Byte-for-byte the same ten lines, timestamps included. The namespace inode is a global fact about the process, not a per-viewer one. It has to be: it's how the kernel answers "are these two processes in the same namespace?", and per namespaces(7) it's the documented way for userspace to ask the same question with stat(2).
ls -l /proc/1/ns/
cat /proc/1/comm4026531834–4026531841 is the kernel's initial set, allocated at boot; you'll see those exact numbers on any Linux box. Nine of the ten lines differ from the container's. The tenth — user:[4026531837] — is identical, and Part 6 is about why that one matters more than the other nine.
| The same process, asked two ways | C · inside ns-a | B · in the Linux VM |
|---|---|---|
| Its PID | 1 | 39023 |
| Its parent | 0 (unnameable) | 38999 (containerd-shim) |
| Processes in view | 4 | 137 |
NSpid in its own status | 1 | 39023 1 |
| pid namespace | pid:[4026532695] — identical from both sides | |
| user namespace | user:[4026531837] — and the VM's init has the same one | |
| Its uid | uid=0(root) | root, Uid: 0 0 0 0 |
/proc/self/cgroup | 0::/ | 0::/docker/6a353dfa121c… |
| Hostname | 6a353dfa121c | docker-desktop |
grep -E '^(Name|Pid|NSpid|Uid):' /proc/39023/statusNSpid: 39023 1 — one process, listed once per PID namespace it is visible in, outermost first (proc_pid_status(5): "the leftmost entry shows the value with respect to the PID namespace of the process that mounted this procfs … followed by the value in successively nested inner namespaces"). Not "the host PID and the mapped PID": the kernel genuinely holds both, because PID namespaces nest, and a process gets an ID in every ancestor namespace.
grep -E '^(Name|Pid|NSpid):' /proc/1/statusOne number, not two. The inner view is not redacting 39023; from where this process stands there is no outer layer to have an ID in. That is the whole of pid_namespaces(7)'s "processes in a child PID namespace can't see processes in the parent", visible in four bytes.
ps -o pid,args | grep '[c]ontainerd-shim'One shim per container, and the -id is the container ID — so you can read this list without guessing. 38999 is 39023's parent, and its id 6a353dfa121c… is ns-a's. 39047 is the B shell you are typing in; --pid=host did not exempt it from being an ordinary container. 490 is Docker Desktop's own internal /portainer container, which docker ps never shows you. "Ordinary process supervised by an ordinary parent" is the entire architecture; the shim is what lets dockerd be restarted without killing your containers.
What this part tests: the first bullet of the mental model. A namespace controls what a process can name. It does not put the process out of reach of anything outside.
kill 39023Not "permission denied" — no such process. In this namespace that integer names nothing, and the kernel's answer is a perfectly honest ESRCH.
sleep 4243, whose in-container name is PID 8:
kill 39037ps -o pid,args in C to check.PID 8 is gone, and B printed nothing at all — kill succeeded silently. The container has no way to name 39037 and no way to refuse it either. Isolation runs strictly outward: hierarchical, and one-way.
/proc/<pid>/root is the kernel's door into another process's mount view:
ls /proc/39023/root/
cat /proc/39023/root/etc/hostnameThat's the container's own root filesystem, read from outside with no docker exec involved — and 6a353dfa121c is the container ID, which Docker writes into /etc/hostname at start. This is how host-level tooling (backup agents, security scanners, docker cp) reaches into a container without entering it.
What this part tests: the second bullet of the mental model — that the eight namespaces are independent, and "join a container" is not an atomic thing you do.
docker run --rm --pid=container:ns-a alpine:3.20 ps -o pid,argsps list, and whose files does its ls see?A separate container, a separate image, its own writable layer — and it can see ns-a's process tree with ns-a's numbering. Check what it did not inherit:
docker run --rm --pid=container:ns-a alpine:3.20 sh -c \
'echo "pid ns: $(readlink /proc/self/ns/pid)"; \
echo "mnt ns: $(readlink /proc/self/ns/mnt)"; \
echo "net ns: $(readlink /proc/self/ns/net)"; \
echo "uts ns: $(readlink /proc/self/ns/uts)"; cat /etc/hostname'pid:[4026532695] is ns-a's, from step 4. The other three are fresh, and the hostname is its own container ID rather than ns-a's 6a353dfa121c. One namespace shared, seven fresh. This is exactly a Kubernetes pod: containers in a pod share the net (and optionally pid) namespace so they can reach each other on localhost and see each other's processes, while keeping separate filesystems.
nsenter is the setns(2) syscall with a CLI. Enter 39023's pid and mount namespaces:
nsenter -t 39023 -p -m ps -o pid,argsThat is docker exec, in one command, with no daemon in the loop.
nsenter -t 39023 -p ps -o pid,args | head -4
nsenter -t 39023 -p ps -o pid,args | wc -lps should still be filtered to the container, right?The VM's entire process list — 137 processes plus a header — from a process that really is in the container's PID namespace. ps doesn't ask the kernel who exists; it reads /proc, and /proc is a mount, still the VM's. The PID namespace changed which processes the kernel would report; nobody changed which /proc was being read. Remember this for Part 7.
What this part tests: that the remaining namespace types work the same way, and that "disabled" and "empty" are different states.
docker run -d --name ns-b --network none --hostname ns-b-box alpine:3.20 sleep 3600
docker exec ns-b ls -l /proc/self/ns/
docker exec ns-b ip -o addr show--network none. Does ns-b get no network namespace, or an empty one?Its own private network namespace — net:[4026532967], where ns-a is 4026532697 and the VM is 4026531840 — containing a loopback interface and nothing else. --network none doesn't remove the namespace, it declines to plug anything into it; every namespace exists in exactly one state, and "empty" is a normal one. user is 4026531837, the same as everyone.
mnt:[4026532962] and uts:[4026532963] and compare them to step 13. Those are the same integers the short-lived --pid=container:ns-a container had, one of them for a different namespace type — that container exited, its nsfs inodes were freed, and the kernel handed the numbers straight back out. Inode equality only means anything between two things alive at the same moment.
Compare ns-a, which has the same kind of namespace with a veth in it:
docker exec ns-a ip -o addr showSame lo, plus one interface. And the hostnames — ns-b-box vs ns-a's 6a353dfa121c — differ because of one namespace whose entire contents is a string.
docker run --rm alpine:3.20 sh -c 'hostname changed-me; hostname'
docker run --rm --cap-add SYS_ADMIN alpine:3.20 sh -c 'hostname changed-me; hostname'Namespaces decide which hostname you affect; capabilities decide whether you may set one at all. Docker drops CAP_SYS_ADMIN by default, so the answer is no — two independent mechanisms, and real container security is always both.
docker run --rm --network host alpine:3.20 sh -c \
'echo "net ns: $(readlink /proc/self/ns/net)"; ip -o addr show | head -4'4026531840 is the initial namespace from step 6 — not an empty namespace, the VM's own. Note what "host" means here on a Mac: 192.168.65.3 is the VM's address, not your Mac's. A third reminder that "host" is the middle box in the diagram.
docker exec ns-a cat /proc/self/cgroup # A
cat /proc/39023/cgroup # BThe container believes it's at the root of the hierarchy. Nothing about its limits changed; only the path it's allowed to see did. (Your B shell needs --cgroupns=host for this, or it gets its own relative view and prints 0::/../6a353dfa….)
What this part tests: the one line in step 6's output that was identical between the container and the VM's init. Seven namespaces were unshared for you; the security-critical one was not.
docker run -d --name ns-c --user 1000 alpine:3.20 sleep 5150
docker exec ns-a id
docker exec ns-c idps -o pid,user,args | grep -E '[s]leep 5150|[s]leep 4242'
grep '^Uid:' /proc/39023/status
grep -E '^(Pid|Uid|NSpid):' /proc/40102/statusNo translation whatsoever. The container's root is the host's root — the same uid 0, the same credential the kernel checks on every file access — and ns-c's 1000 inside is a literal 1000 outside. user:[4026531837] was the initial user namespace, and Docker left every container in it. Note that ns-c is also NSpid: 40102 1: the PID namespace is doing its job, it is only the user namespace that isn't there.
--privileged --pid=host in Session B handed you the whole VM — nothing had to be defeated. And why --user 1000 is the cheapest hardening step available: 1000 outside is a nobody. The real fix Linux offers is a user namespace with a UID map, so uid 0 inside is uid 100000 outside — Docker can do it (dockerd --userns-remap=default), Podman does it by default for rootless containers, and both pay for it in volume-permission pain, which is roughly why it isn't the default here. Docker's own docs list the price: with userns-remap on, --pid=host, --network=host and plain --privileged all stop working — which is to say, Session B of this exercise would be impossible.
What this part tests: the claim in the concept — that docker run is clone() with flags, and every namespace above is a kernel feature you can reach directly.
readlink /proc/self/ns/pid
ps -o pid,args | wc -lThe initial PID namespace, 140 processes plus a header. (It was 137 in step 15; ns-b and ns-c have since started.)
unshare --pid --fork --mount-proc sh -c 'readlink /proc/self/ns/pid; ps -o pid,args'docker anywhere. What does ps print inside it?A brand-new PID namespace and a world containing one process. That's the entire trick a container plays, in a busybox one-liner. Docker's remaining value-add is images, networking, and lifecycle management — not isolation, which is 30 characters of unshare.
unshare --pid --fork sh -c 'echo "pid ns: $(readlink /proc/self/ns/pid)"; echo "my pid: $$"; ps -o pid,args | wc -l; ps -o pid,args | head -3'/proc mount is different. What does ps show?PID 1 that can see 141 processes. Two different processes are both "PID 1" in that listing — the shell (correctly, in its new namespace) and initd (as read from a /proc mounted in the old one). The namespace is completely real; ps is reading a stale filesystem. --mount-proc is shorthand for "also unshare the mount namespace and mount a fresh /proc", which is why every container gets a mount namespace whether or not it wants a private filesystem. Step 15 was the same bug from the other direction.
hostname
unshare --uts sh -c 'hostname ns-by-hand; hostname'
hostnameA namespace whose whole content is one string, unshared and modified and discarded in one line, with the outer hostname untouched. (This works because B is --privileged and therefore holds CAP_SYS_ADMIN — in an ordinary container it fails, exactly as in step 17. 796a8bdda9ec is B's own container ID: even the shell that can see the whole host still has a private UTS namespace, because --pid=host hands back exactly one of the eight.)
One sh process that is PID 1 with two children when it looks at itself, and PID 39023 with a containerd-shim parent among ~137 processes when the VM looks at it — with NSpid: 39023 1 in /proc/39023/status and a bare NSpid: 1 in the container's own /proc/1/status, the same field answering differently depending on who reads it. Identical namespace inodes (pid:[4026532695], all ten lines, timestamps included) read from either side, all differing from the VM's initial pid:[4026531836] — except user:[4026531837], which matches. kill 39023 from inside failing with No such process, while kill 39037 from outside silently removes the container's PID 8. A second container joining exactly one of ns-a's eight namespaces and keeping its own hostname. And unshare --pid --fork --mount-proc producing a one-process world with no Docker involved — plus, without --mount-proc, a PID 1 that still lists 141 processes.
Because namespaces virtualize names, not resources. A PID namespace gives the kernel a second table in which to register your process, and /proc shows a process only the table entries reachable from its own. So the container's ps is not lying and the VM's ps is not privileged information — they're two indexes into one list of tasks, and NSpid is the row that holds both keys.
That design forces the asymmetry you saw in Part 3, and pid_namespaces(7) states it outright: a process is visible in its own namespace and every ancestor, and "the processes in a child PID namespace can't see processes in the parent". Naming therefore works outward-in only. Permission checks, meanwhile, are unaffected by namespaces at all — kill 39037 succeeded because the caller was uid 0 and the target was uid 0, exactly as it would have with no containers in the picture. Nothing consulted the boundary.
It also explains why isolation has to be assembled rather than switched on. Each namespace type is a separate kernel object with a separate CLONE_NEW* flag, and docker run sets seven of them. The eighth, user, it leaves alone — so the uid 0 in your container is the uid 0 that owns the VM. Every practical container-security control (--user, seccomp, dropped capabilities, read-only rootfs, userns-remap) exists to shore up that gap, because namespaces were built to make processes not see each other, which is a different goal from making them safe from each other.
--mount-proc surprise in Part 7 is the general case: the kernel state changed, and something that caches or re-reads the old view didn't. Same reason a process that had a file open before the mount namespace was unshared keeps reading it, and the same reason /proc/<pid>/root in Part 3 works at all.
apk add iproute2 first — busybox's ip has no netns subcommand): ip netns add ns-lab && ip netns exec ns-lab ip -o link show lists a namespace with nothing but lo and the tunnel stubs. ip netns is a thin wrapper over unshare --net plus a bind mount of /proc/self/ns/net onto /var/run/netns/ns-lab — which is how a namespace outlives every process in it.docker run --rm --uts=host alpine:3.20 hostname prints docker-desktop, the VM's hostname, against a random container ID (cb76412e42bd on the run captured here) for the default. The same flag exists for every namespace type; work through them and note which ones Docker will let you share (--uts, --ipc, --pid, --network, --cgroupns) and which it won't (mnt, user).--ipc=container:ns-a: apk add util-linux, create a SysV segment with ipcmk -M 1024 in one and list it with ipcs -m in the other, then repeat without the flag and watch it vanish. This is the namespace Postgres and Oracle containers actually care about.docker run --rm --privileged docker:dind, start a container inside it, and read NSpid for one of its processes from the B shell. NSpid carries one number per PID namespace the task is visible in, so nesting should lengthen the line to three numbers — the mechanism from steps 7 and 8 with the recursion made visible.man 7 namespaces, then man 7 pid_namespaces for the PID 1 semantics this exercise skipped: PID 1 gets no default signal handlers, so docker stop on a shell that ignores SIGTERM waits the full 10 seconds; and when PID 1 exits, the kernel kills the whole namespace. That's why tini / --init exists. For the NSpid/NStgid field definitions, the page you want is man 5 proc_pid_status, not namespaces(7).Sources: namespaces(7) — the authoritative list, and the /proc/pid/ns/ inode semantics · pid_namespaces(7) — nesting and one-way visibility · proc_pid_status(5) — where NSpid is actually defined · user_namespaces(7) · docker container run — --pid and --privileged · Docker: isolate containers with a user namespace
Only the containers this exercise created, by name — never prune:
docker rm -f ns-a ns-b ns-cEvery other container the exercise starts (--pid=container:ns-a, the --cap-add SYS_ADMIN and --network host one-liners, and Session B itself) was launched with --rm and removes itself. Then exit the B shell.