What docker pull actually says

In part one we took an image apart on disk: layer tarballs and a config, each named by the sha256 of its bytes. That left one question hanging. How do those bytes get from Docker Hub to your machine? The honest answer is almost boring: a few HTTP requests, the kind you can send with curl. No special protocol, no persistent socket, no binary handshake. A GET here, a token there, a couple of blob downloads.

This post reads the whole conversation. I captured a real pull of busybox:1.38.0-glibc off Docker Hub and everything below is those exact bytes. It rhymes on purpose with part four of the git series, which read every byte of a git clone. Same spirit: nothing here is a mockup, and every digest the registry names is one you can recompute.

01 · A 401 that hands you the keys

The first surprise is that even a public image makes you authenticate. Ask for the manifest with no credentials and the registry says no. But the way it says no is the whole trick. It returns 401 Unauthorized with a WWW-Authenticate header, and that header isn't a slammed door. It's a set of directions: here is the token server (the realm), here is the service this token is for, and here is the exact scope you're allowed to request.

Hover the three directives below and watch each one turn into a piece of the request the client sends next:

This is the registry token flow, and it's clever because the server tells the client precisely how to satisfy it. There's no guessing which auth endpoint to hit or what to ask for. The 401 is a form the client fills out. For a public repo like busybox, the answer is an anonymous pull token, no username involved.

02 · The token, then everything else

The client GETs that token URL and gets back a small JSON with a token field: a JWT that lives for five minutes. From that point on, every request to the registry carries it in an Authorization: Bearer header. Five minutes is nothing, and it doesn't need to be more: pulling one small image takes seconds.

Now the client asks for the manifest again, same URL as the rejected request, but this time with the token attached and one more header that changes everything: Accept. That header lists the media types the client understands, and it's how the client and registry negotiate what shape the answer takes. The distribution spec builds the whole manifest API on this content negotiation.

03 · The fat manifest

Here's where it gets interesting. The tag 1.38.0-glibc doesn't resolve to an image. It resolves to a list of images, one per CPU architecture. This is the image index, also called the fat manifest, and it exists because busybox runs on your laptop, your Raspberry Pi, and a mainframe, and each needs different machine code.

The client reads the index, finds the entry whose os and architecture match the machine it's running on, and fetches only that one, by digest. My capture was on an arm64 Mac, so it picked linux/arm64. Everything after this point is pinned: the client no longer asks for a tag, it asks for a specific digest, and a digest can only ever return one exact document. Tags drift when maintainers push new builds; digests are frozen. This is exactly why I pinned every figure in this series to digests, never tags.

There's a detail worth naming: the index also lists entries with os and architecture unknown. Those are attestation manifests, build-provenance and SBOM data that modern Docker attaches. The client ignores them when picking a platform, and so do we.

04 · Read the whole pull, byte by byte

Everything is on the table now, so let's watch the entire pull happen. Press play, or step through with the arrow keys once the panel has focus. The left strip fills in as the client gains each piece: token, index, platform choice, config, layer. Click any exchange to inspect its real headers and body. And for the index, the manifest, and the config, the panel hashes the bytes right there in your browser and checks the result against the digest the registry claimed:

Things worth watching for:

• The Docker-Content-Digest header on the index and manifest responses names the very document it's attached to. The green check under the body is your browser confirming it: sha256 of these bytes equals that digest. The registry can't lie about what it sent.
• When the client fetches a blob, the registry answers 307 Temporary Redirect with a Location pointing at CloudFront. The registry doesn't serve blob bytes itself; it signs a short-lived CDN URL and the client follows it. The spec explicitly permits this so registries can offload storage.
• The config's rootfs.diff_ids holds the layer's uncompressed hash, a different number from the layer digest the manifest lists. Two names for one layer: the digest addresses the compressed bytes on the wire, the diff_id addresses the tar you get after gunzip. Same layer, hashed at two stages.
• The last blob is the layer itself, 1.9 MB of gzipped tar, the exact filesystem you dissected in part one.

That's the entire pull: one 401, one token request, one index, one manifest, two blobs. Six round trips and you have every byte of an image. Unpack the layers over each other, the way we did in part one, and it's sitting on your disk.

05 · py-docker, the 240-line version

I have a toy that does exactly this. py-docker is about 240 lines of Python that pull an image and run a command in it, and its pull path is the conversation you just read: get a token, fetch the manifest, follow the index to the right platform, download each layer blob by digest, untar. Reading it next to this figure is a good way to convince yourself there's no hidden machinery. The registry side of Docker really is this small.

What py-docker does after the pull, running a command isolated from the host, is the harder half, and it can't happen in a browser because it's made of Linux syscalls. That's part three: namespaces, chroot versus pivot_root, cgroups, and the overlay filesystem that stacks these layers the way we stacked tars by hand. Every claim in it is a command I ran inside a real Linux container, captured and verified.