Notes on how things actually work, taken apart in the browser.
Part one established that 23andMe never read my DNA, just 643,161 multiple-choice answers. This part reads mine in public: five positions checked against my actual body, my ancestry painted segment by segment onto real chromosomes, and the letters they wrote down without ever measuring.
I paid for my genome and got back a 16 MB text file. The product everyone means by a DNA test does not sequence anything. What it sold me is 643,161 multiple-choice answers, and how those answers get made turns out to be the better story.
The two appendices most people skip contain the largest single training win anyone here measured and a fine-tuning method whose whole trick is initialising one matrix to zero. Then seven things a 2026 model does differently, what each one costs in parameters and cache bytes, and why none of them is where the gains came from.
The training loop from the pretraining posts runs again, byte for byte unchanged, on 935 templated strings. What changes is the string, where the padding goes, and a sentinel value of minus one hundred that removes a position from the denominator. Then the model answers, and the only number anyone can put on the answer is another model's opinion.
The same GPT-2 that cannot answer the question when you ask it in English gets 95.67% of a held-out set of 300 text messages right once you replace its output layer, unfreeze one transformer block, and read the last token instead of all of them. Here is why the last token, and what 95.67% hides.
The training loop is an ordinary PyTorch loop and it finishes in five minutes. Then the training loss dives to 0.391 while the validation loss stalls at 6.452 and the model starts reciting Edith Wharton. Here is what that gap means, what temperature and top-k do to the same frozen weights, and what fourteen rented training runs found when somebody finally re-measured the numbers the book prints without explaining.
Cross entropy for a language model is the log of one probability per position, averaged and negated. Here is that number built by hand out of the book's own six, why 10.7940 was predictable before the model ran, and the case where perplexity says 4 twice and means two completely different things.
Chapter 4 finishes GPT-2 with a layer norm, a GELU, a feed forward and two additions. The feed forward holds twice the parameters of attention, the two additions lift the smallest gradient in the network from 0.00012 to 0.207, and the finished 163,009,536-parameter model says Featureiman Byeswickattribute argue.
The same weighted average as last time, with three learned projections in front of it. Why the score matrix stops being symmetric, why the divisor is 1.4142, why the mask has to be minus infinity rather than zero, and why twelve attention heads are one matrix in stripes.
There is no container object in Linux. There's a normal process that has been fenced in, with its own PID 1, its own root, its own resource budget. We run the real syscalls and watch the fence go up.
Six tokens, three dimensions, and not one trainable weight. A dot product, a softmax and a weighted sum are the entire mechanism, and measuring what it actually produces on the book's own numbers is the argument for everything chapter 3 adds next.
A sliding window turns one 20,479-character short story into 1,286 training rows and nobody labels anything. Then an embedding layer turns each row into floats by doing nothing cleverer than reading a row off a table, and the post ends holding the exact tensor the next one is handed.
The Verdict is 20,479 characters. A regex tokenizer turns it into 4,690 tokens and 1,130 unique ones, and then crashes on the word Hello. This is the chain of repairs that ends at byte pair encoding, with the real GPT-2 tokenizer running live on whatever you type.
GPT-3 was trained on 300 billion tokens at an estimated $4.6 million, to answer one question, which token comes next. This is where the labels come from, why nobody had to write them, and what falls out of an objective that simple.
A docker pull is a handful of HTTP requests you can read with curl. A 401 that hands you the keys, a token, a fat manifest that picks your CPU, and blobs by digest. We read every one.
A container image has no magic in it. It's a few tar files, a JSON, and a rule for naming things by their content. We pull a real one from Docker Hub and read every piece.
A git clone is just two HTTP requests, and you can read every byte of them. Two repos live in this page; watch one clone the other.
The week-2 assignment points one sigmoid unit at 209 photographs and gets 70% of held-out cats right. This is what happens to an image on the way in, why nobody loops over examples, and what a weight looks like when you draw it back into a picture.
Six equations turn a stack of matrix multiplies into something that learns. Here they are, stepped through on real matrices, checked against the definition of a derivative, and then run live on the dataset that beat a single neuron.
Before backprop, before layers, before any of it: one unit, three numbers, and the reason its loss function looks the way it does. Every figure runs a real network in your browser.
Loose objects are a lovely database at rest and a terrible thing to ship. Packfiles put the whole database in one file, and deltas are why it fits.
A branch is a tiny text file, HEAD is a pointer to a pointer, and checkout runs the whole machine in reverse. Let's keep opening up .git.
Git isn't black magic, just a small database made of plain files. Let's open it up, with a real git running in your browser.
MetaMask never sends your transaction. It signs it. A signed Ethereum tx is an RLP list with r, s and one recovery bit, and the `from` address is recomputed from the signature, never transmitted. Built and signed in your browser with real secp256k1, against my own exam dApp's contracts.
In 2022 I wrote a token exchange for a university exam and passed. In 2026 I compiled it and found that sellToken pays out ETH but never pulls your tokens. Here is the drain: the compiled Exchange and its ERC-20 both deployed in the mini-EVM from part seven, the Exchange genuinely calling the token across every sale, over real solc 0.8.9 bytecode.
An ERC-20 is a contract with a mapping(address => uint256), and a balance is one storage slot at keccak256(holder . 5). "Transfer" subtracts from one slot and adds to another; no coin travels anywhere. Real solc bytecode, my own exam token, executed live in your browser.
Ethereum runs one 256-bit stack machine on every node at once, and it stays halted for the strangest reason: it charges rent on computation and stops when you run out. Real solc bytecode, executed opcode by opcode in your browser.
The Merkle Patricia Trie is git's Merkle tree with a keyboard. Every account and every storage slot hangs off one 32-byte root, and changing a single balance rewrites only its path up. Real keccak256, go-ethereum's own test vector reproduced live in your browser.
Bitcoin has no accounts, only unspent outputs. Ethereum threw that out and kept a single mutable table of accounts, each a four-field row. Real accounts, read from a node and re-hashed live in your browser.
A full node is one append-only log plus a couple of rebuildable key-value caches. Real blocks dissected byte by byte, and a payment verified against fifteen years of history with four hashes and an 80-byte header.
A Bloom filter stores no data and still answers questions about it: nine bits per item, no false negatives ever, and a false-positive rate you choose in advance. The formula derived and then measured against a real filter, why deletion is impossible, and why Ethereum's is nearly useless on a busy block.
The padlock from part one opens: a real miner in your browser, a fork that resolves itself, and the arithmetic thermostat that keeps blocks ten minutes apart.
There are no accounts and no balances, only unspent outputs, each locked by a tiny program you unlock with a signature. Real 2010 transactions, re-verified live in your browser.
Every post in this series leaned on one sentence, "your public key is your private key times a point," and then walked away from it. This is the walk back. Modular arithmetic, the group law, a curve over 97 elements you can see every point of, and the real secp256k1 doing the real job, all runnable here.
Take git's content-addressed chain, strip it down to an 80-byte header, and add one rule about leading zeros. Real blocks from 2009, re-hashed live in your browser.
Thirty years of failed digital cash, and every piece of Bitcoin already sitting on a shelf. Chaum's blind signatures, the 1991 chain of blocks, hashcash: all runnable here, plus the four things the whitepaper cites to nobody.
A Merkle tree is one idea applied recursively, and almost every diagram of it is of the broken version. Inclusion proofs, the odd-node bug that is consensus in Bitcoin forever, a second-preimage forgery you can watch succeed, and the append-only proof that keeps the web's certificates honest.
Your machine wakes up knowing one thing, its own MAC address. Four broadcasts later it has an IP, a router, and a lease with an expiry date. Every byte here is a real capture, read whole.
Before your browser can fetch anything, one UDP datagram has to turn a name into an address. It fits in 512 bytes, you can read every one of them, and this page sends one for real.
Every request you've ever sent is a few lines of text over a TCP socket. This page parses them live, and one figure fills itself with the request you just made by opening it.
A real compiler lives under this post. Edit a program and watch it become tokens, a tree, types and JVM bytecode, then step a small JVM through the result.