12f8cd6HEAD → main·2w ago·9 min read
Everyone’s software
AI is about to make software that fits one person, running on a computer in every home. An essay on why I want to build it at Vercel.
#essayeveryones-software.md →
BUILD978e937DEPLOYEDGEOK
Thoughts
Notes on how things actually work, taken apart in the browser.
thoughts.git·branch main·42 commits·last fetched 3m agoSYNCED
6608860·2mo ago·21 min read
what the file says about me
Part one established that 23andMe never read my DNA, just 643,161 multiple-choice answers. This part reads mine in public: five positions checked against my actual body, my ancestry painted segment by segment onto real chromosomes, and the letters they wrote down without ever measuring.
#dna#biology#ancestrywhat-the-file-says.md →
6b3237c·2mo ago·11 min read
23andMe never read my DNA
I paid for my genome and got back a 16 MB text file. The product everyone means by a DNA test does not sequence anything. What it sold me is 643,161 multiple-choice answers, and how those answers get made turns out to be the better story.
#dna#biology#data23andme-never-read-my-dna.md →
682b879·3mo ago·14 min read
Two appendices, and the seven things nobody does any more
The two appendices most people skip contain the largest single training win anyone here measured and a fine-tuning method whose whole trick is initialising one matrix to zero. Then seven things a 2026 model does differently, what each one costs in parameters and cache bytes, and why none of them is where the gains came from.
#machine-learning#neural-networks#math#llmwhat-changed-since.md →
35d01a8·4mo ago·14 min read
Following instructions is a string, not a new objective
The training loop from the pretraining posts runs again, byte for byte unchanged, on 935 templated strings. What changes is the string, where the padding goes, and a sentinel value of minus one hundred that removes a position from the denominator. Then the model answers, and the only number anyone can put on the answer is another model's opinion.
#machine-learning#neural-networks#math#llmfinetuning-to-follow-instructions.md →
2e29cf2·6mo ago·13 min read
Fine-tuning is one new layer and 5.7% of the weights
The same GPT-2 that cannot answer the question when you ask it in English gets 95.67% of a held-out set of 300 text messages right once you replace its output layer, unfreeze one transformer block, and read the last token instead of all of them. Here is why the last token, and what 95.67% hides.
#machine-learning#neural-networks#llmfinetuning-a-classifier.md →
28d036d·7mo ago·rev Apr 2026·14 min read
Eight steps to train it, fourteen runs to tune it
The training loop is an ordinary PyTorch loop and it finishes in five minutes. Then the training loss dives to 0.391 while the validation loss stalls at 6.452 and the model starts reciting Edith Wharton. Here is what that gap means, what temperature and top-k do to the same frozen weights, and what fourteen rented training runs found when somebody finally re-measured the numbers the book prints without explaining.
#machine-learning#neural-networks#math#llmtraining-and-sampling.md →
73560ef·8mo ago·13 min read
The loss reads six numbers out of 301,542
Cross entropy for a language model is the log of one probability per position, averaged and negated. Here is that number built by hand out of the book's own six, why 10.7940 was predictable before the model ran, and the case where perplexity says 4 twice and means two completely different things.
#machine-learning#neural-networks#math#llmwhat-loss-measures.md →
38c63ee·10mo ago·14 min read
The rest of the model is not plumbing
Chapter 4 finishes GPT-2 with a layer norm, a GELU, a feed forward and two additions. The feed forward holds twice the parameters of attention, the two additions lift the smallest gradient in the network from 0.00012 to 0.207, and the finished 163,009,536-parameter model says Featureiman Byeswickattribute argue.
#machine-learning#neural-networks#math#llmthe-gpt-architecture.md →
5eddbf4·11mo ago·14 min read
Three matrices, and a triangle of minus infinity
The same weighted average as last time, with three learned projections in front of it. Why the score matrix stops being symmetric, why the divisor is 1.4142, why the mask has to be minus infinity rather than zero, and why twelve attention heads are one matrix in stripes.
#machine-learning#neural-networks#math#llmattention-with-weights.md →
7cf7ee8·1y ago·13 min read
A container is a lie the kernel tells a process
There is no container object in Linux. There's a normal process that has been fenced in, with its own PID 1, its own root, its own resource budget. We run the real syscalls and watch the fence go up.
#containers#linuxcontainer-runtime.md →
36c631b·1y ago·14 min read
Attention with no parameters at all
Six tokens, three dimensions, and not one trainable weight. A dot product, a softmax and a weighted sum are the entire mechanism, and measuring what it actually produces on the book's own numbers is the argument for everything chapter 3 adds next.
#machine-learning#neural-networks#llmattention-without-weights.md →
6a40c58·1y ago·14 min read
The training labels were already in the file
A sliding window turns one 20,479-character short story into 1,286 training rows and nobody labels anything. Then an embedding layer turns each row into floats by doing nothing cleverer than reading a row off a table, and the post ends holding the exact tensor the next one is handed.
#machine-learning#llm#neural-networks#datatokens-into-vectors.md →
52f2990·1y ago·14 min read
A model has never seen a word
The Verdict is 20,479 characters. A regex tokenizer turns it into 4,690 tokens and 1,130 unique ones, and then crashes on the word Hello. This is the chain of repairs that ends at byte pair encoding, with the real GPT-2 tokenizer running live on whatever you type.
#machine-learning#neural-networks#llmtext-into-tokens.md →
4058f1c·1y ago·12 min read
Next-word prediction is the whole objective
GPT-3 was trained on 300 billion tokens at an estimated $4.6 million, to answer one question, which token comes next. This is where the labels come from, why nobody had to write them, and what falls out of an objective that simple.
#machine-learning#llm#neural-networks#historywhat-an-llm-actually-is.md →
38e617f·1y ago·13 min read
What docker pull actually says
A docker pull is a handful of HTTP requests you can read with curl. A 401 that hands you the keys, a token, a fat manifest that picks your CPU, and blobs by digest. We read every one.
#containers#httpcontainer-registry.md →
cfbbd44·1y ago·14 min read
An image is just tarballs and a digest
A container image has no magic in it. It's a few tar files, a JSON, and a rule for naming things by their content. We pull a real one from Docker Hub and read every piece.
#containers#storagecontainer-images.md →
30d90e7·2y ago·12 min read
What clone actually says
A git clone is just two HTTP requests, and you can read every byte of them. Two repos live in this page; watch one clone the other.
#git#httpgit-over-the-wire.md →
6d21027·2y ago·10 min read
A photograph is 12,288 numbers in a row
The week-2 assignment points one sigmoid unit at 209 photographs and gets 70% of held-out cats right. This is what happens to an image on the way in, why nobody loops over examples, and what a weight looks like when you draw it back into a picture.
#machine-learning#neural-networks#mathneural-pixels.md →
2fe8a51·2y ago·11 min read
Backprop is the chain rule with bookkeeping
Six equations turn a stack of matrix multiplies into something that learns. Here they are, stepped through on real matrices, checked against the definition of a derivative, and then run live on the dataset that beat a single neuron.
#machine-learning#neural-networks#mathneural-backprop.md →
3c95f27·2y ago·9 min read
A neuron is a line and a squash
Before backprop, before layers, before any of it: one unit, three numbers, and the reason its loss function looks the way it does. Every figure runs a real network in your browser.
#machine-learning#neural-networks#mathneural-one-neuron.md →
d30ec3d·2y ago·14 min read
Packfiles: how git ships its database
Loose objects are a lovely database at rest and a terrible thing to ship. Packfiles put the whole database in one file, and deltas are why it fits.
#git#storagegit-packfiles.md →
50c7309·3y ago·12 min read
Branches are 41 bytes
A branch is a tiny text file, HEAD is a pointer to a pointer, and checkout runs the whole machine in reverse. Let's keep opening up .git.
#gitgit-references.md →
5ae89ae·3y ago·15 min read
What happens inside .git?
Git isn't black magic, just a small database made of plain files. Let's open it up, with a real git running in your browser.
#git#storagegit-internals.md →
7844f02·3y ago·rev Jul 2026·14 min read
A transaction has no sender
MetaMask never sends your transaction. It signs it. A signed Ethereum tx is an RLP list with r, s and one recovery bit, and the `from` address is recomputed from the signature, never transmitted. Built and signed in your browser with real secp256k1, against my own exam dApp's contracts.
#ethereum#cryptographyethereum-jsonrpc.md →
57be357·4y ago·rev Jul 2026·14 min read
My exam-grade token exchange hands you its ETH for free
In 2022 I wrote a token exchange for a university exam and passed. In 2026 I compiled it and found that sellToken pays out ETH but never pulls your tokens. Here is the drain: the compiled Exchange and its ERC-20 both deployed in the mini-EVM from part seven, the Exchange genuinely calling the token across every sale, over real solc 0.8.9 bytecode.
#ethereum#solidity#securityethereum-audit.md →
7fedd3d·4y ago·rev Jul 2026·14 min read
Sending a token moves nothing
An ERC-20 is a contract with a mapping(address => uint256), and a balance is one storage slot at keccak256(holder . 5). "Transfer" subtracts from one slot and adds to another; no coin travels anywhere. Real solc bytecode, my own exam token, executed live in your browser.
#ethereum#solidityethereum-tokens.md →
2c655b0·4y ago·rev Jul 2026·16 min read
The world computer is a stack machine
Ethereum runs one 256-bit stack machine on every node at once, and it stays halted for the strangest reason: it charges rent on computation and stops when you run out. Real solc bytecode, executed opcode by opcode in your browser.
#ethereum#virtual-machinesethereum-evm.md →
604cc83·4y ago·rev Jul 2026·14 min read
Ethereum's entire state fits in 32 bytes
The Merkle Patricia Trie is git's Merkle tree with a keyboard. Every account and every storage slot hangs off one 32-byte root, and changing a single balance rewrites only its path up. Real keccak256, go-ethereum's own test vector reproduced live in your browser.
#ethereum#cryptographyethereum-trie.md →
56c10cc·4y ago·rev Jul 2026·13 min read
Ethereum doesn't have coins, it has a database
Bitcoin has no accounts, only unspent outputs. Ethereum threw that out and kept a single mutable table of accounts, each a four-field row. Real accounts, read from a node and re-hashed live in your browser.
#ethereum#storageethereum-state.md →
5ffe04c·4y ago·rev Jul 2026·14 min read
The blockchain is a log file, everything else is a cache
A full node is one append-only log plus a couple of rebuildable key-value caches. Real blocks dissected byte by byte, and a payment verified against fifteen years of history with four hashes and an 80-byte header.
#bitcoin#storagebitcoin-storage.md →
38e49a3·5y ago·rev Jul 2026·18 min read
The data structure that answers "probably not"
A Bloom filter stores no data and still answers questions about it: nine bits per item, no false negatives ever, and a false-positive rate you choose in advance. The formula derived and then measured against a real filter, why deletion is impossible, and why Ethereum's is nearly useless on a busy block.
#data-structures#databases#bitcoinbloom-filters.md →
2612617·5y ago·rev Jul 2026·14 min read
Mining is guessing a number, and the guessing is the security model
The padlock from part one opens: a real miner in your browser, a fork that resolves itself, and the arithmetic thermostat that keeps blocks ten minutes apart.
#bitcoin#consensusbitcoin-consensus.md →
afe5280·5y ago·rev Jul 2026·14 min read
Nobody has a bitcoin balance
There are no accounts and no balances, only unspent outputs, each locked by a tiny program you unlock with a signature. Real 2010 transactions, re-verified live in your browser.
#bitcoin#cryptographybitcoin-ownership.md →
51cf6d0·5y ago·rev Jul 2026·22 min read
You can multiply on this curve, but you cannot divide
Every post in this series leaned on one sentence, "your public key is your private key times a point," and then walked away from it. This is the walk back. Modular arithmetic, the group law, a curve over 97 elements you can see every point of, and the real secp256k1 doing the real job, all runnable here.
#cryptography#bitcoin#mathematicselliptic-curves.md →
586ff95·5y ago·rev Jul 2026·13 min read
A blockchain is a git repo with a hard problem
Take git's content-addressed chain, strip it down to an 80-byte header, and add one rule about leading zeros. Real blocks from 2009, re-hashed live in your browser.
#bitcoin#cryptographybitcoin-chain.md →
2cac11a·5y ago·rev Jul 2026·18 min read
Bitcoin has no new parts
Thirty years of failed digital cash, and every piece of Bitcoin already sitting on a shelf. Chaum's blind signatures, the 1991 chain of blocks, hashcash: all runnable here, plus the four things the whitepaper cites to nobody.
#cryptography#bitcoin#historyblockchain-prehistory.md →
349ce7e·5y ago·rev Jul 2026·20 min read
One hash that vouches for a million things
A Merkle tree is one idea applied recursively, and almost every diagram of it is of the broken version. Inclusion proofs, the odd-node bug that is consensus in Bitcoin forever, a second-preimage forgery you can watch succeed, and the append-only proof that keeps the web's certificates honest.
#data-structures#cryptography#gitmerkle-trees.md →
66599ba·6y ago·rev Jul 2026·12 min read
The packet before the first packet
Your machine wakes up knowing one thing, its own MAC address. Four broadcasts later it has an IP, a router, and a lease with an expiry date. Every byte here is a real capture, read whole.
#networking#udpdhcp-over-the-wire.md →
4cd95f9·6y ago·rev Jul 2026·13 min read
The first packet of everything
Before your browser can fetch anything, one UDP datagram has to turn a name into an address. It fits in 512 bytes, you can read every one of them, and this page sends one for real.
#networking#udpdns-over-the-wire.md →
7ae0d7e·6y ago·rev Jul 2026·18 min read
HTTP is just text
Every request you've ever sent is a few lines of text over a TCP socket. This page parses them live, and one figure fills itself with the request you just made by opening it.
#networking#httphttp-by-hand.md →
5c59d9e·7y ago·18 min read
Watch a compiler think
A real compiler lives under this post. Edit a program and watch it become tokens, a tree, types and JVM bytecode, then step a small JVM through the result.
#compilers#jvm#haskellcompiling-to-bytecode.md →
© 2026 · v2.0 · santiago toscanini