Following instructions is a string, not a new objective

Last time I cut the output head off GPT-2 and bolted a new one on. 768 in, two classes out instead of 50,257 tokens, everything frozen except the final block and the last LayerNorm, and 95.67% of 300 held-out text messages sorted correctly into spam and not-spam. That model does exactly one thing. It cannot answer a question, because it has no vocabulary to answer with any more.

This post puts the head back. Same 50,257 outputs, same cross entropy against the next token, same training loop, and at the end of it the model converts sentences to passive voice and names the author of Pride and Prejudice. Nothing about the architecture changes. Nothing about the loss changes. calc_loss_loader and train_model_simple are imported from the pretraining chapter and never touched. What changes is 1,100 records of JSON, the string they get flattened into, and a sentinel value of -100 that decides which positions of that string the loss is allowed to see.

Three of the six figures compute everything they show: the prompt builder runs the real GPT-2 byte-level BPE from the tokenizer post as soon as you click to fetch it, the collate stepper runs the book's own collate function, and the cross-entropy calculator does its arithmetic in your browser. A fourth puts a live token count beside a table of scores I transcribed, and labels the table. The last two are recordings and say so in their caption bars, because I am not running a 355-million-parameter model in a browser tab.

01 · The model never sees the JSON

The dataset is 1,100 instruction and response pairs written for the book, a 204 KB JSON file, each entry a dict with exactly three keys: instruction, input, output, and input frequently the empty string. format_input flattens one record into one flat string, and the response is not in it:

def format_input(entry):
    instruction_text = (
        f"Below is an instruction that describes a task. "
        f"Write a response that appropriately completes the request."
        f"\n\n### Instruction:\n{entry['instruction']}"
    )

    input_text = (
        f"\n\n### Input:\n{entry['input']}" if entry["input"] else ""
    )
    return instruction_text + input_text

The caller appends "\n\n### Response:\n" + entry['output'] separately. That split is not tidiness. It is what makes the same function usable at inference time, where there is no known output to append.

Empty the input field and watch the ### Input: block disappear, because that ternary is the only branch in the whole function:

The counter is the part worth reading. The Alpaca preamble is 18 GPT-2 tokens, paid on every one of the 1,100 formatted examples before the task arrives. The Phi-3 layout replaces it with chat tags a vocabulary built in 2019 has no special ids for: <|user|> costs five tokens, ids 27, 91, 7220, 91, 29. Raschka's word for that is "very inefficient", a cost rather than a failure, and the figure colours it that way. Two of the four styles are controls I invented. The useful one is none: with no delimiters there is nothing at inference to slice the answer off at.

Giles Thomas works out why Alpaca looks the way it does, which the book asserts and never explains. It is early 2023, downstream of Self-Instruct, when a chat format meant paying for a system prompt and every prior turn against a context of a few thousand tokens. The thinking was one-shot, not conversational.

The split, listing 7.3, is floor arithmetic: $\lfloor 1100 \times 0.85 \rfloor = 935$ train, $\lfloor 1100 \times 0.10 \rfloor = 110$ test, and 55 left over for validation. The array is never shuffled, and test comes before validation in it: data[:935], data[935:1045], data[1045:]. The model is never shown the keys, only the string they were poured into. Which means ### Response: is not a formatting convention. It is a token sequence the model learns to emit, and section 05 has to .replace("### Response:", "") to get the answer back out.

02 · Pad to the batch, not to the dataset

InstructionDataset.__getitem__ returns a plain Python list of ints. Not a tensor, not padded, not truncated. That leaves the batching entirely to the collate function, which is handed one batch at a time, and it is where the chapter earns "from scratch". The function arrives in three drafts, walked on the toy batch [0,1,2,3,4], [5,6], [7,8,9]. Draft 1 pads, draft 2 adds the shifted targets, and custom_collate_fn adds the -100 rewrite and optional truncation. The arithmetic is three lines:

$L_b = \max_{i \in b}\bigl(\operatorname{len}(\text{item}_i) + 1\bigr), \qquad \text{inputs} = \text{padded}[0 : L_b - 1], \qquad \text{targets} = \text{padded}[1 : L_b]$

Set all three rows to the same length, then read the targets, because the row that gets no mask at all is the rule and not the exception:

With equal lengths every row's targets hold exactly one 50256, indices.numel() is 1, and no row is masked at all. That surprises almost everyone. The +1 is the other thing worth stopping on: it is the slot that guarantees the target sequence still ends on the end-of-text token even for the longest item in the batch. Draft 1 computes it and immediately throws it away, which reads like a bug and is not one. Giles Thomas says that unexplained anticipatory code "made me start doubting my sanity for a little while". Drop it in the figure and the longest row's target ends on a real token, so for that row the model is never shown where to stop. The one pad token that survives is the only place the model learns to stop talking.

Because padding is per batch rather than per dataset, the shapes the training loader prints are 8x61, 8x76, 8x73, then after an ellipsis 8x74 and 8x69. Those five average 70.6 columns, so padding everything to the 1,024-token context would cost on the order of fifteen times the cells. That is an estimate from five printed shapes: per-example lengths for the 935 training entries were never published.

One thing the book leaves silent. The inputs keep their padding and no padding attention mask is added anywhere, and the reason is the causal mask this series built four posts ago. The one surviving end-of-text target is predicted from the last real token, and every position downstream of a pad already carries a -100. Nothing the model computes from a pad ever reaches the loss.

03 · Minus one hundred changes the denominator

Cross entropy over a batch is a mean, and ignore_index is a rule about what the mean divides by:

$\mathcal{L} = \frac{1}{|\mathcal{M}|}\sum_{i \in \mathcal{M}} -\log \frac{\exp(z_{i,t_i})}{\sum_{j=1}^{V}\exp(z_{i,j})}, \qquad \mathcal{M} = \{\, i : t_i \neq -100 \,\}$

Set the third target to -100, then switch the reduction to the version that counts it as zero, and watch the denominator rather than the answer:

The book prints three numbers and no intermediates. Position 1 has logits $(-1.0, 1.0)$ and target 0, so $p = 0.11920$ and $-\log p = 2.1269$. Position 2 has logits $(-0.5, 1.5)$ and target 1, so $p = 0.88080$ and $-\log p = 0.12693$. Sum 2.2539, mean over two positions 1.1269. Add a third position identical to the second and the mean over three is 0.7936, lower, because an easy position was averaged in and the model did not get better. Set that third target to -100 and the mean is 1.1269 again. loss_1 == loss_3 prints True.

Now the wrong model of what happened. If ignore_index merely zeroed the term while leaving $N = 3$, the answer would be $2.2539/3 = 0.7513$. Switch the reduction in the figure and that is exactly what you get. The ignored position leaves the numerator and the denominator together, which is the only way the first number comes back exactly. And -100 lives only in the target tensor. It is not a vocabulary entry, it could never be predicted, and feeding it to an embedding layer would fail.

The masking itself is four lines out of listing 7.5:

mask = targets == pad_token_id
indices = torch.nonzero(mask).squeeze()
if indices.numel() > 1:
    targets[indices[1:]] = ignore_index

The phrase "keep the first one" appears nowhere in the chapter. It appears as indices[1:]. The guard does a second job too: torch.nonzero(...).squeeze() on a single hit produces a zero-dimensional tensor, and slicing that would raise, so relaxing the condition to > 0 breaks the code. Giles Thomas is honest about the magic number, which "gives me a bit of an 'ick'", then works out why it is defensible: token indices start at 0, so any negative value is invalid as a class label, and -1 is spoken for by last-element indexing.

04 · What actually gets scored

Figure 7.13 is the only figure in the chapter that shows instruction masking, and it is drawn as two columns for a reason. On the left, the input text, tokenizing to 21106, 318, 281, 12064, 326, …, 13. On the right, the target text, the same sequence shifted with its prefix rewritten: -100, -100, -100, -100, -100, …, 13, 50256. The -100s live only on the right. The chapter discusses it for two paragraphs, declines to do it, cites Shi et al. (2024), "Instruction Tuning With Loss Over Instructions", whose finding is that not masking helps, and leaves the implementation as exercise 7.2, which readers regularly build believing it is the chapter's code.

Turn on instruction masking and read the counter under the target row rather than the colours, because that counter is the denominator:

On figure 7.13's own example the scored share of the target row drops by roughly three quarters, and the input row does not move. Masking does not remove data from the forward pass, it removes positions from the average.

Every number in the next paragraph is a mean of a local Llama 3's 0-to-100 ratings of 110 answers, from a single run, and section 06 is about how far that instrument moves between runs. The repository's exercise runs, read on 2026-06-14, give 48.87 for the Phi-3 template, 47.73 for instruction masking, 48.16 for the 52,002-entry Stanford Alpaca set, and 50.23 for LoRA. The baseline they are measured against is not one number: 50.32 in the printed book, 49.45 in Raschka's notebook, 51.75 in his standalone script pair. So 47.73 sits a few points below a baseline whose own published runs differ by 2.3, and Raschka's guidance is to treat any single judge score as plus or minus two points of noise. The exercise table is not evidence that masking hurts. It is evidence that a 110-example judge evaluation cannot resolve differences this small.

The loss curves under the two regimes share no denominator, so they cannot go on one axis. The judge scores can be compared, and they do not separate.

05 · The training loop is the boring part

The model changes size. CHOOSE_MODEL becomes "gpt2-medium (355M)": emb_dim 1024, n_layers 24, n_heads 16, against small's 768, 12 and 12, and a checkpoint of 1.42 GB. The book's reason is capacity: smaller models "lack the necessary capacity to learn and retain the intricate patterns and nuanced behaviors required".

First the untuned model is asked to convert "The chef cooks the meal every day." to passive voice. It repeats the sentence back and starts hallucinating a fresh ### Instruction: header. That failure is not an accident of prompting. Giles Thomas spent a post trying to coax GPT-2 into chatbot behaviour with no fine-tuning at all, by framing the prompt as a transcript, and all four sizes fail non-monotonically: small emits empty turns, medium is tautological, large emits ಠ_ಠ, xl emits ????.

Then the loop. calc_loss_loader and train_model_simple come in from chapter 5 unmodified. AdamW at lr=0.00005 with weight_decay=0.1, two epochs, eval_freq=5, seed 123. Five batches measured before the loop give train 3.8259 and val 3.7619.

Ep 1 (Step 000000): Train loss 2.637, Val loss 2.626← logged after the first update, not before it
Ep 1 (Step 000005): Train loss 1.174, Val loss 1.103
Ep 1 (Step 000010): Train loss 0.872, Val loss 0.944
Ep 1 (Step 000015): Train loss 0.857, Val loss 0.906
...
Ep 1 (Step 000115): Train loss 0.520, Val loss 0.665
Ep 2 (Step 000120): Train loss 0.438, Val loss 0.670← epoch 2 began at step 116; eval_freq is 5
Ep 2 (Step 000125): Train loss 0.453, Val loss 0.685
Ep 2 (Step 000130): Train loss 0.448, Val loss 0.681
Ep 2 (Step 000135): Train loss 0.408, Val loss 0.677
...
Ep 2 (Step 000230): Train loss 0.300, Val loss 0.657← 37 more lines were logged; the book prints ten
Training completed in 0.87 minutes.

Scrub the run, and at each epoch boundary open the card to read what the eval sampler printed:

After epoch 1 the model says "The meal is prepared every day by the chef." After epoch 2 it says "The meal is cooked every day by the chef.", which now matches the source verb. Across epoch 2 the printed training values bounce, 0.438, 0.453, 0.448, 0.408, while validation goes 0.670, 0.685, 0.681, 0.677 and lands at 0.657. Giles Thomas ran five epochs by accident and watched validation rise steadily after the second, which is presumably why the book stops at two. Every difficult decision in this chapter was made before the optimizer ran.

Two code paths produce text here. The epoch samples come from generate_and_print_sample, which calls generate_text_simple with no eos_id at all and prints with the newlines flattened to spaces. That is the mechanical reason both run past <|endoftext|> into a hallucinated new instruction, and the book does not comment on it. Listing 7.9 uses generate with eos_id=50256, and what comes back is still not an answer: it is the prompt with a continuation glued on. Three string operations peel it. Slice off generated_text[len(input_text):], delete the literal ### Response:, then .strip(). The middle one exists because ### Response: is a token sequence the model learned to produce.

06 · Fifty point three two is not a percentage

The classifier chapter could print "percentage of messages correctly classified". Here there is nothing to divide. The chapter names the three families used in practice, multiple-choice benchmarks in the MMLU style, preference arenas, and LLM-as-a-judge, and takes the third: hand-rating 110 responses does not scale.

The judge is Ollama running an instruction-fine-tuned Llama 3 8B, a 4.7 GB download wanting roughly 16 GB of RAM, driven over http://localhost:11434/api/chat at temperature 0. The scoring prompt has one nice property: the version that averages differs from the version that explains by one appended sentence, "Respond with the integer number only." All 110 responses parsed as integers, which Giles Thomas points out was not a given at the time.

Score the three answers yourself first, then reveal what Llama 3 said about them:

85 for the bullet simile, 40 for cumulus instead of cumulonimbus, 95 for Jane Austen docked only for wordiness. The third verdict opens "I'd rate my own response as 95 out of 100": the judge has partly confused itself with the model it is scoring. Over all 110 answers the average is 50.32. 50.32 is not a percentage of anything, it is a mean of opinions, and the scale has no zero anyone agreed on. Llama 3 8B base, never instruction-tuned at all, scores 58.51 on the same scale, and Llama 3 8B instruct scores 82.6. A reader who takes 50.32 as "half right" concludes the fine-tuning failed.

Then the reruns. Raschka's notebook reports 49.45 and his standalone script pair 51.75, one recipe run by two scripts rather than one script run twice. Giles Thomas got 48.95 on his own machine. He also caught the judge scoring a model 40 for an answer that never contained the word its justification credited it with, because the word came from the reference. A judge that reads the reference and the answer in one context can lose track of which one it is scoring.

His fix is the methodological point. Score every model's answer to a given item in one call, with the model order shuffled per query, so that between-model noise cancels even though between-item noise does not. On that batched harness, judged by GPT-5.1, GPT-2 medium lands at 39.64 and GPT-2 small at 16.66. Three months later he re-ran it under four protocols with a different judge, and was explicit that absolute scores do not carry between judges. What held was the top two and one anomaly ranked 13th, 13th, 11-equal and 13th. The middle moved by up to five places.

07 · What's next

What I would add, in rough order of usefulness. Preference tuning. Section 7.9.1 names it as the deliberate omission and points at the repository's 04_preference-tuning-with-dpo folder. Leaving it out is the thing here most likely to leave a wrong picture, because you can finish the chapter believing supervised fine-tuning is the end of post-training rather than its first step. Parameter-efficient fine-tuning, exercise 7.4 and appendix E: LoRA at rank 16 trains 7,898,384 of 406,286,336 parameters, scores 50.23 against a baseline published between 49.45 and 51.75, and runs about 28% faster. That, and the two appendices this book still has left, are the next post. A better instrument. Raschka's October 2025 write-up on evaluation replaces the 0-to-100 prompt with a 1-to-5 rubric ending in "Evaluation: ", and puts LLM-as-a-judge alongside benchmarks, verifiers and arenas. His reason judges work at all: evaluating an answer is usually easier than producing one.

The figures run on src/lib/minigpt, dependency-free TypeScript with seeded randomness, every function able to hand back a trace so a figure can single-step it. The collate function in section 02 is the book's, reimplemented, and its verification chip is a live deep comparison against the tensors listing 7.5 prints rather than a stored answer. The one thing this post could not run is the 355M model, which is why the run and the judge are recordings: 355 million parameters is 710 MB at half precision, and the honest cost of that is two figures out of six where you are reading my transcription instead of watching arithmetic happen.