Chapter 0 of 0–21

The whole picture, first

What this site explains, the three mathematical objects at its heart, and how to read the rest.

Sources for this chapter

[1] abstract, Sec. 3
[3] intro
[10]

A small neural network was trained to answer one kind of question: what is a + b, if you count on a clock with 113 hours? For thousands of training steps it seemed only to have memorised the answers it was shown, failing on most others. Then — long after it had stopped improving on its training questions — it suddenly started answering every question correctly. Researchers reverse-engineered what it had learned: it turns numbers into angles and adds them by rotating. In the words of the paper that did this [1], the network's algorithm

“uses discrete Fourier transforms and trigonometric identities to convert addition to rotation about a circle”— Nanda et al. 2023, abstract, p. 1 · source

Here is that story in one view: the paper's own network, from its first training step to just after it learns the task. It plays while it is on screen; pause it or drag its slider to look at one moment, and watch the four views change together.

Grokking in one view

One moment of training seen four ways: the losses, the probability of the right answer on all 12,769 inputs, the numbers on one frequency's plane, and one neuron's activation. ▶ plays the run through grokking.

Loading…

How to read it

<div class="fig-read"><p>All four tiles show the same training step (the dotted line in the first tile). <strong>Tile 2</strong>: each pixel is one

question a + b (a = row, b = column); yellow means the network gives the right answer probability 1. Choose training pairs or test pairs to see the two groups separately. Tile 3: each dot is a number n, placed by projecting its embedding on the plane of one frequency's cosine and sine waves and coloured by n's position on that frequency's clock (k·n mod 113); a circle with the colours in order means the network stores n as an angle. The hands point at a, b and a + b. Tile 4: the selected neuron's activation on every input, recomputed in your browser from the nearest stored weights. While playing, tiles 2 and 3 blend the two stored frames around the current step (stored every 200 steps for the paper's run); paused, every tile shows a stored frame and its note says which.

The objects this site explains

Here they are, all at once. You are not expected to understand them yet — by the last chapter you will understand every symbol. Each symbol has a card: hover it (or read the table below).

1. What the network computes. For an input question (a, b), with t₀, t₁, t₂ standing for the three input tokens a, b and “=”:

x^{(0)}_i = W_E\, t_i + W_{pos}[i] \qquad (i = 0, 1, 2)

A^j = \softmax\!\Big(\tfrac{1}{\sqrt{32}}\, \big(W_Q^j x^{(0)}_2\big)^{\!\top} W_K^j\, x^{(0)}_i \Big)_{i=0,1,2} \qquad (j = 0,1,2,3)

x^{(1)} = x^{(0)}_2 + \sum_{j} W_O^j W_V^j \sum_{i} A^j_i\, x^{(0)}_i

\mathrm{MLP}(a,b) = \ReLU\big(W_{in}\, x^{(1)} + b_{in}\big), \qquad x^{(2)} = x^{(1)} + W_{out}\,\mathrm{MLP}(a,b) + b_{out}

\Logits(a,b) = W_U\, x^{(2)}, \qquad \text{answer} = \argmax_c \Logits(a,b)_c

The same computation as a picture, with the real network's numbers for one question. Each column is one stage; click a stage for its formula (chapter 5 explains every one).

One forward pass, stage by stage

The selected input flowing through the network at the selected step (embeddings, attention, residual stream, neurons, logits and the answer) with the real numbers.

Loading…

How to read it

<p class="fig-expect"><strong>What to expect:</strong> After training the correct answer's logit stands out and the prediction is right with probability close to 1.</p>
<p class="fig-meta">
  <a href="../chapters/05-the-model.html">Learn step by step →</a>
   · Source: <a class="cite" href="../references.html#nanda2023" title="Nanda et al. 2023">[1]</a> App. A
  · Computed by <code>web/js/model/grokmodel.js</code> (<a href="https://github.com/BurnyCoder/visualizing-grokking-modular-addition">code</a>)
</p>

2. What those numbers turn out to mean. For the trained network, to a very good approximation:

\Logits(a,b)_c \;\approx\; \beta + \sum_{k \in \mathcal K} \alpha_k \cos\!\big(w_k\,(a+b-c)\big), \qquad w_k = \frac{2\pi k}{p}

where each term is built from the inputs with two angle-addition identities,

\cos(w_k(a{+}b{-}c)) = \cos(w_k(a{+}b))\cos(w_k c) + \sin(w_k(a{+}b))\sin(w_k c),

\begin{aligned}\cos(w_k(a{+}b)) &= \cos(w_k a)\cos(w_k b) - \sin(w_k a)\sin(w_k b),\\ \sin(w_k(a{+}b)) &= \sin(w_k a)\cos(w_k b) + \cos(w_k a)\sin(w_k b),\end{aligned}

and the sum over the set 𝒦 of a few “key” frequencies is largest exactly when c = (a + b) mod p.

The trick at the heart of it, on a clock with 113 hours: turning by a and then by b lands exactly where a + b is. Move the candidate answer c (or press ▶ next to it) and watch its score reach its top only where c meets a + b.

3. How it got there. Training repeats, tens of thousands of times, the update

\theta \;\leftarrow\; \theta\,(1 - \gamma\lambda) \;-\; \gamma\cdot \text{AdamStep}\big(\nabla_\theta \mathcal{L}\big)

and along the way passes through three phases — memorisation, circuit formation, cleanup — the first two of which can be measured before the sudden jump in test accuracy, which happens during cleanup.

Symbol	Meaning (one sentence)
p	The prime the numbers wrap around at; p = 113, so the possible answers are 0, 1, …, 112.
a, b	The two numbers of the question “a + b = ?”, each from 0 to p − 1.
c, c^*	A candidate answer c that the network scores; the correct one is c* = (a + b) mod p.
\texttt{=}	The third input token; the answer is read off at its position.
W_E	The embedding matrix: column t is the list of 128 numbers that represents token t.
W_{pos}	Positional embeddings: one list of 128 numbers per position, added to the token's list.
x^{(0)}_i	The list of numbers at position i before any processing.
W_Q^j, W_K^j, W_V^j, W_O^j	The four matrices of attention head j (query, key, value, output).
A^j	How strongly the “=” position reads from positions a, b and “=” in head j (three numbers summing to 1).
x^{(1)}	The “=” position's list of numbers after the attention step.
W_{in}, b_{in}	Input weights and offsets of the 512 hidden “neurons”.
\mathrm{MLP}(a,b)	The 512 neuron values for input (a, b).
W_{out}, b_{out}	How the neurons write back into the list of numbers.
x^{(2)}	The final list of 128 numbers at the “=” position.
W_U	The unembedding matrix that turns the final list into one score per answer.
\mathrm{Logits}(a,b)	The 113 scores, one per candidate answer c.
k, w_k	A frequency k (a whole number 1–56) and its angle step w_k = 2πk/p.
\mathcal{K}	The few key frequencies the trained network uses.
\alpha_k	How strongly frequency k contributes to the scores.
\theta	All 226,816 learned numbers of the network.
\mathcal{L}	The loss: how wrong the network's scores are.
\gamma, \lambda	The learning rate (0.001) and the weight-decay strength (1).
d, n, H	Sizes: 128 numbers per position, 512 neurons, 4 attention heads.

Symbol

Meaning (one sentence)

The prime the numbers wrap around at; p = 113, so the possible answers are 0, 1, …, 112.

a, b

The two numbers of the question “a + b = ?”, each from 0 to p − 1.

c, c^*

A candidate answer c that the network scores; the correct one is c* = (a + b) mod p.

\texttt{=}

The third input token; the answer is read off at its position.

W_E

The embedding matrix: column t is the list of 128 numbers that represents token t.

W_{pos}

Positional embeddings: one list of 128 numbers per position, added to the token's list.

x^{(0)}_i

The list of numbers at position i before any processing.

W_Q^j, W_K^j, W_V^j, W_O^j

The four matrices of attention head j (query, key, value, output).

A^j

How strongly the “=” position reads from positions a, b and “=” in head j (three numbers summing to 1).

x^{(1)}

The “=” position's list of numbers after the attention step.

W_{in}, b_{in}

Input weights and offsets of the 512 hidden “neurons”.

\mathrm{MLP}(a,b)

The 512 neuron values for input (a, b).

W_{out}, b_{out}

How the neurons write back into the list of numbers.

x^{(2)}

The final list of 128 numbers at the “=” position.

W_U

The unembedding matrix that turns the final list into one score per answer.

\mathrm{Logits}(a,b)

The 113 scores, one per candidate answer c.

k, w_k

A frequency k (a whole number 1–56) and its angle step w_k = 2πk/p.

\mathcal{K}

The few key frequencies the trained network uses.

\alpha_k

How strongly frequency k contributes to the scores.

\theta

All 226,816 learned numbers of the network.

\mathcal{L}

The loss: how wrong the network's scores are.

\gamma, \lambda

The learning rate (0.001) and the weight-decay strength (1).

d, n, H

Sizes: 128 numbers per position, 512 neurons, 4 attention heads.

How to read this site

Chapters 1–11 build the background: clock arithmetic, the dataset, vectors and matrices, probabilities, the network itself, how it learns, what grokking is, waves, the Fourier basis, and how to measure a fit. Each chapter only uses what earlier chapters defined.

Chapters 12–16 derive the algorithm and test it against the trained network in four ways, following the paper [1] and the ARENA course [3].

Chapters 17–20 go back in time: how the algorithm forms during training, what drives it, and whether it happens the same way for other random seeds (we retrained 22 networks to check).

Chapter 21 returns to the three objects above, now with every piece explained, plus the limitations and open questions.

The dashboard shows every visualization on one page, linked together and scrubbable through training.

Every measured number on this site is computed from real networks: by default the paper's own published training run [7], with our replications one click away. Our analysis code reproduces most of the paper's reported numbers on the paper's own network, and the few it does not are listed with explanations (see the reproduction page). Where a step or claim comes from somewhere, it is cited; quotations are exact and short.

This model is a toy: one task, one layer, 226,816 learned numbers. Its value is that its algorithm has been reverse-engineered in unusual detail — the paper's authors call their work a proof of concept for using mechanistic interpretability to understand emergent behaviour in training. The three videos that inspired this site tell the same story from different angles [8] [9] [10].