Fig. 36. §3 AI and Computing
Looped Transformer
A looped transformer runs the same transformer block again and again, feeding its output back in as its next input, instead of stacking many different layers.
1Facts
- Shelf
- AI and Computing
- Kind
- Built from a recipe
- Sound
- Yes
- Licenses
- Own work (made in the page)
2How to play
Tap it: the tiles loop through one block, sharper each lap, until 3 + 4 = 7.
3Its controls
Table I. Every control on this Splash.
| Control | Kind | Range | Start |
|---|---|---|---|
| Loop until it's sure | Tap | ||
| Think | Button | ||
| View | List | 2 choices | Poster (2D) |
4About it
A looped transformer runs the same transformer block again and again, feeding its output back in as its next input, instead of stacking many different layers. Each pass through the loop can refine the answer a little more, a bit like checking your work, so a small model can spend longer thinking about a harder problem.
Tap it: five tiles ride three laps around a track through one block. Each pass sharpens every tile one step, from noise to a blocky mosaic, to nearly right, to exact, until “3 + 4 = 7” settles with the 7 in gold. Pick the poster or a 3D model in the Splash tab.
5Details
- Splats
- Opens at 200,000 at Mid Detail, 56,377 at Low, and up to 660,000 at Max
6Is it right?
Repeating one block is a valid idea in recurrent transformer designs. Here three laps reveal stored pictures of 3 + 4 = 7; no transformer block or arithmetic inference runs.
Simplified.
Transformer computation can repeat across depth.[1]
The single block and returning track illustrate a recurrent-depth architecture.
Correct.
The splash ends with the correct arithmetic statement 3 + 4 = 7.[2]
The five stored marks are 3, +, 4, =, 7; the last tile images contain only finished bitmap values.
Simplified.
The three laps illustrate refinement rather than learned reasoning.[1]
Each lap chooses another stored bitmap level, with no attention, recurrent tensor update, or learned stopping rule.
- [1]Universal Transformers, Dehghani and coauthors, 2018
- [2]Multilayer perceptrons, Dive into Deep Learning authors
- Simplified
- The noise, coarse, nearly finished, and exact tile images are prepared stages. The lap count is fixed at three and is not chosen by difficulty.
7Sources and credits
Made in Splashery. Its code is MIT.
