Inside the Model
Show HN: A 155K-param transformer builds a map of a world it's never shown
Last checked:
What is Inside the Model?
Inside the Model is an interactive research visualization that demonstrates how neural networks build internal world models through a 155,000-parameter network trained only on movement symbols. The tool lets you decode and manipulate the model's internal spatial beliefs in real-time using mechanistic interpretability techniques, showing how false beliefs causally affect the network's behavior.
Inside the Model pricing
Pricing model: Freemium
Inside the Model pros
- Fully interactive, browser-runnable demo making abstract mechanistic interpretability concepts tangible and testable
- End-to-end provable with high accuracy metrics (98.8% position decoding, 100% legal move prediction, 100% behavior change on belief edits)
- Demonstrates causal relationship between internal representations and behavior through activation patching
- Open-source with full source code, reproduction instructions, and connections to related research (Othello-GPT, Gurnee & Tegmark)
Inside the Model cons
- Limited to a toy grid-world scenario; scaling to larger models and complex domains remains an open question
- Requires background in neural networks and mechanistic interpretability to fully understand and interpret results
- Demonstrates principles on a small model; results may not directly transfer to real-world language model applications
Frequently asked questions about Inside the Model
Can I run this in my browser?
Yes, the interactive demo runs entirely in your browser at ankur-chr.github.io/inside-the-model with no signup required.
What does 'belief' mean in this context?
Here it means a decoded internal state representation (spatial position, lamp location), not consciousness or human-like understanding. It's a measurable, causally relevant world-state representation.
How does this relate to large language models?
Similar patterns appear in larger systems: Othello-GPT builds internal board representations from move sequences, and Llama-class models encode linear maps of real-world place and time.
Can I see the source code?
Yes, the full source code is available on GitHub (github.com/ankur-chr/ankur-chr.github.io/tree/main/inside-the-model) with instructions for reproducing the results.