3D visualization of a GPT-style transformer network showing attention layers, embedding matrices, and data flow during inference
If you've ever wanted to truly see how a GPT-style language model works, Brendan Bycroft's LLM Visualization is one of the most impressive educational tools out there. It renders a fully working transformer network in 3D, letting you zoom in and out of every layer, every attention head, every matrix multiply. The default demo uses a tiny model from Andrej Karpathy's minGPT (trained to sort the letters A, B, and C), so you can actually watch the entire inference process from input tokens to output without your browser catching fire. It's been praised widely, including by Karpathy himself in one of his videos.
The project is built by Brendan Bycroft, a New Zealand-based software engineer with a background in math, physics, and a career spanning everything from CUDA to web dev to 3D graphics. He's the roll-your-own type of developer who optimizes for understanding and performance over convenience, and it really shows here. You can find him on X at @BrendanBycroft, browse his work on GitHub, or check out his personal site at bbycroft.net.
On the technical side, the whole thing is written in TypeScript and the renderer is flexible enough to visualize arbitrary network sizes (there's even a GPT-2 option, though the weights aren't bundled since they'd be hundreds of megabytes). What's particularly cool is how it traces the flow of data through attention mechanisms and feedforward layers with animated highlights and detailed walkthrough text. The source code is on GitHub if you want to dig into how he structured the visualization, and definitely give Brendan a follow if you're into this kind of deep-dive graphics work.
- Live Demo: https://bbycroft.net/llm
- Source Code: https://github.com/bbycroft/llm-viz
- Author: Brendan Bycroft