Under The Hood: Build Every Layer of a Large Language Model from Scratch

Most books explain how large language models work. This one makes you build one - every layer, from a scalar .backward() to a production serving stack you understand all the way down.36 projects, one method: build it, break it, measure it. You write every system from scratch, run an experiment designed to make it fail, and read the numbers to learn why. Autograd and attention from first principles. A GPT you train yourself. Tokenizers, KV caches, speculative decoding, and fp8 serving. Quantization, MoE, and scaling laws. RLHF, RLVR, and test-time reasoning. Diffusion decoding and hybrid SSM/attention models - the 2026 frontier included.No hand-waving. Every number traces to a primary source. If it's slow, you profile it. If it's wrong, you break it on purpose to see how.For engineers, researchers, and students who refuse to treat AI as a black box. If you can write Python and remember a little calculus, you can build everything in this book - and by the last page, nothing between the prompt and the token will be a mystery to you.