Stanford's graduate course on building language models end to end, covering architecture, training, scaling laws, systems and kernel optimization, with assignments. Commenters point to its lecture notes and assignments as the way to learn the full pipeline.

Commenters call it a deep, theory-and-systems-oriented route into LLMs, with one saying the lecture notes made computational bottlenecks much more intuitive.

What commenters said (2)

1. Strongly recommendsjvican2026in “Train Your Own LLM from Scratch”

   If you're interested in this resource, I highly recommend checking out Stanford's CS336 class. It covers all this curriculum in a lot more depth, introduces you into a lot of theoretical aspects (scaling laws, intuitions) and systems thinking (kernel optimization/profiling). For this, you have to do the assignments, of course... https://cs336.stanford.edu/

2. Recommendsdanbrooks2026in “I were 17, I'd learn how to build LLMs from scratch”

   I had the same inspiration!

   I spent a days reviewing the lecture notes for CS336: Language Modeling from Scratch - and then trained a nanoGPT-esque model in PyTorch.

   I'd recommend trying it for those who are curious. Computational bottlenecks become much more intuitive when you've looked at the overall process.

Related courses

• Machine Learning (Andrew Ng)Coursera · 240 recommenders
• Practical Deep Learning for Codersfast.ai · 112 recommenders
• CS231n: Convolutional Neural Networks for Visual RecognitionStanford University · 39 recommenders
• Deep Learning Specialization (Andrew Ng)Coursera · 31 recommenders
• Neural Networks for Machine Learning (Geoffrey Hinton)Coursera · 25 recommenders
• CS229 Machine LearningStanford University · 20 recommenders
• Learning From DataCaltech · 20 recommenders
• CS188 Introduction to Artificial IntelligenceUC Berkeley · 18 recommenders