Learn

Large Language Models

From zero: the math to ML to neural networks ladder, built by hand.

How modern AI actually works, starting from zero background. Climb the ladder from what learning even means, through regression and gradient descent, up to neurons, networks, and backpropagation. I want to reason from first principles, not analogies, and see the machinery that later becomes a transformer.

Written by Claude Opus 4.8 57 lessons, 10 modules updated Jul 2026

Module 0 What Learning Means

start here

What is machine learning?

Fitting a function from examples, not rules

Linear regression

The simplest model: a line that predicts

The cost function

Turning 'wrong' into a number to minimize

Gradient descent

Rolling downhill to the best parameters

Module 1 Machine Learning Foundations

the ML groundwork

Supervised, unsupervised, and a peek at reinforcement

The three ways a model can learn

Features, labels, and datasets

Turning raw data into what a model eats

Feature scaling and encoding

Putting inputs on a comparable footing

Evaluating classifiers: precision, recall, and ROC

Measuring a classifier past raw accuracy

Classic ML: k-NN, decision trees, k-means, and PCA

The workhorses before deep learning

Loss functions in depth: MSE, cross-entropy, softmax

The objective the whole model chases

Module 2 From Neurons to Networks

the machinery

Logistic regression

From a line to a probability, a decision

The perceptron

One artificial neuron, and its limits

The forward pass

Stacking neurons into layers of computation

Activation functions

The nonlinearity that makes depth matter

Backpropagation

The chain rule that teaches the network

Module 3 Training Deep Networks

the mechanics of learning

Tensors, broadcasting, and autograd

The array machinery frameworks are built on

The training loop: SGD, mini-batches, epochs

The rhythm every model is trained by

Optimizers: momentum, RMSProp, and Adam

Smarter ways to step down the gradient

Overfitting, bias-variance, and regularization

Generalizing instead of memorizing

Module 4 Architectures Before Transformers

how we got here

Convolutional neural networks

Sharing weights across space for vision

RNNs and LSTMs, the pre-transformer world

Processing sequences one step at a time

Encoder-decoder and the fixed-vector bottleneck

The problem attention was invented to solve

Module 5 Tokens, Embeddings, and Attention

how transformers see text

Tokenization and BPE

Splitting text into the units a model reads

Word embeddings

Turning words into vectors that carry meaning

The attention mechanism

How a model decides what matters in context

Self-attention: query, key, value

The three projections that let tokens talk

Multi-head attention

Many parallel attention patterns, one layer

Module 6 The Transformer

the architecture

The transformer architecture

Encoder, decoder, and the stack that scales

Residuals, LayerNorm, and FFN

The three extras that make deep stacks trainable

Positional encodings

How the model knows word order without recurrence

Module 7 Language Modeling and Decoding

predicting the next token

Language modeling and next-token prediction

The training objective behind every LLM

Decoding and sampling strategies

How a model picks which token comes next

Module 8 Bridges: DSA to LLMs

the same ideas, reused

Arrays and matrices to embeddings and tensors

The data structure the whole field runs on

Matrix multiplication: the shared engine

The same operation running ML and attention

Big-O to the O(n squared) attention wall

Why context length is so expensive

Hash tables to vector search and ANN

Fast lookup, from exact keys to nearest neighbors

Tries and hash maps to BPE tokenization

Prefix structures behind the tokenizer

Dynamic programming to Viterbi and edit distance

The same table, decoding sequences

Graphs and adjacency matrices to attention

How graph structure becomes attention weights

Sorting and selection to top-k sampling

Picking the best few from a distribution

Prefix sums and binary search to top-p sampling

Cumulative mass and the nucleus

Priority queues and Dijkstra to beam search

Exploring the best partial sequences

Recursion and recurrence to autoregression

From call stacks to next-token loops

Sliding window to local attention

Bounding the window to tame quadratic cost

Memoization and queues to the KV cache

Caching past work to speed generation

Topological sort and DAGs to backpropagation

Ordering a computation graph to differentiate it

Bit manipulation to quantization

Squeezing models into fewer bits

Module 9 From GPT to Production

the full LLM stack

Decoder-only models and the GPT family

Why everything became a causal LM

Pretraining at scale

Data, compute, and the next-token objective

Scaling laws and emergent abilities

What gets better when you make it bigger

Fine-tuning and instruction tuning

Specializing a foundation model for tasks

Alignment: RLHF, reward models, and DPO

Teaching a model what humans prefer

Prompting and in-context learning

Steering a frozen model with examples

Embeddings, vector DBs, and RAG

Retrieval-augmented generation end to end

Agents and tool use

When the LLM calls functions and loops

Evaluating LLMs

Benchmarks, evals, and measuring what matters

Hallucination, limits, and safety

Where LLMs fail and how we guard them

Reference shelf

Glossary

Every load-bearing term across the 57 lessons, from features and loss up through attention, scaling, and alignment.

Formula sheet

The identities the course actually derives, grouped by module, each linked to where it is built.

Map of the field

How it all fits: a word2vec-to-agents timeline linking each leap to its lessons, plus the primary papers.

How to use this course

Do the widgets. The quizzes and flashcards are not decoration: trying to recall before you reveal is what makes it stick.
Space it out. Come back a day later and answer them cold.
Read critically. An LLM wrote these, so verify anything before you rely on it.