Reference shelf

Glossary

Every load-bearing term across the 57 lessons, from features and loss up through attention, scaling, and alignment.

Written by Claude Opus 4.8

Every load-bearing term in this course, from features and loss up through attention, scaling, and alignment, defined crisply. Filter to find one fast, or skim it end to end to watch the ladder connect.

Activation function: The nonlinearity applied at each neuron (sigmoid, tanh, ReLU, GELU). Without it, stacking layers collapses to a single linear map.
Adam: The workhorse optimizer that keeps per-parameter running averages of the gradient and its square, giving each weight its own adaptive step size.
Agent: A system that uses an LLM in a loop to decide its own actions, calling tools and reacting to their results toward a goal instead of following a fixed script.
Approximate nearest neighbor: Search that finds close-enough vectors without scanning every one, trading a little recall for a huge speedup, which is what makes vector search scale.
Attention: A mechanism that lets each position gather information from other positions, weighting them by relevance instead of by fixed distance.
Autograd: The engine that records the computational graph during the forward pass and replays it backward to hand you gradients automatically.
Autoregressive: Generating a sequence one token at a time, each new token conditioned on all the tokens produced so far.
Backpropagation: Computing the gradient of the loss with respect to every parameter by applying the chain rule backward through the network: grad(parent) = grad(child) times local derivative.
Beam search: Keeping the k most probable partial sequences at each step instead of committing greedily, trading compute for higher-probability completions.
Benchmark: A fixed dataset and metric used to score and compare models, useful for tracking progress but vulnerable to test-set contamination and overfitting to the leaderboard.
Bias-variance tradeoff: The tension between a model too simple to capture the pattern (high bias, underfitting) and one so flexible it fits noise (high variance, overfitting).
Byte-pair encoding: The tokenizer that starts from characters and greedily merges the most frequent adjacent pair over and over, landing between characters and whole words.
Causal mask: The triangular mask that blocks each position from attending to future tokens, so a decoder only ever conditions on the past.
Chain-of-thought: Prompting the model to write intermediate reasoning steps before its answer, which sharply improves accuracy on multi-step problems.
Computational graph: The DAG of operations that produced a value. Backprop is a reverse topological walk over this graph.
Compute (6ND): Training compute in FLOPs is about 6 times parameters times tokens (C ~ 6ND), the back-of-envelope rule behind every scaling decision.
Compute-optimal (Chinchilla): The Chinchilla result that for a fixed compute budget, model size and training tokens should grow together (roughly 20 tokens per parameter), correcting earlier undertrained giants.
Cost function: The average loss over a whole dataset or batch, the single scalar that gradient descent walks downhill.
Cross-entropy: The loss for classification and language modeling: the negative log probability the model assigned to the true class. Minimizing it pushes probability mass onto the right answer.
Decision boundary: The surface where a classifier flips its prediction, for example where a logistic model crosses probability 0.5.
Decoder-only: A single stack of causally masked transformer blocks that just predicts the next token, the GPT design that scaled cleanest and won.
Decoding: The procedure that turns the model's per-step probability distribution into actual generated tokens, via greedy choice, sampling, or search.
DPO: Direct preference optimization: skip the separate reward model and RL loop and train directly on preference pairs with a simple classification-style loss, reaching a similar objective more stably.
Embedding: A dense vector that represents a token or item so that similar meanings sit close together, letting the model do arithmetic on meaning.
Emergent ability: A capability that is near-absent in small models and appears fairly suddenly past some scale, though measurement choices shape how sharp the jump looks.
Encoder-decoder: A two-tower design where an encoder reads the whole input and a decoder generates the output attending to it, the original transformer setup for translation.
Epoch: One full pass through the entire training set. Training usually runs for many epochs.
F1 score: The harmonic mean of precision and recall, a single number that is high only when both are high.
Feature: One measured input variable the model reads, like a pixel value or a token. A stack of features is the vector that represents an example.
Feature scaling: Rescaling inputs to a common range (standardizing to zero mean and unit variance, or squashing to 0..1) so no single feature dominates the gradient and training converges faster.
Feed-forward network: The per-position two-layer MLP inside each transformer block, usually widening to 4x the model dimension and back, where much of the model's knowledge lives.
Few-shot and zero-shot: Prompting styles: few-shot includes a handful of worked examples for the model to imitate, zero-shot gives only the instruction.
Fine-tuning: Continuing to train a pretrained model on a smaller, targeted dataset so it specializes to a task, domain, or style.
Forward pass: Running inputs through the network layer by layer to produce an output and a loss, before any gradients are computed.
GELU: A smooth ReLU-like activation, x times the Gaussian CDF of x, used in the feed-forward blocks of most transformers.
Gradient descent: The core training loop: repeatedly nudge each parameter a small step against the gradient of the loss, so the loss keeps dropping.
Hallucination: A fluent, confident output that is simply false, an expected failure of a model trained to predict plausible text rather than to check truth.
In-context learning: A model adapting to a task purely from examples and instructions in the prompt, with no weight updates. The behavior that made prompting powerful.
Instruction tuning: Fine-tuning on many (instruction, response) pairs so a raw next-token model learns to follow requests instead of merely continuing text.
KV cache: Storing the keys and values already computed for past tokens so each new token does O(n) work instead of recomputing the whole triangle, at the cost of memory that grows with context.
Label: The correct answer attached to a training example, the target the model is asked to predict. Data with labels is what makes learning supervised.
Language modeling: Training a model to predict the next token given the previous ones, factoring the probability of a whole sequence into a product of per-token probabilities.
Layer normalization: Rescaling each token's vector to zero mean and unit variance (then applying learned gain and bias), which stabilizes the scale of activations across a deep network.
Learning rate: The step size in gradient descent. Too small crawls, too large overshoots and can diverge, so it is the most important dial to tune.
Linear regression: Fitting a straight line y = wx + b to data by minimizing squared error. The simplest supervised model and the seed of everything after it.
Logistic regression: Linear regression passed through a sigmoid to output a probability, turning a line into a classifier with a decision boundary.
Logits: The raw, unnormalized scores a model outputs for each class or vocabulary token, before softmax turns them into probabilities.
LoRA: Low-rank adaptation: freeze the base weights and train small low-rank update matrices, cutting the cost of fine-tuning to a tiny fraction of full training.
Loss function: A number that measures how wrong one prediction is. Training is the search for parameters that make the average loss small.
Mean squared error: The average squared gap between predictions and targets, the standard loss for regression. Squaring punishes large misses hardest.
Mini-batch: A small chunk of examples used to estimate the gradient for one update, trading the noise of single examples against the cost of the full dataset.
Momentum: Accumulating a running average of past gradients so updates keep rolling in a consistent direction, smoothing noise and speeding descent through ravines.
Multi-head attention: Running several attention operations in parallel on split projections, so different heads can capture different relationships, then concatenating and mixing the results.
Negative log-likelihood: The same quantity as cross-entropy seen from probability: minimizing the negative log of the likelihood the model gave the data.
Optimizer: The rule that turns gradients into parameter updates. Plain SGD is the baseline; momentum and Adam are the common upgrades.
Overfitting: When a model memorizes the training set, including its noise, and fails to generalize to new data. The gap between training and validation loss is the tell.
Perceptron: A single artificial neuron: a weighted sum of inputs passed through a step or activation. Stack and connect them and you get a neural network.
Perplexity: The exponential of the average cross-entropy loss, read as the model's effective branching factor: how many equally likely next tokens it is choosing among.
Positional encoding: Information about token order added to embeddings, since attention itself is order-blind. Can be fixed sinusoids, learned vectors, or rotations.
Precision: Of the items the model flagged positive, the fraction that truly were. It answers how much you can trust a positive prediction.
Pretraining: The first, expensive stage: training a model on a vast corpus with next-token prediction to absorb general language and knowledge before any task-specific tuning.
Quantization: Storing weights (and sometimes activations) in fewer bits, mapping a float range onto a small integer grid, to shrink memory and speed inference for a little accuracy loss.
Query, key, value: The three projections of each token in attention: a query asks, keys advertise, and values carry the content that gets mixed. Match is query dotted with key.
RAG: Retrieval-augmented generation: fetch relevant passages from an external store and put them in the prompt so the model answers from real, current data rather than memory.
Recall: Of the items that truly were positive, the fraction the model caught. Precision and recall trade off against each other.
Regularization: Anything that discourages a model from fitting noise, such as penalizing large weights, dropout, or early stopping, trading a little training fit for better generalization.
ReLU: max(0, x): pass positives through, zero out negatives. Cheap, non-saturating, and the default hidden activation for years.
Residual connection: Adding a sublayer's input back to its output, y = x + Sublayer(x), so gradients flow straight through and deep stacks stay trainable.
Reward model: A model trained on human preference pairs to score outputs, standing in for a human rater so the policy can be optimized against it at scale.
RLHF: Reinforcement learning from human feedback: train a reward model on human preferences, then optimize the language model to score well under it while staying near the base model.
ROC and AUC: The ROC curve plots true-positive rate against false-positive rate across every threshold; the area under it (AUC) scores a classifier independent of any one cutoff.
RoPE: Rotary position embedding: encode position by rotating the query and key vectors by an angle proportional to their index, so attention naturally sees relative distance.
Scaled dot-product attention: The attention formula softmax(QK^T / sqrt(d_k)) V. The dot products score relevance, the sqrt(d_k) keeps them from saturating softmax, and the weights blend the values.
Scaling laws: The empirical finding that loss falls as a smooth power law in model size, data, and compute, making the payoff of scale predictable in advance.
Self-attention: Attention where the queries, keys, and values all come from the same sequence, so every token can look at every other token in that sequence.
Sigmoid: The S-curve 1 / (1 + e^-z) that squashes any real number into (0, 1), reading a raw score as a probability.
Softmax: Turns a vector of scores into a probability distribution by exponentiating and normalizing, so the outputs are positive and sum to one.
Stochastic gradient descent: Gradient descent that updates on mini-batches rather than the whole dataset, so each step is cheap and the noise can help escape shallow minima.
Supervised learning: Learning from input-output pairs, adjusting the model until its predictions match the given labels. Regression and classification are the two classic shapes.
Temperature: A knob that divides the logits before softmax: near zero makes generation nearly deterministic, higher values flatten the distribution and add variety.
Tensor: A multi-dimensional array, the universal data container in deep learning. A scalar, vector, and matrix are just rank-0, 1, and 2 tensors.
Tokenization: Splitting text into the discrete units (tokens) the model reads and writes. It quietly sets vocabulary size, sequence length, and cost.
Tool use: Letting a model call external functions (search, code, an API) so an agent can act on the world and read back results, rather than only producing text.
Top-k sampling: Restricting each step's choice to the k highest-probability tokens, then sampling among them, cutting off the unlikely tail.
Top-p sampling: Nucleus sampling: keep the smallest set of tokens whose probabilities sum to at least p, then sample within it, so the cutoff adapts to how peaked the distribution is.
Training, validation, and test sets: The three splits of data: the training set fits the model, the validation set tunes choices, and the untouched test set gives an honest final score.
Transformer: The attention-based architecture that stacks self-attention and feed-forward blocks with residuals and normalization, the backbone of every modern LLM.
Unsupervised learning: Finding structure in data that has no labels, such as clustering similar points or learning to predict the next token from raw text.
Vector database: A store that indexes embeddings for fast nearest-neighbor search, the retrieval engine behind semantic search and RAG.
Word2vec: An early method that learns word embeddings by predicting neighboring words, famous for the analogy king - man + woman is close to queen.