Reference shelf

Formula sheet

The core formulas of the course, on one page to revisit.

Written by Claude Opus 4.8

The core formulas of the course on one page. Come back to these cold and try to say what each one means before you read on.

Foundations

f : X \to Y, (x, y_{1}), (x, y_{2}) \in f \Rightarrow y_{1} = y_{2} A function pairs each input with exactly one output.

lo g_{b} (b^{x}) = x, b^{l o g_{b} x} = x, lo g_{b} (2 N) = 1 + lo g_{b} N Exponential and logarithm cancel; a log counts halvings.

lo g_{b} (x y) = lo g_{b} x + lo g_{b} y, lo g_{b} (x^{k}) = k lo g_{b} x, lo g_{b} x = \frac{lo g _{c} x}{lo g _{c} b} The log laws turn multiplication into addition.

Linear algebra

v + w = [v_{1} + w_{1} v_{2} + w_{2}], a v + b w Add and scale, then combine: the two moves of a vector space.

det [v_{1} v_{2} w_{1} w_{2}] = v_{1} w_{2} - v_{2} w_{1} Independent in 2-D when the determinant is nonzero.

a \cdot b = i \sum a_{i} b_{i} = ∣ a ∣∣ b ∣ cos θ, cos θ = \frac{a \cdot b}{∣ a ∣∣ b ∣} The dot product, two ways, and cosine similarity.

C_{ij} = k \sum A_{ik} B_{k j}, (m \times n) (n \times p) = (m \times p) Each cell of a product is a row dotted with a column.

Calculus for learning

f^{'} (x) = h \to 0 lim \frac{f ( x + h ) - f ( x )}{h}, f (x + Δ x) \approx f (x) + f^{'} (x) Δ x The derivative is the limiting secant slope; nearby, f is linear.

\frac{d}{d x} x^{n} = n x^{n - 1}, (f \circ g)^{'} (x) = f^{'} (g (x)) g^{'} (x) The power rule and the chain rule.

\nabla f = [\partial f / \partial x \partial f / \partial y], p := p - α \nabla f (p) The gradient stacks the partials; descent steps against it.

The gradient is perpendicular to the contour, and $D_{u} f = \nabla f \cdot u = ∥ \nabla f ∥ cos θ$ is maximized when you face along it.

Probability and statistics

μ = \frac{1}{N} i \sum x_{i}, σ^{2} = \frac{1}{N} i \sum (x_{i} - μ)^{2}, f (x) = \frac{1}{σ 2 π} e^{- \frac{( x - μ ) ^{2}}{2 σ ^{2}}} Mean, variance, standard deviation, and the Gaussian bell.

P (A) \geq 0, P (Ω) = 1, E [X] = x \sum x p (x), Var (X) = E [X^{2}] - μ^{2} Probability axioms; expectation and variance of a random variable.

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}, P (A ∣ B) = \frac{P ( B ∣ A ) P ( A )}{P ( B )} Conditional probability and Bayes' theorem.

L (θ) = i \prod p (x_{i} ∣ θ), ℓ (θ) = i \sum lo g p (x_{i} ∣ θ), - i \sum lo g p_{θ} (y_{i} ∣ x_{i}) Likelihood, log-likelihood, and why minimizing NLL is cross-entropy.

H (p) = - x \sum p (x) lo g_{2} p (x), H (p, q) = - x \sum p (x) lo g_{2} q (x), D_{KL} (p ∥ q) = H (p, q) - H (p) Information, entropy, cross-entropy, and KL divergence.

Discrete math and proof

p \Rightarrow q \equiv \neg p \lor q \equiv \neg q \Rightarrow \neg p, \neg (\forall x P (x)) \equiv \exists x \neg P (x) Implication, its contrapositive (equivalent) and converse (not), and quantifier negation.

P (1) \land (\forall k \geq 1, P (k) \Rightarrow P (k + 1)) ⟹ \forall n \geq 1, P (n) The induction template: base case plus inductive step proves it for all n.

i = 1 \sum n i = \frac{n ( n + 1 )}{2}, i = 1 \sum n i^{2} = \frac{n ( n + 1 ) ( 2 n + 1 )}{6}, i = 0 \sum n - 1 r^{i} = \frac{1 - r ^{n}}{1 - r}, i = 0 \sum \infty r^{i} = \frac{1}{1 - r} The sums you meet in complexity analysis.

P (n, k) = \frac{n !}{( n - k )!}, (k n) = \frac{n !}{k ! ( n - k )!}, (a + b)^{n} = k = 0 \sum n (k n) a^{n - k} b^{k} Counting: permutations, combinations, Pascal's rule, and the binomial theorem.

Advanced linear algebra

T (u + v) = T (u) + T (v), T (c v) = c T (v), M = [T (\overset{}{^}) T (\overset{}{^})] A linear map is fixed by where it sends the basis vectors; its matrix is those images as columns.

A v = λ v, det (A - λ I) = 0, A = Q Λ Q^{- 1} Eigenvectors: the directions a matrix only scales; the characteristic equation finds them.

Numerical foundations

x = (- 1)^{s} (1 + \frac{f}{2 ^{23}}) \times 2^{e - 127}, softmax (z)_{i} = \frac{e ^{z_{i} - m}}{\sum _{j} e ^{z_{j} - m}}, m = j max z_{j} A 32-bit float, and the numerically stable softmax that subtracts the max first.