Vectors and vector spaces

An arrow you can add and stretch, and a list of numbers you can compute with: the same object wearing two hats. Master this and every embedding in the LLM path stops being magic.

A vector is the rare idea that is simultaneously visual and numerical. Draw it as an arrow from the origin and you can add arrows tip-to-tail and stretch them. Write it as a column of numbers and a computer can do the exact same operations with plain arithmetic. Those two pictures never disagree, and that agreement is the whole engine of machine learning.

One object, two faces

An arrow in the plane is pinned down by two numbers: how far right and how far up its tip sits. So we write it as a column, its coordinates.

Everything below is just: what happens to the arrow, and what happens to the list, checking that they always match.

Adding and scaling, the only two moves

A vector space allows exactly two operations, and everything is built from them. Addition is tip-to-tail: slide $\mathbf{w}$ so its tail sits on the tip of $\mathbf{v}$; the arrow from the origin to where you land is the sum. In coordinates you add componentwise. Scalar multiplication stretches, or with a negative number flips, an arrow without turning it.

Combine both moves and you get a linear combination, the star of this lesson: $a\mathbf{v}+b\mathbf{w}$. Pick two knobs $a$ and $b$, scale each arrow, add them tip-to-tail, and see where you land. The demo below is that expression, made draggable.

Span, basis, dimension

The span of a set of vectors is every linear combination you can build from them.

When neither vector is a scaled copy of the other, we call them linearly independent, and a quick test in 2-D is that the determinant is nonzero.

Two independent vectors that span the whole plane form a basis: a minimal toolkit where every point has exactly one address $(a,b)$. The number of vectors in a basis is the dimension. The plane is 2-D because it takes exactly two; make them parallel and you drop to a 1-D span, a line.

A vector is literally a list, and add and scale are three lines each.

# a vector is just a list of numbers
v = [2, 1]
w = [-1, 2]

def add(u, x):        # tip-to-tail addition
    return [u[i] + x[i] for i in range(len(u))]

def scale(a, x):      # stretch or flip
    return [a * c for c in x]

# linear combination a*v + b*w with a = 1.5, b = -0.5
combo = add(scale(1.5, v), scale(-0.5, w))   # -> [3.5, 0.5]

# a word embedding is the SAME object, just longer:
king = [0.21, -0.44, 0.05]   # imagine 300 numbers = one point in 300-D

Check yourself