Bridges: DSA to LLMs

Arrays and matrices to embeddings and tensors

The data structure the whole field runs on

~11 minLesson 33 of 57Written by Claude Opus 4.8

A word embedding is a float array. A sentence is a matrix. A batch is a tensor. It is arrays all the way down - the exact structure you met in Arrays and memory.

Here is a small shock: the array - a contiguous block of same-sized boxes in memory, indexed in $O (1)$ by pure arithmetic - is the shared atom of data structures and of deep learning. A word embedding is a 1-D array of floats. A batch of token embeddings is a 2-D array - a matrix. A batch of those is a 3-D array - a tensor. Nothing new is invented on the LLM side; the same row of numbers just gets more index dimensions and a fancier name.

The one-line idea

A tensor is just an array with more than one index. A vector needs one number to reach an element (

v_{i}

), a matrix needs two (

M_{ij}

), a rank-3 tensor needs three (

T_{bij}

). But underneath, every one of them is a single flat run of floats in memory. The extra indices are bookkeeping - a promise about how to fold that one line of numbers into a shape.

The array is the shared atom

In Word embeddings a word became a point in space: cat = a list of, say, 512 floats. That list is literally an array - $d$ boxes, contiguous, each holding one number. Line up one such array per token in a sentence and you have stacked rows into a matrix of shape $(tokens, d)$ . Line up one matrix per sentence in a batch and you have a rank-3 tensor of shape $(batch, tokens, d)$ . Every weight in every layer, every activation flowing through, is one of these - a plain array of numbers with a declared shape.

Row-major: the shape is a story the strides tell

A 2-D matrix is not really 2-D in memory - memory is one line. The array library stores the first row's elements, then the second row's, then the third: row-major order (C-order, NumPy's default). To find element $(i, j)$ it does the same trick as a 1-D array - one multiply, one add - just with the row width $W$ folded in:

From index to address (why it stays O(1))

For a matrix with $W$ columns and element size $s$ bytes, and for a rank-3 tensor with $R$ rows per matrix:

addr (i, j) = base + (i \cdot W + j) \cdot s, addr (b, i, j) = base + (b \cdot R W + i \cdot W + j) \cdot s .

The multipliers $(R W s, W s, s)$ are the strides: how many bytes to jump to advance one step along each axis. The formula never mentions the array's length, so reaching any element is $O (1)$ - exactly the arrays result, one dimension richer.

The demo below makes this concrete. Hover a token to light up its row of the embedding matrix. Flip on Overlay linear memory to watch that row become a contiguous run of boxes on the flat strip. Then switch to Tensor 3-D and slice the cube along each axis - and watch a fancy multi-axis slice turn into a simple, or strided, pattern of memory offsets.

From token to tensor hover a token · overlay memory · slice the cube

Overlay linear memory

sentence:

shape (4, 4)

the

0.1

-0.2

0.0

0.3

cat

0.9

0.4

-0.1

0.2

sat

-0.2

0.7

0.5

-0.3

0.0

-0.5

0.8

0.1

shape (4, 4) = (tokens, dims) · row-major, one contiguous block

Stack one embedding per token → a 2-D array (a matrix). Hover a token to light its row; toggle memory to see the row-major layout.

Top labels are embedding dimensions d0…d3; left labels are tokens. The number over each memory box is its flat offset - the value of i·W + j (plus a batch stride in 3-D).

The aha: a slice is a stride pattern

Look at what the memory strip does in Tensor mode. Slicing axis 0 (pick a sentence) lights up one contiguous block - that whole matrix sits back-to-back. Slicing axis 2 (pick a dimension) lights up every fourth box - that feature is scattered by the stride. Same cube, same flat memory; the axis you fix decides whether the GPU reads a smooth run or a strided gather. That is why array layout, not just array shape, decides how fast a model runs.

Shapes, axes & broadcasting

Once everything is an array, the whole game is keeping shapes consistent. An axis is one index direction; a shape is the length along each axis. Reshaping never moves a single float - it only relabels how the flat line is folded. And broadcasting is the rule that lets a small array act on a big one without copying: a bias of shape $(d,)$ is stretched across the batch and token axes to add to every embedding at once.

import numpy as np

# a word embedding is a 1-D array of floats - one contiguous block in memory
cat = np.array([0.9, 0.4, -0.1, 0.2], dtype=np.float32)   # shape (4,)

# a sentence: one embedding row per token -> 2-D array (a matrix)
sentence = np.array([
    [ 0.1, -0.2,  0.0,  0.3],   # the
    [ 0.9,  0.4, -0.1,  0.2],   # cat
    [-0.2,  0.7,  0.5, -0.3],   # sat
    [ 0.0, -0.5,  0.8,  0.1],   # on
], dtype=np.float32)                # shape (4, 4) = (tokens, dims)

# a batch of sentences: stack matrices -> 3-D array (a tensor)
batch = np.stack([sentence, sentence])   # shape (2, 4, 4) = (batch, tokens, dims)

batch.shape      # (2, 4, 4)
batch.strides    # (64, 16, 4) bytes: step 1 sentence / token / dim
batch.ravel()[:4]  # the-row floats, back-to-back: it is one contiguous block

# broadcasting: a (4,) bias added to every token of every sentence
bias = np.array([0.0, 0.1, -0.1, 0.05], dtype=np.float32)
(batch + bias).shape   # (2, 4, 4) - bias stretched across axes 0 and 1

One block of floats, three shapes

Check yourself

Reading A[i][j] from a row-major matrix is O(1) because its flat offset is:

A batch of 8 sentences, each 12 tokens, each token a 512-dim embedding, is a tensor of shape:

Recall: how are a word embedding, a sentence, and a batch each an array - and where does element (b, i, j) of the batch tensor live in memory?

Try to state it, then check.

Lock it in

The array - a contiguous block of same-sized cells indexed in $O (1)$ by arithmetic - is the shared atom of both data structures and deep learning.
A word embedding is a 1-D array, a sentence is a 2-D matrix $(tokens, d)$ , a batch is a 3-D tensor $(batch, tokens, d)$ - a tensor is just an array with more indices.
Every tensor is one flat run of floats; strides turn a multi-axis index into a single memory offset, so reaching any element stays $O (1)$ .
Layout, not just shape, sets speed: a slice is a stride pattern, and broadcasting stretches a small array over a big one without copying.

Primary source

For arrays as contiguous, O(1)-indexed memory and the address arithmetic behind it, see Sedgewick & Wayne's algs4 booksite (Princeton). For the same arrays wearing their deep-learning clothes - token embeddings as matrix rows, batches of them flowing through a Transformer as tensors - read Jay Alammar's The Illustrated Transformer, whose diagrams are exactly the grids in the demo above.

Ask your teacher

This is why arrays and matrices are the LLM: every embedding, weight, and activation is an array in contiguous memory that a GPU streams through - the matmuls are just those arrays multiplied. Ask me why row-major vs. column-major changes GPU speed, what a "view" vs. a "copy" is, how reshape can be free while transpose is not, or why the whole field settled on the

(batch, tokens, dims)

axis order.