Machine Learning Foundations

Features, labels, and datasets

Turning raw data into what a model eats

~10 minLesson 6 of 57Written by Claude Opus 4.8

Strip away the buzzwords and a machine-learning model never really sees a house, an email, or a face - it sees a spreadsheet. Each row is one example; each column is one feature; and one designated column is the label, the answer you want it to predict. Understand that table and you have understood the raw material of every algorithm ahead.

Everything is a table of numbers

A feature is an input - a measurable property of an example (a house's size, an email's word counts). The label is the target you're trying to predict (the sale price, spam-or-not). Stack all the feature columns into a matrix and you get the $X$ ; stack the answers into a column and you get the target vector $y$ . Training a supervised model is just: given $X$ , learn a function that reproduces $y$ .

The design matrix, precisely

With $m$ examples and $n$ features, the data is

X \in R^{m \times n}, y \in R^{m} .

Row $i$ is one example, written $x^{(i)} = (x_{1}^{(i)}, x_{2}^{(i)}, \dots, x_{n}^{(i)})$ . Column $j$ is one feature across every example. So $X_{ij}$ is "the value of feature $j$ for example $i$ ." The whole of supervised learning is finding weights so that a prediction $\overset{y}{^}^{(i)}$ built from row $x^{(i)}$ lands close to the true label $y^{(i)}$ .

Which column actually predicts the price?

Here is a tiny housing dataset - 12 rows, 6 columns. Pick which column becomes the horizontal axis and which becomes the vertical, and the scatter re-plots instantly. Hover any table row to light up its dot (and hover a dot to light up its row). The bar under the plot measures how strongly your chosen X-column correlates with the label, Price - that is the difference between a good feature and useless noise.

Linked table & scatterpick X & Y · hover a row

X axisY axis

Size (sqft) vs Price: r = 0.99 - strong predictor - a good feature

Lot #	Size	Bedrooms	Age	Dist. to city	Price
3,310	850	2	35	9	210
8,823	1,200	3	20	6	320
1,145	1,500	3	12	5	410
6,650	1,800	4	8	4	500
2,204	2,400	4	3	3	640
7,781	950	2	40	11	180
4,471	1,350	3	25	7	300
9,912	2,000	4	15	5	520
1,029	1,650	3	18	6	430
5,567	2,800	5	2	2	760
8,034	1,100	2	30	8	250
3,390	2,200	4	6	3	600

Try Size vs Price: the dots climb in a tidy diagonal - big signal, a great feature. Now switch X to Lot # (a parcel ID): the cloud scatters with no shape, the bar collapses toward zero. Same model, same effort, but that column carries nothing. This is why feature choice, not model choice, is where beginners win or lose.

Three flavors of feature - and turning them into numbers

The plot only works because every column is already a number. Real features come in three shapes, and part of the job is vectorizing the ones that aren't numeric yet:

Numerical - a real quantity where arithmetic and distance mean something: size in sqft, price, age. Use as-is (usually after scaling, next lesson).
Categorical - an unordered label like neighborhood ∈ {A, B, C}. There is no "B > A", so you can't just number them 1/2/3 - that would invent a fake order. Instead one-hot encode: one 0/1 column per category.
Ordinal - categories with a natural order, like condition ∈ {poor, fair, good}. Here mapping to 1/2/3 is fair game, because the order is real.

import pandas as pd

df = pd.read_csv("housing.csv")

# features (inputs) go in X, the label (target) in y
X = df[["size", "beds", "age", "dist"]]   # design matrix, shape (m, n)
y = df["price"]                            # target vector,  shape (m,)

# a categorical column becomes several 0/1 columns
X = pd.get_dummies(df, columns=["neighborhood"])  # one-hot

print(X.shape, y.shape)   # (12, 4) (12,)

Splitting a table into X and y

Garbage in, garbage out

A model can only be as good as the table it learns from. Wrong units mixed in one column, a leaked column that secretly contains the answer, mislabeled rows, or a feature with zero signal - none of these are fixed by a bigger network. Data quality (and honest, predictive features) is the single highest-leverage thing you control. The fanciest architecture trained on noise still learns noise.

Check yourself

A single row of the design matrix X is:

The Lot # column barely correlates with price, so it is:

Recall: what do the rows, the columns, and y each represent in a dataset?

Try to state it, then check.

Lock it in

A model never sees a house or an email - it sees a table: rows are examples, columns are features, and one designated column is the label. Features stack into the design matrix $X \in R^{m \times n}$ and the answers into the target vector $y$ .
Feature choice beats model choice: Size correlates tightly with price, while an arbitrary Lot # id sits near zero correlation and carries no signal.
Three feature types: numerical (use as-is), categorical/unordered (one-hot - never number them 1/2/3, which invents a fake order), and ordinal/ordered (mapping to 1/2/3 is fair because the order is real).
Garbage in, garbage out: wrong units, a leaked column, mislabeled rows, or a zero-signal feature are not fixed by a bigger network.

Primary source

Google's Machine Learning Crash Course opens with exactly this framing - examples, features, and labels as the raw material of a model. For hands-on practice wrangling real feature tables, work through the free Kaggle Learn micro-courses on intro ML and feature engineering.

Ask your teacher

Want to see how a leaked column silently inflates accuracy, or how one-hot encoding blows up column count? Ask and I'll build the follow-up. Here's the through-line to keep in mind: features are choices. The word embedding from Lesson 12 is a learned feature representation - the model discovers good features instead of you hand-crafting them. That is precisely why deep learning is so powerful, and it's the idea this whole path builds toward.