Descriptive statistics and the Gaussian

You cannot look at a million numbers at once, so you summarise them. Where is the cloud centred? How wide is it? What shape does it take? Three answers - the mean (centre), the standard deviation (width), and the Gaussian (the bell shape a great many clouds fall into) - carry almost all the everyday statistics that machine learning leans on. Get these into your fingers and terms like "normalize the features," "unit variance," and "standard normal" stop being jargon and start being pictures.

Mean: the balance point

The mean (or average) $\mu$ of $N$ values is their sum divided by how many there are:

The physical picture is a seesaw. Place each data point as an equal weight along a ruler; the mean is the single spot where the ruler balances. Slide one point far to the right and the balance point drifts right to compensate - which is exactly why the mean is sensitive to outliers. It is the centre of mass of the data.

Spread: variance and standard deviation

Two clouds can share a mean yet look nothing alike - one tight, one sprawling. To capture width, measure how far each point sits from the mean, $x_i-\mu$. Those gaps are positive and negative and would cancel to zero if we just added them, so we square them first, then average. That average squared gap is the variance:

Squaring fixed the sign problem but left the units wrong - if $x$ is in metres, ${\sigma }^2$ is in metres squared. So take the square root to get back into the data's own units. That is the standard deviation $\sigma$, the number you actually quote as "the spread":

The bell curve and the 68-95-99.7 rule

Astonishingly often, when many small independent effects add up, the resulting spread takes one particular smooth shape: the normal distribution, or Gaussian. It is a symmetric bell, tallest at the mean and thinning out on both sides. It is written $\mathcal{N}(\mu ,{\sigma }^2)$ - fully pinned down by just the centre $\mu$ and the width $\sigma$ - and its height at a value $x$ is

Because the bell is fixed once you know $\mu$ and $\sigma$, distances are naturally measured in standard deviations. That gives the famous 68-95-99.7 rule: about 68% of a Gaussian's values fall within $\pm 1\sigma$ of the mean, about 95% within $\pm 2\sigma$, and about 99.7% within $\pm 3\sigma$. Land more than three standard deviations out and you are in genuinely rare territory.

Drag the points below. Watch the mean line, the shaded $\pm 1\sigma$ band, the variance readout, and the fitted bell all respond to the cloud you build - then hit normalize to standardise it.

The four numbers in NumPy:

import numpy as np

x = np.array([4, 6, 7, 8, 9, 10, 11, 12, 14])

mu    = x.mean()                 # balance point  -> 9.0
var   = ((x - mu) ** 2).mean()   # mean squared distance -> 8.67
sigma = np.sqrt(var)             # spread, in x's own units -> 2.94

z = (x - mu) / sigma             # z-scores: recenter, then rescale
# z.mean() -> 0.0     z.std() -> 1.0

Check yourself