The perceptron

Weighted inputs, a bias, a threshold - the smallest unit that can learn to draw one decision line, and the atom every neural network is built from.

Strip a neural network down to a single cell and you get the perceptron: it multiplies each input by a weight, adds them up with a bias, and fires a yes/no depending on whether the total clears a threshold. That's the whole machine. Yet this one tiny unit can learn - nudging its own weights every time it's wrong until it carves the plane into two clean halves. Understand this atom completely and you understand what every layer of every large language model is quietly doing, billions of times over.

One neuron, in math

Give the neuron two inputs, $x_1$ and $x_2$. It holds two weights $w_1,w_2$ and one bias $b$. First it computes a single number - the weighted sum, or pre-activation $z$ - which is just a dot product plus the bias:

A neuron is a line

Here is the geometric heart of it. The neuron flips from "no" to "yes" precisely where $z$ crosses zero. The set of all inputs that sit exactly on that knife-edge - where the neuron is perfectly undecided - is

$\mathbf{w}\cdot \mathbf{x}+b=0$,

and in two dimensions that equation is a straight line (in higher dimensions, a flat hyperplane). The weight vector $\mathbf{w}=(w_1,w_2)$ is the line's normal - it points perpendicular to the line, toward the "yes" side - and the bias $b$ slides the line toward or away from the origin. So the three knobs have crisp meanings: $w_1,w_2$ rotate the boundary; $b$ shifts it. A single neuron can only ever draw one straight cut through the world.

Teaching the line to place itself

The magic is that the neuron doesn't need us to pick good weights - it can find them. Show it a labelled point, let it guess $\hat{y}$, and if it's wrong, shove the weights a little toward getting that point right. Correct guesses are left untouched. That single sentence is the perceptron learning rule:

Drive it yourself. Drag the three sliders to aim the line by hand, then hit Train to let the rule take over: it hunts for a misclassified point (ringed in amber), nudges the weights toward fixing it, and you watch the line pivot. Press Run to animate the whole descent to zero mistakes. Then switch the data to XOR and try again.

This exact failure, published in 1969, froze neural-network research for over a decade. The escape was not a cleverer single neuron but more of them: stack a hidden layer of neurons and each one draws its own line, then a second neuron combines their yes/no answers. Two lines can box in a corner; XOR falls instantly. That's the entire reason deep networks need depth and non-linear activations - the subject of Lessons 20 to 21.

import numpy as np

def step(z):
    return 1 if z >= 0 else 0

w = np.array([-1.0, -1.0])         # weights w1, w2 (a deliberately bad start)
b = 0.0                          # bias
eta = 0.15                       # learning rate

for epoch in range(100):             # sweep the data repeatedly
    mistakes = 0
    for x, y in data:              # x = [x1, x2],  y = 0 or 1
        y_hat = step(w @ x + b)     # z = w·x + b, then threshold
        err = y - y_hat            # +1, 0, or -1
        if err != 0:
            w += eta * err * x     # nudge weights toward this point
            b += eta * err         # and shift the bias
            mistakes += 1
    if mistakes == 0:
        break                      # separated - the line is done

Check yourself