Entropy and the second law

A hot coffee cooling to room temperature is so ordinary you never question it. But watch it backwards: a lukewarm coffee spontaneously pulling heat out of the room to reheat itself, the mug growing warm as the air chills. Every physics law you have met so far would allow it - energy would still be conserved. Yet it never happens, not once in the history of the universe. Something beyond energy is choosing time's direction, and it is the deepest idea in this whole course.

Play: watch order dissolve, then try to rewind it

Every particle in this box starts crammed into the left half. Press play. Nothing in the code pulls them rightward - they just bounce and wander - yet they spread to fill the whole box, and the entropy trace underneath climbs and then sits at a ceiling. When it is well mixed, hit "reverse time" and watch what the microscopic laws actually permit.

Two lessons live in that widget. First, spreading is not caused by any force - it happens because there are vastly more ways to be spread out than bunched up, so a wandering system stumbles into spread-out arrangements far more often. Second, reverse-time really does un-mix the gas for a while: the laws are reversible. It only looks impossible because reversing reality would mean flipping every molecule's velocity at the exact same instant, and the faintest error sends it spreading again.

Entropy is a count of arrangements

Here is the move that makes everything click. Separate two ideas. A microstate is the full molecular detail. A macrostate is the blurry big-picture view. The key quantity is the multiplicity $W$: how many microstates look the same at the macro level.

Count it for the box. "All 160 particles on the left" is one very special macrostate - there is essentially one way to arrange it. "Roughly half on each side" can be built an astronomical number of ways, because it does not matter which particles are where. In the widget that count is exactly $\left(\genfrac{}{}{0ex}{}{N}{n_{\text{left}}}\right)$, and it is largest at the even split - the arrangements-W readout jumps from a handful to something like $10^{47}$ as the gas mixes. Entropy is defined as the logarithm of that count:

Why the logarithm? Two reasons, and both will feel familiar if you have met logs before in exponentials and logarithms. It tames those absurd counts into manageable numbers, and it makes entropy add up: put two independent systems together and their arrangement counts multiply, so their logs - their entropies - add. Entropy is the log of possibility.

This is why the coffee never reheats. For it to un-cool, its slow molecules would have to happen to collect all the fast air molecules' energy - one specific, rare arrangement - when there are overwhelmingly more arrangements where the energy stays spread out. And it is why no engine can be perfect: turn heat into work and you are trying to herd spread-out energy back into organized motion, which the counting forbids you from doing completely. The best possible efficiency, the Carnot limit, is $1-T_{\text{cold}}/T_{\text{hot}}$, and it is stated, not derived here - but it falls straight out of entropy bookkeeping.^[2]

The same math runs your compression codec

Now the payoff a programmer waits for. Look again at $S=k\ln W$: entropy is a logarithm of a count of possibilities. In information theory you met a quantity that is the same shape:

These are not cousins; they are the same idea in two costumes. When $W$ arrangements are all equally likely, each has probability $p=1/W$, and Shannon's formula collapses to $H={\log }_{2}W$ - a plain log of the count of possibilities, exactly like Boltzmann's, differing only in the base of the log and the constant out front. Ludwig Boltzmann counted molecular arrangements in the 1870s; Claude Shannon counted message arrangements in 1948 and arrived at the identical mathematics.^[3]

import math

def entropy_bits(probabilities):
    # Shannon entropy H = -sum p log2 p, the average surprise in bits.
    return -sum(p * math.log2(p) for p in probabilities if p > 0)

# For W equally likely arrangements, each has p = 1/W, and the sum collapses
# to a plain log of the count: 4 equal options -> exactly 2 bits.
print(entropy_bits([0.25, 0.25, 0.25, 0.25]))   # 2.0 == log2(4)

# The thermodynamic version is the SAME log of a count, scaled by k:
W = 10**12                                       # microstates in a macrostate
k = 1.380649e-23                                 # Boltzmann constant, J/K
S = k * math.log(W)                              # Boltzmann's S = k ln W
print(S)                                         # joules per kelvin

Check yourself