Entropy as area: one rectangle per symbol, width its probability and height its surprise, so every area is a contribution and the total is H — rare symbols tall and thin, common ones short and wide. Evidence: the binary entropy curve against Gini impurity; the ladder is bits per symbol from a coin to a byte.
How many yes-or-no questions until you know?
I'm thinking of one of your toys. If you only have two, you need one question: 'Is it the bear?' If you have eight, you need three clever questions. Entropy is how many questions the game needs on average — and if I almost always pick the same toy, you barely need to ask at all.
A game you can always guess has zero entropy — there is nothing left to learn.
hᵢ = pᵢ · log₂(1/pᵢ) H = Σhᵢ
Fair coin
2 outcomes
1.00 BIT
A letter, as written
English text
≈1.3 BITS
DNA base
A C G T
2.00 BITS
Six-sided die
log₂ 6
2.58 BITS
A letter, if uniform
26 equally likely
4.70 BITS
A byte
256 values
8.00 BITS
Linear bits — the ceiling for n equally likely symbols is log₂ n; real text falls far below its alphabet's because letters are predictable