Welcome back
Last time, we went birdwatching and met convolutional networks. Small filters slide across an image patch by patch, reusing the same weights everywhere, and each one learns to spot a single local clue, such as an edge or a dab of colour.
But spotting an edge is a long way from recognising a bird. Today we’ll see how a network gets from one to the other, and the answer looks a lot like how cathedrals get built.
Brick by brick
Walk around an old cathedral slowly. Look closely and you’ll see that it’s made of simple bricks and blocks of stone. A few bricks form a wall. Walls with an opening form an arch. Arches in a row form an aisle. Aisles, towers and a great vault together form the cathedral.
No single brick knows it’s part of a cathedral. Each level is built only from the level below, and each level is a little bigger and more meaningful than the one before.
Layers upon layers of seeing
A convolutional network stacks its layers in the same way, and each layer’s filters look at the feature maps made by the layer before.
- Early layers find the bricks: edges, corners, spots of colour.
- Middle layers combine those into walls and arches: textures, curves, simple shapes such as a circle or a stripe.
- Later layers combine those into whole parts: an eye, a beak, a feather pattern.
- The final layers put the parts together into a judgement: “That’s a robin.”
This ladder from simple to complex is called a feature hierarchy. Nobody designed it. It emerges on its own during training, because building big ideas out of smaller ones is simply the most efficient way to reduce the loss. Researchers who peek inside trained networks really do find edge detectors near the bottom and something close to “face detectors” near the top.
Stepping back to see the whole
There’s a puzzle, though. A filter only sees a small patch. How does a later layer ever take in something as large as a whole bird?
Part of the answer is stacking: each layer looks at a patch of the previous layer’s map, which itself summarised a patch, so the view widens as you climb. The other part is a simple step called pooling.
Pooling shrinks a feature map by summarising small neighbourhoods. The most common kind, max pooling, looks at each little 2-by-2 square and keeps only the strongest value. It’s like an architect stepping back from the scaffolding. She loses some brick-level detail, but now sees the shape of the whole wall.
Pooling brings three gifts:
- A wider view. After pooling, each step of the next filter covers more of the original image.
- Less to compute. A 2-by-2 pool keeps just one value in four, so the maps get smaller and faster to work with.
- A little forgiveness. If a feature shifts by a pixel or two, the strongest value in its neighbourhood barely changes. The network cares that the beak is there, not its exact pixel.
What to carry with you
Stacked convolutional layers form a feature hierarchy, from edges to shapes to parts to whole objects. Pooling summarises small neighbourhoods so later layers can see more of the picture and forgive small shifts.