
Ben Austen describes it:
Human faces are all built pretty much the same: two eyes above a nose that’s above a mouth, the features varying from person to person generally by mere millimeters. So what our brains look for, according to vision scientists, are the outlying features—those characteristics that deviate most from the ideal face we carry around in our heads, the running average of every visage we’ve ever seen. We code each new face we encounter not in absolute terms but in the several ways it differs markedly from the mean.
In other words, to beat what vision scientists call the homogeneity problem, we accentuate what’s most important for recognition and largely ignore what isn’t. Our perception fixates on the upturned nose, rendering it more porcine, the sunken eyes or the fleshy cheeks, making them loom larger. To better identify and remember people, we turn them into caricatures.
(Image: Participants of the New Hampshire debate by Flickr user DonkeyHotey)