I can’t simplify the title beyond that, but don’t run away yet, the idea itself is straight forward once the terminology is explained. Skip ahead two paragraphs if you know what domain specificity means.

Recognition of objects in the visual scene is thought to arise in inferior temporal and occipital cortex, along the ventral stream (see also this planned Scholarpedia article on the topic by Ungerleider and Pessoa – might be worth waiting for). That general notion is pretty much where consensus ends, with the issue of how different object categories are represented remaining controversial. Currently, the dominant paradigm is that of Nancy Kanwisher and colleagues, who hold that a number of domain-specific (that is, modular) areas exist, which each deal with the recognition of one particular object category. The most widely accepted among these are the fusiform face area (FFA), the parahippocampal place area (PPA), the occipital face area (OFA), the extrastriate body area (EBA), and the lateral occipital complex (LO), which is a bit of a catch-all region for the recognition of any object category that doesn’t fall into one of the domains with their own area. Usually, the face-selective part of the superior temporal sulcus (STS) is also included.

Typical locations of object areas by category. B is an upside down down, C is flattened

This modular view of the visual recognition has received a lot of criticism. However, the undeniable success of the functional localiser approach to fMRI analysis, in which responses are averaged across all voxels in each of the previously-mentioned areas, has led to widespread acceptance of the approach. Essentially, then, the domain specific account seems to be accepted because recording from a functionally-defined FFA, for instance, seems to yield results that make a lot of sense for face perception.

When you think about it, the domain specific account in itself is a pretty lousy theory of object recognition. It does map object categories onto cortex, but it is considerably more difficult to explain how such a specific representation might be built on input from earlier, non-object specific visual areas. This brings us to today’s paper, which proposes a possible solution (op de Beeck et al, 2008). The bulk of the paper is a review of previous research in this area, so give it a read for that reason if you want to get up to speed. The focus of this post is on the theoretical proposal that op de Beeck et al (2008) make towards the end of the paper, which goes something like this:

Ventral stream areas contain a number of overlapped and aligned topographical maps, where each maps encodes one functional property of the stimulus. Op de Beeck et al (2008) suggest that properties might include shape, functional connectivity, process, and eccentricity. Let’s go through each of those suggestions in turn (the following is based on my own ideas – op de Beeck et al don’t really specify how the topography of these featural maps might work):

A shape map might encode continuous variations of for instance angularity and orientation of parts of the stimulus. So one imaginary neuron in this map might be tuned to a sharp corner presented at an upright orientation (see Pasupathy & Connor, 2002 for an example of such tuning in V4), and topographically, the map might be laid out with angularity and curvature as the x and y dimensions in the simplest case.

Functional connectivity is hard to explain – read the article I just linked if you’re curious, but let’s just call it brain connectivity here. A map of brain connectivity is a topographical layout of connections to other areas – for instance, one part of the map might be more connected to earlier visual areas (such as V4), while another part of the map might connect more with higher-order areas that deal with memory or emotion (e.g., hippocampus, amygdala).

The process map is a tip of the hat to some of Kanwisher’s strongest critics, such as Tarr & Gauthier (2000), who argued that the ventral stream isn’t divided by object category, but by the visual processing that is used. So for example, the FFA is actually an area specialised for expert within-category discrimination of objects (faces or otherwise), which happens to appear face-specific because we have more experience with faces than with other categories. Some parts of the map might deal with such expertise discriminations, while others might deal with more general between-category classification.

Eccentricity is a fancy term for distance from the fixation point (ie, the fovea) in retinal coordinates. If you hold your finger slightly left of your fixation point and continue to move it left, you are increasing the eccentricity of the stimulus. Eccentricity and its complicated partner polarity (visual angle) reflect the two basic large-scale topographical principles in early visual areas, but such maps can be found throughout the visual system.

Incidentally, the eccentricity map is the only of these proposed maps for which there is currently good evidence in this part of the brain (Levy et al, 2001). The part that corresponds to the FFA has a foveal (or central) representation of the visual field, which makes sense considering that we tend to look directly at faces. Conversely, the PPA has a peripheral representation, as might be expected since most of us don’t spend much time fixating on the scenery.

The central proposal is that in an area such as the FFA, the face-specific response is actually the combination of the concurrent, aligned activation of a number of different maps. For example, the FFA might correspond to responses tuned to rounded shapes in the shape map, to input from earlier visual areas in the functional connectivity map, to expert within-category discrimination in the process map, and to a foveal (central) representation in the eccentricity map.

To really get the kind of strong domain-specificity that is observed, these maps must display multiplicative interactions – op de Beeck et al (2008) suggest that if their simultaneous activations were just added to make up the fMRI response, you wouldn’t get the strong selectivity that is observed (so by implication, less strict modularists could do away with the multiplicative bit and get a map that corresponds better to their view of ventral areas).

This is a pretty interesting idea, although wildly speculative. Note that with the exception of eccentricity, there really is very little evidence for this form of organisation. In other words, this theory is a theory not just in the scientific sense, but also in the creationist sense of the word. It definitely is an inspiring source of possible future experiments, however.

References
Levy, I., Hasson, U., Avidan, G., Hendler, T., & Malach, R. (2001). Center-periphery organization of human object areas. Nature Neuroscience, 4, 533-539. DOI: 10.1038/87490

Op de Beeck, H.P., Haushofer, J., Kanwisher, N.G. (2008). Interpreting fMRI data: maps, modules and dimensions. Nature Reviews Neuroscience, 9, 123-135. DOI: 10.1038/nrn2314

Pasupathy, A., & Connor, C.E. (2002) Population coding of shape in area V4. Nature Neuroscience, 5, 1332-1338. Link

Tarr, M.J., & Gauthier, I. (2000). FFA: a flexible fusiform area for subordinate-level visual processing automated by expertise. Nature Neuroscience, 3, 764-769. DOI: 10.1038/77666

That image certainly piques your interest, doesn’t it? Sugita (2008) was interested in addressing one of the ancient debates in face perception: the role of early experience versus innate mechanisms. In a nutshell, some investigators hold that face perception is a hardwired process, others that every apparently special face perception result can be explained by invoking the massive expertise we all possess with faces, compared to other stimuli. Finally, there is some support for a critical period during infancy, where a lack of face exposure produces irreparable face recognition deficits (see for example Le Grand et al, 2004). Unfortunately, save for a few unfortunate children who are born with cataracts, there is no real way to address this question in humans.

Enter the monkeys, and the masked man. Sugita (2008) isolated monkeys soon after birth, and raised them in a face-free environment for 6, 12 or 24 months. After this, the monkeys were exposed to strictly monkey or human faces for an additional month.

At various points during this time, Sugita (2008) tested the monkeys on two tasks that were originally pioneered in developmental psychology as means of studying pre-lingual infants. In the preferential looking paradigm, two items are presented, and the time spent looking at either item in the pair is recorded. The monkeys viewed human faces, monkey faces, and objects, in various combinations. It is assumed that the monkey (or infant) prefers whichever item it looks at more. In the paired-comparison procedure, the monkey is primed with the presentation of a face, after which it views a face pair, where one of the faces is the same as that viewed before. If the monkey views the novel face more, it is inferred that the monkey has recognised the other face as familiar. So the preferential looking paradigm measures preference between categories, while the paired-comparison procedure measures the ability to discriminate items within a category.

Immediately following deprivation, the monkeys showed equal preference for human and monkey faces. By contrast, a group of control monkeys who had not been deprived of face exposure showed a preference for monkey faces. This finding suggests that at the very least, the orthodox hard-wired face perception account is wrong, since the monkeys should then prefer monkey faces even without previous exposure to them.

In the paired-comparison procedure, the control monkeys could discriminate between monkey faces but not human faces. By contrast, the face-deprived monkeys could discriminate between both human and monkey faces. This suggests the possibility of perceptual narrowing (the Wikipedia article on it that I just linked is probably the worst I’ve read – if you know this stuff, please fix it!), that is, a tendency for infants to lose their ability to discriminate between categories which are not distinguished in their environment. The classic example occurs in speech sounds, where infants can initially discriminate phoneme boundaries (e.g., the difference between /bah/ and /pah/ in English) that aren’t used in their own language, although this ability is lost relatively early on in the absence of exposure to those boundaries (Aslin et al, 1981). But if this is what happens, surely the face-deprived monkeys should lose their ability to discriminate non-exposed faces, after exposure to faces of the other species?

Indeed, this is what Sugita (2008) found. When monkeys were tested after one month of exposure to either monkey or human faces, they now preferred the face type that they had been exposed to over the other face type and non-face objects. Likewise, they could now only discriminate between faces from the category they had been exposed to.

Sugita (2008) didn’t stop there. The monkeys were now placed in a general monkey population for a year, where they had plenty of exposure to both monkey and human faces. Even after a year of this, the results were essentially identical as immediately following the month of face experience. This implies that once the monkeys had been tuned to one face type, that developmental door was shut, and no re-tuning occurred. Note that in this case, one month of exposure to one type trumped one year of exposure to both types, which shows that as far as face recognition goes, what comes first seems to matter more than what you get the most of.

Note a little quirk in Sugita’s (2008) results – although the monkeys were face-deprived for durations ranging from 6 to 24 months, these groups did not differ significantly on any measures. In other words, however the perceptual narrowing system works for faces, it seems to be flexible about when it kicks in – it’s not a strictly maturational process that kicks in at a genetically-specified time. This conflicts quite harshly with the cataract studies I discussed above, where human infants seem to lose face processing ability quite permanently when they miss out on face exposure in their first year. One can’t help but wonder if Sugita’s (2008) results could be replicated with cars, houses, or any other object category instead of faces, although this is veering into the old ‘are faces special’ debate… It’s possible that the perceptual narrowing observed here is a general object recognition process, unlike the (supposedly) special mechanism with which human infants learn to recognise faces particularly well.

On the applied side, Sugita (2008) suggests that his study indicates a mechanism for how the other-race effect occurs – that is, the advantage that most people display in recognising people of their own ethnicity. If you’ve only viewed faces of one ethnicity during infancy (e.g., your family), perhaps this effect has less to do with racism or living in a segregated society, and more to do with perceptual narrowing.

References
Sugita, Y. (2008). Face perception in monkeys reared with no exposure to faces. Proceedings of the National Academy of Sciences (USA), 105, 394-398.

I came across this figure in a review by Grill-Spector and Malach (2004). It condenses an already-dense 40-page review into a single figure, so I would have to write a post of similar length to explain it entirely in laymen’s terms. This may be one post to skip if you haven’t the slightest idea of visual perception.

Even if you know your vision, this figure isn’t entirely straightforward. Still, I think it serves as a useful reference for those dense vision papers. With one or two notable exceptions, vision scientists insist on ridiculous naming conventions (the motion sensitive area hMT+ being the case in point), so this might help you remember the plot.

This map is only schematic. However, it represents the rough relationships, as they stood in 2004. The areas are mapped onto the right hemisphere occipital lobe, which has been flattened so that the dark areas represent sulcii (grooves), and the light gyrii. The posterior-anterior axis is sort of bottom-left to top-right, so V1 is (predictably) at the very back of the brain, while the Parahippocampal Place Area (PPA) is on the ventral (bottom) side.Height in this picture represents hierarchy in the processing, as Grill-Spector conceives of it. In other words, the first area is V1, and then we move up the stairs to V2, V3, and so on.

The colours code specialization. In the early areas (V1-V3), this is represented by central versus peripheral mappings, where the cortical magnification factor ensures that the centre is largest, and the highest acuity. Helpfully, they are labelled P-D (down) for the superior end of each map, and P-U (up) for the ventral end (your retinal image of the world is upside down, and apparently the visual system has no need to reverse this representation in later areas).

In later areas, the areas are filled in with one colour, presumably for simplicity – Grill-Spector actually believes that these have some retinotopic organisation as well. The colours still reflect specialization though – we can see that areas such as the Fusiform Face Area (FFA) and the Lateral Occipital complex (LO) are based on central, high-acuity representations, while other areas such as the PPA are based on more peripheral, lower-resolution representations. The letters that are strewn over the areas are meant to approximate locations of sensitivity to certain object categories: places (Pl), objects (O), and faces (F).

Do note that the Superior Temporal Sulcus (STS) is treated as somewhat of a black sheep, placed out in the corner with no colouring or height. This is probably because it is relatively poorly understood. The STS responds to biological motion, such a Johansson figures (see a demo), but its activation also appears to be strongly modulated by the social significance of the stimulus. For instance, Pelphrey et al (2005) found that the STS response in normal controls was greater when a face looked away from an obvious object rather than when gaze was directed towards it, which suggests that the STS does more than merely detect biological motion. Interestingly, people with Autism failed to show the same modulation by expectation in the STS.

The poor understanding of the STS is in part because it responds so specifically to biological motion, which makes conventional retinotopy techniques impossible. Also, I suspect there is a deep-rooted fear in some vision scientists of anything that starts with “Social.”

Another thing to note is the chasm between the last V area and the STS. Presumably, the intermittent areas are also involved in vision, but we don’t know much about what they do yet.

References
Grill-Spector, K, and Malach, R. (2004). The Human Visual Cortex. Annual Review of Neuroscience, 27, 649-677.

Pelphrey, K.A., Morris, J.P., and McCarthy, G. (2005). Neural Basis of Eye Gaze Processing Deficits in Autism. Brain, 128, 1038-1048.

As a continuation of the recent post on audiophiles, let’s look closer at how good we are detecting the compression in digital music formats.

Most music formats, such as MP3 or the AAC format used by iTunes, define the rate of compression as the number of bits that is used to encode each second of music. The standard bitrate, as used by the iTunes Music Store and elsewhere, is 128 kbit/s. Music geeks (myself included) tend to use slightly higher bitrates, while the proper audiophiles use lossless formats that compress the file without actually removing any information. Recently, Radiohead released their new album as a free download, only to experience some fan backlash for their choice of a 160 kbit/s bitrate. Critics bemoaned the fact that this was half as much as the 320 kbit/s rate that is used on the mp3s available for purchase on their website. By comparison, the bitrate of a normal audio CD is approximately 1411 kbit/s, so clearly a lot of information is removed.

But can you tell the difference? I dug out a few non-peer-reviewed sources to get an idea – if someone knows of peer-reviewed studies into this, I’d be interested to hear about them. The most serious source is probably this 1998 report from the international organisation for standardisation (PDF), which reports some evidence that participants could distinguish 128 kbit/s compression from the original, uncompressed source. Unfortunately, no tests were made above 128 kbit/s. More recent, but less rigorous tests have been reported by Maximum PC and PC World.

Maximum PC elected to report their results participant-by-participant, and with a sample size of 4, maybe that’s just as well. There isn’t enough data reported in this article to actually run a binomial or another significance test, but the overall conclusion seems to be that none of the testers did well at distinguishing 160 kbit/s from the original source.

PC world’s test actually contains some descriptives, and used a sample size of 30. However, they used some fairly obscure ways of reporting their results. Clearly, in a case like this one, the optimal method is to ask the participants to guess which file is the mp3 and which is the cd, and run a number of trials without feedback. With this approach, you can easily assess whether performance is over the level of chance (50%) for each bitrate. With this in mind, here are their results:

The percentages represent the proportion of listeners who “felt they couldn’t tell the difference” – once again, this measure is far from ideal. While we have no idea which of these differences are significant, the trend is that the differences in ratings flatten off: there appears to be no difference in quality between 192 kbit/s and 256 kbit/s, and in the case of MP3s, no real difference between 128 and 192.

These studies aren’t exactly hard science, they do seem to indicate that those complaining about Radiohead’s 160 kbit/s bitrate wouldn’t necessarily be able to distinguish it from CD quality, let alone a 256 kbit/s mp3. This illustrates the human tendency to overestimate our own perceptual ability – if we know that two things are different, we will find differences, imagined or otherwise. Blind testing is the only way to establish whether a genuine difference in sound quality exists, yet, this is very rarely done.

If you want to test your own ears, try these examples. With the above in mind, it would be best to get a friend to operate the playback, so that you can’t tell from the outset which file is which. If you run a large number of trials, you can also look up whether your performance is above chance in this Binomial probability table. In psychology, .05 is the commonly accepted p value, so as an example you would need to get 15 out of 20 trials correct for your performance to be significantly better than chance at this level.

Update: Dave over at Cognitive Daily has answered my prayers by carrying out a nicely designed test of performance at discriminating different bitrates. In a nutshell, his results confirm the ones reported here – Although there participants rated the 64 kbit/s tracks as significantly poorer in quality, no differences appeared between 128 and 256 kbit/s. Read the complete write-up here.

The other week Gizmodo posted an amusing rant about a set of $7250 speaker cables, and the gushing review they received. Among other things, the reviewer referred to the cables as “danceable.” James Randi soon popped around to offer his $1 million prize to the cable company, if they could prove that their cables outperform “normal” Monster cables in a double-blind test.

This is actually an issue of the limits of human perception. Is it really possible to tell the difference between normal high-end equipment, and equipment that veers into the audiophile range? It’s clear that according to many audiophiles, the answer is going to be yes. Wikipedia informs us that there are actually two schools among audio enthusiasts: the objectivist school, which favours double-blind testing, and subjectivists, who favour a more philosophical approach. The review that caused so much ire comes from Positive Feedback, an online magazine that concerns itself with the “audio arts” – guess which school they subscribe to.

Among subjectivist audiophiles, there is a belief that almost any change to the stereo setup results in a perceptible difference in sound. This results in bizarre behaviours, as in this picture from the Positive Feedback staff page:

Note how the speaker cables are carefully propped up on stilts to keep them off the floor, and how what looks like power amplifiers are propped up on massive slabs of wood (I can assure you those didn’t come from the local lumberyard). Another nice example comes from an article in Hi-Fi magazine Masters on Video and Audio:

“The [product] tightened up the sounds of a wide variety of equipment, the improvements often most noticeable in the bass. Imaging and focus usually improved, as did the interstitial quiet, which raised the level of overall palpability, air, and transparency.”

The product? Shelves.

There is one obvious objection to raise here: judging by the pictures of the reviewers on sites such as Positive Feedback, most of them are in their 40’s and beyond. As the following figure shows, this spells trouble:

I grabbed this from a lecture handout, so unfortunately I don’t know the source. The lines plot performance at detecting sounds over age, with each line representing a frequency. In this case “hearing level” is a standardised measure where normal hearing is at 0 dB. The clear pattern is that the higher frequencies disappear with age. This figure only goes up to 6 khz, but it’s worth noting that the human ear can hear up to 20 khz, and that the loss is more dramatic the higher up you go.

In other words, it’s not really worth trusting an audio reviewer who is older than you are, because there is a range of higher frequencies that you can hear while they cannot.

Apart from the overall lack of evidence and the sheer physical implausibility of some of the products, there is some classic research in social psychology that have implications for this topic.

Cognitive dissonance theory was primarily developed by Festinger. Briefly, the idea is that when the individual finds himself in a state where internal beliefs conflict with reality, there is dissonance, which is an unpleasant state. The individual may then employ a number of mechanisms to get around the dissonance, ranging from simply acknowledging that the beliefs were wrong to attacking the reality of external events, or devaluing the conflict.

The classic cognitive dissonance study is one where students perform a dull experiment, and are then paid a small or a large amount of money for telling the next participant that the experiment is actually fun (Festinger & Carlsmith, 1959). Surprisingly, students who are paid less actually rate the dull task as more interesting. In this case, the student finds himself (all males in Psychology studies in those days, generally) in a conflict: he has just done a boring experiment and lied to a fellow student for a very small reward. According to Festinger and Carlsmith, the student then reduces dissonance by re-evaluating the task. If the task was actually fun, then there is no dissonance between the student’s actions and beliefs.

The implication for consumer behaviour is that when your green $7250 cables arrive in the mail and you plug them in, finding that they do nothing would result in unacceptable dissonance. In fact, cognitive dissonance theory predicts that the more you pay for the cables, the more inclined you will be to conclude that they sound good, regardless of the actual quality of the cables. In this context, it is worth noting that the Positive Feedback website states that their policy is that reviewers should own the equipment they review, which is a very unusual policy in light of cognitive dissonance theory.

There is another classic social psychology study that is relevant here: Sherif’s investigation of the autokinetic effect (1935). To observe this effect, place yourself in an absolutely dark room with a single, faint light source. The spot of light will appear to move around as a result of small eye movements that your brain normally filters out. Sherif’s participants didn’t know about this however, so they really thought the light moved.

When the participants rated when the light moved individually, there was considerable variation between the participants in how far the light moved. Sherif then placed participants in groups and asked them to call out the movements of the light. Now, there was a convergence effect, so that the estimates of the different participants came closer to each other, and remained close in subsequent individual re-tests.

If you and your friend are listening to a new stereo and she mentions that the low bass sounds a bit flat, you are going to hear it too. The sound itself is ambiguous, not to mention the terminology that audiophiles use, so Sherif’s study suggests that in such situations, you will align with the group. You can imagine that this tendency to conform is quite useful in many real-life contexts, but it does mean that wine sampling and stereo testing are unlikely to reflect anything other than your tendency toward conformity. That doesn’t mean it can’t be fun, of course.

You can test this out yourself if you ever find yourself at a wine sampling. Make up associations: say the wine tastes like blackcurrant (always a a winner), sandal wood, tobacco, myrrh. As long as your ideas aren’t too far off, you will find that others suddenly experience the taste too.

While our senses are rather limited, our ability to fool ourselves is almost endless.

References
Festinger, L., and Carlsmith, J.M. (1959). Cognitive Consequences of Forced Compliance. Journal of Abnormal and Social Psychology, 58, 203-210.

Sherif, M. (1935). A study of some social factors in perception. Archives of Psychology, 27.

A while back I blogged a paper by Thierry et al (2007 – see also the Neurocritic’s post). Some controversy is brewing about the paper now, so I thought I’d offer an update.

To recap: the Thierry et al (2007a) paper is interesting because it challenges the notion that a specific component of the EEG waveform called the N170 (since it is negative, and occurs at 170 ms) is specific to faces. Thierry et al found that the N170 disappeared when they controlled for inter-stimulus perceptual variance (ISPV), that is, the fact that faces tend to be presented in portraits, while other stimuli are often shown at various angles and sizes.

As the comments on the Neurocritic’s post suggest, some investigators were not entirely convinced by Thierry et al’s (2007a) demonstration. Now one of the critical comments has made its way into the latest issue of Nature Neuroscience, along with a reply from Thierry et al (2007b).

Bentin et al (2007) point to previous research that shows how controlling for ISPV does not in fact explain the N170 specificity to faces. They’ve packed a bit too much information into this figure than necessary, but it’s worth a look:

The black line indicates the mean pixel-wise correlations within each stimulus group, that is, which is an indicator of how much ISPV there was. The grey bars show the N170 amplitude. If Thierry et al (2007) is correct in their argument that the N170 reflects ISPV rather than face specificity, you would expect the black line (indicating ISPV) and the grey bars (indicating N170 amplitude) to match up reasonably well. This isn’t really happening. To make this point even clearer, they show in the supplements that Thierry et al’s (2007) own data seems to indicate the same thing. Bentin et al (2007) also take Thierry et al (2007a) to task for failing to note how their conceptualisation of the N170 contradicts a bulk of well-known effects in previous literature.

So how could Thierry et al (2007a) get such contradictory results? Bentin et al (2007) suggest that one reason may be Thierry et al’s (2007) choice of EEG recording sites, which differ from those generally used by other investigators.

In their reply, Thierry et al (2007b) argue that, among other things, the recording site explanation doesn’t hold, since the same pattern of results appeared across all electrodes. They also contend Bentin et al’s (2007) notion that there is a generally agreed standard for electrode selection.

As for the lack of similarity between the pixel-wise correlations and the N170, this is explained (if you can call it that) by arguing that pixel-wise correlations may not be a perfect measure of ISPV – individual pixels may have very different effects depending on their location, which is not taken into account with such an analysis.

Confused yet? I don’t think there are any clear answers at this point. At the most basic level, the findings of Thierry et al (2007a) contradict previous findings, something that they appear to have failed to mention themselves. There is also some disagreement over what type of analysis is appropriate for this kind of research. Personally, I would like to see an independent replication of the results before I make any further attempts to understand what’s going on.

References
Bentin, S., Taylor, M.J., Rousselet, G.A., Itier, R.J., Caldara, R., Schyns, P.G., Jacques, C., and Rossion, B. (2007). Controlling interstimulus perceptual variance does not abolish N170 face sensitivity. Nature Neuroscience, 10, 801-802.

Thierry, G., Martin, C.D., Downing, P., & Pegna, A.J. (2007a). Controlling for Interstimulus Perceptual Variance Abolishes N170 Face Selectivity. Nature Neuroscience, 10, 505-511.

Thierry, G., Martin, C.D., Downing, P., & Pegna, A.J. (2007b). Is the N170 sensitive to the human face or to several intertwined perceptual and conceptual factors? Nature Neuroscience, 10, 802-803.

Google has somewhat secretly rolled out a special filter for its image searches, that enables you to restrict your searches to faces. Geeks in the know say that this technology comes from their purchase of Neven Vision, a company I’ve never heard of before.

To use this technology, simply add &imgtype=face to the search URL – adding it in the search box won’t work (although Quicksilver users will find that it does work if you use the Google search plugin – I have no idea why).

Face detection algorithms are all the rage in computer science, and there are lots of implementations out there. I thought I’d compare Google to the competition.

An initial test suggests that Google does rather well. Compare a search for “house” with the image filter and without it. There are only faces (including a certain british-comedian turned american-doctor) in the former, and only houses in the latter.

However, it turns out that it’s quite easy to trip Google up once you throw faces of non-human animals at it. According to Google, this is a face:

But it’s not. So how about the competition? There are many online demos of face detection algorithms, but I quite like PittPatt by the Face Group at Carnegie Mellon. You can just plug in a URL rather than having to upload anything, and it’s quite fast. Pittpatt puts rectangles over each part of the image where it thinks a face is. Nothing lights up for the poor dog above.

Let’s make things more difficult for Pittpatt. Gorillas look more like people, right? Google produces mostly faces, but also this:

Ah, Pittpatt seems confused:

Apparently there are a few facial features in there, but I think the blue colour of the frame indicates uncertainty – from the gallery of uploaded pictures on Pittpatt, it seems the most obvious faces are framed in green or at times yellow.

How about chimps, then? Google fails this one spectacularly – all the faces on the first page are non-human, except for a bearded guy and a few pictures of George W. Bush. Go figure. To be fair, this probably has less to do with Dubya’s chimp-like features and more to do with Google’s page rank system, where a picture that is linked a number of times with the word “chimp” nearby ends up on page one of the results for that term. So Bush doesn’t necessarily look like a chimp, it’s just that the Internet thinks he does.

Google thinks this is a face. And you might be inclined to agree, but for the sake of argument I’m assuming that Google wants their algorithm to pick out human faces, not general primate or mammal faces. Pittpatt produces only another weak blue cube, this time restricted to the mouth of the chimp.

So what can we conclude here? This test is of course terribly unfair – I’ve only picked faces that Google failed so at best, Google could only have done as badly as Pittpatt. But it seems like Google’s face detection algorithm isn’t all that great compared to the alternatives. Another possibility is that they’re just trying not to be specist, but I somewhat doubt that. Also, we have learned that a fake statue of King Kong apparently looks more human than a fat dog or a young chimp, according to Pittpatt. I’m not sure what to do with that information.

In any case, it is pretty fascinating that computers can get this good at face detection. While primate faces can look like a human face to us, there is no way you would confuse the two. I guess you might interpret the ambiguous output of Pittpatt as something similar – there is something face-like there, but it’s not quite it.

Update: Other geeks inform me that the face detection algorithm that Google uses is probably trained by either of their two online games Peekaboom or Image Labeler. I couldn’t figure out how Peekaboom works without registering, but the point of the image labeler game is basically for both players to give the same name to a picture. With a training procedure like that, you suddenly start to see how the chimp above might appear under a “face” search – while “chimp” might be the first word you try there, then possibly “ape,” “face” isn’t going to be far behind. This actually points to a problem of human-trained computers – they acquire all our idiosyncrasies and imperfections, for better and for worse.

In this video, James Haxby talks about how a multivariate techique called pattern classification can be used to extract more data from fMRI than was previously thought possible.

Haxby’s research has an interesting angle – while most cognitive neuroscientists are going for strict modularity by creating conditions with contrasts that show how individual areas contribute to a given process (for instance, the fusiform face area), Haxby looks at distributed representations. The central idea is that information can be coded in a distributed pattern of activation, so for example, face processing is not exclusively localised to the FFA, but instead occurs as the result of a distributed pattern of activation that includes but is not limited to the FFA. This was the result reported in Haxby et al’s 2001 paper in Science. Haxby et al were able to show, among other things, that they could predict from the patterns of nonmaximal responses (ie, excluding the FFA and areas that responded maximally to objects) what type of stimulus the participant was looking at. So clearly, the brain may use the information that is being conveyed here, in the areas that aren’t conventionally considered to be important for faces or objects.

As I understand it, the key difference between this technique and the classic subtraction method is as follows:

In typical fMRI subtraction analysis, you look at which areas are uniquely involved in task A but not task B. For instance, if you subtract the activation when the participant views houses from the activation when they view faces, the FFA emerges, while the reverse calculation produces an area known as the Parahippocampal Place Area. Thus, you get the tidy, isolated spots of activation that are so typical of fMRI results – but in reality, the activation that is produced by faces goes far beyond the FFA. So essentially, the subtraction technique discards the activation that doesn’t contribute uniquely to faces but not places, to continue this example.

In pattern matching, you instead carry out statistical tests to see if the pattern of activation when viewing faces is different from the pattern of activation when viewing objects. This makes the neural activation patterns far more difficult to interpret with the naked eye, but if Haxby and other critics of strict modularity are correct in arguing that information is coded in distributed representations rather than in separate modules, this is pretty much the only way of getting at those representations.

The talk is a little bit technical, but I think anyone with more than a passing interest in neuroscience will find it worthwhile to spend half an hour watching it.

UPDATE: I wrote this post at a much earlier point in my studies, and it’s now clear that the comparison between different analyses is a bit inaccurate. The video is still worth watching, so I’ve kept this post on file, but be aware that the post itself may be way off. See this review by Norman et al (2006) for a more authoritative take.

References
Haxby, J.V., Gobbini, M.I., Furey, M.L., Ishai, A., Schouten, J.L., and Pietrini, P. (2001). Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex. Science, 293, 2425-2430.

James Haxby lecture: “Implications of decoding for theories of neural representation“

You might suppose that processing faces and facial expressions should be done in the same place – a face centre, perhaps the conveniently named fusiform face area? On the other hand, recognising a face should reasonably require different types of processing than recognising an expression – while recognising an individual face calls on constant features and fine detail, recognising an expression requires an ability to process motion quickly, and not necessarily with as much detail.

Vuilleumier et al (2003) capitalised on the fact that the situation I just described sounds a lot like the broad split in processing that occurs early in the visual system, in the lateral geniculate nucleus (LGN). In order to understand this paper, you may need a crash course on the basic visual system – skip ahead 4 paragraphs if you’re comfortable with the geniculostriate and retinotectal pathways. You may also want to look at a previous post on the LGN, where I also tried to explain what it’s all about.

The LGN is located close to the centre of each hemisphere, and serves as a relay station for visual input. It receives input from the retina (via the optic chiasm), and sends this signal on to the primary visual cortex. The LGN is laid out in a retinotopic fashion, meaning that a circle that falls on the retina should result in a circle of firing neurons in the LGN. Most important to the topic at hand, the LGN is divided into 6 layers, producing what essentially is 6 maps of the retina on top of one another. As an aside, smaller koniocellular neurons have been discovered between each layer, though most textbooks still pretend that these cells do not represent yet another 6 layers.

The first two layers (counting from the bottom in the figure above) are known as magnocellular. The cells here are not so good for detail or colour, but they respond well to motion. The remaining 4 layers are parvocellular. The cells in these layers have the converse characteristics: they are sensitive to detail and colour, but not to motion.

Unfortunately, it gets even more complicated. It turns out that there is yet another visual pathway, which bypasses the LGN and primary visual cortex altogether. The retinotectal pathway is believed to project from the retina and on, via the superior colliculus and the pulvinar. This pathway appears to have much the same characteristics as the magnocellular pathway.

So to summarise: some retinal ganglion cells are wired to cells in the magnocellular layers, others to the parvocellular layers, and yet others to the retinotectal pathway. The different layers of the LGN each project on separately to the primary visual cortex, where this tidy distinction is quickly blurred (to the dismay of anyone who was hoping for a tidy explanation of the visual system). There is also a separate retinotectal pathway, which is believed to have the same characteristics as the magnocellular layers of the LGN.

Still with me? We are now at a point where we can get to the topic at hand. To test the relative contributions of these various pathways to the perception of faces and facial expressions, Vuilleumier et al (2003) generated sets of faces that were altered to be preferentially processed by one of the pathways. Examples of the faces are below.

By taking out all low spatial frequency information from the faces, you end up with the pictures in the middle column below. These faces would be detected easily by the parvocellular pathway, but not by the magnocellular or the retinotectal pathways. Conversely, the low spatial frequency faces in the right column contain less information than the normal faces in the left column – and this is entirely the parvocellular pathway’s loss. The magnocellular and retinotectal pathways are presumably coded in such coarse resolution that they cannot discriminate between the faces in the left and right columns.

Vuilleumier et al (2003) used fMRI to investigate the neural responses to the three types of faces (normal, high spatial frequency, low spatial frequency), which were either neutral or fearful. The participants were asked to judge the gender of the faces, but this was mainly to keep their attention on the faces, which were presented in random order. The researcher hypothesised that areas in the ventral visual cortex (ie the fusiform face area) would be sensitive to the high spatial frequency faces, while the amygdala would be sensitive to the low spatial frequency faces.

This latter hypothesis is based on the idea that there is a kind of shortcut in the brain to enable potentially dangerous stimuli such as fearful faces to get to the amygdala as quickly as possible, where the fear response is produced that then guides the response (fight/flee etc).

Vuilleumier et al (2003) found ample support for both hypotheses – across neutral and fearful expressions, the fusiform face area responded more strongly to the high spatial frequency faces than the low spatial frequency faces, while the opposite was true of the amygdala.

To further the evidence for low spatial frequency coding in the amygdala, Vuilleumier et al (2003) also showed that the amygdala only responded to fearful faces (as compared to neutral faces) when the faces were presented at low spatial frequency. No significant difference appeared for the high spatial frequency comparison between fearful and neutral faces. Better yet, the same activation patterns appeared for a part of the thalamus corresponding to the pulvinar and the superior colliculus, which, as you may recall, was hypothesised to be part of the retinotectal pathway that enables the amygdala response. The figure below gives the mean activity across the conditions for the amygdala (d), and the pulvinar-colliculus area (e).

I think this study is a nice example of how vision research can be used to generate hypotheses for affective neuroscience. While this study is by no means conclusive (there were a few other brain activation patterns that are not so easily interpreted, which I have spared you), it does provide rather good evidence of a coarsely-coded quick shortcut in the visual system, which may enable fast amygdala-mediated fear responses.

While the idea of a continued strict division between the magno- and parvocellular pathways in “higher” visual areas has been abandoned (it turns out that even a recognition-based area like the fusiform face area receives some input from the magnocellular pathway as well), it is interesting to think that an amygdala with blurry visual input could explain why we are so easily scared of sudden movements. Before your higher-resolution, geniculostriate-mediated visual system has determined that it’s actually just clothes blowing in the wind on a laundry line, your retinotectal amygdala shortcut has already produced quite a fright based on its own, low-resolution input.

References
Vuilleumier, P., Armony, J.L., Driver, J., & Dolan, R.J. (2003). Distinct Spatial Frequency Sensitivities for processing Faces and Emotional Expressions. Nature Neuroscience, 6, 624-631.

A recent paper in PNAS by Pezaris and Reed (2007) outlines a potential way of providing artificial vision to the blind. This in itself is nothing new – other teams are working on producing artificial input to the visual system via the retina or by stimulating the primary visual cortex directly. The novel aspect of this paper is that Pezaris and Reed (2007) are probing the lateral geniculate nucleus (LGN) instead.

The LGN (pictured in a macaque monkey above, courtesy of the wonderful BrainMaps) serves as somewhat of a relay point, half-way between the retina and the primary visual cortex. Conveniently, it has a retinotopic organisation, meaning that the receptive areas of the neurons in this nucleus match the layout of the retina – so a straight line across the retina would be represented by a straight line (more or less!) of firing neurons in the LGN. While the primary visual cortex and even higher-level areas have the same organisation, the neurons in these areas handle more complex processing. As early as the primary visual cortex, neurons respond preferentially to specific angles of orientation, which may make it challenging to provide useful artificial input. In the LGN, simple blobs still elicit solid responses, far closer to the pixels that a camera could provide as input.

As an experiment, the Pezaris and Reed (2007) design is simple. Macaque monkeys were trained to make saccades (fast orienting eye movements) towards spots of light. The experimenters inserted electrodes into the LGN, and used a clever technique to pinpoint the location that their electrode corresponded to in the monkey’s visual field. The normal light-orienting trials were mixed with trials where a small current was administered into the electrode, stimulating neurons in the LGN. The monkeys produced the same gaze-orienting response to this stimulation, even though they viewed no spot of light whatsoever. The figure below shows the response patterns.

This result could be interpreted as a simple reflex – we know that primates will orient their eyes automatically to an unexpected stimulus, if it is salient. Such orienting responses may be mediated by the retinotectal pathway, which connects the retina to the superior colliculus and on to the dorsal visual stream, without passing the LGN. However, Pezaris and Reed (2007) could show that this was no orienting reflex in a second task, where the monkey was flashed two lights, and trained to look at one followed by the next. The monkeys could carry out this task just fine even when the second stimulus was actually electrical rather than visual.

It’s worth noting that since the monkeys responded to the electrical stimulus the way that they had been trained to respond to the visual stimulus, the two types of stimuli must have been perceived as fairly similar – stimulus generalization only goes so far. While we can’t answer this question until we get trials on humans, it’s tempting to try to imagine what the monkeys might have seen. Given what is known about processing in the LGN, it was probably just a highly specific blob of light, similar to the afterimage you can get after staring at a light bulb.

Another interesting thought is what the monkeys may have perceived when they moved their eyes. Unfortunately, this experiment used a 80-200 ms stimulus presentation, meaning that the monkeys simply didn’t have time to move their eyes until the stimulus was gone. Given Helmholz’ Outflow Theory, this is probably a good thing, as we will see next!

Essentially, Helmholz tried to understand why the world doesn’t seem to move when you move your eyes. Objectively, the images that fall on your retina as you move your eyes are the same that would be produced by the entire world shifting slightly to the side. Yet, your perceptual experience is quite different – try poking at the side of your eye and note how different it feels when your eye moves this way. Helmholz assumed that when the brain sends a signal to the eye muscle to move, it also sends a copy of that signal to a comparitor, which compares the copy of the movement order with the perceived retinal movements. As long as the movement signal copy and the retinal movement matches, the comparitor cancels the perceived motion, making the world seem stable. When you poke at your eye, there is no movement signal from the brain, so the comparitor can only conclude that the retinal image is moving because the world is moving. While Helmholz first outlined this theory back in the 19th century, it has been largely confirmed by subsequent research.

So what does this mean for our monkeys, had we given them more time to view the stimuli? Well, the monkeys are moving their eyes, but there is no change in the retinal image – the same one spot of the LGN is being stimulated, no matter where you look. So the comparitor gets a movement signal from the eye, but no retinal movement signal. The outcome of this, according to Outflow Theory, is that every time the poor monkey shifts his gaze, it will appear as if the entire world moved, just like when you poke your eye.

But how do we know this is what happens? Rather good evidence that it does comes from experiments where (unfortunate) participants have their eyes rendered immobile. This can be done by various means, and all of them are unpleasant. One favourite is by injection of Curare, which essentially produces temporary paralysis. When the paralysed participant tries to move their eyes, the experience is one of the world moving, although neither their eyes nor the retinal image moves (e.g., Matin et al, 1982). Presumably, the movement order copy is still sent to the comparitor, which concludes that since your eyes are (supposedly) moving but the world isn’t, this means the world is moving. As an aside, the effect isn’t as strong under daylight conditions when movement can be judged relative to the background, which suggests that the visual system is doing something a bit more complex than Helmholz’s comparator.

As a technical demonstration, the Pezaris and Reed (2007) paper is quite impressive. However, it’s undeniably true that some hurdles remain before anything like this will help the blind. As outlined above, a device using this technology would likely appear disturbing and even nauseating to users initially, as every eye movement causes the world to shake around violently. Perhaps it would be possible to learn to avoid eye movements, but as was mentioned above, the orienting response in primates is quite automatic and is unlikely to disappear as a result of training. Also, Pezaris and Reed (2007) don’t make much of the fact that both these input points are more or less available to direct probing, being covered at most by bone. The LGN, on the other hand, is pretty much in the middle of the brain. While single electrodes can be passed down through the cortex without causing damage (as was done in this study), you soon run into problems when you want to achieve something better than a few blobs of light. To achieve a modest 640*480 pixels resolution, you would need 307 200 input points, ie, electrodes. While I’m no neurobiologist, running such an amount of wire through the brain seems like a challenge.

On the other hand, this line of research could lead to important insights into the workings of the retinotectal pathway. As was mentioned above, this pathway goes from the retina, on to the superior colliculus, terminating in the dorsal stream, which codes spatial localisation or visually guided action (depending on if you ask Ungerleider and Mishkin or Milner and Goodale, respectively). By electrode stimulation in the LGN, you could essentially present stimuli selectively to the geniculostriate pathway (LGN to primary visual cortex, and on). The dorsal stream appears to receive input from both the retinotectal and the geniculostriate pathway, so by looking at performance differences when the stimulation is visual versus electronic, you could start to tease apart the relative contributions of these two pathways to dorsal-stream processing.

References
Matin, L., Picoult, E., Stevens, J.K., Edwards, M.W., Young, D., & MacArthur, R. (1982). Oculoparalytic illusion: visual-field dependent spatial mislocalizations by humans partially paralyzed with curare. Science, 216, 198-201.

Pezaris, J.S., & Reed, R.C. (2007). Demonstration of artificial visual percepts generated through thalamic microstimulation. Proceedings of the National Academy of Sciences of the United States of America, 104, 7670-7675.