From Sight to Recognition: Researchers Map How the Brain Processes Faces

Researchers at Carnegie Mellon University are getting closer to understanding how your brain perceives faces and recognizes old friends you haven't seen in years. 

In a study published in the Dec. 26, 2016, issue of the Proceedings of the National Academy of Sciences (PNAS), scientists used highly sophisticated brain imaging tools and computational methods to measure the real-time brain processes that convert the appearance of a face into the recognition of an individual. The research team is hopeful the findings might be used in the near future to locate the exact point at which the visual perception system breaks down in different disorders and injuries, ranging from developmental dyslexia to prosopagnosia, or face blindness.

"Our results provide a step toward understanding the stages of information processing that begin when an image of a face first enters a person's eye and unfolds over the next few hundred milliseconds, until the person is able to recognize the identity of the face," said Mark D. Vida, a postdoctoral research fellow in the Dietrich College of Humanities and Social Sciences' Department of Psychology and Center for the Neural Basis of Cognition (CNBC).

To determine how the brain rapidly distinguishes faces, the researchers scanned the brains of four people using magnetoencephalography (MEG). MEG allowed them to measure ongoing brain activity each millisecond while participants viewed images of 91 different individuals each having two facial expressions: happy and neutral. The participants indicated when they recognized that the same individual's face was repeated, regardless of expression.

The MEG scans allowed the researchers to map out, for each of many points in time, which parts of the brain encode appearance-based information and which encode identity-based information. The team also compared the neural data to behavioral judgments of the face images from humans, whose judgments were based mainly on identity-based information. Then, they validated the results by comparing the neural data to the information present in different parts of a computational simulation of an artificial neural network that was trained to recognize individuals from the same face images.

"Combining the detailed timing information from MEG imaging with computational models of how the visual system works has the potential to provide insight into the real-time brain processes underlying many other abilities beyond face recognition," said David C. Plaut, professor of psychology and a member of the CNBC.

In addition to Vida and Plaut, CMU's Marlene Behrmann and University of Toronto Scarborough's Adrian Nestor participated in the study.

The Natural Sciences and Engineering Research Council, the Pennsylvania Department of Health's Commonwealth Universal Research Enhancement Program and the National Science Foundation funded the research.

Using MEG and computational tools to map how the brain processes faces from sight to recognition is one example of Carnegie Mellon's strengths in combining cutting-edge cognitive neuroscience with big data and analytics. The university's BrainHub initiative, which is designed to leverage these strengths, focuses on how the structure and activity of the brain give rise to complex behaviors.