Carnegie Mellon University
Skip navigation and jump directly to page content

Cracking the Code

The Brain's Dictionary

Cracking the Brain's Code

Two hundred years ago, archaeologists used the Rosetta Stone to understand the ancient Egyptian hieroglyphs. Now, a team of Carnegie Mellon University scientists has discovered the beginnings of a neural Rosetta Stone.

By combining brain imaging and machine learning techniques, neuroscientists Marcel Just and Vladimir Cherkassky and computer scientists Tom Mitchell and Sandesh Aryal determined how the brain arranges noun representations. Understanding how the brain codes nouns is important for treating psychiatric and neurological illnesses.
     
"In effect, we discovered how the brain's dictionary is organized," said Just, the D.O. Hebb Professor of Psychology and director of Carnegie Mellon's Center for Cognitive Brain Imaging. "It isn't alphabetical or ordered by the sizes of objects or their colors. It's through the three basic features that the brain uses to define common nouns like 'apartment,' 'hammer' and 'carrot.'"
     
As the researchers report in the journal PLoS One, the three codes or factors concern basic human fundamentals: (1) how you physically interact with the object (e.g. how you hold it, kick it, twist it); (2) how it is related to eating (biting, sipping, tasting, swallowing); and (3) how it is related to shelter or enclosure.

The three factors, each coded in three to five different locations in the brain, were found by a computer algorithm. The algorithm searched for commonalities among brain areas in how participants responded to 60 different nouns describing physical objects. For example, the word 'apartment' evoked high activation in the five areas that code shelter-related words.
     
In the case of hammer, the motor cortex was the brain area activated to code the physical interaction. "To the brain, a key part of the meaning of 'hammer' is how you hold it, and it is the sensory-motor cortex that represents 'hammer holding,'" said Cherkassky, who has a background in both computer science and neuroscience.
     
The research also showed that the noun meanings were coded similarly in all of the participants' brains.

"This result demonstrates that when two people think about the word 'hammer' or 'house,' their brain activation patterns are very similar. But beyond that, our results show that these three discovered brain codes capture key building blocks also shared across people," said Mitchell, head of the Machine Learning Department in the School of Computer Science.
     
This study marked the first time that the thoughts stimulated by words alone were accurately identified using brain imaging, in contrast to earlier studies that used picture stimuli or pictures together with words. The programs were able to identify the thought without benefit of a picture representation in the visual area of the brain, focusing instead on the semantic or conceptual representation of the objects.
           
"With psychiatric and neurological illnesses, the meanings of certain concepts are sometimes distorted," Just said. "These new techniques make it possible to measure those distortions and point toward a way to 'undistort' them. For example, a person with agoraphobia, the fear of open spaces, might have an exaggerated coding of the shelter dimension. A person with autism might have a weaker coding of social contact."
 
The research was funded by grants from the W.M. Keck Foundation and the National Science Foundation.

Related Links: Download the PLoS Article  |  Read the LA Times Blog  |  School of Computer Science  |  Center for Cognitive Brain Imaging


Homepage Story Archives