Decoding the Brain’s Hand ‘Alphabet’ Could Transform Prosthetics and Brain-Computer Interfaces

Researchers from Carnegie Mellon University and the University of Coimbra have discovered that the human brain organizes hand movements much like an alphabet — combining a small set of simple motions to create a vocabulary of everyday actions. Understanding how people use tools so fluidly could lead to better prosthetics, brain-computer interfaces, robots and new ways to treat problems caused by brain injury.

Seemingly simple actions, like using a corkscrew, writing a letter with a pen or unlocking a door by turning a key, require a complex orchestration of precise movements, the research showed. So, how does the brain do it? It builds these actions in a systematic way.

The researchers found that a part of the brain called the supramarginal gyrus (SMG) acts like an assembly hub. Instead of storing every possible movement separately, it combines simple finger, hand, wrist and arm patterns, called kinematic synergies, to build the full range of actions humans perform.

For example, the way a person holds scissors is similar to how they hold pliers, even though the tools do different things. The SMG recognizes and reuses these similarities to simplify how actions are planned. 

“Just as brain regions supporting language function combine sounds, or phonemes, to form words, the brain also combines kinematic synergies to form complex, object-directed actions,” said Leyla Caglar, lead author of the study. “From this closed set of basic building blocks, the brain constructs the full repertoire of actions that can be performed with the human hand.”

Caglar is currently a postdoctoral fellow at Mount Sinai Medical Center. She led this research effort while she was a postdoctoral fellow, jointly at Carnegie Mellon University’s Department of Psychology(opens in new window) and the University of Coimbra, Portugal.

Implications for robotics, brain-computer interfaces and action deficits caused by brain injury

Understanding how the brain assembles hand movements could pave the way for prosthetics that feel more natural, new treatments for movement problems caused by injury or disease, and improved brain-computer interfaces — systems that let people control devices like prosthetic limbs, wheelchairs or computer cursors with their thoughts. While the idea that the brain uses a combinatorial structure of motor synergies is not new to this study, the new evidence provided by this study may have far-reaching implications for robotics and the development of effective brain-computer interfaces.

“If we can map these synergies directly from neural activity, we could build more efficient brain-machine interfaces that allow users to control prosthetics with greater naturalness, precision and flexibility,” said study coauthor Jorge Almeida, a professor at the University of Coimbra. “This also moves us closer to creating artificial systems capable of acting with agility, efficiency and intelligence comparable to that of humans.”

The discovery might also offer new perspectives on disorders such as apraxia — a neurological condition in which patients can lose the ability to use objects correctly, despite being able to recognize the objects they cannot use.

“Just as a deficit to the ability to properly assemble the sounds of language into words impairs language function, damage to this brain area can make it difficult for people to plan and carry out complex actions with objects,” said Almeida.

Integration of cognition, perception and action

When people use their hands to grasp objects they don’t have to “think” about building up actions out of their elemental parts — just like native speakers of a language don’t have to think about how to say the words they want to use. The processes supported by the SMG are always running automatically in the background, behind what people are thinking about at any given moment.

A key aspect of the system that supports complex hand actions is that its location — about one inch above and behind the left ear — is strategically located in the brain to receive and integrate many different types of information. These include visual, tactile, motor and conceptual information about the world and the status of the body.

“Imagine you reach out and take a sip from the cup on your desk,” said Brad Mahon(opens in new window), study coauthor and a professor in the Department of Psychology at Carnegie Mellon. “Before you start that movement of your hand to grasp the cup, your brain has already computed all kinds of properties about the cup and its contents, including its weight, how slippery it is, where its surface is likely to be hot, its location relative to your hand and how big a sip you plan to take.

“This study shows that the final posture of the hand when grasping an object is built from a vocabulary of basic building blocks and that vocabulary is the same across individuals," said Mahon.

Despite the very different kinds of interactions with objects that different people have, and despite differences in manual dexterity across individuals, all humans have a common neural system that supports complex, object-directed interactions. Similarly, while human infants are not born speaking any particular language, human infants are able to become a native speaker of any language in the world — and all humans have a common neural system that supports language.

“This study moves us one step closer to understanding the fundamental principles of brain organization that make human tool use possible,” said Caglar.  

This research was funded by grants from the National Institutes of Health, Pennsylvania Department of Health and European Research Council.