Setting the SCENE for Neuroscience Breakthroughs

Carnegie Mellon University and University of Pittsburgh researchers are working to understand how the brain turns perception into action, paving the way for smarter machines, improved brain-computer interfaces, and new insights into how people move through the world.

The new $80 million Simons Collaboration on Ecological Neuroscience  (SCENE) is a 10-year initiative that brings together neuroscience and machine learning to study how brains interpret their environments and decide what to do next. 

Among the six interdisciplinary teams selected for this initiative are Xaq Pitkow, associate professor at Carnegie Mellon’s Neuroscience Institute, and Aaron Batista, professor of bioengineering at the University of Pittsburgh Swanson School of Engineering. 

SCENE builds on principles from ecological psychology, which posit that one of the brain’s core functions is to encode affordances. An affordance is an opportunity for action available in an environment — for example, a chair affords the opportunity to sit, and a perch affords the opportunity for a bat to land. By encoding affordances, the brain closely links perception with action in an efficient manner. Identifying how the brain encodes and uses this information will bridge gaps in our understanding of the neural basis of cognition.

“This project aims to bring in modern neuroscience and computational methods to move beyond the laboratory for understanding perception and action in natural environments,” Batista said. His team will focus on collecting neural data that captures cognitive performance during tasks requiring precision and skill.

 By combining Batista’s findings with additional recorded data from other brain regions involved in perception, like the visual system and prefrontal cortex, the team will develop a whole-brain dataset. A team of theorists and data scientists, including Pitkow, will use artificial systems and machine learning to study this dataset and mathematically describe how the brain represents affordances and prompts action. 

 “Our team is using a mathematical framework known as the Partially Observable Markov Decision Process (POMDP) to investigate what the brain cares about when making these decisions, how it encodes useful information to select an action or behavior, and how the way that brains make choices relate to each other across tasks, brain areas, and even species,” Pitkow said. 

 The multi-institutional collaboration begins July 1 and will provide over $8 million per year across the six teams of researchers. Simons Neuroscience Collaborations are designed to span 10 years, enabling scientists to conduct large-scale longitudinal studies that typically aren’t feasible under conventional grants.

 “This is an exciting opportunity because the time scale of this grant allows for extensive progress not usually possible,” Pitkow said. “The team we have is really fun to work with, and with such a rich depth of expertise we can cover a wide range of things like brain computer interface and augmented reality experiments.”

 The Simons Foundation is dedicated to supporting research in neuroscience and fostering collaboration among teams with diverse expertise and a shared commitment to scientific progress. SCENE follows on their successful funding model, adding to their ongoing neuroscience programs including the Simons Collaboration on the Global Brain and the Simons Collaboration on Plasticity and the Aging Brain. Like those previous endeavors, SCENE will bring together groups of outstanding scientists to address fundamental questions about brain function, focusing on fields in which new developments can create novel opportunities for exploration that reverberate through the broader field of neuroscience.

 “We received hundreds of intriguing proposals and are truly excited by the many outstanding scientific directions put forward by our community,” said Alyssa Picchini Schaffer, vice president and senior scientist of the Simons Collaborations in Neuroscience. “It was a rigorous evaluation process, and we are confident that SCENE will push the entire field forward by reshaping our understanding of cognition and behavior.”