Dataset Bridges Human Vision and Machine Learning
Neuroscience, Computer Vision Collaborate To Better Understand Visual Information Processing
By Byron Spice
CMNI constituents including Michael Tarr, John Pyles, and Abhinav Gupta collaborated with other neuroscientists and computer vision scientists at Carnegie Mellon University and Fordham University to produce a new dataset of unprecedented size.
The dataset comprises brain scans of four volunteers who each viewed 5,000 images. It will help researchers better understand how the brain processes images.
Researchers at Carnegie Mellon University and Fordham University, reporting today in the journal Scientific Data, said acquiring functional magnetic resonance imaging (fMRI) scans at this scale presented unique challenges.
Each volunteer participated in 20 or more hours of MRI scanning, challenging both their perseverance and the experimenters' ability to coordinate across scanning sessions. The extreme design decision to run the same individuals over so many sessions was necessary for disentangling the neural responses associated with individual images.
The resulting dataset, dubbed BOLD5000, allows cognitive neuroscientists to better leverage the deep learning models that have dramatically improved artificial vision systems. Originally inspired by the architecture of the human visual system, deep learning may be further improved by pursuing new insights into how human vision works and by having studies of human vision better reflect modern computer vision methods. To that end, BOLD5000 measured neural activity arising from viewing images taken from two popular computer vision datasets: ImageNet and COCO.
"The intertwining of brain science and computer science means that scientific discoveries can flow in both directions," said co-author Michael J. Tarr, the Kavčić-Moura Professor of Cognitive and Brain Science and head of CMU's Department of Psychology. "Future studies of vision that employ the BOLD5000 dataset should help neuroscientists better understand the organization of knowledge in the human brain. As we learn more about the neural basis of visual recognition, we will also be better positioned to contribute to advances in artificial vision."
