April 18, 2016
The Klatzky Lab
By Emily Stimmel
Roberta Klatzky, the Charles J. Queenan Jr. Professor of Psychology, is a world-renowned expert in cognition. Her work — which spans CMU’s Psychology Department, Human-Computer Interaction Institute and the Center for the Neural Basis of Cognition (CNBC) — examines the relationships between human perception and action, with a focus on touch.
“It’s not what most people think about psychology,” she said.
Whether through navigation aids for the blind, tools to improve image-based surgery or interactive games for children, Klatzky’s research seeks to link people’s sensory capabilities to technology, emulating real-world tactile experiences through the use of haptic devices.
Alongside George Stetten, a research professor in CMU’s Robotics Institute, and students from CMU and the University of Pittsburgh, Klatzky demonstrated that ultrasound images projected onto the body by using augmented reality help surgeons with delicate, precise procedures like inserting a catheter. Now, over a decade later, a current project in Klatzky’s lab is building on this research, replacing ultrasound images with a new technology called optical coherence tomography (OCT).
The research, funded by the National Institutes of Health (NIH), utilizes augmented reality to place images in the eye. Unlike virtual reality, which substitutes a simulated environment for the real world, augmented reality overlays perceived space, allowing participants to manipulate elements in their environment.
“The augmented reality image that the surgeon sees will precisely match the orientation and location of the slice of eye tissue that is being measured with OCT,” said Klatzky.
The surgeon must be able to see surface orientation — or slant — under the microscope. A body of psychological research can tell us the accuracy with which people see slant, but there is very little existing research about the way slant is perceived when it is distorted under a microscope. As a principal investigator, Klatzky is relating these issues to what we already know about basic psychological science. She is working with Samantha Horvath, a Ph.D. student in CMU’s Robotics Institute, and John Galeotti, a research scientist who received his doctorate degree in robotics from CMU in 2007.
In another recent series of studies, Klatzky and her fellow researchers at Disney Research Pittsburgh developed a vocabulary of “feel effects” — patterns of haptic stimulation produced by tactile simulators, or tactors, embedded in a vest or gloves. For example, brief pulses across the body would be interpreted as “rain.” By varying speed and pressure, these rain tactors may be perceived as a sprinkle or a downpour.
This research is bringing early reading experiences to life for young children. When pre-reading children listened to stories accompanied by feel effects, they demonstrated an improvement in comprehension and memory. Children who were actively learning to read also benefited from oral reading experiences enhanced by feel effects.
Interdisciplinary collaboration is a hallmark of Klatzky’s work, which is not only broad in scope, but in its geographical reach. Together with Ed Colgate and Michael Peshkin at Northwestern University and two undergraduate computer science students at CMU, Klatzky is working to develop active touch technology for glass screens. The project, funded by the National Science Foundation (NSF), involves altering the surfaces of commercial flat-screen devices like tablets and smartphones. As the friction on the glass changes under a person’s fingers, illusions can be created to evoke edges, bumps, holes or textures. Like much of Klatzky’s work, this technology has practical applications, including graphical displays for the blind, enhanced games and tactile communication of products like fabrics.
“The mind is grounded on a very adaptive and successful sensory system,” she explained. “The goal of much of my work is to guide complex behaviors directly through the senses, reducing demands on thought processes. Perhaps it seems strange for a psychologist to say, but I want to relieve people from thinking when they interact with technology.”