CMU group works on movies that reach out and touch you - CTTEC - Carnegie Mellon University

Wednesday, June 8, 2011

CMU group works on movies that reach out and touch you


Sitting in a movie theater watching 3-D superheroes leap off the screen and soar overhead, it's hard to imagine what direction the next level of media enhancement could take.

Maybe viewers feeling wind whipping their hair as the hero flies by or the heat of his laser-beamed eyes warming their cheeks? What if an audience could feel the moisture of a tear as it streams down a heroine's face, followed by a gentle brush of hand wiping it away?

Sounds like the stuff of science fiction, but a team at the Disney Research Lab in Carnegie Mellon University's Collaborative Innovation Center in Oakland is working at making those sensations seem real. The lab is part of CMU's Entertainment Technology Center and one of six research facilities Disney operates internationally.

As a corporate partner to the university and led by Jessica Hodgins, a professor of computer science and robotics, the lab conducts research and development directed toward improving Disney rides and products. The arrangement gives Disney access to the university's laboratory and facilities in addition to the opportunity to collaborate with some of the finest minds in the computer science industry.

The new project, designed to add sensations to entertainment technology, is in the early stages, but the team already has gotten some interest from a video game developer.

The Tactile Brush algorithm, as the team has named its project, is meant to create sensations varying from a single point of vibration to continuous vibrations that can travel the length of a person's spine.

It opens the door for illusions that go far beyond vibrations, said Ali Israr, principal investigator, and Ivan Poupyrev, Interface Technology Group director.

"The problem with today's technology is the vibrations communicated by normal devices -- iPhones, joysticks, vibrating chairs people use in movies -- tend to be very one-dimensional," explained Mr. Poupyrev.

"The whole thing vibrates. So you can vibrate on, off or maybe increase intensity of vibration, but there's not really much information you can create."

One of the key differences between current tactile technologies and what's available through Tactile Brush, said Mr. Poupyrev, is how vibrating actuators -- the sources found in cell phones and other vibrating technologies -- are used.

Through the algorithm, the team discovered that changes in speed, intensity, timing and other variables create two new kinds of so-called haptic illusions, or illusions of sensation on the skin.

An illusion called "apparent tactile motion" allows for a continuous stream of vibration akin to a finger stroking the skin, while "phantom tactile sensation" creates the feeling of single-points of vibrations in spaces where no actuators are present.

Mr. Israr, a perceptual psychologist and mechanical engineer, developed the psychological model of how to create and manipulate the illusions.

Calling the combined technologies "Surround Haptics," the two researchers said the next step was to try to use visual imagery and research the brain's reactions to touch to create more authentic sensations and experiences.

"Our hypothesis is that people would feel different tactile sensations differently depending on what they see on the screen," said Mr. Poupyrev. "In a sense, our visual perception system would help us to disambiguate what sensation it is.

"In one situation, the sensation could feel like water streaming; in another, it could feel like a bug walking on your skin. In one kind of scenario, you'll feel it as hitting pavement as you're driving a car; in another case, it would feel like a gunshot."

What people see as they're experiencing sensations affects how they feel them, but how the brain processes those sensations is equally important.

Opportunities to introduce tactile sensations to industries already looking to enhance technologies are ripe.

The Motion Picture Association of America reported that 2010 global box office receipts reached a record of $31.8 billion, with 3-D films in the United States and Canada alone accounting for $2.2 billion.

The video game industry, which economist Paul Heydon of Avista Partners estimated to be worth about $105 billion in 2010, already has seen 3-D releases from Nintendo and firmware that allows 3-D viewing through Sony's Playstation 3.

Since low-level vibrating technologies already are used in some theater seats and video game joysticks, the notion of combining 3-D technologies with a wider range of vibrations and sensations could be the next logical step.

Deciding exactly how much sensation is too much for viewers is another hurdle for the Disney Research Lab team.

Should they actually create an illusion that simulates the pain of a fallen avatar's gunshot wound to the chest?

Mr. Poupyrev answered with an enthusiastic "why not?"; but Mr. Israr said they could find ways to mimic the sensation with considerably less pain.

"We don't want to create this emotional feeling of hurt or hurting, we just want to trigger these mechanical receptors that could create the sensation that something started to go in [the skin]," he said. "We don't want to go beyond that to make people suffer."

The duo is set to debut their prototype in August at the Association for Computing Machinery's Special Interest Group on Computer Graphics and Interactive Techniques conference in Vancouver, British Columbia. It's a three-day annual conference that draws thousands of computer science professionals, and highlights emerging technical and creative initiatives in research, science, art, animation, gaming and more.

The prototype -- a kitchen chair fitted with 12 actuators lined in three rows of four and swathed in cushion -- is connected to a simple video game that allows players to feel new types of vibrations each time a robot crosses a barrier on the screen.

That model already has led to an informal arrangement with Disney Interactive gaming developer, Black Rock Studio, to examine how the technology could be used in one of its games.

While the team envisions similar chairs with enhanced features in homes and movie theaters throughout the country someday, team members also recognize the potential to use the technology outside of entertainment.

Surround Haptic jackets could work as maps that use vibrations to notify wearers which direction to travel.

Someone wearing a haptic helmet could feel the sensation of a soothing temple massage.

Since the technology can use actuators as small and inexpensive as those found in pager motors (the prototype uses actuators made from speaker motors), both creators say the possibilities are ultimately as large or as small as the aspirations of the individuals who approach them with a plan.

"It can be a hand-held device such as a cell phone, it can be a sleeve. Actually, anywhere you have skin, this device can work," said Mr. Israr.

"Only imagination is the limiting factor here," Mr. Poupyrev said.



Article courtesy of Pittsburgh Post-Gazette