Carnegie Mellon University

The Piper

CMU Community News

Piper Logo
October 20, 2011

Touchable Technology

Researchers from CMU, Disney and Microsoft are Featured Presenters at ACM Symposium

Researchers from Carnegie Mellon, Disney Research, Pittsburgh and Microsoft Research presented their work, including a wearable projection system, an interactive projection system and touchscreen technology at this week's Association for Computing Machinery's Symposium on User Interface Software and Technology in Santa Barbara, Calif.


OmnitouchOmniTouch, a wearable projection system developed by researchers at Microsoft Research and Carnegie Mellon, enables users to turn pads of paper, walls or even their own hands, arms and legs into graphical, interactive surfaces.

OmniTouch employs a depth-sensing camera, similar to the Microsoft Kinect, to track the user's fingers on everyday surfaces. This allows users to control interactive applications by tapping or dragging their fingers, much as they would with touchscreens found on smartphones or tablet computers. The projector can superimpose keyboards, keypads and other controls onto any surface.

"It's conceivable that anything you can do on today's mobile devices, you will be able to do on your hand using OmniTouch," said Chris Harrison, a Ph.D. student in Carnegie Mellon's Human-Computer Interaction Institute

Read more:


sidebysideResearchers at Disney Research, Pittsburgh, and Carnegie Mellon devised a system called SideBySide that enables animated images from two separate handheld projectors to interact with each other on the same surface.

The system, suitable for games, education and a variety of other applications, is self-contained in special handheld devices. No external cameras or other sensors are required, which enables people to use the projectors to interact with each other anywhere and at anytime.

"Smartphones have made it possible for us to communicate, play games and retrieve information from the Web wherever we might be, but our interaction with the devices remains a largely solitary, single user experience," said Karl D.D. Willis, a Ph.D. candidate in computational design at Carnegie Mellon and a lab associate at Disney Research. "Now that handheld projectors have become a reality, we finally have a technology that allows us to create a new way for people to interact in the real world."

Read more:


tapsenseBy attaching a microphone to a touchscreen, CMU scientists showed they can tell the difference between the tap of a fingertip, the pad of the finger, a fingernail and a knuckle. This technology, called TapSense, enables richer touchscreen interactions. While typing on a virtual keyboard, for instance, users might capitalize letters simply by tapping with a fingernail instead of a finger tip, or might switch to numerals by using the pad of a finger, rather toggling to a different set of keys.

Another possible use would be a painting app that uses a variety of tapping modes and finger motions to control a pallet of colors, or switch between drawing and erasing without having to press buttons. 

TapSense was developed by Harrison, fellow Ph.D. student Julia Schwarz, and Scott Hudson, a professor in the HCII.

Read more:

Other Presentations

Additional Carnegie Mellon projects at the ACM included:

Instrumenting the crowd: Using implicit behavioral measures to predict task performance
Jeffrey Rzeszotarski, Carnegie Mellon
Aniket Kittur, Carnegie Mellon

CrowdForge: Crowdsourcing Complex Work
Aniket Kittur, Carnegie Mellon
Boris Smus, Carnegie Mellon, Google
Robert Kraut, Carnegie Mellon
Susheel Khamkar, Carnegie Mellon

A Tongue Input Device for Creating Conversations
Ronit Slyper, Carnegie Mellon, Disney Research
Jill Lehman, Carnegie Mellon, Disney Research
Jodi Forlizzi, Carnegie Mellon, Disney Research
Jessica Hodgins, Carnegie Mellon, Disney Research

Monte Carlo Methods for Managing Interactive State, Action and Feedback Under Uncertainty
Julia Schwarz
, Human Computer Interaction Institute, Carnegie Mellon
Jennifer Mankoff, Human Computer Interaction Institute, Carnegie Mellon
Scott Hudson, Human Computer Interaction Institute, Carnegie Mellon

PocketTouch: Through-Fabric Capacitive Touch Input
T. Scott Saponas, Microsoft Research
Chris Harrison, Microsoft Research, Carnegie Mellon
Hrvoje Benko, Microsoft Research

Gligor Honored for Work in Computer Privacy

At another ACM event, Virgil Gligor, co-director of Carnegie Mellon CyLab and a professor of electrical and computer engineering, received the Outstanding Innovation Award for computer privacy and security expertise on Wednesday at the annual ACM meeting in Chicago.

Gligor received the award from the ACM's Special Interest Group on Security, Audit and Control. The award recognizes Gligor's innovations in secure operating systems as well as covert channel analysis, intrusion detection and secure wireless sensor networks.