ETC Explores Augmenting Reality
Students at CMU's Entertainment Technology Center studied when people found augmented reality helpers useful. To do this, they created Mixy, an avatar, to help explain a museum exhibit.
It may not feel revolutionary to ask Siri a question, but by interacting with her voice you're actually using augmented reality, a transformative technology that is working its way into our daily lives.
Known as AR, augmented reality involves a layer of digital information that interacts with the immediate physical surroundings of the user.
"The power of AR isn't to radically change our lives, but to make us better at what we already can do," said Will Hagen, a student at Carnegie Mellon University's Entertainment Technology Center (ETC), which is exploring its possibilities. The ETC is part of CMU's Integrative Design, Arts and Technology (IDeATe) Network.
ETC Director Drew Davidson is interested in how augmented reality differs from virtual reality, which involves a person entering a completely different visual environment.
"Augmented reality is in some ways potentially more immersive than virtual reality," Davidson said. "I think AR is opening up these new potential ways to think about and develop these [virtual reality] games."
Popular games like Pokemon Go feature AR, but the technology is not just for play. At the ETC, graduate students work with interdisciplinary teams on student- and client-driven projects. Last semester several teams experimented with AR, including mixAR and Music Everywhere.
The mixAR team studied how users interact with a virtual agent — in this case a spherical robot — to examine how an interactive agent can be useful, when it becomes a nuisance and how the robot could emotionally connect with the user.
"Right now the field is truly in its infancy," said Hagen, the producer for the mixAR team. "Imagine a paramedic arriving at an accident with an augmented reality device that can provide all the vitals of an unconscious individual, saving them the moments they may need to prevent that person's death. Dramatic perhaps, but one of the best examples of how we could use augmented reality to improve our society right now."
Two major AR platforms are HoloLens and Google Tango. They are difficult to acquire, but Hagen said he sees the speed of development increasing each year. MixAR used Microsoft's HoloLens, a cross between a visor and sunglasses worn to experience the augmented reality.
The Music Everywhere team used the HoloLens to help teach or learn an instrument. They worked with a piano to understand how AR can help engage users, demonstrate concepts and guide users through interactive exercises.
Carl Rosendahl, director of the ETC's Silicon Valley program, said augmented reality will be more visible within the next five to 10 years, but is in use in products like BMW's Head-Up display, Amazon's echo and Apple's Siri.
"The ETC brings in both artists and programmers — think the left brain and the right brain — with a diverse set of skills to work on a project for a whole semester and create something interesting and meaningful," Rosendahl said. "Building skills of working in a team with different expertise, training and background is so crucial to how you communicate and share your ideas."
In addition to building valuable communication skills and meaningful projects in class, ETC's Silicon Valley program is hosted by Electronic Arts, so students on site are next door to a gaming company with many opportunities.
"The ETC helps prepare students for employment through lifelong learning as well as developing professional connections and the ability to collaborate with other professionals with a hugely varied skillset and succeed," Hagen said. "In the gaming industry that is an important thing to be able to do, and the ETC does a great job enabling students to do it."