Leveraging ‘Unreality’ for Good

A Grand Challenge Seminar examines how virtual technologies can be used to benefit society

Augmented reality is an interactive experience where objects in the real world are enhanced by computer-generated perceptual information. While immersive and spatial media are not commonly used in our day-to-day lives, Carnegie Mellon University’s Lauren Herckis, believes it is only a matter of time.

To help her students understand how this technology can affect society, Herckis co-taught a Grand Challenge Seminar with Tom Corbett, assistant teaching professor with the Entertainment Technology Center, titled “Unreality: Immersive and Spatial Media.”

Grand Challenge Seminars harness the intellectual power of humanists and social scientists from Dietrich College of Humanities and Social Sciences in collaboration with faculty from disciplines across the CMU campus. The faculty present information through a multi-disciplinary, problem-solving lens, while students gain experience in critical thinking, communication and collaboration.

“I am an anthropologist who specializes in the ways people interact with the material space around them,” said Herckis, who holds appointments in the Dietrich College, Simon Initiative, Human-Computer Interaction Institute and Block Center for Technology and Society. “So much of our lives has moved online in the last couple decades, especially during the pandemic, I was interested in how people interact in these spaceless places to work together.”

Herckis and Corbett were joined by CP Moreau, a doctoral student in the Department of English, in constructing the course content to initiate complex conversations about how to use these powerful technologies for the common good.

“Our students are in a position to think about shaping the future of immersive and spatial media,” said Herckis. “They are in a position to focus on how to use these tools in an ethical and culturally equitable way.”

And then the pandemic hit.

While the team originally envisioned dividing the class between virtual space and meeting together as a class, they had to do a hard pivot to meet the needs of remote learning as the pandemic continued to restrict large gatherings.  

“We wanted students to have a wide variety of immersive experiences to reflect on the larger conversation for the course,” said Herckis. “In order to do this, we had to identify and remove barriers to access so our students could engage with the virtual media.”

To accomplish this task, they turned to Vincent Sha, assistant dean of Information Technology and Operations for the Dietrich College. Sha and his team began assembling kits that included Cardboard virtual reality (VR) viewers and Oculus Quest IIs for every student in the class. 

“It is one thing to read about how our experiences are mediated through electrical signals that are interpreted by our brain to [working with the technology to see] how your mind is registering a virtual reality,” said Moreau. “A part of you recognizes that it is a virtual space but the bodily response is that it is real.”

Instructors, Lauren Herckis, Tom Corbett and CP Moreau, practice using the VR headsets prior to teaching the class.

Connecting from a Distance

The pandemic offered a powerful moment to explore how immersive media and technology

can be used to reduce isolation by supporting a feelings of togetherness and co-presence even at a distance. 

“I have seen VR, and immersive reality in a lot of dystopian books and movies, and was wondering what truth or fiction was behind the media,” said Lana Kim, a rising sophomore who participated in the course.

Key to the course was a series of group projects to help the students not only learn the tools but learn to work together from a distance. Most of the students in the class lacked experience with these technologies. To help their students gain confidence with these tools and build a feeling of connection with the university, the first project tasked the student groups to build an immersive experience around the CMU campus.

“[Our first-year students] are trying to figure out what it means to be part of a campus community when they have not been on campus yet,” said Herckis. “Immersive media can support, enhance and provide extra tools for a feeling of togetherness and co-presence even when people are not in the same place.”

The students took this charge and handed in interactive tours of campus and even the wider Pittsburgh community. One group even created a virtual Fence, which required virtual camping to protect their creation.  

Kim enjoyed dissecting the media aspects of the virtual world, especially as it related to her interest in the truth or fiction behind how VR is portrayed by the media.

“We would watch a clip from a movie trailer or a part of a video game,” said Kim. “We would see how movies take aspects of VR and conflate it to create drama, or see how game developers made their games more immersive.”

Each new project reshuffled students into different groups to ensure everyone had an opportunity to work together, grow together and learn from one another as they completed their tasks. Another project tasked groups to design visuals to reflect real-world spaces in a fictional setting with an eye toward representation and stereotypes.

The students groups met for their final projects and presented to the class. The projects were rendered in VR space and covered a wide range of experiences, from virtual dressing rooms to how AR can help people parallel park. 

“The class offered a certain degree of escapism for the students, and it gave me the insight that there were categories of things that will be of interest to different people,” said Corbett.  “It was great to see the technology from the newly initiated perspective.”