Carnegie Mellon University
January 11, 2016

Educational Engagement

EarthShake

Educational computer games are evolving from nose-to-the-screen experiences. Using research, artificial intelligence and data, new teaching models are giving instructors feedback on their students' stumbling blocks, helping students learn how to develop their character alongside their book smarts, and joining the digital to reality with games that have human and virtual interactions.

Carnegie Mellon University alumnus Derek Lomas (HCI 2014) has created 35 educational games and is developing a new personalized learning system Brain Builder, as a complement to traditional learning games. In addition to helping students be successful with educational goals, he's tackling how to help students be successful in life.

Brain Builder is a children's web browser meant to engage students in the "four Cs" of curiosity, character, creativity and cognitive skills.

"We want to bring kids something new and interesting every day from the Internet. We'll use AI to make these recommendations and follow kids' progress," Lomas said.

Lomas was a student of Ken Koedinger, a professor of human-computer interaction and psychology, who developed Cognitive Tutors, educational technology that uses problem-solving to support school learning.

Koedinger's Cognitive Tutors were some of the first of their kind to be implemented in schools in the early 1990s, and their popularity grew quickly. They are used by nearly half a million students annually.

An important component to the Cognitive Tutors is the data gathered as students use them. Researchers and teachers use it to see models of the learning patterns. It follows a CMU tradition from the late Nobel and Turing laureate Herbert Simon, who created computer models of how people think and solve problems. CMU's Simon Initiative, named for him, aims to measurably improve student learning outcomes by harnessing the university's learning engineering ecosystem that has developed over several decades.

For Koedinger, the data helps him discover what he calls "hidden skills," meaning the stumbling blocks students have with a given task.

"The instructor is trying to make it a smooth process, but there might be something that is hard that they didn't realize is hard. The data can help identify that so they can develop new instructional tasks to practice those hidden skills," Koedinger said.

Implementing technology into learning is not without its challenges — one of which is the concern that computers isolate young learners.

Nesra Yannier (HCI 2016), a doctoral student of Koedinger and Scott Hudson, is addressing that concern by mixing technology and human interaction with her game EarthShake, which uses mixed reality learning. The project started with the goal of bridging physical and virtual learning to help teachers teach early physics principles in a more engaging and collaborative way.

EarthShake has a platform that connects physical experimentation in the real world with an interactive screen, and with AI vision algorithms is able to watch what kids do and give them feedback and support. In a demonstration the students are asked to place different physical towers on the platform and make a prediction about which tower might fall first when the earthquake table shakes. The system detects what happens in the physical world and gives them interactive feedback as they observe the results in the real world. Students also try to build their own structures that can withstand the movements.  Students learn five times more from the mixed reality system, Yannier's experiments show, than from the same guidance through video on a flat screen tablet or laptop.

"It was really exciting to see how they were thinking like scientists, collaborating and trying to make decisions," Yannier said.

Yannier is now developing EarthShake as an exhibit in the Children's Museum of Pittsburgh. She also was recent chosen as an Innovation Fellow through CMU's Center for Innovation and Entrepreneurship to commercialize her research. Previously she received the Mister Rogers Scholarship from the College Television Awards in New Media for Children, is part of CMU's NSF I-Corps program and Project Olympus.

These are just a few of the examples of how interactive experiences and using feedback improves learning.

"Learning by doing gives students deliberative opportunities to address objectives," Koedinger said.


Related:
Learning in the Real World Tops Learning From a Tablet

Learning is not a Spectator Sport