Fun and Games
Vincent Aleven & Yanjin Long
For Carnegie Mellon University's Vincent Aleven, the claim was too good to be true.
"It only takes about 42 minutes to learn algebra with video games" boasted a Forbes Magazine headline singing the praises of DragonBox, a popular learning game.
So Aleven, a learning scientist and associate professor of Human-Computer Interaction, and Ph.D. student Yanjin Long put it to the test.
Selecting a random group of seventh and eighth grade students, they compared the effectiveness of DragonBox to an intelligent algebra tutoring system created at Carnegie Mellon. Half the children played DragonBox for a specified period of time and half worked through equations using the intelligent tutors. They were then tested on solving equations.
As Aleven and Long suspected, when it came to actual learning, DragonBox was a lot more smoke than fire.
Aleven and Long presented their research results in early June at the 12th International Conference on Intelligent Tutoring Systems in Honolulu. CMU has been a pioneer in the development of so-called cognitive tutors for nearly three decades. Carnegie Learning, co-founded in 1998 by HCII professor Ken Koedinger, is a mathematics cognitive tutor in use by more than 600,000 students in 3,000 schools. The company was acquired by Apollo Group, parent of the University of Phoenix, in 2011, for $75 million.
Aleven and Long have a suspicion about where DragonBox loses steam when it comes to learning outcomes.
"The game provides too much 'scaffolding,'" Aleven said.
For example, in a problem that requires the player to subtract from both sides of an equation, DragonBox automatically prompts the player to do so. He added that DragonBox at no point requires the player to devise his or her own equation.
"The game is somewhat effortless, but that is part of what makes it enjoyable," he said.
Aleven said games can be effective instructional tools. In fact, he teaches a course in educational game design.
"It's hard to make something that's effective, but still engaging," Aleven said. "There is a way to intelligently integrate gaming features into intelligent tutor systems. You have to get input from game designers as well as teachers."
Schools across the nation are ramping up investments in technology, including many that have put laptops or tablets in the hands of all students. As a result, the so-called "EdTech" market is soaring. For example, Renaissance Learning Inc., a K-12 assessment and learning analytics company, was acquired for $1.1 billion in March, just three years after it was acquired for less than half that amount.
The Software & Information Industry Association estimates the K-12 software market at nearly $8 billion, with many new entrants on the way. Of those new entrants, a growing number are makers of educational games, a phenomenon that's being supported by the federal government. Of the 24 projects awarded Small Business Innovation Research awards by the U.S. Department of Education last year, half were games and game-related projects.
Aleven warns that those who market education games, like any educational product, must be held accountable for learning outcomes. This requires the kind of rigorous study that he and Long applied to DragonBox.
"We need a way to judge the effectiveness of educational products," Aleven said.
Improving learning outcomes through the science of learning is the focus of the Simon Initiative at Carnegie Mellon.
Long is ready to take up the challenge of designing education games that combine the effectiveness of cognitive tutors with the engagement/entertainment qualities of a game like DragonBox, making it the subject of her thesis.
"I look forward to collaborating with people at CMU from other disciplines to design games that incorporate both the scientific and creative aspects," she said.
Related Links: Simon Initiative | Carnegie Learning | Human-Computer Interaction Institute | Gamification Paper by Long and Aleven