Carnegie Mellon University

Educators Tap Into Data

August 29, 2017

Ken Koedinger and Susan Gracia

LearnLab Director Ken Koedinger noticed his lab's summer school was different from years past. Interests spiked in creating online courses and interpreting the data collected by digital tools to provide evidence that students are learning.

“Many participants wanted to work directly in building or improving their own online courses during summer school, compared to earlier years where most were building prototypes or analyzing course data produced by others,” said Koedinger, who is a professor of human-computer interaction and psychology at Carnegie Mellon University.

The Simon Initiative's LearnLab Summer School makes it possible for educators to become acquainted with new teaching methods and digital tools, such as intelligent tutoring systems, while also interacting with CMU’s learning engineering researchers.

Susan Gracia, professor of education at Northeastern University, developed two modules of a six-part course during the week and plans to continue developing it.

"I gained a better understanding of the data collected by online courses and how to use assessments to improve learning," she said.

Ricardo Poza from the University of Massachusetts, Amherst, also benefitted by learning about assessments. He had already built an online course that teaches instructors on how to teach online, and LearnLab enabled him to apply data driven improvements.

Caitlin Augustin, a senior data scientist at Kaplan, said her employer is reliant on data. She came to learn about new research in learning science and best practices in online education and to see how others engaged with the data collected from the digital tools.

"CMU's large body of research in technology-enhanced learning was eye-opening," she said.

Dozens of researchers, faculty and graduate students came from across the U.S., as well as Qatar and Hong Kong. The five-day workshop is made up of lectures and time dedicated to producing a small prototype experiment. Everyone presents work on the last day.

Carolyn Rosé, a professor of human-computer interaction and language technologies in CMU's School of Computer Science, ran the Computer-Supported Collaborative Learning track, one of the summer school's four tracks.

“I encouraged my teams to learn new tools and collaborate, rather than focus on completing a finished project. The emphasis is to acquire new knowledge and skills they can apply long after the workshop is over,” Rosé said.

Several participants experimented with Carnegie Mellon's Open Learning Initiative (OLI) development environment. Other state-of-the-art tools available included the Cognitive Tutor Authoring Tools, tools for authoring natural language dialog, TagHelper tools for semi-automated coding of verbal data, and DataShop for storage of student interaction data and analysis of student knowledge and performance.

The use of DataShop attracted participants on the RoboTutor team, a CMU project led by Jack Mostow. It is one of 11 projects competing internationally as a semifinalist in the $15 million Global Learning XPRIZE competition. A prototype of RoboTutor currently teaches basic reading, writing and mathematics to a test group of children in Tanzania.

Vishnu Rajan Tejus, a 15 year-old college student from Silicon Valley, queried Mostow on collaborating together for the XPRIZE. They began the project in May 2015. At the summer school, Tejus applied DataShop to analyze some of the video data collected from the user testing.

"We were excited by the results showing students' progress of learning. We saw the students applied concepts that they learned in one area of the tutor to other activities," Tejus said.

He added, "With all the lectures by pioneers in the field and then implementing what we've learned for analyzing our data, the week went by very fast! Using DataShop was a great experience for me to expand my understanding of data mining to educational applications."

Ji-Eun Lee, a graduate student from Utah State University, analyzed students' online interactions with the Canvas learning management system. She measured students’ learning behaviors, such as reading, collaborating, discussion reading and discussion writing, and then analyzed how their actions influenced learning.

“The workshop gave me new skills in data analysis that I will be able to apply to my research. I learned a great deal,” said Lee.

Hong Liu, a professor of mathematics at Embry-Riddle Aeronautical University, said collecting data on students to see what they are learning is an important aspect of his job as faculty.

“That’s why I’m here. LearnLab shows us new ways of interpreting the data and how to use it to enhance educational experiences,” he said.

Related Articles

Summer School Helps Students Build Tech Tools To Improve Learning
Simon Summer Institute Helps Pittsburgh-area Professors Build Ed Tech Tools for Their Classrooms
RoboTutor Advances to Global Learning XPRIZE Semifinals
LearnLab Experts Present Education Research Accomplishments at NSF Meeting
By Ann Lyon Ritchie

Pictured above are CMU Professor Ken Koedinger and Simon Initiative LearnLab Summer School student Susan Gracia.