SUCCESS STORY
Accelerating student learning with data-driven instruction
Learning effectiveness studies conducted during three semesters between 2005 and 2007 at the Carnegie Mellon University’s Open Learning Initiative found that students in a typical college-level, non-calculus-based “Introduction to Statistics” course spent more than 100 hours on the material, but saw only about a 3 percent gain in their learning outcomes.
In contrast, the studies found, students in an adaptive, data-driven course spent less than 50 hours on the same material — that is, under half the time of the traditional course — while consistently and reliably achieving learning gains on the order of 18 percent.
How it worked:
Students focus their practice where they need it most. Teachers adapt their instruction to meet students’ needs.
In the typical classroom model, students attend a lecture, complete homework some days later, turn it in and then get feedback from the instructor the following week. In this adapted, data-driven class, the course material for the “Introduction to Statistics” was presented to the students through an online tutor rather than through classroom lectures.
The tutor incorporated assessment exercises that gave the students the opportunity to demonstrate their knowledge. The tutor provided the students immediate and targeted feedback (for example, hints), reacting to what each student did and did not know and presenting additional exercises as necessary. The instructor then reviewed reports on the students’ performance generated by the online tutor to see where students were struggling. Afterward, she spent her time in class — two 50-minute sessions held each week — on answering students’ questions and helping the students review the more challenging material.
The data used:
The instructor designed the course by setting overall learning objectives. (For example, an overall learning objective in the statistics course was “Relate measures of center and spread to the shape of the distribution, and choose the appropriate measures in different contexts.”)
Each chapter and assessment exercise was also tied to a sub-learning objective that, if mastered, would help achieve the overall learning objective. (Example of sub-objectives in the statistics course: “Predicting,” “mean vs. median,” “compute median,” “identify outlier.”)
By setting objectives and designing items and activities to support those objectives, the tutor could collect real-time, interaction-level data on how the students were learning.
The instructor then used a "Learning Dashboard" to track student progress through the course modules and tailored intervention based on feedback from the software indicating where students were having difficulty.
The dashboard, for example, displayed data measuring:
- Each student’s performance on individual questions and assignments.
- The number of attempts each student made to comprehend the learning objective (a low number of attempts typically indicates a student who did not try, while a high number of attempts indicates a student who is truly confused.)
- How much time each student is spending on each assignment and on the course material in general.
- The overall class accuracy by sub-objective.
The larger impact
The dashboard helped the instructor:
- See in real-time which concepts students were struggling to understand.
- Use analytics to assess retention over the time of the course.
- Make face-to-face time more meaningfully engaging for the students.
Find out more
Read the peer reviewed paper:
Quotes from students and instructors involved:
Student Quote:
“This is so much better than reading a textbook or listening to a lecture! My mind didn’t wander, and I was not bored while doing the lessons. I actually learned something.”
Instructor Quotes:
“The format [of the accelerated learning study] was among the best teaching experiences I’ve had in my 15 years of teaching statistics.”
“Using OLI, we’ve developed what we think is a really innovative, inquiry-based approach to teaching stats.”
“There is generally a third of the class that hates statistics and doesn’t want to be there. Before [I used OLI], I didn’t know who those students were or how to support them.”
“The software not only taught procedures but helped students understand their possible applications. It answers the ‘Why do I care?’ question.”
“As an adjunct math professor, I was able to jump into a brand new course at my college ONLY because I had access to OLI materials.”
“The learning curve is sharp and managing the resources was difficult at first but having access to what students are really learning and not is excellent…Great for both instructors and students to have access to the ‘truth’ and not just the perceived truth about the learning. This has given me an opportunity to grow as an instructor out of my usual comfort zone.”