Writing practical pre- and post-tests for concepts in introductory courses

Burckhardt, P., Elliot, P. W., Evans, C., et al.

When revising a course, it can be helpful to know exactly how students struggle with the material. While revising an introductory statistics and data science course taken by all students in Dietrich College, we developed questions to assess student understanding and misconceptions of fundamental statistics concepts. Our goal is to determine how students think about these concepts, using a mixed-methods design to refine the questions and develop new ones. We present results from roughly 40 think-aloud interviews and from the administration of draft pre- and post-tests to hundreds of students at Carnegie Mellon and at Colby College. These assessment results have revealed new student misconceptions and changed our perception of how students learn statistics, which we will illustrate with several real examples. Our results suggest practical changes in how we teach statistics, and reveal how concept inventories can inform instructors in any discipline.

Philipp Burckhardt, Statistics & Data Science
Peter W Elliott, Statistics & Data Science
Ciaran Evans, Statistics & Data Science
Amanda Luby, Mathematics and Statistics, Swarthmore College
Mikaela Meyer, Statistics & Data Science
Josue Orellana, Center for the Neural Basis of Cognition and Machine Learning 
Jerzy Wieczorek, Mathematics and Statistics, Colby College 
Rebecca Nugent, Statistics & Data Science
Alex Reinhart, Statistics & Data Science