Analyzing, Aligning, Assessing: A portable framework for corpus-based writing pedagogy at scale

Author: Michael Laudenbach

Degree: Ph.D. in Rhetoric, Carnegie Mellon University, 2024

The teaching of writing in disciplinary content areas reinforces learning objectives and encourages students to engage in active problem-solving applicable to future professional or academic settings. Within Writing Across the Curriculum (WAC) and Writing in the Disciplines (WID) scholarship, subject areas like civil or mechanical engineering have received thorough attention. However, the presently expanding field of statistics and data science has yet to receive explicit attention in WAC/WID, technical and professional communication, and rhetorical genre studies more broadly. Statistical analyses appear in a variety of interdisciplinary and professional contexts, but relatively recent calls for curricular shifts to promote data science initiatives have widened the responsibilities of statistics departments and instructors. Recent scholarship has nevertheless outlined the dearth of writing-intensive data science classes across various programs in higher education, highlighting a variety of factors, including the additional labor burden placed on instructors of technical courses that are often composed of 70 to 150 students. Responding to these conversations, the research presented in my dissertation is the product of an ongoing collaboration between the Department of English and the Department of Statistics and Data Science at Carnegie Mellon University.

My dissertation describes the first large-scale study of its kind to describe writing in the discipline of statistics and to use those descriptive results to pilot a technology-enhanced learning intervention using DocuScope Write & Audit. First, I collected and analyzed a collection of over 1,200 texts, consisting of published statistics research and authentic student writing samples from six courses offered by CMU’s Department of Statistics & Data Science. With corpus linguistics methods, I am able to explicitly name the rhetorical and linguistic features of client-facing and expert-facing statistics reports. Using the more granular results from the corpus analysis, our interdisciplinary team designed learning materials for voluntary revision workshops offered to students of 36-200: Reasoning with Data. These workshops asked participants (n = 30) to revise their papers using DocuScope: Write & Audit, a student-facing tool that can visualize task-specific features in real-time as students write and revise. This exploratory study showed promising results in the comparison of pre- and post-workshop survey responses: significant positive increases in subscales of self-efficacy and beliefs about the importance of writing content after using the tool. To better contextualize the survey results, I pair them with a close analysis of think-aloud interviews. This dissertation project contributes to corpus linguistics studies, WAC/WID research, genre studies, and areas of formative writing assessment. Ultimately, though, I sketch a portable framework for analyzing and teaching disciplinary writing at scale, one which I plan to refine and reapply in future collaborations.