Carnegie Mellon University
Eberly Center

Teaching Excellence & Educational Innovation

Eberly Center  ›  GAITAR Initiative  ›  GAITAR@Scale

GAITAR@Scale

GAITAR@Scale projects focus on reproducibility and generalization of scholarship across many teaching contexts for a few high-priority learning objectives and research questions. In contrast to GAITAR Institutes and Fellowships, cohorts of instructors collaborate with Eberly colleagues to study Eberly-generated research questions and teaching interventions at scale, such that data collection can directly facilitate higher-level pattern induction regarding student learning outcomes.

GAITAR@Scale projects test a common research question at scale across diverse CMU courses:

To what extent does student use of generative AI (genAI) impact student outcomes?

GAITAR@Scale 1

Impacts of asynchronous learning modules on genAI competency in college students (Fall 2024)

Designed interactive, online learning modules about generative AI and measured their impact on students’ generative AI competency (knowledge, skills, and self-efficacy).
SCOPE: 46 instructors, 65 sections of 53 unique courses, and 2,324 students
RESULTS: Modules significantly increased students’ generative AI knowledge, prompt engineering skills, and self-efficacy above and beyond a control group.

Link to study ›

GAITAR@Scale 2

Impacts of Generative AI as a “Feedback Partner” on Students’ Writing (Spring 2025)

Tested the impact of generative AI feedback versus peer feedback on students’ ability to support a claim with evidence, their self-efficacy, and their perceptions of the revision process in first-year writing courses.
SCOPE: 16 instructors, 28 courses, and 424 students.
RESULTS: Students receiving multiple rounds of genAI feedback appeared to perform better than students receiving a single round of human peer feedback. Students in the human peer group reported a different and more positive experience than their peers in the genAI group.

 

Link to study ›

GAITAR@Scale 3

Currently testing the impact of student’s practicing with and receiving feedback from generative AI on writing for a specific audience.

 

Study results coming soon...

GAITAR@Scale 4

Will test the impact of generative AI on students' scientific estimation skills and is currently under development in collaboration with subject matter experts from the College of Engineering.

 

Study results coming soon...

More About GAITAR@Scale

To support data-informed teaching and learning at CMU and beyond, we gather cohorts of 10 or more CMU instructors of record across disciplines to conduct a controlled teaching as research (TAR) study that is designed and supported by the Eberly Center, but implemented by CMU instructors. 

First, Eberly colleagues identify high-priority teaching as research (TAR) questions that are relevant across disciplines and teaching contexts. An example research question might be: How does student use of GAI during low stakes practice exercises impact their ability to articulate an argument in writing and support it with evidence on a higher stakes assignment? Research questions targeted by GAITAR@Scale projects are selected based on gaps in the education literature and the needs of CMU instructors that surface through Eberly programs and services,  GAITAR Fellowship projects, GAITAR Institutes, or targeted needs assessments.

Next, Eberly colleagues design teaching interventions and TAR study designs to help answer the research question at scale, across numerous courses and teaching contexts. We identify a standardized educational intervention, study design, and data sources on student learning and other outcomes.

Then, we recruit CMU instructors of record from 10 or more courses across disciplines to collaboratively implement the Eberly-generated TAR study design. This approach facilitates testing TAR questions at scale and helps overcome the challenges of reproducibility and generalization in applied education research.

To further illustrate how TAR@Scale works, please see how the Eberly Center is implementing a similar project with 12 faculty regarding the impacts of specifications grading on teaching and learning.

Starting in May 2024, 1-3 times annually for three years, we will gather cohorts of 10 or more CMU instructors across disciplines to conduct a controlled TAR study, designed by the Eberly Center, but implemented by CMU instructors at scale. Initial potential GAITAR@Scale projects may include, but are not limited to, testing hypotheses about:

  • how best to leverage online learning modules and classroom exercises to enhance students’ knowledge and skills regarding GAI, and
  • the effects of GAI on student outcomes for learning objectives regarding written communication, coding, creation of art, and more.
gaitardiagram-rd-highlight.png