Eberly Center › GAITAR › GAITAR Fellows | Project Descriptions

GAITAR Fellows | Project Descriptions

Scott Andrew

Adjunct Faculty
Art
College of Fine Arts

60-424 AI Animation (Spring 2024)

Generative AI Tool(s) Used

Runway, Deforum Stable Diffusion, ChatGPT, ElevenLabs, Midjourney, Dall-E

Research Questions

How does student use of generative AI tools to make animations impact their:
1. technical and aesthetic control over their art?
2. self-efficacy as animators and generative AI users?

Teaching Intervention with Generative AI

Scott’s students used generative AI tools as collaborators to create animations, especially during creative editing and stylization decisions. Applications during and between class sessions included generating storyboards, scripts, animated sequences, synthesized voice narration and voice acting, and sound designs, resulting in both narrative and experimental works of animation. The suite of generative AI tools included Runway, Deforum Stable Diffusion, ChatGPT, ElevenLabs, Midjourney, Dall-E and more.

Study design

Students used a suite of generative AI tools across all animation assignments. The first assignment required students to recreate an animation from a previous course and Scott will compare student deliverables created with (treatment) and without (control) the assistance of generative AI. He will also measure changes in students’ performance and attitudes regarding animation across the individual assignments in the course.

Data Sources

Students’ deliverables from animation assignments, scored via a rubric with criteria for technical and artistic control
Pre/post surveys of students’ self-efficacy regarding skills using generative AI and course learning objectives

Emily DeJeu

Assistant Teaching Professor
Tepper School of Business

70-340 Business Communications (Spring 24)

Generative AI Tool(s) Used

ChatGPT, Copilot

Research Questions

How, and to what extent, do students feel they do or do not benefit from learning about ethical and effective generative AI use cases for the kinds of professional communication tasks they are likely to face in their future jobs?
To what extent does inviting students to use a generative AI tool in specific ways help or hinder their writing-related skill-building?
How are students using generative AI tools when it comes to planning, drafting, and revising course assignments?

Teaching Intervention with Generative AI

DeJeu introduced four mini-lectures in one of the two sections of her course that showcase generative AI use cases in professional communication contexts. Specifically, these lessons provided instruction and modeling on using ChatGPT or Copilot to revise a document, create model documents, identify "lexical bundles" (i.e., phrases and sentences that are used often in particular genres of writing), and generate ideas. Mini-lectures each occurred in tandem with one of the four major writing assignments that included reflection questions and documentation regarding the writing process.

Study design

DeJeu had two sections, one of which received in-class scaffolding for ethical and effective generative AI tool use while the other did not. In both sections, students had the option to use generative AI tools and document their use on each of their four standard writing assignments. DeJeu will compare students’ perceptions, documented practices using generative AI tools, and writing deliverables across the two sections. In addition, in a third section taught by a colleague, students were asked to not use generative AI tools for an assignment that will be used to compare students’ writing to her sections (i.e., with vs. without the reported use of generative AI).

Data Sources

Pre/post surveys of students’ experiences with and perceptions of generative AI tool use
Students’ reflections on and documentation of generative AI tool use
Transcripts of students' interactions with generative AI tools
Students’ writing assignments, scored with a rubric for various writing skills

Sébastien Dubreil

Teaching Professor
Modern Languages
Dietrich College of Humanities and Social Sciences

82-304 French and Francophone Sociolinguistics Oral Language and Storytelling (Spring 24)

Generative AI Tool(s) Used

ChatGPT

Research Questions

Do various use cases of generative AI yield different linguistic accuracy and complexity in French students’ writing?
What are French students’ perceptions of using generative AI to complete writing assignments?

Teaching Intervention with Generative AI

Dubreil introduced generative AI (ChatGPT) as a support for students writing in a foreign language. In one condition, he instructed students to create their initial draft while using AI as a language assistant for support to suggest vocabulary or specific language features (e.g., a rhyme, an alliteration), check the accuracy of sentences, or to edit. In the other condition, he instructed students to use AI as a creative assistant, prompting the AI to create their initial draft. They adjusted their prompting to create three different drafts that the students then refined into a single, final deliverable.

Study design

All students in the course prepared a writing assignment in both AI conditions. Dubriel randomly assigned the order in which students experienced conditions, which counterbalanced the type of AI usage across different writing genres.

Data Sources

Students' two writing assignments scored with a rubric for linguistic accuracy in vocabulary, grammar, and syntax as well as genre conventions, emotional impact, and originality
Students’ reflections on their writing process and the quality of their written assignments
Pre/post surveys about students’ familiarity, competency, and confidence working with genAI

Larry Heimann

Teaching Professor
Information Systems
Heinz College of
Information Systems
and Public Policy

Houda Bouamor

Associate Teaching Professor
Information Systems
CMU-Qatar

Shihong Huang

Teaching Professor
Information Systems
Heinz College of
Information Systems
and Public Policy

67-272 Application Design and Development (Spring 24)

Generative AI Tool(s) Used

ChatGPT, Copilot

Research Question

Does generative AI tool us affect equity in student outcomes, giving less-experienced students a better chance to be successful in technical courses?

Teaching Intervention with Generative AI

Heimann, Bouamor, and Huang introduced generative AI tools (Copilot, ChatGPT) in their course and encouraged students to leverage these tools for solving computer lab assignments and the main course project during the semester. Instructors demonstrated effective generative AI tool use during class to help scaffold students’ learning. They required students to document the frequency of generative AI usage while completing course assignments.

Study design

To gauge students’ level of programming and programming-related experience, Heimann, Bouamor, and Huang surveyed their Spring 2024 students as well as students from the past two iterations of the course (Spring 2022 and Spring 2023) when there was no formal policy for generative AI use and such tools were not as omnipresent in the academic landscape. Then, they encouraged Spring 2024 students to use generative AI tools while completing course assignments. To determine the extent to which generative AI tool use impacts less experienced students, these instructors will compare student work between the past and present cohorts using prior level of experience as a hypothesized moderator.

Data Sources

Surveys of students’ background programming experience
Students’ documentation of generative AI tool use frequency during coursework
Students’ deliverables from coding exercises, exams, and a course project

Alan Thomas Kohler

Senior Lecturer
English
Dietrich College of Humanities and Social Sciences

76-270 Writing for the Professions (Spring 24, Fall 24)

Generative AI Tool(s) Used

Copilot

Research Question

To what extent can the use of generative AI tools improve the student peer review process for students in an intermediate level undergraduate writing course?

Teaching Intervention with Generative AI

Kohler’s students completed a peer-review feedback process for each of five writing projects in his course. For two of the projects in Spring 2024, students completed this process using a generative AI tool (Copilot), rather than another student, as the source of feedback. Students submitted their writing along with the rubric and an instructor-engineered prompt to receive feedback from the AI tool on their writing sample. Additionally, students submitted an instructor-engineered prompt to the AI tool to generate a writing sample. Then, students practiced providing feedback on that sample. Kohler introduced Copilot during class and provided all pre-engineered AI prompts. For each project, students documented the feedback they received and gave, as well as their perceptions on the usefulness of each experience for learning.

Study design

Students used traditional peer review for the first three projects (control) in Spring 2024, but substituted generative AI for peer reviewers (treatment) during the fourth and fifth projects. In Fall 2024, this design will be counterbalanced, with the first three projects using the generative AI-based peer review (treatment) and the fourth and fifth projects using traditional peer review (control). Kohler will compare student perceptions of the feedback process and the quality writing deliverables across conditions.

Data Sources

Pre/post surveys of students’ perceptions of the usefulness of the peer review process
Students’ reflections on the feedback process for each project
Transcripts of feedback given and received for traditional and AI-based peer review
Students’ deliverables for all writing projects, scored with rubrics measuring writing skills

Steven Moore

Graduate Student Instructor
Human Computer Interaction Institute
School of Computer Science

05-840 Tools for Online Learning (Spring 2024)

Generative AI Tool(s) Used

ChatGPT

Research Questions

How does student use of generative AI while creating micro lessons affect:

the quality of their lesson designs?
their learning of fundamental teaching and learning principles?
their self-efficacy as educators and generative AI users?

Teaching Intervention with Generative AI

Moore’s students engage with four interactive, online learning modules on fundamental teaching and learning principles. Each module contains two micro lesson design activities, in which he challenges students to apply the learning principles to their practice. For particular micro lesson activities, he instructs students to use generative AI (ChatGPT) as a collaborator in their design process.

Study design

Moore implemented two conditions, generative AI used (treatment) or not (control) by students, in the single section of his course. For the first micro lesson assignment in each of four online learning modules, randomly, he assigned half of the students to the treatment condition and the half to the control condition. For the second micro lesson assignment in each module, students switched to the other condition. Data sources will be compared for each student between conditions, across modules and micro lesson assignments.

Data Sources

Students’ deliverables from micro lesson assignments, scored via a rubric with criteria for cohesion, correct application of learning principles, diversity of names and pronouns referenced, and diversity of lesson topics
Concept- and application-based multiple choice questions embedded in online learning modules and aligned with course learning objectives
Transcripts of students' interactions with generative AI
Pre/post surveys of students’ self-efficacy regarding skills using generative AI and course learning objectives

Carrington Motley

Assistant Professor
Tepper School of Business

70-415 Introduction to Entrepreneurship (Spring 2024)

Generative AI Tool(s) Used

Copilot

Research Questions

Do students generate more or fewer distinct ideas when using AI while brainstorming?
How does the nature of students’ ideas change when using AI while brainstorming?
How does brainstorming with AI impact student self-efficacy regarding AI use and course learning objectives?

Teaching Intervention with Generative AI

Motley implemented scaffolded brainstorming sessions during class to support ideation for entrepreneurship projects (by individuals). Students then leveraged generative AI tools (Copilot) to support both the generation and evaluation of ideas for new business ventures. Individual students created “pitch decks” (slides) to present their ideas to their peers to recruit collaborators to design a business implementation plan. Teams of students then collaboratively designed implementation plans for the entrepreneurship projects chosen.

Study design

All students in two concurrent course sections received training on brainstorming techniques. Motley randomly assigned two conditions to sections: students used (treatment) or did not use (control) generative AI tools in brainstorming exercises during class. Treatment groups received training on generative AI use focused on prompt engineering. Data sources will be compared between course sections, statistically controlling for variation in students between conditions.

Data Sources

Artifacts of brainstorming sessions, including google docs (control and treatment) and transcripts from generative AI use (treatment)
Students’ pitch decks (slides from student presentations), scored using a rubric with criteria for uniqueness of the problem being solved, the solution, and the customer segment targeted
Pre/post surveys of students’ self-efficacy regarding skills using generative AI tools and course learning objectives

Fethiye Ozis

Assistant Teaching Professor
Civil and Environmental Engineering
College of Engineering

12-333 Experimental and Sensing Systems Design and Computation for Infrastructure Systems (Spring 2024)

Generative AI Tool(s) Used

PerplexityAI

Research Questions

Does utilization of AI tools impact students’ skills for data processing, cleaning, and visualization of large data sets?
What are the attitudes, perceptions, and experiences of students regarding AI-powered tools for data processing and visualization?

Teaching Intervention with Generative AI

Ozis introduced generative AI (PerplexityAI) as a possible support tool during students’ multi-week, big-data group project. Students had the option to use AI during their data cleaning and visualization tasks. They were not restricted in how they could choose to use the tool but were given some possible uses, such as a coach to provide advice or a tool to detect outliers in the dataset or to provide code to create data visualization plots in Python.

Study design

Students could choose to opt into using AI during their data project, creating a self-selected group of AI users to compare to a group of non-AI users within the course. Additional comparisons will be made to a previous iteration of the course that did not use AI.

Data Sources

Students’ data visualizations, cleaned datasets, and documentation of process scored with a rubric for ability to clean, analyze, and visualize large data sets
Students’ reflections on how effective, challenging, and rewarding their data cleaning process was and whether or not they used AI in their process

Jordan Usdan

Adjunct Faculty
Heinz College of Information Systems and Public Policy

94-816 Generative AI: Applications, Implications, and Governance (Spring 2024)

Generative AI Tool(s) Used

ChatGPT

Research Questions

How effective is AI at enhancing student thinking, research, and writing skills?
How does AI impact research and writing efficiency?
Are there different impacts of AI across writers with different English language proficiencies or other characteristics?

Teaching Intervention with Generative AI

Usdan provided students with an overview of prompt engineering and ways to use generative AI (e.g., ChatGPT), as a tool for summarization, information synthesis, research, explanation, idea generation, and more. Classroom demonstrations of potential student applications of generative AI included as a prose assistant, editor, thought partner, and critic. Across three policy memo assignments, he instructed students to not use, to use extensively, and finally to choose whether or not to use a genAI tool while preparing their written submission.

Study design

All students in the course prepared a writing assignment in each of three conditions: first without genAI, then with the use of genAI, and finally with their choice of using genAI or not. While the order of these conditions did not vary, Usdan counterbalanced (randomized) the specific policy scenario assigned across these conditions.

Data Sources

Students’ three writing assignments, scored with rubrics measuring thinking (e.g., logical policy, argument to support policy) and writing skills (e.g., quality of summary, concision, organization)
Students’ self-report of writing efficiency, i.e., students tracked the time they spent actively engaged in completing each writing assignment
Pre/post survey about students’ writing confidence and perspectives on AI as an educational tool
Post survey about students’ perceived improvement in their writing and attribution of improvement to repeated writing practice versus use of AI

Liz Walker

Graduate Student Instructor
English
Dietrich College of
Humanities and Social Sciences

Bonnie Youngs

Teaching Professor
Department of Languages, Cultures, and Applied Linguistics
Dietrich College of
Humanities and Social Sciences

66-139 DC Grand Challenge Seminar: Reducing Conflict Around Identity and Positionality (Spring 2024)

Generative AI Tool(s) Used

PerplexityAI

Research Questions

To what extent does student use of generative AI impact their:
1. ability to critically read and analyze academic papers?
2. self-efficacy as critical readers and generative AI users?

Teaching Intervention with Generative AI

Walker and Youngs provided classroom training on how to read academic papers as well as how to engineer generative AI prompts and evaluate generative AI output. Students then used generative AI (Perplexity AI) as a reading support tool prior to class discussions by uploading assigned readings and individually engaging with the AI as a dialogue partner, asking questions to clarify paper content and potential interpretations of the text.

Study design

Walker and Youngs required every student to use the AI tool for each assigned reading in Spring 2024. They will compare student responses to reading questions to a previous semester, when students did not use generative AI, but only for questions used in both semesters. Student self-efficacy will be measured at the beginning and end of Spring 2024.

Data Sources

Students’ responses to assigned reading questions, scored with rubrics for academic reading skills (e.g., reading comprehension, metacognition, critical analysis of text)
Transcripts of students' interactions with generative AI, analyzed for ability to ask productive questions
Pre/post surveys of students’ self-efficacy regarding skills using generative AI and course learning objectives