Carnegie Mellon University

Carnegie Mellon Robotics Academy

Use educational affordances of robotics to create CS-STEM opportunities for all learners

FACILITATE

Formative Assessment using Computationally-driven Insights of Learner Intent via Transdisciplinary AI for Technology Education

Summary

Intelligent Tutoring Systems and Analytics Dashboards have shown the potential of technology in education far beyond being a topic of instruction or method of delivery. Machine Learning and Artificial Intelligence (ML/AI) have long been employed in support of those roles. However, these approaches are strongly student- or teacher-facing; they do not focus on instructional interactions between instructors and students. FACILITATE explores an approach in which AI targets the entire instructional triangle: students, teachers, and curriculum content as a contextualized, interacting triad.

convening_ai_diagram.png
Contrasting models of AI roles under intended system usage. ITS AI is student-facing through curriculum. Dashboard AI is teacher-facing through information display. A convening approach brings students, teachers, and curricular context together around excerpts of student work-in-progress.

Specifically, we propose an AI-driven tool could bring students and teachers together around specific “moments” in student work, playing a convening role. This leaves intact the core of the triadic interaction, including instructors’ preferred methods, rapport, awareness of external circumstances, and special expertise, which are out of scope for course-level AI.

FACILITATE addresses the known and unknown technical and sociotechnical integration challenges of an AI-driven convener through design-based research, by developing a proof of concept Formative Assessment Suggestion Tool (FAST) in the context of middle school robotics programming. By examining patterns in student code and simulated robot output, a convening AI could identify students who are stuck, infer students’ intended solution pathway, locate the first point of divergence from a working solution, then brief the teacher and call a focused help session with the student. These map closely onto areas teachers identified as most time-consuming in practice.

fast_system_mockup.png
Conceptual mockup of the FAST system. Student code and simulator output (A) are analyzed by the FAST Convening AI. Key segments are excerpted (B) and used to convene teachers and students to troubleshoot around the focal clip (C).

Thus, we hypothesize that a convening AI would accelerate and focus instructional facilitation for open-ended problem solving. We also expect that retained freedom over methods will increase instructor comfort, acceptance, and correct use of the technology; in turn, more students should be helped, in a shorter time; yielding equal or better learning and motivational outcomes.

Intellectual Merit

FACILITATE explores an alternative convening approach to the design of AI for educational settings, a significant departure from existing bodies of work such as Intelligent Tutoring Systems – which effectively replace the teacher for routine interactions – and Analytics Dashboards, which inform teachers but do not connect strongly to well-formed and accepted teaching routines. Design-based exploration of this concept also includes an unpacking of, and comparison to, the de facto manual methods teachers to locate such moments today. Research products are expected to include technical papers on the ML implementation, design publications, contextual models of troubleshooting workflow with and without FAST, and exploration of novel concerns and phenomena surfaced through the work.

Broader Impacts

FAST prototypes are expected to directly impact 500 students per year through directly collaborating teachers. A significant number will be minority, low-SES participants in an urban school system. We further anticipate this population will be concentrated in remote and hybrid instruction settings, as FAST and its accompanying toolset are entirely virtual and optimized for low-cost computing hardware such as Chromebooks. Troubleshooting individual student programs is particularly difficult in remote settings, so FAST may be a unique solution, encouraging further adoption.

Indirect impact is expected through use in Robotics Academy professional development activities, which reach approximately 500 teachers per year. Untrained facilitators (e.g. in informal education, or teaching out-of-discipline) may also benefit from FAST’s automatic identification of important areas of student work and solution pathways.

The structure of FAST makes it rapidly portable to any brand of hardware and software, and our lab has done similar work in the past. The convening concept is also generalizable to any content domain with an analogous simulation structure. Design documentation and source code are open-sourced in support of this.

Finally, because convening leaves existing micro-instruction intact, it is fully compatible with ITS systems and other pedagogical enhancements, both technological and non-technological. It can thus be adapted to benefit teachers and learners in many other settings, and potentially accelerate interaction research taking place in those settings by reducing classroom time spent per interaction.

This material is based upon work supported by the National Science Foundation under Grant Number 2118883.