Carnegie Mellon University

Carnegie Mellon Robotics Academy

Use educational affordances of robotics to create CS-STEM opportunities for all learners

p3g_mock_04-03.png P3G: Player-Programmed Partner Games for Low-Resource Learners

p3g-collage-v3.1.png

Player-Programmed Partner Games (P3G) for Low-Resource Learners is an Innovations in Development project that seeks to increase access to informal STEM education in out-of-school time (OST) settings, while simultaneously bringing the human-robotics interaction paradigms of robotics education up to date. It addresses gaps in existing robotics STEM education programs through Design-Based Research (DBR) into a video game-based OST learning experience that teaches coding and computational thinking in a natural co-robotics context.

Publications

  • Chen, Y., Kemper, J., Harpstead, E., Higashi, R., & Uchidiuno, J. (2023). Designing Black Children in Video Games. ACM Interactions
  • Solyst, J., Odili Uchidiuno, J., Harpstead, E., Kemper, J., & Higashi, R. (2023). Comparative Design-Based Research: How Afterschool Programs Impact Learners' Engagement with a Video Game Codesign. arXiv e-prints, arXiv-2307.
  • Uchidiuno, Judith Odili and Solyst, Jaemarie and Harpstead, Erik and Higashi, Ross. (2023). “I'm a little less inclined to do it”: How Afterschool Programs’ Culture Impact Co-Design Processes and Outcomes. Designing Interactive Systems Conference (DIS '23). 1263 to 1276.
  • Harpstead, E. (2023). Learning within Gameplay Loops: Considering Adaptive Educational Technology from a Game Design Lens. Proceedings of the 25th International Conference on Human-Computer Interaction, to appear.
  • Solyst, J., Uchidiuno, J. O., Harpstead, E., Kemper, J., & Higashi, R. (2023). Comparative design-based research: How afterschool programs impact learners’ engagement with a video game codesign. To appear in the Proceedings of the International Conference of the Learning Sciences (ICLS), June 10-15, 2023, Montreal, Canada.
  • Uchidiuno, J. O., Solyst, J., Kemper, J., Harpstead, E., Higashi, R., & Hammer, J. (2023). “What's Your Name Again?”: How Race and Gender Dynamics Impact Codesign Processes and Output. ACM Transactions on Computer-Human Interaction.
  • Flot, J., Harpstead, E. (2022, August 7-9). Beyond the Block: Sustained Engagement and Accessibility with Coding Games in Afterschool Programs [Conference presentation]. 2022 Play Make Learn Conference, Madison, WI, USA. (Presentation)
  • Higashi, R., Uchidiuno, J. Kemper, J. (2022, August 7-9). (Co-)Designing for Diversity: Games across afterschool programs [Conference presentation]. 2022 Play Make Learn Conference, Madison, WI, USA. (Presentation)
  • Kemper, J. (2022, August 7-9). Real Recognizes Real: Toward Eliminating Barriers to Children's Generative Ideas in Co-Design [Conference presentation]. 2022 Play Make Learn Conference, Madison, WI, USA. (Presentation)
  • Higashi, R., Harpstead, E., Solyst, J., Kemper, J., Odili Uchidiuno, J., & Hammer, J. (2021, October). The Design of Co-Robotic Games for Computer Science Education. In Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play (pp. 111-116).
  • Odili Uchidiuno, J., Solyst, J., Kemper, J., Harpstead, E., Higashi, R., & Hammer, J. (2021, October). Negotiating Systemic Racial and Gender Bias as a Minoritized Adult Design Researcher. In Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play (pp. 203-208).

This material is based upon work supported by the National Science Foundation under Grant Number 1906753.

P3Gs utilize a novel hybrid interactivity approach in which players both control an on-screen avatar in real time, but also program one or more computational agents to actively assist in accomplishing in-game objectives. This strategy allows designers to leverage benefits of games-based instruction such as the ability to act in unfamiliar roles (role-taking), lowered psychological costs of failure, and natural scaffolding through progression. As use of the game is situated within OST spaces, an additional layer of social interaction design utilizes the affordances of out-of-school learning spaces in low-resource neighborhoods to promote long-term re-engagement by learners.

A number of variations (e.g. different themes) and extensions (e.g. adding digital badges) are planned in addition to continuous iteration on the core P3G design. Data collection methods include automated in-game telemetry and archiving of student source code, in-game surveys, qualitative observation, co-design with students and providers, and focus groups.

A small subsample of participants are given pre-post computational thinking knowledge tests. Research products include documentation of design features that are effective in low-resource OST contexts at producing increases in learner engagement, competency beliefs, and proficiency in coding, computational thinking skills around algorithms, and career interest in STEM fields.

The end goal is to build both design theory and exemplar products that allow OST learners who cannot afford robot kits and have never thought of themselves as robot programmers to adopt that role, learn and apply the computational thinking skills therein, and perhaps come to see themselves as co-robot developers.

p3g.png

Figure 1. Conceptual mockup of the “mining” P3G user interface showing: (A) the command library from which players select commands; (B) the current program the player has written for the “miningdrill” agent (note tabs above to select between agents); (C) the player’s manually-controlled avatar in the mine; (D) the minecart co-robot agent in the mine; (E) the miningdrill co-robot agent in the mine.

Intellectual Merit

We use design-based research methods to provide ecologically valid answers on central issues of design for low-resource learners in informal education settings. Specifically, how games can be best designed to achieve STEM learning and motivational outcomes for populations with low initial awareness and motivation toward STEM, in out-of-school time spaces where trained adult facilitators and mentors cannot be assumed. Organic and planned variation in the design of the game, combined with analysis of differential learner responses to the intervention by age, sex, race, prior interest, and other factors may also advance our understanding of OST interventions more generally. Further, exploration of the design of games in a novel genre based around real-time cooperative interactions between human players and computational agents they program themselves represents a contribution to the field of games-based learning. Additionally, a planned set of P3G extensions test diverse hypotheses across numerous disciplines.

For example, the Online Asynchronous Social Interactions extension asks whether the situation of P3G participation within broader online social event structures (e.g. online competitions) affects learner engagement, whether effects vary based on competitive vs. celebratory vs. collaborative framings of that broader structure, and whether agency to choose a framing has an independent effect. This work contributes to research on learning motivation. Other extensions test adaptive tasks (intelligent tutoring systems, games-based learning) and digital badges (motivation, microcredentials). Finally, the project employs novel large-scale data collection and formative analytics methods that are of interest to their respective fields.

Broader Impacts

We anticipate that a successful implementation will give other researchers and commercial developers a basis for developing games within the new P3G genre; as the genre is inherently based on principles of co-robotics and coding, such follow-on activity should also increase exposure to these concepts and opportunities to learn STEM skills in the general population. 

Additionally, the work is intended to become self-sustaining after the period of grant support. While the game and online event structure will remain free for informal education, successful uptake will attract demand from formal educators who are able to access school funding to purchase classroom-adapted versions (e.g. for intramurals), expanded topic coverage, accompanying formal lessons, professional development, and so on.

Project Team Game Jam

Description: As a new genre, the Player-Programmed-Partner Games presents a number of unknowns both in terms of methodology and technology. To begin to explore both, the project team ran an internal “game jam” to drive the development of prototypes. The game jam resulted in 12 initial game pitches that explored different relationships between the player and co-bot, a variety of settings, and possible technology implementations. 

gamejam-1.png
gamejam-2.png
gamejam-3.png

Technical Prototype 1: Masterpiece by Numbers

Game Description: In this game, we explore a 2D design where the co-bot is an assistant to the player. The co-bot is implemented as 1 (or more) brushes which are programmable through simple logic to create sketches for the player. This allowed us to test the feasibility of simple in-game programming environments as well as large numbers of programmable agents. 


Game Link: https://cs2n-unity-games.s3.amazonaws.com/Paint-by-numbers/index.html


How to Play

  • Add paint brushes using commands from the programming interface. Use the parameters of the commands to change the behavior of the brush. Drag the command to the black box on the right, and click run program to create the brush.
  • Once you have a brush selected, you can click or drag anywhere on the black canvas. The pen will start to follow the cursor defined by the behavior you gave it in the programming interface. 
  • You can add more pens by clicking the ‘+ paint brush’. The programming interface will pop back up and you will be able to add another brush.
  • Controls:
    • Use the mouse to draw paths on the black canvas
    • ‘U’ reloads the scene
    • ‘`’ (tic) toggles Utility Overlay

Screenshots:

mbn-1.png
mbn-2.png
mbn-3.png

Technical Prototype 2: iSuffer (so you don't have to)

Game Description: In this game, we explore a 2D design where the co-bot is expendable in service to the player. The co-bot is implemented as a programmable agent that will take damage, allowing the player to safely navigate a “fog of war” style maze. This allowed us to test the player-programmed-partner design with event-based programming logic and world triggers. 


Game Link: http://corobotpoc.s3-website-us-east-1.amazonaws.com/


How to Play

  • Make a search bot by pressing the plus button on the left. 
  • Once you click this button, you will be able to program the robot. The yellow blocks are event blocks. The Green blocks are command blocks.
  • Using these mechanics, navigate the maze until you get to the green circle:
    • When Searching - This event triggers by default. Any block under the ‘when searching’ block will be executed as long as the other events do not get triggered. 
    • When Enemy -  Any code under this block will execute when the search bot sees an enemy. 
    • Move To (x, y) [click] -  This block commands the search robot to go to a specific location in the maze. Since users do not know the coordinate system, they can take advantage of the click button on this command. If the user clicks the command, the PI (Programming Interface) will collapse and instructions will ask the user to click a point on the screen. When the user clicks an area, the PI will pop back up and the coordinates for this block will automatically populate. 

Screenshots:

isuffer-1.png
isuffer-2.png

Technical Prototype 3: Ruins of Atlantis with Blockly

Game Description: In this game, we explore a 3D design where the co-bot is an underwater agent that can move, collect treasure, and shoot torpedoes. The co-bot programmable through a full implementation of Blockly. This allowed us to test the feasibility of combining a rich 3D environment with a full featured, block-based programming environment that many students have some prior experience with. 


Game Link: https://cs2n-blockly.s3.amazonaws.com/index.html


How to Play

  • Drag the blocks from Robot and Logic section of the Blockly command list into the area on the right to form a program. 
  • Press Run Code to execute the program on the virtual robot in the 3D game space. 
  • Press Escape to reset the world state. 

Screenshots:

roa-1.jpg

Naive Game 1: Untitled Generation Space Game

  • Game Pitch
  • Description: Untitled Generation Space Game is designed and developed by expert game designers and researchers at Carnegie Mellon University. In this game, students have a cobot partner to help them complete tasks such as cleaning up a messy bay on a spaceship. 
  • Screenshots:




Co-Designed Game 1: Super Slime Battle

  • Play on CS2N!
  • Game Pitch
  • Description: Super Slime Battle is a game that was co-designed between researchers at Carnegie Mellon University and students from several Boys and Girls Clubs of Western Pennsylvania locations. In this game, students play as one of 4 Halloween-themed characters. They program their creature-themed cobot partner to help them fend of baddies who are trying to invade their town and steal their candy.
  • Screenshots:






Co-Designed Game 2: Acceleration City

  • Game Pitch
  • Description: Acceleration City is currently being co-designed between researchers at Carnegie Mellon University and students from the Boys and Girls Clubs of Western Pennsylvania. In this game, students are able to freely drive and explore a diverse digital space with the aid of their cobot. 
  • Screenshots:






Co-Designed Game 3: Zillah City

  • Game Pitch
  • Description: Zillah City is currently being co-designed between researchers at Carnegie Mellon University and students from the Boys and Girls Clubs of Western Pennsylvania. In this game, students take back their city from unwanted pests with the help of their cobot. 
  • Screenshots:




 

Co-Designed Game 3.5: Zillah Beats

  • Play on CS2N!
  • Description: Zillah Beats is a game that was co-designed between researchers at Carnegie Mellon University and students from several Boys and Girls Clubs of Western Pennsylvania locations. Players program their cobot hoverbots to change lanes, avoid obstacles, and take out many Toxibots in this rhythm-styled game.
  • Screenshots:
    zillah-beats-game.png

    zillah-beats-programming.png

Co-Designed Game 4: Friends Forever

  • Game Pitch
  • Description: Friends Forever is a game that was co-designed between researchers at Carnegie Mellon University and students from Center that Cares. Friends Forever is a creepy escape-the-room style puzzle game where a cobot doll helps you to navigate and escape a haunted house. 
  • Screenshots:
    friends-forever-gameplay.png

    friends-forever-coding.jpg

Co-Designed Game 5: Battle for the Hill

  • Game Pitch
  • Description: Battle for the Hill is an analog game that was co-designed between researchers at Carnegie Mellon University and students from Center that Cares
  • Photos:
    battlebox.jpg

    battleboard.jpg

Description: A design affordance that goes hand-in-hand with informal education is student agency. Students have an incredible amount of choice in how they spend their time in informal settings such as after-school clubs, community organizations, libraries, and museums. For games to make a meaningful impact in these spaces, they must be something that students choose to engage (and re-engage) with. To design games that take our core audience and the spaces they play into account, we began our research by partnering with a variety of informal education partner sites located in Pittsburgh, Pennsylvania. In each site, we collected data described in the following categories:

Site summary: After each site visit, the researchers that conducted the observation use the data collected in the following categories to produce a summary document. 

Space Layout The space is sketched from a top-down view. Each sketch will be complemented with an album of photos. 

Behavioral Observations During this observation, researchers conduct a behavioral observation of the site when students are using the space. Researchers note the information in regards to how the students interact with the site. Some examples a researcher might observe are:

  • What applications the students are using in the space?
  • Are there any adults present? If so, what are their interactions with the students like?
  • Is the space crowded?
  • What are the students interacting with the most?

Technical Observation This observation can be a mix of quantitative and qualitative data. Researchers note down technical statistics about space. For example:

  • How many machines are available in the space?
  • What are the machines in the space that can be identified? (i.e. phones, tablets, Xbox, PC, Chromebooks)
  • Do the students have access to the machines? (only observed during peak hours)
  • Can you visually see other programs installed on the machines? 
  • Are the machines older, newer, too hard to tell?
  • What input devices are observable? (ie mouse and keyboard, controller/joystick, Kinect)

Internet Speedtest Two “speed test inquiries” will test Internet connection speed on the computing devices and networks available at the site. The first will be conducted when the space is empty. The second test will be conducted when the space is full/peak hours of the facility.

Description: Due to restrictions and precautions brought on by COVID19, some planned face-to-face co-design and observation research was not possible. In response, we pivoted some of our research instrumentation to cultural probes that can be disseminated digitally and through the mail, called CoBoxes

Probes at their base: 

  • Are a collection of small activities designed around a theme with everything inside to allow the participant to engage with research needs. 
  • Should be playful and encourage engagement. 
  • Should be self-contained. 
  • Should allow researchers to collect useful data and code their responses. 
  • Have a clear way to return the artifacts. 

probe-ideas.jpg

Designing Probes for Children 

Cultural probes are used frequently with children, but require careful thoughts and planning. According to Wyeth and Diercke: 

 “The number of activities included in the probe should be carefully considered with the view of facilitating a better activity completion rate. Activities which focus on topics to which children easily relate are well received and responses are useful in guiding the early design process. However, participants are likely to avoid ‘difficult’ activities or complete them with little attempt at thoughtful interpretation. In addition, cultural probes are not tools that provide a view of specific learning preferences. Such information may be better gathered using alternate techniques.” 


The Design of Our Probes

  • We want our probes to be engaging, fun, and not be a burden to children or parents during the pandemic lockdown. 
  • We want our probes to answer the research questions we have gathered. 
  • We want them to be accessible to our target population, and not leave behind marginalized groups.
  • We want them to specifically engage in our themes and topics without overwhelming our population. 
  • We want them done simply in a quick time frame in order to gather research results and start game design. 

Implementation: CoBox

  • Like a free subscription box, CoBox includes five simple activities for a child to create and send in. The child does *not* have to complete all five activities. 
  • According to Wyeth and Dierke, the most engaging activities were creative and constructive with an emphasis on roleplay, invention and design. CoBoxes have five simple activities that revolve around those. 
  • CoBoxes include easy-to-understand instructions that would describe each probe and have a singular place to upload photos of the artifacts where children can view other responses. 
  • CoBoxes include a short child-centered explanation of cobotics.

Produced CoBoxes:

The CoBoxes produced and implemented contained engaging activity cards, along with all materials necessary for children to complete them. Digital versions of the activity cards from CoBoxes 1 - 3 are included here. 

coboxfrontclosed.jpg cobox1shot.jpg

How We Get Data Back

As children complete the engaging activities in the box, the activity cards direct them to upload pictures and narrations of their work using the cobox.me web application. Each child receives a unique code as part of their first box that allows them to log in and share their work and ideas.

Once pandemic restrictions were relaxed and students were able to re-enter their OST spaces, the research team collaborated with OST staff to deliver in-person codesign sessions. The research team created scaffolded activities with just-in-time professional development and instructions for OST facilitators to run activities and collect student codesign data. These activities fostered student thinking and engagement around programming, game design, and corobotics, while allowing students to share their designs, ideas, and preferences in codesign games. 

cs2n_group.png cs2n_page.png

While many of the students that we codesigned with played a variety of games, we noticed that 1) many did not have a game-based vocabulary that allowed them to express their preferences, 2) sometimes expressed preferences were really something entirely different upon further analysis, and 3) there was sometimes a disconnect between the games students were playing and the research team was familiar with. 

To help students grow their gaming vocabulary while also more meaningfully understanding their ideas and preferences, we created a curated list of 40 games across 7 loosely-defined genres. These games were available to students as an unstructured play activity at the end of co-design sessions. Researchers could then observe their preferences empirically and have conversations around game features with shared understanding.

cobox_game_site.png