Carnegie Mellon University

Eberly Center

Teaching Excellence & Educational Innovation

R&S Digest Header Image

Archive of Past Articles

Gouvea, J. S. (2024). Ethical dilemmas in current uses of AI in science education. CBE—Life Sciences Education, 23(1), fe3. 

This piece briefly reviews some recent papers on the use of AI tools to assess student work. It begins by discussing a piece laying out how machine learning assessment is not neutral, and can reinforce a bias towards certain White norms of communication. Then, the review explores the controversy around a study in which an AI tool was trained to use a rubric to evaluate the scoring of scientific explanations and diagrams. Although the automated scores were similar to those given by humans, the model was easily confused by unexpected details, and the authors conclude that the diversity of study work may need to be constrained to allow for the automation of scoring. The series of responses to this paper are presented. Lastly, the author presents a paper which argues that although the harms of using AI to automate assessments or otherwise replace human educators outweigh the benefits, there are more promising ways to use AI as a collaborator with unique abilities. At the end, the author reminds us that these conversations will of course be ongoing, and that they should remain "not just about what AI *can* do in a technical sense, but also about making ethical choices about how or if it *should* be used and to what ends."

Payne, T., Muenks, K., & Aguayo, E. (2023). “Just because I am first gen doesn’t mean I’m not asking for help”: A thematic analysis of first-generation college students’ academic help-seeking behaviors. Journal of Diversity in Higher Education, 16(6), 792–803.  

Many studies about first-generation students tend to contribute to a deficit narrative, including that they are less likely to seek academic support (e.g., attend office hours, use campus resources) when compared to continuing generation students. In this study, the authors conducted focus groups with seventeen first-generation students about their academic help-seeking behaviors. A thematic analysis of the transcripts of the focus groups found that these students do effectively seek help, but often the help they sought was not within formal institutional structures. Instead, these students actively engaged in help-seeking behavior by, for example, looking for appropriate online resources or relying on their peer networks, thus countering the deficit narrative and emphasizing the skills first-generation students possess. Whether students sought out more “formal” help from professors depended, in part, on the extent to which students found their professors approachable. The authors encourage instructors to provide opportunities for their students to develop strong peer relationships and to normalize seeking help, including by explicitly stating that they expect students to have questions and responding positively to those questions.

Tise, J. C., Hernandez, P. R., & Schultz, P. W. (2023). Mentoring underrepresented students for success: Self-regulated learning strategies as a critical link between mentor support and educational attainment. Contemporary Educational Psychology, 75, 102233.

This longitudinal study conducted over a 12-year period across 38 universities surveyed over 1400 predominantly undergraduate students from historically underrepresented groups to examine the extent to which the presence and quality of faculty mentorship, self-regulated learning (SRL) strategy use, and student growth mindset is associated with academic achievement. Students were surveyed biannually while enrolled as students and tracked over a 12-year period to monitor degree attainment (e.g., Baccalaureate, Master’s, Ph.D., etc.). The researchers had four main research questions: 

1) Does student use of SRL strategies change over time? 

2) Does growth mindset, having a faculty mentor, and level of mentor support predict SRL strategy use? 

3) Does SRL strategy use predict degree attainment?

4) Does SRL strategy use mediate the relationship between mentoring or mindset and degree attainment?

The results showed that students’ SRL strategy use was relatively stable over time, and was positively associated with the presence (vs. absence) of a faculty mentor, as well as the degree of mentor support, but not growth mindset. SRL strategy use also positively predicted academic achievement (a previously well documented finding). Moreover, it was found that the effect of having a faculty mentor, as well as the quality of mentor support, on academic achievement was mediated by SRL strategy use. More plainly, these results show that one of the mechanisms through which faculty mentorship affects academic achievement is the greater use of SRL strategies. The authors point out that this work represents some of the first empirical exploration of how faculty mentorship can benefit underrepresented students, and the results can hopefully be leveraged to further advocate for its importance.

Xie, H., Lin, D., He, W., & Chen, Q. (2024). The aesthetics at a pencil tip: The effects of drawing on learning poems. Learning and Instruction, 91, 101881. 

Does asking students to draw pictorial representations of humanities texts help them remember what they read? In this study, students read fragments of poems and were prompted to draw a representative illustration, read it out loud, or rewrite it verbatim on a new sheet. They then took an item recognition test (which asked them whether a specific item had been “studied” or “not studied”), a source memory test (where students had to identify whether a “studied” item had been “drawn”, “written,” or “read”), and a motivation questionnaire.  In this experiment, the performance of item recognition accuracy as well as source memory in the drawing condition was better than that in the reading and writing conditions. Additionally, the learning motivation in the drawing condition was higher than that in the reading and writing conditions.

The researchers also conducted a second experiment that compared the drawing strategy to an explanation strategy, where students were asked to paraphrase the presented poems. They also tested delayed (1-week) performance in addition to immediate performance. In this experiment,  immediate performance of item recognition accuracy as well as source memory in the drawing condition was not as good as that in the explaining condition. However, drawing succeeded in assisting students with slower forgetting and higher motivation when compared to explaining.

Laflen, A. (2023). Exploring how response technologies shape instructor feedback: A comparison of Canvas Speedgrader, Google Docs, and Turnitin GradeMark. Computers and composition, 68, 102777.

How might the choice of technology influence instructor feedback on students’ written work? This study analyzed an instructor’s comments in Canvas Speedgrader, Google Docs, and Turnitin GradeMark to see if there were differences in comment location, type, focus, and form across the three technologies. The instructor cycled through the tools in three assignments in three sections of a first-year writing course so that each student received feedback in all of the programs. The researchers found notable differences in the instructor’s commentary. For example, comments in Google were more likely to be suggestions than in Canvas or Turnitin, while comments in Canvas were more likely to provide info/criticism than in Google or Turnitin. Furthermore, comments in Turnitin were more likely to provide praise or clarification than in Canvas or Google. The author acknowledges that the observed effects are not necessarily generalizable to other contexts or instructors, but the study provides a valuable framework for considering how a technology might influence the focus/form of instructor feedback (in addition to the technological affordances of each tool).

Samudra, S., Walters, C., Williams-Dobosz, D., Shah, A., & Brickman, P. (2024). Try Before You Buy: Are There Benefits to a Random Trial Period before Students Choose Their Collaborative Teams?. CBE—Life Sciences Education, 23(1), ar2. 

A question that many instructors face is whether, and to what extent, students should have choice in forming groups for collaborative work. The authors of this paper wanted to investigate whether the patterns that students show when choosing their own groups would be different if they were first forced to work in randomly assigned groups for a “trial period”. They set up their Biology lab such that students in some sections were allowed to form groups on their own (unstructured), and in others the students were only allowed to choose their groups after a trial period of working in randomly assigned groups for four weeks (structured). Students in unstructured sections were allowed to change their groups each week during the first four weeks if they desired. Using student ratings of group members and social network analysis, the authors found that students in the structured and unstructured sections reported no differences in any quantitative outcomes (e.g., group satisfaction, conflict, grades), and that students in both sections used the same criteria to form their groups when given the choice (e.g., demographic similarities, friend status). In addition, students in structured sections were more likely to report negative group member behavior during the trial period and less likely to remain with those individuals when the trial period ended. Overall, although the authors found no impact of the random assignment trial period in terms of group functioning or how students selected their groups, they point out that the use of randomly assigned groups could help to surface problematic group behavior and present opportunities for instructor-mediated intervention in some cases.

Webb, D. J., & Paul, C. A. (2023). Attributing equity gaps to course structure in introductory physics. Physical Review Physics Education Research, 19(2), 020126.

The authors argue that differences in achievement between groups of students are not due to student deficiencies, but rather to course structures, and therefore that changing the structure of a course may remove equity gaps without changing the course’s topics or rigor. They present two examples, both from physics classes at a large public research university in the United States: a change to the order in which course material is covered, and the addition of retake exams. First, the authors found that in one section where content was reorganized so that all concepts are explored before any calculations begin, there was no gap in final exam scores between URM and non-URM students, whereas three sections that worked through the material in order have an average 0.79 SD grade gap. Second, they show that in four classes that offered an optional exam retakes, women had slightly higher grades than men, but in 54 classes without retakes the opposite was true. Neither intervention closed both achievement gaps, but they support a "course deficit," rather than student deficit, model. Additionally, the authors discuss at length the difficulties of measuring student preparation, how the variables often used as measures of preparation are linked to demographics, and cautions for anyone interested in controlling for preparation.

Connell, G. L., Donovan, D. A., & Theobald, E. J. (2023). Forming Groups in a Large-Enrollment Biology Class: Group Permanence Matters More than Group Size. CBE—Life Sciences Education22(4), ar37.

Many forms of active learning involve students talking and working in groups of peers, but what is the best way to form these groups? The instructors of this large-enrollment biology course had previously established that creating heterogeneous groups of high- and low-performing students benefit the initially low-performing students more than working in homogenous groups. In this study, the instructional team tested the impact of two more variables, group size and permanence, on both test performance and attitudes towards group work. In the first experiment, 364 students completed both in-class work and the group phase of a two-part exam in groups of either 3 or 6 (with all groups being heterogeneous and balanced so that women and BIPOC students were not isolated). Students in larger groups did slightly better on the group portion of exams, although it is not possible to determine whether this is because larger groups happened to contain higher-performing students or because knowledge was being pooled, and there was no effect of group size on either content acquisition or student attitudes. The authors feel that this is not strong evidence for larger groups, and the best group size will most likely depend on contextual factors such as room layout and the size that best allows the creation of heterogeneous, non-isolating groups. In the second experiment, students were assigned to groups of six which either stayed the same throughout the quarter, or reshuffled three times. Students in the permanent groups reported being less frustrated with their group members, having a greater sense of interdependence, and felt more support from their peers, with no interactions with demographics (although BIPOC students reported lower satisfaction and peer support regardless of how groups were formed). 88.5% of students in the permanent groups said they preferred not being shuffled, with the most common reason given being that "students felt more comfortable sharing ideas and asking questions when they were in groups with people that they knew well." The authors recommend that instructors form permanent groups, although with an option in place for shuffling highly dysfunctional groups.

Emberley, A. C., Choi, D. S., Williams, T., & Loui, M. C. Engineering survivors: Students who persisted through academic failures. Journal of Engineering Education

How do engineering students who persist through academic failure respond to failure? The researchers in this study interviewed 26 undergraduate students who continued to pursue an engineering degree after failing a required technical course. The interviews consisted of questions designed to elicit students’ attitudes related to three guiding theories: goal orientation theory, mindset theory, and grit. The authors constructed four themes that describe the responses (unresponsive, avoidant, floundering, and rebounding) and suggest strategies to support students who might respond to failure in different ways. For example, for unresponsive and avoidant students, instructors can try to normalize failure and frame it as an opportunity to grow through exam corrections.

Estefan, M., Selbin, J. C., & Macdonald, S. (2023). From inclusive to equitable pedagogy: How to design course assignments and learning activities that address structural inequalities. Teaching Sociology, 51(3), 262–274.

Some students, including first generation and working class students, face structural barriers to their learning. For example, students with long commutes or other obligations may not be able to take advantage of office hours. The authors encourage instructors to think beyond communicative strategies (such as promoting a growth mindset) that can make courses more inclusive and to examine how they can structurally (re)design aspects of their courses to make them more equitable. The authors provide several strategies they have used in their courses. One example is to dedicate some class time to convey information and insights discussed during office hours to all students. Another example is incorporating a discussion board activity to help students prepare collaboratively for exams (and providing extra credit to promote participation). All teaching strategies have pros and cons that need to be considered within particular course contexts (for example, offering extra credit to incentivize participation may add to inequities by advantaging students who already have the time to spend on extra course activities).

Idsardi, R. C., Luft, J. A., Wingfield, J. L., Whitt, B., Barriga, P. A., & Lang, J. D. (2023). Relationships between undergraduate instructors' conceptions of how students learn and their instructional practices. Journal of Research in Science Teaching.

A key question for educational developers is: Does changing instructors’ beliefs about teaching and learning actually cause them to use more effective teaching methods? And if not, what prevents them from doing so? This study examined the beliefs and the teaching methods of 40 STEM faculty at a large American university, the majority of whom were participants in some form of pedagogical professional development program (workshops, teaching mentor pairings, etc). Researchers interviewed instructors with questions such as "How do students learn in your class?" and "Can you describe the relationship between your teaching and student learning?," and assessed their teaching using the Classroom Observation Protocol for Undergraduate STEM, or COPUS, 3 times over the course of the semester. There was a great deal of variation in both how instructors conceptualize learning and how they spend their class time. Two groups of instructors had aligned beliefs and practices: those with teacher-centered conceptions of learning who primarily lectured, and those with student-centered conceptions who used more active learning approaches. Interestingly, there was also a third group, who espoused highly student-centered beliefs but nevertheless used as much lecturing as the first group (see Figures 4 and 5). In other words, these instructors believe that students learn through active engagement, but their teaching was misaligned. Instructors in this third group offered several reasons for not implementing more active learning, including personal lack of training and experience, large class sizes and a lack of TAs, and the need to cover content. The instructors conclude that student-centered conceptions are necessary but not sufficient for instructors to implement active learning, and that other personal and contextual barriers must be addressed in order for all instructors who wish to use active learning to do so.

Scherer, D., Verkühlen, A., & Dutke, S. (2023). Effects of related decorative pictures on learning and metacognition. Instructional Science, 1-24. 

Research generally suggests that extraneous, non-explanatory graphics hinder student learning (e.g., Clark & Mayer, 2016; Sundararajan & Adesope, 2020). This article examines the assumptions/limits of this instructional principle by studying the effect of related decorative pictures on (1) knowledge acquisition and (2) metacognitive monitoring. These images, while thematically related to the main ideas in the text, do not transmit relevant information about the learning content. The researchers found that non-explanatory decorative pictures that are thematically related to the text promoted learning, which is likely due to spreading activation processes that lead to deeper encoding of related content into long-term memory. The related decorative images also supported a metacognitive monitoring task. In general, this study supports the use of related decorative images, but instructional designers should still be cautious of seductive illustrations (irrelevant pictures that are not related to the learning content).

Students as victim-survivors: the enduring impacts of gender-based violence for students in higher education

The authors interviewed college students and postgraduates in Australia, where almost half of women experience gender-based violence by the age of 15 and sexual violence is also widespread on campuses, about how their experiences of violence shape their experiences of higher education. Participants were mainly women, mainly White, and had a variety of sexual orientations. They described how they developed lasting stress and anxiety, lost their sense of capability, and had difficulties meeting deadlines and academic expectations. The authors argue that gender-based violence, not just during college but also before, is a key issue impacting gender equity in education.

Coffey, J., Burke, P. J., Hardacre, S., Parker, J., Coccuzoli, F., & Shaw, J. (2023). Gender and education, 35(6-7), 623-637. 

[link to article]

Learning from each other’s lives: First-generation college students of color engaging with diversity course content

What kinds of prior knowledge do first-generation students of color view as especially meaningful in their learning of subject matter in diversity courses? The researcher in this study selected 10 such students from a 4-year, highly diverse public higher education institution and observed them in class. The students also participated in interviews and voluntarily provided their completed assignments. All 10 participants noted that their classmates’ prior knowledge and experiences helped them relate to the content of the class on a personal level (e.g., explaining topics like racial identity and political lobbying using familiar language, pop culture, and current events). They all also connected diversity course material to their lives outside of school (e.g., awareness of quality of homes by neighborhood and inequities in gender roles). The paper also dives deeper into 3 specific student experiences to demonstrate how first-generation college students of color draw from their classmates’ prior knowledge and experiences to gain understanding of subject matter.

Delima, D. G. (2023). Active Learning in Higher Education

[link to article]

Establishing a new standard of care for calculus using trials with randomized student allocation

Calculus is a foundational course for many STEM disciplines and often relies on traditional lecture-based practices, acting as a “filter” in the STEM pipeline that has historically excluded many communities of students. Using a large-scale experimental approach, this research presents strong evidence for the effectiveness of active learning (AL) approaches in calculus courses. Over three semesters, students were randomly assigned to treatment (AL) or control (lecture) conditions that featured multiple sections of 40 students each (total n = 1019). The treatment condition implemented a pedagogical approach called “modeling practices in calculus” (MPC) that devotes class time to working on pre-designed activities and minimizing lecture. For the two primary outcomes, measures aligned with learning objectives (d = .77) and overall course performance (d = .30), results showed significantly better outcomes for the MPC courses across all three semesters. The same pattern and magnitude of results were consistent across racial and ethnic groups, genders, and academic majors. The authors discuss the implications of these findings both in terms of having a new standard for teaching calculus as well as a high standard of evidence for assessing impact of interventions on student learning. The supplementary materials provide in-depth descriptions of everything from the interventions, implementation procedures, and descriptions of the data for readers interested in a more comprehensive summary.

Kramer, L., Fuller, E., Watson, C., Castillo, A., Oliva, P. D., & Potvin, G. (2023). Science, 381(6661), 995-998.

[link to article]

Busch, C. A., Wiesenthal, N. J., Mohammed, T. F., Anderson, S., Barstow, M., Custalow, C., ... & Cooper, K. M. (2023). The Disproportionate Impact of Fear of Negative Evaluation on First-Generation College Students, LGBTQ+ Students, and Students with Disabilities in College Science Courses. CBE—Life Sciences Education, 22(3), ar31. 

Anxiety can be a barrier to students' full engagement in active learning courses. One major contributor to anxiety is the fear of being judged by others in a social setting--for example, the fear of being seen as "stupid" by classmates. This fear can increase cognitive load, and reduce willingness to participate in class, and cause doubt about belonging. This survey of 566 undergraduates in large-enrollment science courses found that 99.9% of students report some level of fear, and that this was more severe for first-generation college students, LGBTQ+ students, and students with disabilities, with a non-significant trend in the same direction for non-male students (p = 0.08). The forms of participation that elicited the highest fear were being cold-called and presenting alone. Survey participants suggested that instructors could make them more comfortable by allowing students to choose their own seats in class, allowing students to work alone instead of in a group, welcoming questions, and responding constructively to student questions and answers without criticism.

Wilson, K. E., Martinez, M., Mills, C., D'Mello, S., Smilek, D., & Risko, E. F. (2018). Instructor presence effect: Liking does not always lead to learning. Computers & Education, 122, 205-220.

With online education growing in popularity, one of the research questions of interest when it comes to content delivery is modality and format of lecture videos. In four experiments with online participants (total n = 571), the authors tested different formats of online lecture videos that included different combinations of audio, text, and visuals of the instructor. Experiments 1 and 2 used a between-subjects design, where participants received one lecture video as audio only, audio with text, or audio with visuals of the instructor. Outcome variables of interest  were participants’ self-reported mind wandering, and performance on a comprehension test following exposure to the video. Results of these two experiments found no evidence that having visuals of the instructor was beneficial for mind-wandering nor comprehension; to the contrary, findings suggest that having instructor visuals may actually be a “seductive detail” (i.e., a distraction), as comprehension decreased in those instances. Experiments 3 and 4 used a within-subjects design where participants were presented with four different lecture videos, which were chopped up segments of the videos used in experiments 1 and 2. The four videos were presented in different formats with varying combinations of audio, text, and instructor visuals. After watching the videos, participants were asked for their perceptions of each one, including to what extent they enjoyed the format, found it interesting, and thought it facilitated their learning. These results showed that participants found the formats with instructor visuals to be more enjoyable, interesting, and helpful for their learning than the formats without the instructor visuals. Altogether, these results suggest that although students may have a preference for particular formats or a belief that something is best for learning, instructors should be wary of including additional stimuli in their videos that do not directly pertain to the target content.

Burnette, J. L., Billingsley, J., Banks, G. C., Knouse, L. E., Hoyt, C. L., Pollack, J. M., & Simon, S. (2023). A systematic review and meta-analysis of growth mindset interventions: For whom, how, and why might such interventions work?. Psychological Bulletin, 149(3-4), 174-205.

In meta-analyses, the goal is often to distill a large group of research studies on a particular topic to assess the overall evidence and obtain a single summary estimate of the effect in question. In addition to an overall effect, however, the questions of “for whom”, “how”, and “why” can be just as important. The authors of this paper review the literature on growth mindset interventions and take a closer look at those latter questions with their own meta-analysis. The authors highlight the high levels of heterogeneity, or variation, in the extent to which growth mindset interventions are shown to be effective in the literature, and their analyses focus on two particular factors that may influence such variability: focal groups (i.e., individuals for whom the intervention is expected to be most effective), and implementation fidelity (i.e., the presence of intervention components thought to be important for effectiveness). For interventions with high implementation fidelity and targeted focal groups, the results showed a significant effect for academic achievement (d = .15) and mental health (d = .32). The authors conclude with a discussion about intervention standards, heterogeneity, and general limitations of meta-analyses when it comes to estimating a true effect for a particular intervention.

Macnamara, B. N., & Burgoyne, A. P. (2023). Do growth mindset interventions impact students’ academic achievement? A systematic review and meta-analysis with recommendations for best practices. Psychological Bulletin, 149(3-4), 133-173.

Research on mindset theory has led to many claims that growth mindset interventions improve students’ academic achievement, and has also generated arguments cautioning against drawing that conclusion. Although prior meta-analyses (Sisk et al., 2018) have revealed a small but significant overall effect size to support the effectiveness of mindset interventions, the authors of this paper point out several limitations of those analyses and attempt to account for them here. The authors conducted three different meta-analyses examining the effect of growth mindset interventions on academic achievement. When examining all available studies meeting basic meta-analytic criteria, a relatively small (d = .05) but significant effect was found. When examining only studies where there was a measured increase in growth mindset, however, there was no significant overall effect. Finally, when only examining available studies that met at least some of the criteria for best practices for designing and evaluating evidence from interventions (of which there were very few), there was no significant effect found. The authors conclude that the current evidence does not support the claim that growth mindset interventions positively impact students’ academic achievement.

A growth-theory-of-interest intervention increases interest in math and science coursework among liberal arts undergraduates

The term “growth mindset” is often used to describe individuals’ implicit attitudes toward general intelligence on a continuum of being able to cultivate and improve it versus it being innately “fixed'' at whatever level they were born with. As plenty of research has shown, however, these implicit attitudes are often domain-specific (i.e., one can hold different implicit attitudes toward different areas of their lives). The authors of this paper investigated this idea in the domain of student interests - the beliefs that a student can cultivate or develop a particular passion or interest, rather than merely “finding” it, which suggests that it is a fixed or predetermined entity. Indeed, students are often told to “find their passion”, which the authors speculate may discourage students from developing interests they have not already “found”. In two quasi-experiments, the authors found that after completing a brief (30 minutes) intervention in the form of an online module about the “growth theory” of interest development prior to the start of school, first-year students who did not initially report having strong interest in their required math and science courses reported having stronger interests at the end of the year, as well as better grades, when compared to controls. These findings suggest that more deliberately framing how students’ interests are developed (not simply found, fully intact) may help broaden student interest development as well as achievement. 

O'Keefe, P. A., Horberg, E. J., Dweck, C. S., & Walton, G. M. (2023). Journal of Educational Psychology.

[link to article

Non-English academics face inequality via AI-generated essays and countermeasure tools

The authors present the results of an experiment showing that artificial intelligence detection tools perform differently depending on whether the writing being analyzed was originally written in English or one of three other languages. The tools correctly differentiated GPT-generated vs. human-authored essays written in English, and gave intermediate scores for essays written in English by a human and then run through a paraphrasing tool. But GPT-generated essays written in Malay, Mandarin, and Japanese were not detected. Perhaps of more concern in our local context, running human-authored essays through AI translation and paraphrasing tools (Google Translate and Quillbot AI) led to them being flagged as AI-generated. Any aid from AI tools was flagged--a potential equity concern because these tools are anecdotally reported to be used more by non-native English writers. Current AI detection tools are unable to differentiate between different kinds of AI contribution (full writing vs. translation and paraphrasing). The authors call for a nuanced discussion and clear standards around what forms of AI assistance are acceptable in specific contexts. 

BioScience, biad034.

[link to article]

Civil engineering students as avoidant actors: Using culturally relevant problem‐solving to increase critical action attitudes

In what ways does culturally relevant problem-solving (CRPS) influence critical action attitudes in undergraduate engineering students? Students responded to an open-ended engineering scenario that either included or did not include culturally relevant context (such as income level, population size, and location). Researchers coded their responses based on the transformative action (TA) framework, which consists of three levels of action: (1) “destructive” actions directly perpetuate oppression and systemic inequities; (2) “avoidant” actions ignore oppression that indirectly reinforces systemic inequities; and (3) “critical” actions intentionally uproot oppression and systemic inequities.

Generally, students avoided discussing critical action and evaded acknowledgement of sociopolitical factors. Still, when exposed to culturally relevant problem-solving, they showed a statistically significant increase in both critical and destructive action responses. The increase in destructive factors may have been influenced by “beliefs of superiority” that permeate engineering culture. In effect, CRPS may be a useful tool, but it is not sufficient to only give students historical context in problem sets—they also need additional instruction and practice to unpack biases and link engineering work with systemic inequities.

Broadly, the researchers recommend integrating ethics and social justice topics with technical aspects of engineering curricula. Students need practice using a critical lens to analyze contextualized, historical accounts of the field and its complicity with oppression.

Drake, R., Poleacovschi, C., Faust, K. M., True‐Funk, A., & Kaminsky, J. (2023). Journal of Engineering Education.

[link to article]

Implicit-bias remedies: Treating discriminatory bias as a public-health problem

Implicit bias, a form of automatic and/or unintentional bias, has become a more prominent topic in the public awareness in recent years. The authors argue that there are several misunderstandings when it comes to public awareness of implicit bias, in terms of what it is, how it works, and how it may be remedied. They outline and attempt to correct five prominent misunderstandings, and offer a review of what is known, and not yet known, about implicit bias based on the available evidence. This review includes research on two popularly sought remedies: individual treatment interventions expected to weaken or eradicate implicit biases, and group-administered training programs to overcome implicit bias and bias in general. Regarding these remedies, the authors conclude that there is a lack of compelling evidence to suggest that these interventions have a durable debiasing effect or impact on the discriminatory consequences of implicit biases. The authors are optimistic, however, that by utilizing strategies that have been successful in the public-health domain, such as focusing on preventative methods (as opposed to curative), and a more proactive approach to specifically identify disparities using available data, that further progress can be made. The paper concludes with recommendations for discrimination reduction strategies and some disparity-finding tools that organizations may find useful.

Greenwald, A. G., Dasgupta, N., Dovidio, J. F., Kang, J., Moss-Racusin, C. A., & Teachman, B. A. (2022). Psychological Science in the Public Interest, 23(1), 7-40.

[link to article]

The benefit of reflection prompts for encouraging learning with hints in an online programming course. 

Building on the literature supporting productive reflection as a means to deep, process-based learning, this article seeks to find ways to build reflection into online classes. The authors particularly ask if providing hints in online programming classes can be more useful when paired with a reflection prompt. They use a two-condition randomized field study where condition one offered learners only the option to use hints, and condition two gave learners both hints and reflection prompts. “Think-aloud” and interview data were used during a pilot study, prior to the ultimate study, in order to fine-tune both the instruments and the framing to participants. The finalized study was conducted over a four-week long introductory data science course among 165 learners in a fully online Master’s program out of the University of Michigan. Learners were randomly assigned to the hint (control) and hint-reflection (treatment) condition. The final results, using instruments for student success such as task performance and submission delays, showed that the hint-reflection condition had a positive impact on task performance overall, notably with fewer submissions (i.e. fewer guesses). The authors also found the hint-reflection combination did not notably decrease enjoyment of the tasks, and even had a long-lasting effect on task submissions for students in the treatment group who continued the class after the treatment was removed. Finally, this study suggests that reflection prompts also affected how students engaged with hints, using them less frequently as time went on. 

Choi, H., Jovanovic, J., Poquet, O., Brooks, C., Joksimović, S., & Williams, J. J. (2023). The Internet and Higher Education, 58, 100903. 

[link to article]

Empowering early career academics to overcome low confidence

Early career academics often suffer from imposter syndrome and low self-efficacy in teaching. This can impact the methods they try implementing: some evidence suggests that instructor confidence is at least as important as teaching philosophy in determining whether they adopt student-centered approaches. Given this, the authors encourage educational developers to think about how we can support new instructors in developing greater self-efficacy. They outline evidence-based strategies, both programmatic and individual, for increasing instructor confidence. 

Dore, E., & Richards, A. (2022). International Journal for Academic Development, 1-13, DOI: 10.1080/1360144X.2022.2082435

[link to article]

The power of successive relearning and how to implement it with fidelity using pencil and paper and web-based programs

Successive relearning (SR), a combination of retrieval practice and spaced practice, is an evidence-based learning strategy that has been shown to be effective in both laboratory and authentic classroom settings, in particular for foundational content. The authors of this paper review the critical components of SR, and present the evidence for its efficacy across a broad range of content and for diverse groups of students. This “teacher-ready” review also provides specific directions for implementing this strategy for an in-person format using paper and pencil, as well as online, and outlines potential barriers for implementation fidelity. Lastly, the authors provide an evaluation of several different tools based on their attributes that are aligned with key components of SR that are available for instructional use.

Dunlosky, J., & O'Brien, A. (2020). Scholarship of Teaching and Learning in Psychology.

[link to article]

In their own words: What scholars and teachers want you to know about why and how to apply the science of learning in your academic setting

This book offers a comprehensive examination of contemporary scholarship on the science of learning and its application in primarily college and university settings. The focus of the book is on research from the viewpoint of cognitive psychology, but some other perspectives are represented - for example, two chapters on discipline-based educational research (in chemistry and physics). The book is organized into five parts. In Part 1, authors provide their views on the “Past, Present, and Future of Applying the Science of Learning in Education.” In Part 2, “Principles and Approaches,” authors describe their own and related work on a wide range of topics that will be of interest to teachers who want to improve their students’ learning, retention, and transfer of academic knowledge. The chapters in Part 3 focus on “Preparing Faculty, Educational Developers, Student Success Professionals, and others to Apply the Science of Learning.” The chapters in Part 4 describe authors’ work on “Preparing Students to Apply the Science of Learning.” In Part 5, “Putting the Science of Learning into Practice,” authors discuss studies they completed in academic settings that examined one or more science of learning principles. Authors were asked to write their chapters in a way that the content is accessible to instructors and others from any field or discipline, including teachers who may have limited or no formal background in science of learning or its application in educational contexts.

Overson, C. E., Hakala, C. M., Kordonowy, L. L., & Benassi, V. A. (Eds.). (2023). Society for the Teaching of Psychology.

[link to article]

Varied practice testing is associated with better learning outcomes in self-regulated online learning

Although student learning has been shown in laboratory studies to benefit more from retrieval practice compared to rereading, this finding has not been demonstrated in real world educational environments at scale. Using the Open Learning Initiative (OLI), researchers leveraged data from over 1,000 students (and more than 10,000 data points) in two different online courses to test whether students' decisions to practice with the material, rather than reread, are related to better learning outcomes. Results showed that, indeed, compared to reading only, practicing with the material was associated with improved learning outcomes. There was also a dosage effect, such that the more activities were completed, the more learning outcomes improved. Students tended to practice with a variety of different activities, which differs from most laboratory experiments where repeated practice tends to be with the same problems, suggesting that exact repetition is not necessary to achieve this effect. This paper provides an example of how data from naturally occurring educational environments can be used to further our understanding of learning science.

Carvalho, P. F., McLaughlin, E. A., & Koedinger, K. R. (2022). Journal of Educational Psychology.

[link to article]

Internet or archive? Expertise in searching for digital sources on a contentious historical question

This study explored what skills are involved in searching online for information, particularly on contentious historical and political questions. Through a think-aloud task, the author compared how experts (professional fact checkers and historians) and college students approached researching the following prompt: “Some people have claimed that Margaret Sanger, the founder of Planned Parenthood, supported euthanasia. Use any resources online to help you decide if you think this claim is true.”

Analyses of screen recordings and transcripts revealed that experts used the search results as a source to help them understand the political context of their queries and to select reliable sources. Furthermore, they searched for evidence from authoritative secondary sources. In contrast, students clicked on sites near the top of search results and weighted evidence from primary sources more heavily. In courses where research is a key component, students may benefit from practice with identifying how search engine algorithms filter results, contextualizing claims, and using snippets (the small block of phrases pulled from the website and provided below the URL in a search result) to make selections about which sites to open.

McGrew, S. (2022). Cognition and Instruction, 40(4), 488-516.

[link to article]

Varied practice testing is associated with better learning outcomes in self-regulated online learning

Although student learning has been shown in laboratory studies to benefit more from retrieval practice compared to rereading, this finding has not been demonstrated in real world educational environments at scale. Using the Open Learning Initiative (OLI), researchers leveraged data from over 1,000 students (and more than 10,000 data points) in two different online courses to test whether students' decisions to practice with the material, rather than reread, are related to better learning outcomes. Results showed that, indeed, compared to reading only, practicing with the material was associated with improved learning outcomes. There was also a dosage effect, such that the more activities were completed, the more learning outcomes improved. Students tended to practice with a variety of different activities, which differs from most laboratory experiments where repeated practice tends to be with the same problems, suggesting that exact repetition is not necessary to achieve this effect. This paper provides an example of how data from naturally occurring educational environments can be used to further our understanding of learning science.

Carvalho, P. F., McLaughlin, E. A., & Koedinger, K. R. (2022). Journal of Educational Psychology.

[link to article]

Internet or archive? Expertise in searching for digital sources on a contentious historical question

This study explored what skills are involved in searching online for information, particularly on contentious historical and political questions. Through a think-aloud task, the author compared how experts (professional fact checkers and historians) and college students approached researching the following prompt: “Some people have claimed that Margaret Sanger, the founder of Planned Parenthood, supported euthanasia. Use any resources online to help you decide if you think this claim is true.”

Analyses of screen recordings and transcripts revealed that experts used the search results as a source to help them understand the political context of their queries and to select reliable sources. Furthermore, they searched for evidence from authoritative secondary sources. In contrast, students clicked on sites near the top of search results and weighted evidence from primary sources more heavily. In courses where research is a key component, students may benefit from practice with identifying how search engine algorithms filter results, contextualizing claims, and using snippets (the small block of phrases pulled from the website and provided below the URL in a search result) to make selections about which sites to open.

McGrew, S. (2022). Cognition and Instruction, 40(4), 488-516.

[link to article]

Evidence of weight bias in the college classroom: A call for inclusive teaching practices for students of all sizes

This study is one of the first to investigate how classroom dynamics are shaped by fat-phobia, or stereotypes about and discrimination against anyone with a body weight above what is considered typical. The authors were interested in how larger students may be evaluated and excluded by their peers. Two hundred undergraduate psychology students read short vignettes describing hypothetical peers which included names, pronouns, race, and weight category (“overweight” or “normal weight”), and hobbies. They then estimated the described person's GPA and ranked their desirability as a partner in class group work. Participants estimated normal weight peers as having higher GPAs than overweight peers, and expressed a preference to work with normal weight peers. These results show that student prejudices may result in exclusion when students are allowed to select team members for group work, and the authors argue that instructor-chosen groups may therefore lead to more inclusion. More broadly, this study is experimental evidence that fat-phobia exists and likely shapes the experiences of many college students, roughly a third of whom are categorized as overweight or obese. Instructors must be aware of this axis of discrimination in order to combat it and create inclusive classrooms.

Alexander, K. E., & Alexander, R. G. (2022). College Teaching70(4), 461-468.

[link to article]

Do students know what they think they know?: Evaluating the relationships between online practice questions, knowledge monitoring, and course outcomes

The researchers investigated whether having students retrieve information for low-stakes question sets and assess their own confidence in the answers can promote learning and metacognitive skills. Data came from a large, content-heavy Human Anatomy class in which students could complete online modules for extra credit as part of their studying. In these modules students were asked to both answer questions and assess their confidence in their answers, and they received immediate feedback. The >350 students studied completed between 0 and 1088 questions over the course of the semester. The researchers found that both the number of practice questions a student completed and their ability to accurately rate whether or not they knew the answer were positively correlated with overall grade in the course. However, there was no correlation between number of questions completed and self-assessment ability, and students did not improve in their self-assessment skills over time. This suggests that although retrieval practice is associated with better learning outcomes, it does not necessarily support the development of beneficial metacognitive skills, even when students assess their confidence as they practice. The authors suggest incorporating additional metacognitive scaffolding questions to increase the benefits of low-stakes practice.

Husmann, P. R., & Smith, T. C. (2022). College Teaching70(4), 482-492.

[link to article]

A sociological lens on linguistic diversity: Implications for writing inclusive multiple-choice assessments

This article inquires into the accessibility of multiple-choice questions (MCQs), particularly for students who use English as an Additional Language (EAL). First, the authors use a normative sociological approach to problematize how MCQs often “pack” coded language derived from academic disciplines into their question stems. This linguistic complexity, as they define it, contributes to the variability of construct-irrelevancy. Construct irrelevancy lowers the validity and reliability of MCQ exams, which contributes to lower student performance as an artifact of the question itself, not the learning the question is meant to measure. In addition to their theoretical contribution, the authors conducted a set of two-stage randomized exams, contrasting performance on packed and unpacked MCQs. They sampled from 606 EAL students, with differing levels of language proficiency. To complement the exam data, the authors also conducted a series of focus groups to understand how students experienced the MCQs. The findings revealed that unpacking MCQs increased students’ ability to demonstrate their knowledge, and answer more MCQs correctly. This effect increased further for students also enrolled in pathway programs. The focus groups revealed an interesting divergence in student perception of unpacked MCQs: first students perceived unpacked MCQs as containing more content; second, since the stem of an unpacked question is often longer, they also perceived it as more difficult to answer in a timed setting. The aggregate results left the authors to conclude that university-wide MCQ-writing workshops should be available for instructors who frequently find need to use them.

Lyon, K., Roberson, N., Lam, M., Riccardi, D., Lightfoot, J., & Lolliot, S. (2022). Teaching Sociology, 0092055X221134126.

[link to article]

Undergraduate international students’ challenges in a flipped classroom environment: An Australian perspective

In a "flipped classroom," students first encounter new material outside of class in readings or videos, while classroom time is devoted to active learning activities such as discussions. Although this structure has potential benefits to student learning, the authors were concerned that flipped classrooms might pose unique challenges to international students. They interviewed thirty-two international students at an Australian university and identified three main themes: 1) language issues, 2) high need for autonomy and little guidance, and 3) unfamiliar technology. International students with lower English proficiency reported that they struggled with long readings and videos, timed online quizzes requiring quick comprehension, and contributing in class discussions. They suggested that instructors in flipped classrooms assign succinct readings and videos with subtitles and transcripts. Additionally, the authors suggest that instructors design and moderate in-class activities to encourage interactional student participation. The second issue raised was that many international students reported being surprised and unprepared for the high degree of autonomous learning required by flipped classrooms. Although they were able to adjust with time, they suggested that instructors prepare students for how they will be expected to learn and provide an avenue to be able to ask questions while working outside of class. Lastly, students reported that they struggled with navigating online Learning Management Systems. They suggested that instructors design LMS sites carefully and explicitly teach students how to use them. Although international student experiences are extremely diverse and more studies like this in other contexts are needed, instructors can use these suggestions as a starting point of how to make their flipped classrooms more accessible and inclusive.

Singh, J. K. N., Jacob-John, J., Nagpal, S., & Inglis, S. (2022). Innovations in Education and Teaching International59(6), 724-735.

[link to article]

Resilience, not grit, predicts college student retention following academic probation

This study tries to dissever the overlapping notions of student “grit” (perseverance and internal motivation) and “resilience” (being able to recover from stress or trauma) and which promotes students success, particularly retention. The authors conducted a large-N survey which elicited measures of relative grit and resilience, then tracked survey respondents’ enrollment and retention in the following years. The survey respondents represented a geographically and institutionally diverse sample of over 4,000 undergraduate students, with and without histories of academic probation. The majority of participants identified as White and as female. The authors ultimately find that resilience is more predictive of student success and retention, even more so among students who had been on academic probation. Based on their findings, the authors note that grit may be helpful for students who are not struggling, but interventions designed to promote student resilience will be more effective for helping retain students who are struggling.

Caporale-Berkowitz, N. A., Boyer, B. P., Muenks, K., & Brownson, C. B. (2022). Journal of Educational Psychology.
[link to article]

Do introductory courses disproportionately drive minoritized students out of STEM pathways?

There has long been an association between performance in introductory level STEM courses and degree attainment in STEM fields (i.e., “weeding out” or “gatekeeping”), which appears to have a disproportionate effect on underrepresented minority (URM) students. The authors of this paper offer a quantitative analysis of this issue using data from over 100,000 students across six different institutions from 2005-2012. The primary research question driving this analysis was, to what extent are students’ race/ethnicity, sex, and the number of Ds, Fs, or withdrawals (DFWs) in a first year STEM course associated with STEM degree attainment? The models used in this analysis also took into account intersectionality and were able to produce probabilities of STEM degree attainment for different demographic groups (e.g., 49% for white male students and 28% for black female students). Results suggest that, even after controlling for background variables such as high school preparation and intent to study STEM, the effects from these “weed out” courses drive URM students out of STEM fields at a disproportionate rate. The authors argue for the need to critically examine the institutional structures and policies that facilitate these inequities and that larger scale change is needed to address them.

Hatfield, N., Brown, N., & Topaz, C. M. (2022). PNAS Nexus1(4), pgac167.

Li, W., Sun, K., Schaub, F., & Brooks, C. (2022). Disparities in students’ propensity to consent to learning analytics. International Journal of Artificial Intelligence in Education32(3), 564-608.
Through a survey sent to 4000 University of Michigan students, stratified into 5 different race categories, the study found that students have three factors on whether they'd opt in or out of allowing their data to be used in learning analytics: trust in the institution, concerns with privacy, and comfort with instructors using the data to improve their teaching. It also found a clear overrepresentation of whites and a clear underrepresentation of blacks in the final consenting sample (+31% and -40% respectively). Digging deeper into the three factors, the study found black students tend to be less trusting of the institution compared to white students, and female students tend to be more concerned with privacy issues compared to male students. However, comfort with instructors’ use of the data seems to be a factor which can bring those who have concerns with either the institution or privacy to consent to having their learning data used. The study shows that there are clear representation issues in student data, which means modeling on such data can compound biases. However, institutional, instructional and tool transparency, data access and opportunities to object may help alleviate the issues.
Moè, A., & Katz, I. (2022). Need satisfied teachers adopt a motivating style: The mediation of teacher enthusiasm. Learning and Individual Differences99, 102203.
This article posits that teacher enthusiasm, both experience and displayed, acts as a mediator for adopting motivating teaching styles. However, it finds that when teachers' basic needs are frustrated or not satisfied, they are less likely to teach with enthusiasm, and less likely to adopt a motivating teaching style as well. Using a representative sample of 341 Italian high school teachers, survey results indicated that when teachers’ felt their basic needs were satisfied they tended to adopt teaching strategies linked with student motivation, such as supporting student autonomy and providing class structure. Teachers who experienced need frustration were associated with chaotic and controlling teaching styles. Teachers who were need-frustrated often displayed enthusiasm, but did not experience it, leading the others to conclude teachers' needs should be satisfied in order for them to experience enthusiasm, genuinely display it, and adopt more student-motivating teaching styles. 
Pardos, Z. A., Borchers, C., & Yu, R. (2023). Credit hours is not enough: Explaining undergraduate perceptions of course workload using LMS records. The Internet and Higher Education56, 100882.
This article fundamentally reexamines the credit hour as a valuable measure of student work for a college course. It briefly explores the history of the credit hour, and makes claims for the comparative value of a holistic construction of course load versus the traditional credit hour. This construction includes many of their variables of interest such as student perception of workload, including mental effort and psychological stress, degree persistence, and LMS metrics. In Spring 2021, the authors collected potential determinants of course load across 596 courses taken by 127 students, responding to survey questions measuring self-reported time load, mental effort, and psychological stress. They conducted a correlational analysis of these survey responses with features of LMS data and enrollment records. The authors find that credit hours explain less variance in students’ perception of course load as compared to measures imputed from LMS features, such as number of assignments. This leads the authors to conclude that course design, rather than credit hours, is one of chief importance for course load, rather than the number of credit hours.
Quinn, D. M., & Desruisseaux, T. M. (2022). Replicating and Extending Effects of “Achievement Gap” Discourse. Educational Researcher, 0013189X221118054.
This article builds on critiques of “achievement gap” discourse (see Gouvea 2020 and Shukla 2022 in this annotated bibliography) and offers an alternative. The authors replicate their own past work showing that both teachers and non-teachers prioritize these disparities more when they are framed as “inequalities in educational outcomes” rather than “achievement gaps.” They use implicit association tests to show that people with higher implicit bias are more likely to deprioritize “achievement gaps,” but the “inequality” framing appears to neutralize this effect. Even more notably, exposure to the achievement gap language actually increased endorsement of explicitly racist stereotypes of Black people. This article makes a strong case for reframing how we discuss racial disparities to motivate change and avoid entrenching racist beliefs about student ability.

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251-19257.

Although active learning has been shown repeatedly to lead to better learning outcomes when compared to passive instruction (e.g., lecture), this may not always be consistent with the students' perceptions of their learning, which may hinder the more widespread adoption of active learning practices. The authors of this paper randomly assigned students (n=149) to either an active or passive learning section for two topics in a physics course, keeping all course content and materials the same for both, as well as counter-balancing the instructors for both groups across topics. In addition, active learning as a superior form of learning was never mentioned or promoted by the instructors. Results showed that when it comes to direct measures of learning, their findings were consistent with what we would expect - more learning for the students in the active section. When examining students’ perceptions, however, they found that students perceived both the quality of instruction and how much they learned to be lower in the active section. The authors point to 3 factors that may be involved with this finding, and propose some specific strategies to improve students’ engagement with active learning. All of the authors’ materials are shared in a supplementary appendix.

Lewis, D. (2022). Impacts of Standards-Based Grading on Students’ Mindset and Test Anxiety. Journal of the Scholarship of Teaching and Learning22(2).

Standards-based grading (SBG), in which course grades are primarily determined by the number of content standards a student masters, has been hypothesized to lower student test anxiety and support growth mindsets when compared to traditional grading. This study attempted to test those hypotheses. The author compared change in test anxiety and in mindset over the semester between students taking traditionally graded and standards-based mathematics courses. Students were ~65% white and ~75% male. The results showed significantly lower test anxiety after the SBG course (effect size of r = 0.27), and this effect was driven by female students, whose anxiety was initially higher. The result was that the initial gender imbalance in test anxiety disappeared after the course. Comparisons to a past study suggest that these effects may be course-specific, meaning that students feel better about tests in their SBG course but not in other courses they are taking. In the second major research question, this study found no effect of SBG on growth mindset, either overall or when broken down into different measures of orientation towards goals. Overall, this paper suggests that standards-based grading can reduce test anxiety for women and improve equity, but that the effects may be limited to the courses that employ these grading practices.

Britt, L. L., Ball, T. C., Whitfield, T. S., & Woo, C. W. (2022). Students’ Perception of the Classroom Environment: A Comparison between Innovative and Traditional Classrooms. Journal of the Scholarship of Teaching and Learning, 22(1).

Although the use of innovative classrooms (e.g., multiple white boards, flexible seating arrangements, A/V capabilities) as space for active learning is often geared toward student engagement, much of the research using these rooms has focused on other variables such as academic outcomes. Through the use of surveys, focus groups, and coded observations, the authors of this paper conducted a study comparing multiple course sections (some in innovative classrooms and some in traditional classrooms) on students’ perceptions of the environment and how the space impacted students’ interactivity and behavior. Results showed that students in the innovative classrooms reported higher levels of satisfaction and cohesiveness, and had generally more positive attitudes toward the classroom environment. Observations revealed that students in the innovative classrooms were clearly able to collaborate more effectively, although social distractions were also more frequent. Overall, the researchers concluded that the use of the innovative classroom space on its own is not sufficient to generate positive outcomes. Instructors need to also teach innovatively, which includes setting certain norms for engagement and empowering students to use the agency the space provides to take responsibility for their learning and shaping the collaborative environment.

Witherby, A. E., & Carpenter, S. K. (2021). The rich-get-richer effect: Prior knowledge predicts new learning of domain-relevant information. Journal of Experimental Psychology: Learning, Memory, and Cognition.

This paper examines the role of students’ prior knowledge in how they learn new information. In three experiments, students completed prior knowledge tests for two different domains and completed learning tasks where they answered questions and received feedback to learn new information. Some of the new information was fabricated by the researchers to ensure that the new information was not previously known. Students also made judgments about their own learning. Following the learning task all students were given a self-paced final test that featured the same information from the learning phase. For experiment 3, students also rated their level of curiosity about learning the new information for each item during the learning task. Results across the experiments showed that prior knowledge in a specific domain significantly predicted final test performance for that same domain, but not for the other domain. Results from experiment 3 where students rated their curiosity showed students with more prior knowledge tended to be more curious about learning new information in that domain. Furthermore, curiosity level mediated the relationship between prior knowledge and new learning. This result suggests that one of the mechanisms by which prior knowledge impacts new learning in a particular domain is through students’ curiosity. The authors offer a rich discussion exploring the theoretical explanations for these findings, including meta-cognitive implications, and outline challenges for future research.

Bol, L., Esqueda, M. C., Ryan, D., & Kimmel, S. C. (2021). A Comparison of Academic Outcomes in Courses Taught With Open Educational Resources and Publisher Content. Educational Researcher, 0013189X211052563.

The use of Open Educational Resources (OER) has been shown to have benefits for students, including saving them money, completing courses more often, and remaining enrolled at a higher rate. The authors of this study conducted an experiment at a community college where they randomly assigned 215 students in online classes to sections that either used OER or publisher/non-OER materials, and compared the sections on several outcomes, including retention, course completion, and exam scores. Four instructors from different disciplines (communications, math, psychology, history) were recruited to teach an online section of their course using OER and another section using non-OER while keeping all other course elements the same. To control for the advantage of saving money, the experiment was arranged so that the students in the non-OER sections did not have to pay for their course materials. Results showed that students in the OER sections were retained past the drop date and persisted (i.e., completed the course) at a significantly higher rate than those in the non-OER sections. When comparing exam scores and the number of students who completed the course with a grade of C or better, however, no statistically significant difference was found. The results suggest that, in addition to cost savings, there may be considerable benefits associated with using OER, and although performance was not significantly different, the lack of evidence for OER being associated with poorer performance is consistent with a “do no harm” approach.

Krüger, J. T., Höffler, T. N., Wahl, M., Knickmeier, K., & Parchmann, I. (2022). Two comparative studies of computer simulations and experiments as learning tools in school and out-of-school education. Instructional Science50(2), 169-197.

Experiments are regularly used in science classrooms to convey complex and abstract concepts and have been shown to increase learning. More recently, computer simulations are being developed as a replacement for hands-on experiments, being able to show several different variables all at once while also showing changes in a particular system in real time as the variables change. This study aims to compare experiments vs. simulations while holding as many other factors constant. The study had two experiments, one studying a one-dimensional relationship of ocean acidification on organisms and the second studying a multi-dimensional relationship of 4 different factors on sea water and their impacts on three different organisms. In both experiments, students were split into a simulation group and an experiment group. Results found that while students experienced higher cognitive load on simulations compared to experiments, they also learned more, particularly in the case of the more complex second experiment. The authors suggested that simulations showed multiple variables while also seeing the effects in real time, which is much more information than that of a physical experiment. However, the easier manipulation of multiple variables is an immense help towards understanding the complex relationship, overcoming the increased cognitive load. Interestingly, students doing physical experiments showed more situational interest after the study compared with the simulation group. The authors believe this is a turning point in the research where students of the current generation no longer see digital activities as novel and thus do not gain increased interest simply by doing activities in this medium.

Lardi, C., & Leopold, C. (2022). Effects of interactive teacher-generated drawings on students’ understanding of plate tectonics. Instructional Science50(2), 273-302.

Often teachers use drawings to help explain relationships between components or objects. Having students generate these drawings themselves is an alternative teaching strategy that has been shown to increase learning. Interactive teacher-generated drawings are a hybrid teaching strategy combining traditional teacher generated drawings by having students give instructions to the teacher on what to draw and the teacher can in turn ask clarifying questions to make a more detailed and accurate drawing, thereby curtailing the main drawbacks of student generated drawings (inaccurate with high cognitive load) while still allowing them to better engage deeply with bringing the abstract concept into the concrete world. This study aims to compare the three types of generated drawings in hopes of showcasing the efficacy of interactive teacher-generated drawings. The study had students study a lesson on plate tectonics and work through the content either through explanation of the concepts (the control group), student generated drawings, interactive teacher generated drawings, or a hybrid of both, where students drew first and then refined through the teacher asking them clarifying questions. Results from the study showed that students that were part of the interactive teacher generated drawing group showed significant effects compared to those without on immediate transfer performance, recall performance and drawing test performance, whereas students that interacted were part of the student generated drawing group showed no significant effects on any of these performances. However, with regards to a 5 week delayed performance test, both drawing groups showed no significant effects on transfer or recall. They both showed significant effects on drawing performance though. The authors believe this is due to how the lesson was set up in a way to focus on accuracy of the drawings, without explicit fostering of transfer or recall.

Nalipay, M., Jenina, N., King, R. B., Mordeno, I. G., Chai, C. S., & Jong, M. S. Y. (2021). Teachers with a growth mindset are motivated and engaged: the relationships among mindsets, motivation, and engagement in teaching. Social Psychology of Education, 24(6), 1663-1684.

Although much of the research we see on mindset typically focuses on students’ global perceptions regarding the malleability of intelligence, or even teachers’ perceptions of the malleability of their students' intelligence, mindset research has also been applied to more domain-specific abilities. This paper focuses on teachers’ mindsets regarding their own teaching ability. Similar to other mindset research, mindsets about teaching ability range between growth (teaching ability can be learned and improved) and fixed (teaching ability is innate) beliefs. The authors hypothesized that having a teaching ability growth mindset would be associated with a greater motivation toward teaching (e.g., “I teach because I have fun doing my job”) and work engagement (e.g., “I am proud of teaching”). Using a sample of 547 Filipino teachers, the authors found that having a teaching ability growth mindset positively predicted autonomous motivation, which, in turn, predicted work engagement. These results were found after controlling for mindset for intelligence as well as other demographic covariates. The adapted teaching ability mindset scale is shared in the paper’s supplementary materials document.

Oslawski-Lopez, J., & Kordsmeier, G. (2021). “Being Able to Listen Makes Me Feel More Engaged”: Best practices for using podcasts as readings. Teaching Sociology, 49(4), 335–347.

The use of podcasts, in lieu of traditional print readings, has increased in recent years; however, little research exists on how podcasts influence students' learning. This study, involving 45 students in two university sociology courses, addressed three research questions 1) Do students listen to the podcast or read the transcript before class? 2) Give the option, do students prefer podcasts or print content and why?, and 3) How does exam performance differ by chosen format? On two occasions, students could choose to listen to a podcast or read the associated transcript before class. On both occasions, listening was the first option (66.67% and 42.03%), followed by reading (20.51% for both), listening + reading (7.69% and 28.21%), or neither (5.13% or 10.26%). On the exam questions, 81.25% of students who read answered correctly, 78.57% who both read and listened, 66.67% of students who neither listened nor read, and 64.29% who only listened. Regardless of format, students indicated that the format they chose made it easier to focus or understand the content, and many appreciated having the choice. Uniquely, students who listened to the podcast reported multitasking. Finally, students who engaged in both noted that they first listened and then read particular parts for clarification. The authors noted that multitasking and a lack of note taking may have influenced lower performance on exam questions for students who only listened to podcasts. Authors recommend offering students the choice, but also teaching students how to use podcasts (i.e., not multitasking, taking notes, and referring back to the transcript).

Nordmann, E., Küepper-Tetzel, C. E., Robson, L., Phillipson, S., Lipan, G. I., & Mcgeorge, P. (2020). Lecture capture: Practical recommendations for students and instructors. Scholarship of Teaching and Learning in Psychology.

Lecture capture (i.e., recording and uploading lectures) can be a controversial topic for instructors and students. While some studies indicate students' desire to have lecture recordings, instructors may worry about attendance rates and overall participation in class if they record lectures. Although research (much of which was conducted before the pandemic) is inconclusive on the effect of lecture capture on student learning, this article provides evidence-based recommendations for instructors on using lecture capture, as well as for teaching students how to engage with lecture captures. Authors suggest that instructors 1) explain the purpose of lecture capture to students, and 2) explicitly teach students strategies for when and how to engage with lecture captures, including effective note-taking skills, using lectures for self-assessment, supplementing live lectures, and using recordings to ask for help. The authors also recommend that instructors use a context-dependent approach to lecture capture (vs. all-or-nothing), investigate the reasons why students fail to attend lectures (vs. blaming lecture recordings), increase the effectiveness of lectures through active learning, seek help/clarification from relevant resources or teaching centers, and consult university policies. These strategies can support instructors and students, for whom lecture capture may be a new resource.

Shukla, S. Y., Theobald, E. J., Abraham, J. K., & Price, R. M. (2022). Reframing Educational Outcomes: Moving beyond Achievement Gaps. CBE—Life Sciences Education, 21(2), es2.

This paper addresses the racialized history of deficit-minded thinking in educational outcomes research. The authors present the negative consequences of and argue why we should avoid using terms such as “achievement gap”, and, moreover, that we should fully embrace a more equity-minded framework that interrogates student outcomes. To this end, the authors review four frameworks centered on historically or currently marginalized groups and highlight the key components at their core. This includes systems-level frameworks, which shifts the focus from comparative deficits to barriers due to systemic discrimination and providing support, and asset-based frameworks, which focus on building students’ strengths, often through the personal or cultural assets associated with their identities. Real-life examples of each of the featured frameworks is provided, as well as a table of questions we can ask ourselves to facilitate the adoption of a more equitable educational system.

Voyer, D., Ronis, S. T., & Byers, N. (2022). The effect of notetaking method on academic performance: A systematic review and meta-analysis. Contemporary Educational Psychology, 68, 102025.

The age-old question, “is the pen mightier than the keyboard?”, is one that has been examined from both sides by educational researchers, leaving us with mixed results over several studies. This paper features a meta-analysis of effect sizes associated with studies comparing the effect of taking written versus digital (e.g., typed) notes on academic performance. Although a prior meta-analysis suggests performance is slightly poorer when taking notes digitally, the authors argue that the analysis of that work was limited by narrow search parameters as well as the confounding of the effect of of note-taking method with the influence of external distractions, which may be particularly problematic with the digital medium (e.g., browsing online). Hence, the analyses in the present paper include a broader review of studies (77 effect sizes compared to 24) as well as considering whether distractions were controlled (e.g., use of a device that has no other capability than taking notes) or manipulated. Overall results indicate that when distractions are accounted for, there is no significant difference in performance based on whether note-taking occurs digitally or by longhand. The authors acknowledge that, although their conclusion appears to be clear, we should not interpret it as “support for the null hypothesis”, as the optimal note-taking method likely depends on a multitude of factors and this research is still progressing.

Brazeal, K. R., Brown, T. L., & Couch, B. A. (2021). Connecting activity implementation characteristics to student buy-in toward and utilization of formative assessments within undergraduate biology courses. Journal for STEM Education Research4(3), 329-362.
This study interviewed 38 students from 8 undergrad biology courses at a large research university on their perceptions, buy-in and utilization of the formative assessments their instructors used. They organized their study by their three research questions:
1) What general behaviors do students describe with respect to FA completion?; 2) How do students perceive that specific implementation characteristics influence their buy-in and utilization?; 3) What combined effects exist across different implementation characteristics?

Of particular interest to instructors are the findings of question 2, including categories such as timing of the formative assessment, content, feedback and grading policies amongst others. Some findings are insightful, such as students finding higher order thinking, particularly real world applications helpful to their learning, while putting in less effort to assignments associated with memorization or other lower order thinking. Others are not novel and unsurprising, such as students preferring explanations alongside correctness feedback. Still others seem to go against the research, such as instructors explaining their rationale for particular types of formative assessments. Here, the study found most students don't remember the instructor doing so, particularly later on in the course. Overall, instructors can use this article to position their formative assessments in ways that will make students buy in more and in turn engage with such assignments in a deeper way, helping increase their learning.
Chan, J. Y.-C., Ottmar, E. R., & Lee, J.-E. (2022). Slow down to speed up: Longer pause time before solving problems relates to higher strategy efficiency. Learning and Individual Differences, 93, 102109.
Strategy efficiency (i.e., selecting a strategy that uses the fewest steps) is an important skill in mathematics. Researchers have identified several factors that may influence strategy efficiency, including pause time, mathematical knowledge, and affective factors, such as mathematics anxiety and self-efficacy. In this study, middle school students (n = 248) completed a 45 min pretest and survey, four 30 min interventions sessions, and a 40-minute posttest to examine the influence of pause time, algebraic knowledge, mathematics anxiety and self-sufficiency on strategy efficiency. During the intervention sessions, students played an online mathematics game designed to improve students' algebraic understanding. The game automatically recorded pause time before students began the problem, number of steps taken to complete a problem, problems solved on the initial attempt, and number of resets to attempt a problem again. Results from an OLS regression indicated that longer pause time promoted more efficient strategy use through fewer solution steps, a higher number of correct initial attempts, and fewer resets. Algebraic knowledge was weakly associated with strategy efficiency, while self-efficacy and anxiety were not significantly associated with strategy efficiency. Although additional research is needed to better understand why pause time is related to strategy efficiency (e.g., a possible relation to metacognitive strategies), taking longer pause times before solving a problem is related to the use of more efficient strategies and can be encouraged or provided in the mathematics classroom.
Diefes‐Dux, H. A., & Cruz Castro, L. M. (2022). Reflection types and students' viewing of feedback in a first‐year engineering course using standards‐based grading. Journal of Engineering Education.
This paper dives deeper into the research that regular reflection helps increase feedback views. The study asks two research questions: How do the total number of feedback views differ based on reflection type, and how do weekly feedback viewing patterns differ based on reflection type? The study took place over two years in a required first year engineering course using a well-developed, tightly aligned, standards based grading system. It covered four different types of reflection: 1. Structured reflections focused on feedback with open ended questions on how to improve specific learning goals and a survey on how they would rate their understanding of specific learning goals (RA) 2. Minute-papers asking what they learned in class today (RB) 3. Structured reflections focused on learning strategies with open ended questions on what learning goals were going well and which weren’t, giving evidence for both and selecting learning strategies from a list they would use the following week (RC) 4. Structured reflections using a combination of RA and RC questions (RD) 5. No reflection (NR) The study ultimately found that RA not only increased feedback viewing the most (10.6 more views than NR), but also was the only one that sustained such viewing habits throughout the semester. Results also suggest that students may infrequently or never view feedback without prompting of some kind, and that instructor behaviors can have a profound effect on students’ viewing habits.

Collaborative learning in sociology research methods courses: Does race matter?

This study examined the effect of lecture vs. collaborative-based learning on student learning by race in a Sociology course on Research Methods. Students enrolled in a lecture-based course (n = 76), in which the majority of class time was structured around a lecture with occasional questions for students, or a collaborative-based course (n = 51), in which students read material before class and then completed collaborative activities to apply course material in groups. Students took a pre- and post-test, consisting of 13 multiple choice questions, seven short answer, and one essay question about research methods. Although there were no significant overall differences between pre and post-tests in the lecture vs. collaborative-based courses, there was a difference in African American students' performance between course types. Controlling for socio-demographic information, gender, parental education, and employment status, regression models indicated that African American students' predicted post-test scores were 22 percentage points lower compared to White students in the lecture-based course, yet this difference did not appear in the collaborative-based learning course. While further research is needed to better understand the effects of collaborative learning based on race, this study suggests that collaborative-based learning has the potential to close the gap in student performance in higher education. 

Morris, P., Ida, A. K., Migliaccio, T., Tsukada, Y.,  & Baker, D. (2020). Teaching Sociology, 48(4), 300–312. 

[link to article]

Learning in double time: The effect of lecture video speed on immediate and delayed comprehension

Is watching lecture videos at an increased speed effective for learning? In three experiments, the authors had participants watch lecture videos at 1x, 1.5x, 2x, and 2.5x speed, followed by immediate and delayed comprehension tests. When it came to watching a video one time at a particular speed, results showed that learning was not significantly different depending on whether the participant watched at 1x, 1.5x, or 2x speed, but performance did decline while watching at 2.5x speed. After watching each video and before taking a test, participants were asked to predict how well they would do on the tests, and the video speed did not have an effect on these predictions. In a second experiment, the authors tested whether watching the same video twice (back to back) at 2x speed would lead to more or less learning than watching it once at 1x speed (i.e., time on task was identical). In this case, participants who only watched the video one time at 1x speed had higher expectations of performance than participants who watched twice at 2x speed, but there were no differences in their actual performance. When the participants who watched twice at 2x speed were able to space out their viewings, however, watching the video for the second time right before the test, they did perform better than individuals who only watched once at 1x speed. This finding is consistent with research on the spacing effect. Overall these results suggest that (for videos featuring around 150 words per minute at 1x), watching lecture videos at up to 2x speed does not incur a significant cost when it comes to comprehension of the content.

Murphy, D. H., Hoover, K. M., Agadzhanyan, K., Kuehn, J. C., & Castel, A. D. (2022). Applied Cognitive Psychology, 36(1), 69-82.
[link to article]

Up close and personal: Examining effects of instructor video presence on student’s sense of connection

Students' sense of connection with instructors is something that has been of concern since the shift to a mostly remote learning environment, where recorded lecture videos are a frequent course medium. Even as in-person learning comes back into the fold, the use of recorded lecture videos is an element from the shift to remote learning that may be retained in some capacity for many instructors. The authors of this paper conducted two studies in which they had students watch a segment of an introductory psychology lecture that was recorded in different ways. Students either watched the lecture recorded with the instructor looking at and speaking directly into the camera, from a more distant vantage point of the camera embedded in the lecture hall (instructor present but not speaking into the camera), or a clip with audio only. After watching the lecture video, students were asked to complete several measures, including the overall likeability of and likelihood of certain behaviors from that instructor (e.g., this instructor would invite students to call or meet outside of class if they have questions or want to discuss something), perceptions of the instructor’s knowledge, and questions about how the students would feel in that class (e.g., connection to and comfort with the instructor). Results showed that, when compared to the lecture hall video and audio-only clip, students who watched the eye-contact video rated the instructor as more likable and more likely to engage in a series of positive behaviors, but did not perceive the instructor as more knowledgeable. In addition, students who watched the eye-contact video reported being more likely to have a series of positive feelings in that class, including feeling connected to the instructor and feeling comfortable raising their hand. Interestingly, the lecture hall video where the instructor was present (but not speaking into the camera) did not show advantages in these measures when compared to the audio only group. These results suggest that being able to view the instructor’s eyes and facial expressions up close in lecture videos may be beneficial for facilitating a positive learning experience for students.

Wong, M., Marshall, L. M., Blank, H. C., & Hard, B. M. (2021). Scholarship of Teaching and Learning in Psychology
[link to article]

Effects of collaborative versus individual preparation on learning by teaching

Evidence suggests that students benefit from teaching others. This study investigated how 1) collaboratively vs. individually preparing to teach, and 2) teaching vs. being taught impacted student learning. Undergraduate students (n = 92) were randomly assigned same-gender pairs, in which they had to prepare and teach a psychology lesson on social loafing and coordinator loss. Pairs were assigned to a Collaborative Preparation (CP) condition (n=23 pairs) or an Individual Preparation (IP) condition (n=23 pairs). Participants completed a pre-test measuring prior conceptual knowledge. Pairs then read and prepared an explanation about the concepts. CP participants were able to collaborate with their partner to design the explanation, while IP participants worked individually. Next, one participant was randomly assigned to explain the material, while the other participant listened. After the presentation, students completed a post-test with the same items from the pre-test, as well as eight inference questions and two transfer questions. Researchers then scored students' explanations based on comprehensiveness (i.e., including statements from the text in their explanation) and quality of conceptual statements and interpretations. Results indicated that there were no significant differences in conceptual knowledge and transfer across either condition, between students who explained vs. listened. However, explainers in the CP condition did include higher-quality result interpretations after working collaboratively, and the quality of explainer participants' explanations significantly predicted learner outcomes. In other words, results support the idea that collaboration before teaching promotes better explanations and deeper processing of learning materials when learning by teaching.

Kobayashi, K. (2021) Instructional Science, 49, 811–829.
[link to article]

Diversity interventions in the classroom: From resistance to action

This paper presents a framework for understanding and providing support for faculty resistance to implementing diversity-enhancing interventions in their classrooms. As empirical support for low-cost diversity-enhancing interventions with potentially long-term effects grows, understanding how faculty think and make decisions about these issues is necessary in order to achieve successful implementation of these strategies at scale. A sample of 40 biology faculty were surveyed and given in-depth, semi-structured interviews which led to rich qualitative narratives that provided insight on the faculty thought and decision process. The resulting framework has four key inputs for faculty in the decision process: 1) noticing underrepresentation is a problem in their field; 2) interpreting underrepresentation as requiring immediate action; 3) assuming responsibility; and 4) knowing how to help. The details of these inputs are analyzed and discussed throughout the paper. The authors argue that faculty are the systemic gatekeepers to widespread use of interventions designed to close equity gaps, and such interventions are not likely to be widely implemented without further scrutinizing and engaging with the faculty decision-making mindset.

Thoman, D. B., Yap, M. J., Herrera, F. A., & Smith, J. L. (2021). CBE—Life Sciences Education, 20(4), ar52.
[link to article]

Scrum methodology in context-based secondary chemistry classes: effects on students’ achievement and on students’ perceptions of affective and metacognitive dimensions of their learning

Three principles underline Scrum methodology: transparency, inspection and adaptation. These principles suggest its' use may help students' metacognitive and executive function skills and initial anecdotal evidence seems to suggest this. This study is meant to be a more formal exploration on Scrum's effects on learning.
10 teachers in the Netherlands teaching a total of 320 students were separated into two groups, one group of 6 teachers would teach their Chemistry module implementing Scrum, while the other 4 would teach as normal. The Scrum teachers attended Scrum training sessions. This group would be later separated into four teachers who had trouble implementing Scrum into their classrooms (growth teachers) and two who did not (top teachers). The Scrum curriculum was split into three sprints of two weeks each, with each sprint having a collection of learning tasks to be completed on scrum board and students choosing who would complete what. Each sprint ends with a formative assessment for students to receive feedback on their understanding of the concepts, followed by a debriefing where students reflect on how to improve learning and collaboration for the next sprint.
Results showed that compared with the control group, the students from the top teachers achieved a large effect size increase in post test scores and the growth teachers achieved medium effect size increases. This is particularly remarkable when considering the growth teachers felt like they had trouble implementing Scrum in their classrooms, as well as the fact that all the teachers had only recently learned about Scrum. Top teacher students also showed significantly higher scores on learning climate, self-regulation, attitude towards chemistry and collaboration, whereas growth teacher students showed significantly higher scores on self efficacy and attitude towards chemistry. The control group showed no significant increases.
The authors acknowledged the fact that the study didn't differentiate the components of Scrum and thus did not know which component affected which result and that there was potential bias in the fact that teachers were put into the treatment or control group based on their interest in getting Scrum professional development. They hope further studies can shed more light on these areas.

Vogelzang, J., Admiraal, W.F. & van Driel, J.H. (2021). Instr Sci 49, 719–746 .
[link to article]

Considering the effects of assignment choices on equity gaps

A disproportionately high number of students leaving college without degrees are from underrepresented ethnic minority populations (URM) and transfer students. One contributing factor in leaving college is low grades on assignments, which can negatively influence students' sense of self-efficacy and belonging. In order to better understand the effects of different assignments, this article investigates whether certain assignment types were associated with inequitable grade distributions for underrepresented minority (URM) and transfer students. Researchers analyzed 745 students' final grades and scores on different assignments across 8 courses. Assignments were divided into six categories: exams, quizzes, or tests; homework; writing; group projects; class activity; oral report. All grades (final grades and across each type of assignment) were lower for URM and transfer students than for non-URM and non-transfer students. In particular, there were significant differences in grades on multiple choice quizzes and exams, homework involving reading, and writing assignments. While not all differences in grades were significant, authors argue that lower grades in general can negatively affect URM and transfer students. To create more equitable assessment practices, instructors should carefully select and use a variety of assignment types, create assignments that are professionally, personally, or academically useful, and provide clear and detailed instructions and rubrics.

Hobbs, H. T., Singer-Freeman, K. E., Robinson, C. Research & Practice in Assessment, 16(1) 49-62.
[link to article]

Uncertain instruction: Effects on curiosity, learning, and transfer

Research suggests that instruction with uncertain elements can promote learning and transfer better than instruction that is certain. In addition, research on curiosity suggests that uncertainty is a key trigger, and that curiosity can motivate learning. However, educational research has not looked into the link between uncertainty, curiosity and learning at the same time, nor has it looked into curiosity on the specific task at hand (state-level), focusing more on it as a personality trait, which is far harder to manipulate and increase.The study took n=208 7th and 8th graders from 10 classes in New Jersey with a 96% Hispanic demographic. Students were asked to derive the formulas of Hooke's Law and Newton's Second Law through experimentation over a course of 4 days. The Low Uncertainty (LU) group were given information about and examples of formulas while the High Uncertainty (HU) group received information not directly related to their learning task. The tasks were followed with direct instruction on the 3rd day and a learning posttest and transfer posttest on the last day.

The study showed that while the HU group had higher state level curiosity on the first day compared with the LU group, both groups state level curiosity decreased as the days went on and became equal by the end of day 2. This suggests that students naturally become less curious as they learn more about a topic. On a post-test, there were no group differences for retention, but the HU group showed a significant benefit for transfer compared with the LU group. The authors suggest that uncertain situations may help learners adapt to other uncertain situations, such as transfer tasks. They conclude by hoping that further studies can find the sweet spot of how much uncertainty to include in instruction and the negative effects of such instruction.

Lamnina, M., Chase, C.C. (2021) Instructional Science 49, 661–685.
[link to article]

Multimedia effects during retrieval practice: Images that reveal the answer reduce vocabulary learning

Multimedia learning theory has shown that providing (relevant) images and words together can benefit learning, and in some instances has even shown that student recall is better when items are presented as images alone rather than words. Importantly, however, this multimedia effect may vary based on whether the student is in the encoding or retrieving phase of learning. In three experiments (N=293), this research provides evidence that including images during retrieval practice may hinder learning. The presence and timing of images was experimentally manipulated for secondary school students (mean age approx. 13 years old) during retrieval practice for foreign language vocabulary that they had been exposed to previously. The foreign words were presented to the students in one of three ways: by themselves (condition 1), with an accompanying image that provided a hint for the words’ translation (condition 2), or by themselves with the image appearing after their retrieval attempt (condition 3). Results showed that recall was significantly better for students who practiced retrieval with no images, compared to those who had images accompanying the foreign words or those who had images presented after retrieval. In addition, students overestimated how helpful the images were, perceiving the images as helpful when in fact they were a hindrance to their learning. These findings suggest that although providing both words and images may be beneficial for learning while encoding, it may have a negative impact during retrieval practice.

van den Broek, G. S. E., van Gog, T., Jansen, E., Pleijsant, M., & Kester, L. (2021). Journal of Educational Psychology. Advance online publication.
[link to article]

The importance of adolescents' sense of belonging to mathematics for algebra learning

Sense of belonging has been shown to influence student learning, especially among underrepresented minority (URM) students. Sense of belonging may be particularly important in mathematics, where there is an achievement gap between URM students (Black, Latinx, and Native American) and non-URM students (defined as White and Asian in this study). Through a survey and pre and posttest, this study examined the predictive utility of sense of belonging to mathematics, on middle-school algebra students' (N=206) performance. Students took a survey measuring their sense of belonging to math, math self-concept, perceptions of math importance, interest in math, and entity view of math. Students then completed a pre-test, a lesson on quadratic equations, and a posttest. The study also controlled for socioeconomic status (SES), gender, and URM status. Results indicate that sense of belonging and URM status were the only significant predictors of posttest scores. URM students had a significantly lower sense of belonging and lower posttest scores; however, URM students did not differ from non-URM students on prior knowledge, math self-concept, perception of math importance, interest in math, or entity view of math. In addition, URM students' lower sense of belonging accounted for some of the differences in posttest scores (controlling for SES and prior math knowledge). In summary, a sense of belonging to math was a significant predictor of algebra learning for all students. Lower sense of belonging was related to lower posttest scores for URM students. These differences could not be accounted for by prior knowledge, SES, interest in math, perception of math importance, math self-concept, or entity view of math. Fostering students' sense of belonging can be an influential factor in supporting student learning.

Barbieri, C.A., Miller-Cotto, D. (2021). Learning and Individual Differences87, 101993.
[link to article]

Increasing intercultural competence among psychology students using experiential learning activities with international student partners. 

This research examines the impact of an experiential learning approach called Crossing Borders on students' intercultural competence. The Crossing Borders program, part of a cultural psychology course, facilitates the grouping of domestic (U.S.) and international students together to participate in a series of in-class and out-of-class experiential learning activities during a semester (see examples in the appendix of the paper). The goals of the Crossing Borders program are for students to become more self-aware of their own cultures, learn about the cultures of other group members, and practice the skills and empathy needed to meaningfully converse with others that have different cultural backgrounds. Data was collected over 8 years (n=207) using pre- and post-measures of intercultural competence that tap three different domains (cognitive, intrapersonal, and interpersonal) with two subscales within each domain. Pre- and post-scores, along with supplemental narrative analyses from some students' writing, revealed significant increases in subscales for cognitive (knowledge) and intrapersonal (identity) areas. In addition, narrative analysis indicated three different themes regarding the positive impact of the Crossing Borders program: breaking down barriers, self-awareness, and awareness of others.

Wickline, V., Wiese, D. L., & Aggarwal, P. (2021). Scholarship of Learning in Psychology. Advance online publication.
[link to article]

Integrating Rubric-Based Metacognitive Reflection to Improve Scientific Research Presentations

Researchers describe the process they implemented for helping research students develop presentation skills and share the rubric they used to provide feedback to students. Eleven students in the study gave two research presentations, for which a trait analytic rubric was used for 1) self, 2) peer, and 3) instructor evaluation and feedback. After the first presentation, students were given the chance to discuss the feedback with their research mentor, and identify areas for growth before the next presentation. Overall, student scores on all three evaluations improved significantly between the first and second presentations. Although the methods and results of this study are not completely novel, it provides a helpful description of their process, as well as a solid rubric for student presentations in a STEM context. This article may be particularly interesting for undergraduate research mentors.

Gunderson, J. E. C.; MacDonald, L. J., Gunderson., W. A. (2021). Journal of Chemical Education.
[link to article]

Ultra-Short Presentations with Immediate in-Class Public Feedback to Enhance Skill Development with Low Class Time and Instructor Time

The performance of in-class oral presentations can be time-consuming, intimidating for some students, and typically has a low amount of feedback relative to the time it takes for students to prepare and perform their talks. In this research students received repeated opportunities to perform 1-minute presentations throughout the semester. Each presentation received either one positive comment only, or an additional suggestion of something to work on. Preparation for these talks usually took place in class for only 8 minutes. The short duration of preparation and performance gave students the opportunity to experiment with new speaking strategies in a low-stakes environment. It also meant that there was much more time available for students to have additional, new opportunities to present throughout the semester. The limited amount of feedback helped students focus on a single, actionable suggestion. The process also alleviated performance anxiety for students by habituating them to the experience through multiple, low-stakes opportunities while the instructor also assessed the student audience for their supportive listening behaviors. This alleviated some of the pressure from students with less experience speaking in front of peers. Immediate and delayed surveys showed that students valued this process, that it helped them feel less stressed about presentations, and that they believed they learned more about presenting than other students that they knew.

Heideman, P. D., & Laury, J. E. (2021). College Teaching, 1-10.
[link to article]

Moonshot: Redesigning NASA's High School Aerospace Scholars Experience at Johnson Space Center for Online Delivery

This article describes a case study on the NASA STEM Pathway Activities--Consortium for Education's High School Aerospace Scholars program (HAS). Under normal circumstances, HAS ends its program with an all-expense-paid trip to the Johnson Space Center. Due to the pandemic, however, this interaction was forced to move online. The HAS team chose to do this through a gamified version of sending the first woman and another man to the moon by 2024. The article details the various learning technologies used to achieve this, as well as some key feedback from students, including the importance of a more concrete schedule and deadlines, the use of smaller teams, and accountability issues. The authors also discuss various aspects of the in-person program that were lost due to the online format.
Neumann, K. L., Stansberry, S. L., Del Rosso, C. L., Welch, S. S., & Ivey, T. A. (2021). International Journal of Designs for Learning, 12(1), 140-157.

Fostering Metacognition to Support Student Learning and Performance

Helping students develop metacognitive skills to support their learning is important, but it is also challenging. Designing and implementing interventions in this area is often difficult due to there being a very broad, noncohesive research landscape, coupled with the fact that a student's thoughts, which are often implicit in nature, are difficult to measure. This paper provides a summary of some of the key research findings on student metacognition, and translates that work into generalizable instructional recommendations. The authors also created a website resource that provides summaries of relevant research papers and organizes all of the information from the paper into a highly accessible evidenced based teaching guide.
Stanton, J. D., Sebesta, A. J., & Dunlosky, J. (2021). CBE—Life Sciences Education, 20(2), fe3.
[link to article]

An inclusive pedagogy in Arts and Humanities university classrooms: What faculty members do.

Inclusive teaching strategies are at the forefront of many higher ed conversations, including how to support students with disabilities. In this qualitative study, students with disabilities identified instructors that they felt were supportive and created engaging, inclusive environments in their classrooms. Instructors (n=119) were then contacted and asked to participate in individual semi-structured interviews about classroom practices, in particular for students with disabilities. Results focused on the humanities instructors (n=24). Interview data was coded and two major categories were identified - teaching-learning process design and methodological strategies. Results indicated that one of the main elements that the instructors had in common was the planning process before the semester and their communication to students at the beginning of the semester about the course schedule and expectations, especially for students with disabilities. to allow them to plan and prepare for the semester. Faculty also expressed the importance of getting to know students in order to adjust course materials for student needs. For students with disabilities, this involved a direct and continuous conversation about their specific needs. In terms of teaching methodology, instructors highlighted the need for active learning, and in particular, group work. Instructors emphasized the importance of group work to recognize students' unique abilities and promote more equality among students. Finally, instructors discussed flexibility, when appropriate and possible, for all students, but especially students with disabilities.

Carbello, R., Cotán, A., & Spinola-Elias, Y. (2021). Arts and Humanities in Higher Education, 20(1), 21-41.
[link to article]

Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. 

In the laboratory, the 'testing effect' is one of the most robust and well-documented interventions to enhance student learning. As we all know, however, real-life classrooms are often far 'noisier' environments than what is typically experienced in the lab. To reconcile this issue, the present research examines data collected from over 48k students in over 200 independent studies to better understand how the testing effect works (e.g., effect size, moderators, theoretical mechanisms, etc.) in a live classroom setting. Results from a meta-analytic review reveal a moderate overall effect size (approx d=.5), as well as evidence for several moderating variables (e.g., feedback given, frequency of testing). In addition, support for several theoretical mechanisms was found, including transfer-appropriate processing, and increasing motivation to learn unknown material, but not for retrieval effort. These results provide strong empirical evidence for the effectiveness of the testing effect 'in the wild' and provide a glimpse into how instructors can enhance implementation in their practice. A table on page 7 of the paper provides a convenient summary of all of the research questions that are addressed along with their corresponding findings.

Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Psychological Bulletin. Advance online publication.

Gouvea, J. S. (2020). Antiracism and the problems with "achievement gaps" in STEM education

Measuring and defining "achievement gaps" in STEM teaching often promotes a comparison that holds white students as the norm or as the ideal learners. The authors have identified that this, itself, is a problematic practice that promotes racist ideas and standards. This review article examines "a small collection of recent work [that may] stimulate critical reflection on what we [biological science instructors] mean by “achievement” in STEM, how we can understand the causes of “gaps,” and what we might consider to be productive steps toward racial equity and justice."

Gouvea, J. S. (2020). CBE-Life Sciences Education, 20(1).
[link to article]

Syllabus tone, more than mental health statements, influence intentions to seek help

With the rise of mental illness cases on college campuses, providing and encouraging the use of resources for students using course syllabi is an important practice. The authors of the present research examined the effects of two syllabus variables on the likelihood that students would reach out to the instructor for help if needed. In a 2x2 lab experiment, syllabi were manipulated to have the presence or absence of a special statement addressing mental health, and had either a warm tone or a cold tone. Participants read one of the four types of syllabi and then rated their intentions to reach out for several different kinds of help if needed. Controlling for attitudes toward seeking psychological help, results showed that, overall, both having a statement addressing mental health and having a warm tone can influence student intentions to reach out to the instructor for help, although the effects of the warm tone appeared to carry influence in several types of situations whereas the presence of the mental health statement affected only one. These results suggest that having a warm tone in your syllabus may encourage students to reach out to instructors for help in more types of situations than simply including a statement about mental health.

Gurung, R. A., & Galardi, N. R. (2021). Teaching of Psychology, DOI:0098628321994632
[link to article]

Play like a team in teams: A typology of online cognitive-social learning engagement

The majority of prior research in online learning and student engagement has focused on cognitive, quantitative aspects of online interaction, e.g., number and length of forum contributions. The authors propose that the analysis of online learning environments should include a social lens. They performed a qualitative assessment of over 3,500 discussion posts from 181 students in an online course, in which they reviewed the number of posts made by each student, as well as the length, depth, and conversational sophistication of each post. They also considered the interactive properties of the posts, e.g., turn-taking and concept building. They produced a matrix of participatory categories using team-based language borrowed from the sports world. Bench-sitters post infrequently, using low complexity writing or reactions. Hustlers post or respond frequently, but also with low complexity writing. Strikers post infrequently, but write with high complexity and in collaboration with peers. Champions write frequent, high-complexity posts that also play to the community. By considering the social aspects of engagement, the authors created a framework that expands how online designers may understand their learners, and be able to build activities that are more reflective of the given population of learners. Building assignments this way may help reach students who struggle with engagement.

Prestridge, S., & Cox, D. (2021). Active Learning in Higher Education, 1–18.
[link to article]

It matters how to recall–task differences in retrieval practice

Although retrieval practice has clear evidence of helping students remember learning material, less study has been focused on the type of retrieval practice. The present research shows the type of recall task used may substantially influence the effects of learning by retrieval practice. Students were split up and read two texts, followed either 12 short answer questions or a free recall question. Students were also given a survey that asked about their mental effort and self-efficacy. After a one-week delay, the students took a post-test with short answer and free recall questions. Results found that the short answer and free recall formats had different strengths. Short answer format led to more successful retrieval of targeted information and increased students’ calibration of meta-knowledge (knowing what they know), and free recall was associated with increases in students’ self-efficacy and learning of general sub-content not targeted directly by the short answer prompts. This article shows the usefulness of both targeted short answer questions and more open-ended holistic questions, albeit for different purposes. Educators should consider either type, depending on the context of their learning outcomes.

Endres, T., Kranzdorf, L., Schneider, V., & Renkl, A. (2020). Instructional Science48(6), 699-728.

[link to article]

Comparing Mastery-based Testing with Traditional Testing in Calculus II

Mastery-Based Testing (MBT) is a new form of assessment that builds on prior work in mastery learning techniques. It requires students to show complete mastery of course concepts in order to achieve credit for each of those concepts. Passing grades are only granted when students master a subset of core concepts. Additional concept mastery adds to the final grade. The motivation for this work is to use theories in growth-mindset to improve student learning. This quantitative study compared students across semesters in a Calculus II class who received either traditional assessments or MBT. A significantly higher number of students in the MBT courses finished the semesters with an A. They also reported spending significantly less time studying per week while also achieving a deeper understanding of the material than their counterparts did.

Harsy, A., & Hoofnagle, A. (2020). International Journal for the Scholarship of Teaching & Learning, 14(2).

[link to article]

Example-based learning: should learners receive closed-book or open-book self-explanation prompts?

Self-explanation is an empirically backed technique that students can use to help make connections between old and new material or a concept and an application of the concept they are learning. Less is known, however, about whether this should be done in a closed- or open-book format. Authors conducted two 2x2 experiments to examine the effects of having to self-explain newly learned concepts in either an open- or closed-book format, with the students either being prompted to actively process the important content or not. Results found that when not prompted to actively process the main content, closed book self-explaining elicited fewer self-explanations and led to lower scores on the immediate and delayed posttest.  When prompted to actively process the main content (through the use of a guiding question), however, the detrimental effect of the closed-book format disappeared, but performance did not exceed that of the open-book format. These results suggest that closed-book self-explanation formats are not advantageous and may even be detrimental in some circumstances. The authors also suggest that further studies should be done to see if any of these results might change following a longer retrieval period.

Hiller, S., Rumann, S., Berthold, K., & Roelle, J. (2020). Instructional Science48(6), 623-649.

[link to article]

Effectiveness of guided group work in graduate level quantum mechanics

In a non-randomized study of graduate students taking a quantum physics course, students who voluntarily attended a structured, weekly, group work session (using Peer Led Team Learning principles) achieved higher gains between a pre and post assessment than their peers who did not attend. Students who attended two or more sessions showed even higher gains. The selection bias was explored as a confounding factor, and the results were still significant. This study shows that PLTL group work sessions, which have previously been shown to be helpful for undergraduate learning, are also helpful for more advanced graduate students, and discusses some considerations unique to this audience.
Porter, C. D., & Heckler, A. F. (2020). Physical Review Physics Education Research16(2), 020127.

A psychological intervention strengthens students’ peer social networks and promotes persistence in STEM

Although values affirmations are typically only focused on the individual's knowledge, skills and beliefs to improve persistence (e.g., in STEM), persistence has been shown to be strongly impacted by a social/relational component. The present research draws an important connection between students' relational networks and receiving a values-affirmation treatment. Students who received the values affirmation intervention were more likely to make new friends by the end of the semester, and were more central in their social networks than students who did not receive the intervention. These changes in social networks, in turn, were the driving mechanism behind students' persistence in future courses. The concept of targeting students' social networks with values affirmation interventions, or other similar efforts, ties very nicely into the present increased focus on the importance of psychological climate, not just individual knowledge, skills, and beliefs for fostering student success and equity. This research further informs our understanding of values affirmations, as well as presents a potentially important target for faculty (social/relational systems) to incorporate into their course design.
Turetsky, K. M., Purdie-Greenaway, V., Cook, J. E., Curley, J. P., & Cohen, G. L. (2020). Science advances6(45), eaba9221.

Teaching with social media: Evidence-based strategies for making remote higher education less remote

The authors provide a literature review of social media’s use in remote learning in order to provide some research-based guidelines for instructors looking to strategically leverage the use of social media in their courses. The paper goes into detail about the utility and instructional affordances social media can provide, and offers evidence-based guidelines for instructors who may be looking to complement their existing online teaching approaches.

Greenhow, C., & Galvin, S. (2020). Information and Learning Sciences, 121(7/8) 513-524. 

[link to article]

Training learning strategies to promote self-regulation and transfer: the knowledge, belief, commitment, and planning framework

Unfortunately, many students often report the use of relatively ineffective study strategies. This paper presents a practical framework for guiding the training of students’ learning strategies in ways that will lead to their effective use across a range of content. The framework has four essential components: knowledge, belief, commitment, and planning (KBCP). The authors provide a thorough review of research related to each component, and argue that they must be considered together in order to create a coherent, evidence-based training approach. After summarizing these key components, the authors provide a sample training protocol that can be implemented in a classroom setting.

McDaniel, M. A., & Einstein, G. O. (2020). Perspectives on Psychological Science

[link to article]

Escape the (Remote) Classroom: An Online Escape Room for Remote Learning

This paper highlights a free, innovative approach to remote learning through the use of Google forms to create a virtual room escape game. Students could work together to solve chemistry-based puzzles and advance from “room to room”. More specific details are included in the supplementary information document that accompanies the manuscript (linked below and attached). Students were asked about the experience and reported enjoying the simulation and thought it was a good experience in cooperative learning. There was little to no reporting of partitioning tasks, suggesting that students remained engaged in a collective effort throughout. Students also provided some other feedback regarding the ideal time it should take to solve each puzzle and the ideal total number of “rooms” needed to make the final escape.

Vergne, M. J., Smith, J. D., & Bowen, R. S. (2020). Journal of Chemical Education97(9), 2845-2848.

[link to article]

A learning method for all: The testing effect is independent of cognitive ability

Retrieval practice, or the testing effect, is a robust technique for enhancing learning, although research on its effectiveness in the context of individual differences in cognitive ability is limited. Experiment 1 (N=324) tested retrieval practice compared to restudying across varying levels of cognitive ability at three different testing intervals, and experiment 2 (N=84) used fMRI to examine the brain activity of some participants from experiment 1 at a one-week interval. Results showed no differences in the effectiveness of retrieval practice across different cognitive abilities, and participants were shown to have significantly higher brain activity patterns in the predicted regions for content learned through retrieval practice compared to restudy. The authors argue that due to its easy-to-implement nature and seemingly universal level of cognitive effectiveness, retrieval practice should be a must-have in all teachers’ toolkits.

Jonsson, B., Wiklund-Hörnqvist, C., Stenlund, T., Andersson, M., & Nyberg, L. (2020). Journal of Educational Psychology. Advance online publication. 

[link to article]

Attention Matters: How Orchestrating Attention May Relate to Classroom Learning

This paper presents a new framework for understanding how students pay attention while learning. This framework organizes student attention along two dimensions: internal/external, and on-topic/off-topic. The authors explore these dimensions in the context of classroom learning and apply the framework to several common classroom scenarios. The authors also provide tips for how instructors may leverage students’ attention in the classroom. In particular, it is hoped that this perspective will help shed light on the underlying mechanisms that affect active learning and provide insight for why certain implementations may be more successful than others.

Keller, A. S., Davidesco, I., & Tanner, K. D. (2020). CBE—Life Sciences Education19(3), fe5.

[link to article]

Optimizing the Efficacy of Learning Objectives through Pretests

We all know the importance of learning objectives (LOs) for aligning classroom content and assessments with desired outcomes, but what about their effects on the learning itself? In 3 experiments, (total N=516), the authors show how LOs can be beneficial for students’ learning, and provide evidence for how they can be optimally leveraged in an active fashion. In experiment 1, students who were presented with LOs prior to reading passages performed better on a posttest than students who did not see the LOs. In experiment 2, participants performed better on the posttest when LOs were presented in the form of pretest questions to be answered by the participants, compared to how they were presented in experiment 1. Experiment 3 expanded on the findings from experiment 2 with a 2x2 design, providing students with pretest LOs that were either multiple choice (MC) or short answer (SA), and either provided feedback on the pretest, or did not provide feedback. Results did not show a significant difference in posttest performance between the MC and SA formats, but did show a significant main effect of feedback – students performed better on the posttest when they did notreceive feedback on their pretest LO questions. These findings, as well as limitations of and future directions for this work, are discussed.

Sana, F., Forrin, N. D., Sharma, M., Dubljevic, T., Ho, P., Jalil, E., & Kim, J. A. (2020). CBE—Life Sciences Education19(3), ar43.

[link to article]

A metacognitive retrieval practice intervention to improve undergraduates’ monitoring and control processes and use of performance feedback for classroom learning

This paper provides an example of a practical skills training intervention that can help students better monitor and improve their learning. A classroom study of 103 students examined the effects metacognitive control and retrieval practice training on learning outcomes, metacognitive awareness, and practice habits. Throughout the semester, students received a series of practice assignments and feedback assignments that required the students to self-report their monitoring of their learning. Half of the students were randomly assigned to learn about the benefits of retrieval practice, self-regulation strategies, and evaluation of feedback, and half of the students completed a series readings and activities aligned with the course content. Results showed that students who received the metacognitive training intervention scored higher on novel exam items and this effect was mediated by monitoring accuracy of to-be-learned materials. These results suggest that, compared to spending additional time studying the course content, time spent on developing metacognitive skills can lead to better learning outcomes.

Cogliano, M, Bernacki, M.L., Kardash, C.M. (2020).  Journal of Educational PsychologyDOI: 10.1037/t21885-000

[link to article]

A change of scenery: Does the setting of an instructional video affect learning?

The setting of instructional videos (e.g., authentic to the context or neutral) has been theorized to be a potential learning distraction and, conversely, a potentially helpful retrieval cue. This research tested several hypotheses relevant to those theories that examined the effect of instructional video settings on retention and application of learning content. In two experiments (N=208), instructional videos that were shot in either an authentic setting (e.g., video about flowers shot in a greenhouse) or a neutral one (i.e., a white wall), were viewed by students who then completed tests for retention and application of concepts. One experiment showed increased retention for the authentic setting, but this was not replicated in the other experiment. The results otherwise failed to show benefits for the videos being shot in an authentic setting, although future research is needed to investigate possible boundary conditions where setting may play more of a role.

Merkt, M., Lux, S., Hoogerheide, V., van Gog, T., & Schwan, S. (2020). Journal of Educational Psychology, 112(6), 1273–1283.

[link to article]

Assessing why the testing effect is moderated by experimental design

Although research on the “testing effect” has repeatedly demonstrated that memory for learning material is enhanced by prior testing compared to prior restudying, there have been a variety of experimental designs that have enhanced, reduced, and even reversed this effect. This paper examines these critical differences in an attempt to gain a more complete understanding of the mechanisms at work when testing effects are produced. Three different accounts of these mechanisms were tested (item-order, rehearsal-borrowing, and retrieval distinctiveness). In three experiments (N=134), the researchers found evidence for rehearsal-borrowing, but not item-order or retrieval distinctiveness. This research provides an incremental advancement in testing effect research, and can be used aid future experimental design and understanding patterns of findings that are otherwise considered inconsistent. 

Mulligan, N. W., Buchin, Z. L., & West, J. T. (2020). Journal of Experimental Psychology: Learning, Memory, and Cognition, 46(7), 1293–1308.

[link to article]

Four empirically based reasons not to administer time-limited tests

Although the use of time-limited tests (TLTs) in college courses is a time-honored practice at most institutions, the authors of this paper present four empirically based reasons why the use of such tests may not be in the best interest for the goals of higher education. They argue that TLTs fall short of providing equitable opportunities to be evaluated fairly and often lead to confounded measurements of learning that have questionable legitimacy. The four reasons (elaborated on at length in the paper) for using untimed tests instead of TLTs are 1) TLTs are less valid; 2) TLTs are less reliable; 3) TLTs are less inclusive; and 4) TLTs are less equitable. In addition to presenting these arguments against TLTs, the authors provide a set of practice-based recommendations for faculty in higher education on how to avoid TLTs in their courses.

Gernsbacher, M. A., Soicher, R. N., & Becker-Blease, K. A. (2020). Translational Issues in Psychological Science6(2), 175

[link to article]

Out of Place: Socioeconomic Status, Use of Public Space, and Belonging in Higher Education

Across four studies, the authors examine the relationship between socioeconomic status (SES), use of public spaces in higher education and feelings of belongingness at elite institutions. Consistent with prior research on SES and the use of public space, the authors found that lower-SES students use public spaces on campus less than their higher-SES peers (see supplementary analyses for analyses by race instead of SES). Analyses also showed that the use of those public spaces (especially iconic public spaces) mediates the relationship between SES and feelings of belonging (i.e., lower SES students tend to use the space less, which, in turn, negatively affects their feelings of belonging). Finally, when the use of public space was experimentally manipulated, results showed that the increased use of space can reduce and even eliminate the SES gaps in felt belongingness. Altogether, this work highlights the importance of creating welcoming and inclusive spaces on college campuses and should warrant a closer look at the signals that such spaces are sending to students. 

Trawalter, S., Hoffman, K., & Palmer, L. Journal of personality and social psychology. Advance online publication.

[link to article]

On Students’(Mis) judgments of Learning and Teaching Effectiveness

This is a review of the research on students’ judgement of their learning in higher education and their role in evaluations of teaching effectiveness. The authors summarize research from laboratory studies as well as classroom data that help us understand the factors that contribute to students’ perceptions of their learning and their underlying mechanisms. These factors include the general tendency for overconfidence, perceptions of organization and clarity, and instructional fluency. The authors argue that the ability to accurately evaluate one’s own learning is more important now than ever before, and subjective impressions of learning that are often misused as evaluations of teaching effectiveness represent a barrier to the development of those metacognitive skills.

Carpenter, S. K., Witherby, A. E., & Tauber, S. K. (2020). Journal of Applied Research in Memory and Cognition.

[link to article]

The Tyranny of Content:“Content Coverage” as a Barrier to Evidence-Based Teaching Approaches and Ways to Overcome It

This paper addresses the longstanding, instructor-centered model of “content coverage” that often incentivizes instructors to spend more time “delivering” content in their courses, often at the expense of deeper, more meaningful learning experiences in the classroom. The authors unpack this model from an historical perspective and offer analysis through the lens of what modern learning science has taught us about how students learn. In addition, they propose a three-step process to help instructors move away from the “tyranny of content coverage” and focus on a more learner-centered approach that can help students achieve deeper and longer-lasting learning.

Petersen, C. I., Baepler, P., Beitz, A., Ching, P., Gorman, K. S., Neudauer, C. L., ... & Wingert, D. (2020). CBE—Life Sciences Education19(2), ar17.

[link to article]

No digest this month

Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math

The authors conducted a meta-analysis using data from both published and unpublished studies to examine the effects of active learning on achievement gaps and passing rates in STEM fields for underrepresented students (URS). The data the authors used came from a wide array of STEM disciplines, courses, and instructors and compared student outcomes from a variety of active learning interventions to those coming from traditional lecture formats. Overall, the analyses revealed a 33% reduction in the achievement gap between URS and non-URS, and a 45% reduction in the gap between their respective probability of passing the course. The authors also found extensive variation in these changes among the studies included in the analysis, revealing “active learning intensity” to be a significant moderator for both achievement gaps and passing rates. That is, achievement gap and passing rate reductions occurred more robustly in courses where a higher percentage of class time was spent engaged in active learning activities. The authors posit that, in general, it is likely the synergy between two key elements that ultimately leads to success in reducing these gaps: 1) employing deliberate, engaging practice with important material, and 2) fostering a culture of inclusion. This specific theory has not undergone rigorous testing, however, and will hopefully be a high priority for future research.

Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., ... & Grummer, J. A. (2020). Proceedings of the National Academy of Sciences117(12), 6476-6483.

[link to article]

[supplementary appendix]

Does the Interleaving Effect Extend to Unrelated Concepts? Learners’ Beliefs Versus Empirical Evidence

In three experiments, the authors examined the effects of interleaving learning materials at multiple levels, such as when a student must study multiple concepts embedded in unrelated domains (e.g., a physics course and a psychology course each with multiple concepts to be learned). In this type of scenario, which is common for most college students, decisions need to be made regarding the optimal studying strategy – should the student interleave study materials, and if so, should the interleaving occur at the concept level, domain level, or both? Results suggest that the optimal sequence involves interleaving at either the domain or concept level, but not both. This finding can be explained by existing theories of why interleaving is generally more effective than blocking. Furthermore, the authors found that, when given a choice, students rarely employed the optimal strategy, whether they were tasked with scheduling their studying on a single level (e.g., multiple concepts from a single domain only) or two levels (e.g., multiple concepts from multiple domains). 

Yan, V. X., & Sana, F. (2020, March 12). Journal of Educational Psychology. Advance online publication. 

[link to article]

How Firm Are the Foundations of Mind-Set Theory? The Claims Appear Stronger Than the Evidence

One's mind-set is described as having profound effects on one's approach to life, including one's motivation, learning, and approach to challenges. This study examines the evidentiary strength of six key premises of mind-set theory in 438 participants: (1) people with growth mind-sets hold learning goals, (2) people with fixed mind-sets hold performance goals, (3) people with fixed mind-sets hold performance-avoidance goals, (4) people with fixed mind-sets believe that talent alone - without effort - creates success, (5) people with growth mind-sets persist to overcome challenge, and (6) people with growth mind-sets are more resilient following failure. This study finds that, in all cases, mind-set's effects were significantly weaker than the average effect size found in social-psychological research, and that only 2 of the 6 premises (#1 and 2) were statistically significant in the predicted direction. Given these findings, the authors suggest that mind-set researchers temper strongly worded claims regarding the importance of mind-set.

Burgoyne, A. P., Hambrick, D. Z., & Macnamara, B. N. (2020). Psychological Science

[link to article]

Knowledge or Abilities? How Undergraduates Define Intelligence

Literature on mindset theory, or "implicit theories of intelligence" is divided in terms of showing its power in predicting academic outcomes, with recent meta-analyses suggesting that there is little to no correlation between a student's mindset and their learning outcomes. Often, mindset is measured by student self-reports using validated instruments about their beliefs. However, these authors suggest that because mindset instruments often fail to define what they mean by "intelligence," they lack process validity when used with undergraduates. In this study, students were asked to more explicitly define their views on intelligence through surveys (n = 100) or interviews (n = 20), which were compared to how they responded to items on common mindset scales. Students generally viewed intelligence in terms of either knowledge or abilities, with some viewing it as both or neither. Students who viewed intelligence in terms of knowledge adopted more of a growth mindset, while those who viewed it as ability adopted more of a fixed mindset. This study suggests that when asking students about their beliefs on intelligence, researchers may need to be more specific, and that past research using mindset scales may need to be viewed with a more critical eye.

Limeri, L. B., Choe, J., Harper, H. G., Martin, H. R., Benton, A., & Dolan, E. L. (2020). CBE—Life Sciences Education19(1), ar5.

[link to article]

Think, pair, freeze: The association between social anxiety and student discomfort in the active learning environment

Researchers investigated the role of social anxiety in college students when it comes to benefitting from active learning. Across three studies, (N=569), the authors found that many students met the clinical threshold for social anxiety, and that that social anxiety was positively associated with reported discomfort with active learning. These two variables interacted to predict final course grades in courses that were structured with a variety of active learning strategies. For students with high active learning discomfort, social anxiety negatively predicted course grades, whereas when students reported low active learning discomfort, social anxiety was not related to student performance. These results suggest that for students who are high in both social anxiety and active learning discomfort, active learning may not provide all of its expected learning benefits.

Cohen, M., Buzinski, S. G., Armstrong-Carter, E., Clark, J., Buck, B., & Reuman, L. (2019). Scholarship of Teaching and Learning in Psychology, 5(4), 265–277
[link to article]

An inclusive pedagogy in Arts and Humanities university classrooms: What faculty members do

Researchers interviewed N=24 faculty in the field of Arts and Humanities that had been identified by students with disabilities as being open and supportive to them in classes. The three main takeaways from the qualitative analysis of the interviews are that instructors that are seen as supportive by students with disabilities do: 1. Plan their class and share their planning with students (and listen to their opinions), they get to know their students early on. 2. they use a variety of teaching methods within active learning, and 3. they apply changes to the whole class and assume the diversity of their students. 

Carballo, R., Cotán, A., & Spinola-Elias, Y. (2019). An inclusive pedagogy in Arts and Humanities university classrooms: What faculty members do. Arts and Humanities in Higher Education,0(0), 1-21 DOI:10.1177/1474022219884281
[link to article]

Rethinking the use of tests: A meta-analysis of practice testing.

The "testing effect", or gains in learning that are a result of retrieval practice, is a well documented phenomenon that has been studied under many different conditions. This paper features a meta-analysis of the` research that has been done comparing the testing effect to other "non-testing" conditions (e.g., "restudying"), which included 118 different articles and 272 different effect sizes. The results of the meta-analysis show that practice testing is, indeed, superior to all other non-testing learning conditions. In addition, moderator analyses highlight important features of testing effect studies that have had a significant impact on the overall findings (e.g., test format, time interval, availability of feedback, etc.). These results can be helpful for any teacher, student, or researcher who is interested in leveraging the testing effect to be as impactful as possible.

Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Review of Educational Research87(3), 659-701.
[link to article]

Unpacking “active learning”: A combination of flipped classroom and collaboration support is more effective but collaboration support alone is not

“Active learning” comprises a relatively heterogeneous set of strategies that often get lumped together. This “2nd generation” active learning study aimed to further differentiate which active learning strategies are most effective and investigate beyond the simple hypothesis that “active learning is more effective than traditional lecture”. In a quasi-experiment, the researchers compared the effect of using collaborative activities, and a combination of both collaborative activities and a flipped classroom, to traditional lecture format. Findings suggest that the combination of the flipped classroom and collaborative activities is most effective, and that collaborative activities alone may not always offer an advantage over traditional lecture.
Rau, M. A., Kennedy, K., Oxtoby, L., Bollom, M., & Moore, J. W. (2017). Journal of Chemical Education94(10), 1406-1414.
[link to article]

Gender similarities in the brain during mathematics development

The difference between men and women in their pursuit of careers in STEM fields has long been hypothesized to have biological underpinnings, particularly with regard to aptitude in mathematics. The present research tested this hypothesis using functional magnetic resonance imaging (fMRI) to take real-time images of participants’ brain activity while viewing mathematics educational videos. This was done in three studies with a total of 104 typically-developing 3-to 10-year-old children.

Overall, the results were consistent with prior work on the neural correlates of mathematical thinking; participants engaged in the predicted mathematical neural processes during the viewing tasks. In addition, the results failed to show evidence for any differences in these processes between male and female participants. Comparisons of neural maturity between male and female participants also yielded no evidence of differences based on reported gender, suggesting that the neural processing of mathematics develops at similar rates in boys and girls. Lastly, to the extent that variability in neural processing was observed, analyses suggest that the participants’ patterns of neural activity reflect one large heterogeneous group, rather than two distinct groups. Taken altogether, the results show widespread gender similarities in neural development and processing of mathematical content, suggesting the differences in STEM performance observed later in life are unlikely to originate from early childhood differences in cognition.

Kersey, A. J., Csumitta, K. D., & Cantlon, J. F. (2019). npj Science of Learning4(1), 1-7.
[link to article]

Using Relevant Animations to Counter Stereotype Threat When Learning Science

This research extends work on the roles of stereotype threat and multimedia animations in student learning by testing whether the presence of relevant multimedia imagery can mitigate the negative effects of stereotype threat while learning. Participants in this lab study were 128 college students randomly assigned to one of four conditions in a 2x2 design that manipulated the presence of stereotype threat and multimedia animations that were relevant but redundant with the material. Participants were assigned to either receive a threat or not, and to subsequently read a science-based text that either contained animations or did not. After completing the assigned reading, all participants were tested for learning transfer using an essay format and rated their own effort on a 1-10 scale. Results showed a significant interaction between animation and threat condition on both test performance and reported effort, revealing a buffering effect of animations after receiving a threat. In the threat condition, participants who received animations scored significantly higher on the transfer test and reported expending more effort relative to participants who did not receive animations. These results suggest that the use of relevant animations could be helpful for individuals at risk for stereotype threat in STEM fields.

Sanchez, C. A., & Weber, K. (2019).  Journal of Applied Research in Memory and Cognition8(4), 463-470.
[link to article]

Does interleaved practice enhance foreign language learning? The effects of training schedule on Spanish verb conjugation skills.

Four lab experiments evaluated the benefits of interleaving in the domain of foreign language learning (Spanish). Two different verb tenses were learned using either blocking or interleaving, followed by a delayed test on the content. In two experiments, the learning took place during a single session, and the researchers did not observe learning benefits in the interleaving condition. In two other experiments, however, the learning was increased to two sessions and spread across two weeks, and the results showed the expected learning benefits of interleaving. In addition to providing a novel extension of the interleaving effect to foreign language learning, these results suggest that interleaving may be most beneficial when used across multiple training sessions.

Pan, S. C., Tajran, J., Lovelett, J., Osuna, J., & Rickard, T. C. (2019).  Journal of Educational Psychology, 111(7), 1172-1188.
[link to article]


Instructor presence in video lectures: The role of dynamic drawings, eye contact, and instructor visibility

Two lab experiments investigated the effects of different “instructor presence”features (eye contact, dynamic drawings, and instructor visibility) when learning from video lectures. The first experiment tested whether students learned better from a recorded lecture played over a series of static drawings, or the same lecture but the drawings were dynamically drawn throughout the lesson. No instructor was visibly present in either of these conditions. Results showed that the dynamic drawing condition performed significantly better on the posttest. The second experiment used the same lesson, but with the instructor now visible. In one condition, the instructor made eye contact with the user while drawing on a transparent whiteboard, and in the other condition the instructor did not make eye contact and completed the drawings on a regular whiteboard. As predicted, results showed significantly better learning in the eye contact group relative to the no eye contact group. In addition, the dynamic drawing group from experiment 1 and the eye contact group from experiment 2 were compared and no significant difference in posttest performance was found. These results show that there are additional features to consider beyond merely having an instructor present on the screen that can enhance learning from videos.

Fiorella, L., Stull, A. T., Kuhlmann, S., & Mayer, R. E. (2019). Journal of Educational Psychology111(7), 1162-1171.
[link to article]

Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom

Students in a large introductory physics course (N = 159) were taught identical content by either 1) active instruction or 2) passive instruction. Students' self-reported feelings about their learning were compared to their actual learning in the course, as measured by exam scores. Students in the active instruction sections performed significantly better than those in the passive instruction group, but reported lower "feelings of learning" (FOL). This study nicely addresses student resistance to active learning techniques, and discusses ways to address this resistance directly with students, particularly by acknowledging the discomfort students often feel with active learning and framing it as a "desirable difficulty."

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Proceedings of the National Academy of Sciences116(39), 19251-19257.
[link to article]

Seductive Details in the Flipped Classroom: The Impact of Interesting but Educationally Irrelevant Information on Student Learning and Motivation

Students in a flipped classroom (N = 754) were randomly assigned to either a control group who saw traditional, fact-based videos outside of class for prework and a treatment group who viewed the same videos with seductive details added. The same course instructor presented the information, and everything in class was the same for both groups. The groups did not score differently on quizzes of the material, but the students in the treatment group with high self-reported levels of prior knowledge viewed the videos favorably and believed they were learning more from them than those who did not see the seductive details. This study contradicts other previous observations that seductive details hinder learning, and although it doesn't show that they harm students, they also do not help. If anything students (especially those who highly estimate their skills) may actually overestimate their learning when presented with seductive details.

Maloy, J., Fries, L., Laski, F., & Ramirez, G. (2019). CBE—Life Sciences Education18(3), ar42.
[link to article]

Smaller Classes Promote Equitable Student Participation in STEM

Previous studies identify six course elements that are shown to impact equitable gender participation in STEM courses, including student-instructor interactions, active learning, the instructor’s gender, gender-balanced classrooms, class size, and the type of course (lower vs. upper division). The present research examined which element best supports female student participation in those courses. The authors collected student behavioral data from 44 courses across six American universities. Results indicate that increasing class size has the greatest negative effect on women’s participation in the classroom, and active learning leads to increased, voluntary participation of women.

Ballen, C. J., Aguillon, S. M., Awwad, A., Bjune, A. E., Challou, D., Drake, A. G., ... & Harcombe, W. (2019). BioScience69(8), 669-680.
[link to article]

Can Test Anxiety Interventions Alleviate a Gender Gap in an Undergraduate STEM Course?

Undergraduate students in a large (N = 779) introductory biology course were assigned quasi-randomly into control or treatment groups to study the effects of reappraisal and expressive writing interventions on test anxiety. Reappraisal involved students learning to recognize physiological anxiety responses and their effects on cognitive and physical performance, and expressive writing involved a short writing test prior to taking an exam. The study found that 1) women in the course self-reported higher levels of test anxiety before and after exams compared to men, 2) higher self-reported test anxiety predicted worse exam performance (but better performance on other course assignments), 3) the interventions did not affect self-reported levels of test anxiety, and 4) the interventions resulted in higher exam scores for all participants, regardless of gender.

Harris, R. B., Grunspan, D. Z., Pelch, M. A., Fernandes, G., Ramirez, G., & Freeman, S. (2019). CBE—Life Sciences Education18(3), ar35.
[link to article]

Exploring sequences of learner activities in relation to self-regulated learning in a massive open online course

Researchers tested whether interventions aimed at encouraging Self-Regulated Learning (SRL) in a Massive Open Online Course (MOOC) impacted students’ performance and retention. N=655 participants took the MOOC and N=222 were in the treatment condition where they viewed SRL video prompts every week. Researchers found a significant difference between the intervention group and the control group with participants who viewed the SRL videos having interacted with more course activities and followed the course structure more closely.

Wong, J., Khalil, M., Baars, M., de Koning, B. B., & Paas, F. (2019). Computers & Education140.
[link to article]


What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course

Researchers observed the differences in self-regulated learning behaviors of (N=784) undergraduate students in a first-year biology class. The experimental group received learning analytics based feedback messages in weeks 4 and 8 of the semester while the control group did not receive any such feedback. The feedback reminded students how they did on the previous quiz as well as a reminder that using the online textbooks was crucial for success in the course. Looking at the online metrics, researchers concluded that the treatment group interacted more frequently with the online tools of the class after the intervention and performed better in their final grades than the control group. There was no difference in the impact of this feedback for students based on varying background characteristics.

Lim, L. A., Gentili, S., Pardo, A., Kovanović, V., Whitelock-Wainwright, A., Gašević, D., & Dawson, S. (2019). Learning and Instruction.
[link to article]

Defining and Measuring Engagement and Learning in Science: Conceptual, Theoretical, Methodological, and Analytical Issues

It has been argued that engagement is a construct that is widely overgeneralized and often misused in the learning science literature. This paper reviews the key issues pertaining to the defining and measuring of student engagement and presents the pros and cons of various approaches featured in several related articles. The author highlights important challenges and questions for future research and offers an optimistic perspective on the potential for further progress in this area.

Azevedo, R. (2015). Educational Psychologist50(1), 84-94.
[link to article]


The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes

This paper describes a framework (ICAP) of four different modes of student engagement that are hypothesized to be associated with student learning. The four modes, passive, active, constructive, and interactive, represent progressively increasing levels of engagement that are thought to correspond to progressively increasing levels of learning. The authors outline the underlying assumptions and existing evidence for the ICAP framework, and propose processes for the associated changes in learning. By clearly operationalizing the construct of engagement and providing hypotheses regarding its association with learning, the ICAP framework represents a theoretically promising and widely applicable model for collecting meaningful data on student engagement.

Chi, M. T., & Wylie, R. (2014). Educational psychologist49(4), 219-243.
[link to article]

How the Design of Presentation Slides Affects Audience Comprehension: A Case for the Assertion-evidence Approach

The assertion-evidence structure is a framework for constructing educational slides that aims to integrate six multimedia learning principles for more effective learning. In this study, 110 engineering students viewed technical presentations of slides that were constructed either according to the assertion-evidence structure or with what the authors refer to as the ‘default-driven’ practice, which does not explicitly align with the aforementioned multimedia learning principles. Upon evaluating student learning outcomes, it was found that students showed better comprehension with fewer misconceptions when learning from the assertion-evidence slides compared to the default-driven slides. Student performance showed the same pattern on an unexpected delayed posttest administered one week later. Interestingly, the students rated their perceived mental effort while learning from the slides as significantly less when learning from the assertion-evidence slides relative to the default-driven slides.

Garner, J., & Alley, M. (2013). International Journal of Engineering Education29(6), 1564-1579.
[link to article]


Slide Structure Can Influence the Presenter’s Understanding of the Presentation’s Content

Research has shown that students learn more and have fewer misconceptions when they view educational slides constructed with the assertion-evidence structure compared to when slides are constructed with alternative strategies. In this study, 120 engineering students were asked to create their own slides that could be used to teach other students a lesson, with approximately half of them being assigned to utilize the assertion-evidence structure and half of them being assigned to create their slides with a structure of their choosing. Shortly after completion of this assignment, students were given an unannounced comprehension test on the content of the lesson for which they were designing slides. The authors found that the students who designed their slides using the assertion-evidence structure performed significantly better on the comprehension test than students who were assigned to create slides with a structure of their choosing. These results offer further evidence for the effectiveness of the assertion-evidence structure when presenting educational information.

Garner, J. K., Alley, M., & Aippersbach, S. (2016). International Journal of Engineering Education32(1), 39-54.
[link to article]

Transfer of test-enhanced learning: Meta-analytic review and synthesis

This paper features a comprehensive meta-analysis of the research on how test-enhanced learning (i.e., the "testing effect") promotes learning transfer, or learning that allows students to generalize their knowledge and successfully apply it in different situations. In analyzing data collected over the past 40 years, including 192 effect sizes from 67 papers, the authors found strong evidence for test-enhanced learning yielding transferable learning benefits when compared to non-testing conditions such as restudying or rereading (d = .40). The authors propose a framework for understanding the transfer of test-enhanced learning based on three significant moderators in their analyses: A) response congruency, B) elaborated retrieval practice, and C) initial test performance. According to this framework, transfer is most likely when the answers on the initial test and transfer test are the same (A), when the initial test and transfer test involve a broader level of encoding (e.g., addressing multiple levels of knowledge as described in Bloom's Taxonomy) (B), and when performance on the initial test is high (C). Potential implications as well as directions for future research in this area are discussed.

Pan, S. C., & Rickard, T. C. (2018). Psychological bulletin144 (7), 710.
[link to article]

Equivalent but not the Same: Teaching and Learning in Full Semester and Condensed Summer Courses

Are there differences in student learning outcomes between full and condensed courses? The literature presents mixed findings, which the authors attribute to key differences across study designs (course lengths, course logistics, and measurement tools). To provide clarity, the authors present a mixed methods study design comparing 14-week full semester courses to 6-week summer courses between Summer 2015 and Fall 2016. The study leverages pre/post tests and final grades to measure student performance, a survey design to assess student perceptions, and a focus group to assess faculty perceptions. Findings suggest that there is no difference in student learning between course formats (for pre/post measure), students perceive condensed courses as less stressful and prefer it to full-semester courses, and faculty also perceive condensed courses as less stressful and more beneficial to student learning.

Walsh, K. P., Sanders, M., & Gadgil, S. (2019). College Teaching, 67 (2): 138-149
[link to article]

Comparison of student team dynamics between nonflipped and flipped versions of a large‐enrollment sophomore design engineering course

An undergraduate engineering design course (N=167) using team-based learning was used to study differences across team conflict, peer assessment, and team member satisfaction in flipped (Fall 2015) versus non-flipped (Fall 2014) versions of the course. Self-reported survey data were used to collect student responses on satisfaction and three categories of team conflict: task conflict, relationship conflict, and process conflict. Web-based CATME was used to collect student responses on peer assessment. Findings suggest that flipped courses produce greater student satisfaction and more positive peer assessments than non-flipped courses.

Baughman, J., Hassall, L., & Xu, X. (2019). Journal of Engineering Education108(1), 103-118.
[link to article]

Agile research studios: Orchestrating communities of practice to advance research training

Researchers from the Design, Technology, and Research team at Northwestern University reported on their use of Agile Research Studios (ARS). This model provides an opportunity for faculty to work with large teams of students and researchers effectively and in a way that contributes meaningfully to undergraduate professional development. Students regularly meet in small and then progressively larger groups in between full studio meetings, and are given virtual tools to connect and share resources with each other. Students reported that the experience had a positive impact on their planning and research skills, and increased willingness in seeking and providing help along the way.

Zhang, H., Easterday, M. W., Gerber, E. M., Rees Lewis, D., & Maliakal, L. (2017, February). In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (pp. 220-232). ACM.
[link to article]

STEM Faculty Who Believe Ability is Fixed Have Larger Racial Achievement Gaps and Inspire Less Student Motivation in Their Classes

STEM faculty at one university were surveyed (N=150) and completed a two-item mindset scale measuring perceptions of a growth vs. fixed mindset for their students (e.g., "students have a certain amount of intelligence and can't do much to change it"). Course grades for over 15,000 students were analyzed to examine whether there was a relationship between faculty mindset toward their students and course performance. Overall, a significant relationship was found, with students performing worse, on average, in courses taught by faculty reporting relatively low levels of growth (i.e., fixed) mindset. Underrepresented minority students (URM) showed especially low performance for low-growth mindset faculty, which exacerbated the overall STEM achievement gap for URM students.  In addition, student reports of motivation were lower in courses taught by faculty with relatively low growth mindsets towards their students.

Canning, E. A., Muenks, K., Green, D. J., & Murphy, M. C. (2019). Science Advances5(2), eaau4734.

Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning?

Prior work argues that students need to achieve basic, lower level learning outcomes (e.g., factual knowledge) before moving on to pursue higher order learning outcomes (e.g., application and synthesis) successfully. In two laboratory experiments, the present research evaluates whether building a foundation of factual knowledge enhances students higher order learning. Results show that retrieval practice with lower level learning outcomes did not benefit higher order learning, and that higher order learning was best when students were given the opportunity to practice using higher order quizzes. These results suggest that in order for students to best achieve higher order learning outcomes, establishing a strong lower level foundation is not beneficial, and it may be best to spend more time practicing higher order skills instead. The authors discuss implications and theoretical rationale for their findings, which were also replicated in an authentic K-12 classroom setting. 

Agarwal, P. K. (2019).  Journal of Educational Psychology, 111(2), 189-209.
[link to article]

Knowing is half the battle: Assessments of both student perception and performance are necessary to successfully evaluate curricular transformation

Student perceptions of active, inquiry-based pedagogical approaches in introductory biology courses were measured and compared to their learning gains in the course. Over a four year time period, the 542 students' responses indicated that their perceptions of the instructional approach became more positive over time, as did their learning gains measured by a pre- and post- course concept test. The authors argue that negative perceptions among students, which often accompany curricular changes, may decrease over time as the new practices gain traction and students develop more buy-in. However, tailored instruments designed to probe student attitude and motivation are more informative in measuring this compared to end-of-semester evaluations.

Shaw, T. J., Yang, S., Nash, T. R., Pigg, R. M., & Grim, J. M. (2019). PloS one14(1), e0210030

Investigating the Replicability and Generalizability of the Negative Testing Effect

Some recent research has shown that retrieval practice, known as ‘the testing effect’, can actually be disruptive to learning when it is utilized under specific conditions. Such findings suggest that there are conditions under which taking quizzes or tests is not only not helpful for learning, but actually detracts from it – a so-called ‘negative testing effect’. One limitation of many cognition experiments, however, is a potential lack of generalizability due to limited authenticity of the learning materials that are used in laboratory settings. One goal of the present work was to adapt the design of the aforementioned research in an attempt to replicate the ‘negative testing effect’ using more educationally relevant materials. In multiple experiments, their results show that trials using the educationally relevant learning materials failed to replicate a negative testing effect, which suggests that such an effect may be driven by the lack of educational relevance of the materials in question.

Wissman, K. T., & Peterson, D. J. (2018). Journal of Applied Research in Memory and Cognition (7)3, 352-360.
[link to article]

An adaptive collaboration script for learning with multiple visual representations in chemistry

Researchers assessed whether an adaptive collaboration script (a software providing real-time feedback to student groups) helped students learn visual representations of chemistry better than a traditional paper worksheet setting with delayed feedback (of one week). The research question was tested in a quasi-experiment with 61 undergraduate students involved in an introductory chemistry course. Students in the adaptive collaboration script condition showed significantly higher learning gains on a transfer test immediately after the intervention and with complex concepts on a midterm exam three weeks later.

Rau, M. A., Bowman, H. E., & Moore, J. W. (2017). Computers & Education109, 38-55.

Measurement of cognitive load in multimedia learning: a comparison of different objective measures

Using multiple objective measures of cognitive load, these researchers provide evidence for the seductive details hypothesis by demonstrating that students completing a computer-based multimedia learning task that featured extraneous information (i.e., seductive details) exhibited higher levels of cognitive load as well as a decrease in retention and comprehension (relative to a no-extraneous information condition), despite self-reports to the contrary. These results are novel as few studies of this type feature direct measurements of cognitive load in this context.

Korbach, A., Brünken, R., & Park, B. (2017). Instructional science45(4), 515-536.
[link to article]

Can I get better? Exploring mindset theory in the introductory communication course

In this study, researchers adapt a mindset scale to examine the role of a growth mindset in an introductory communication (i.e. public speaking) course, asking whether having a growth mindset was correlated with public speaking anxiety (PSA), interpersonal communication competence, student engagement, and student performance. They were also particularly interested in whether a growth mindset could help make predictions related to any of these factors. 1037 participants taking one of two, required introductory communication course completed survey instruments at the end of the semester and individual speech grades were also analyzed. Results indicated a significant negative correlation between growth mindset and PSA and significant positive correlations were found for mindset and student engagement, performance, and communication competence. Researchers suggest that next steps include research into possible interventions.

Nordin, K., & Broeckelman-Post, M. A. (2019). Communication Education, 68(1), 44-60

Do Students Overestimate Their Contribution to Class? Congruence of Student and Professor Ratings of Class Participation

Research suggests that students' perceptions of their class particiation are not always congruent with professors' perceptions. The authors conducted two studies to examine (1) the extent to which students' and professors' perceptions are misaligned and (2) whether mid-term feedback would increase congruence. In Study 1, students and professors across nine courses (N = 191) used a rubric to assess student participation. In Study 2, students (N = 87) were non-randomly selected into a treatment group that received mid-term feedback (using the same rubric from Study 1) on their participation and a control group that did not receive feedback. Students and professors in both groups then used the rubric again to evaluate student participation at the end of term. Findings show that student participation is generally correlated with professors' perceptions, but that the largest perception gap occurs for low student participators. Additionally, mid-term feedback was not shown to increase congruence between student and professor ratings of participation. 

Meyer, M. L., McDonald, S. A., DellaPietra, L., Wiechnik, M., & Dasch-Yee, K. B. (2018). Journal of the Scholarship of Teaching and Learning18(3), 44-54.
[link to article]

Dividing attention in the classroom reduces exam performance

118 students were randomly-assigned students in two identical course sections, and were either allowed to use mobile devices during certain (controlled) class periods or not. They completed paired questions in class and later on exams to measure both the short term and long term impacts of device distraction. Although it had no effect on their immediate recall of information (based on in-class questions), students in the no-device sessions retained significantly more information than students in the device-allowed sessions, regardless of whether devices were chosen to be used in those sessions or not.  
Glass, A. L., & Kang, M. (2018). Educational Psychology, 1-14.

Two-stage examinations: Can examinations be more formative experiences?

This paper examines the effect of two-stage examinations, which are exams that feature both an individual and collaborative component, on student performance. Two-stage examinations were administered 11 times to five successive cohorts of students (N = 899) in a variety of courses for both midterm and final examinations between 2013 and 2017. Researchers compared the group's individual average scores to the group score. Researchers used these scores to identify individual-to-group gains in performance and individual-to-top student gains in performance.  Overall, the authors found that two-stage exams offered substantial performance gains for individuals when working in the group stage, and that students perceived two-stage exams as more helpful and less stressful than traditional exams. 

Levy, D., Svoronos, T., & Klinger, M. (2018). Active Learning in Higher Education, 1469787418801668.
[link to article]

The lure of seductive details during lecture learning

The negative impact of seductive details on learning is well documented in the lab. Although there are numerous studies that focus on text passages or narrated multimedia modules in demonstrating those effects, there is a dearth of research on these effects occurring in lectures or other settings that represent high stakes learning environments. The current research (N=259) examined the impact of seductive details across both low stakes (completed the study for research participation credit) and high stakes (completed the study under conditions of social, evaluative, and performance based pressure) learning contexts. Results showed that seductive details were detrimental to learning in the low stakes, but not the high stakes, learning environment. In addition, students who tested high for prior knowledge on the subject matter were not impacted by seductive details in either condition. These results suggest that the impact of seductive details on learning may be more dependent on context than previously thought.

Fries, L., DeCaro, M. S., & Ramirez, G. (2018). Journal of Educational Psychology. Advance online publication.
[link to article]

Using Selective Redundancy to Eliminate the Seductive Details Effect

Although there is much evidence that suggests the inclusion of seductive details in the presentation of a lesson impairs learning, there is also work to suggest that the interest/engagement that results from “hooking” a student’s attention can be beneficial. In this research, the authors attempted to mitigate the negative effects of seductive details by fostering shallower processing of those details, while at the same time fostering a deeper processing of essential information.  In an online experiment, participants (N=69) were presented a lesson in one of three conditions: one with seductive details presented with spoken audio paired with redundant on-screen text, one with seductive details presented without the redundant text, and one with neither seductive details nor redundant text. Results showed that when presented seductive details paired with redundant text, participants did not suffer the same learning decrement experienced when seductive details were presented without redundant text. Although learning in this critical condition was not significantly different from the condition without any seductive details, these findings suggest that by inducing shallower processing through the use of the redundancy principle, it is possible to avoid the negative effects of seductive details on learning.

Yue, C. L., & Bjork, E. L. (2017). Applied Cognitive Psychology31(5), 565-571.
[link to article]

It’s all a matter of perspective: Viewing first-person video modeling examples promotes learning of an assembly task.

This paper examines the effects that different ways of creating videos from an instructional design perspective can have on learning. In one experiment, 105 participants were randomly assigned to watch an instructional video that was filmed either with a first-person perspective or a third-person perspective, with the content being otherwise identical in both videos. It was found that participants who viewed the instructional video from a first-person perspective were able to perform the task from the video better than students who viewed the same video from a third-person perspective. This effect was shown to be even stronger for more complex tasks. In a second experiment, this effect was replicated in a different lab with 120 participants. These findings suggest an instructional design principle in which people learn better from videos constructed from a first-person perspective relative to a third-person perspective.

Fiorella, L., van Gog, T., Hoogerheide, V., & Mayer, R. E. (2017). Journal of Educational Psychology, 109(5), 653.
[link to article]

Faculty drivers and barriers: Laying the groundwork for undergraduate STEM education reform in academic departments.

Researchers sought to describe the response of faculty to efforts of shifting norms in teaching toward evidence-based and pedagogically informed practices. Analysis of responses resulted in ranked lists of issues categorized as barriers (18) or drivers (15). The distribution and frequency of faculty responses to efforts to shift teaching practices provides a useful framework for strategies that may be employed to effect change. The results and discussion offer valuable context for potential dialogue with faculty and may help to anticipate and preemptively address concerns and/or emphasize positive aspects that faculty view as driving forces that can be used to motivate change.

Shadle, S.E., Marker, A., & Earl, B. (2017). International Journal of STEM Education, 4:8, p. 1-13.
[link to article]

Should students assessed as needing remedial mathematics take college-level quantitative courses instead? A randomized controlled trial.

Students needing to take remedial math classes often do not pass those classes and have high levels of college dropout. Consequently, some colleges have started to allow students to take college level classes without first taking remedial courses. The researchers randomly assigned students (n=717) in need of remedial classes to take either a traditional remedial algebra class, a remedial algebra class with workshops, or a college level statistics class with workshops. Students assigned to the college-level statistics class with workshops passed the class at a higher rate than students in remedial algebra with or without workshops (56% vs 39% and 45%). Students assigned to the college-level statistics class accumulated more college credits than those in the remedial class.

Logue, A. W., Watanabe-Rose, M., & Douglas, D. (2016). Educational Evaluation and Policy Analysis38(3), 578-598.
[link to article]

Examining the contributions of desirable difficulty and reminding to the spacing effect. 

Three experiments were conducted (N=134) to evaluate the desirable difficulties (DD) account of the mechanism underlying the benefits of spaced practice on learning. The DD account hypothesizes that retrieving information after a lag in time creates a slightly more difficult task that, in turn, leads to stronger encoding of the information.  Results from the three experiments, however, show that a longer lag time in between study events improves memory performance above and beyond the measured “desirable difficulty” induced by the lag itself, suggesting that the DD account of the spacing effect is either inaccurate or incomplete. That is, increasing the time lag between study events is effective, but this is not simply due to the increased difficulty of acquisition relative to that of a study event following a shorter (or zero) time lag.  The results of experiment 3 suggest that other variables, such as encoding variability, likely play a key role in explaining the spacing effect, although more research is clearly needed before coming to a firm conclusion.

Maddox, G. B., Pyc, M. A., Kauffman, Z. S., Gatewood, J. D., & Schonhoff, A. M. (2018). Memory & cognition, 1-13. , 1-19.
[link to article]

Brain drain: the mere presence of one’s own smartphone reduces available cognitive capacity

Undergraduate students in a laboratory setting were randomly divided into three conditions: those whose phones (turned off) remained 1) in a separate room outside the testing location, 2) in their bags near them, but out of sight, and 3) nearby and in sight. All students then completed a measure of working memory capacity and functional fluid intelligence. Students whose phones were most salient (condition 3) showed the lowest available cognitive capacity, while those whose phones remained in another room (condition 1) displayed the highest. All students self-reported that they did not believe the presence of their phone affected their performance. In a follow-up experiment, similar results held true whether the phones were on or off.

Ward, A. F., Duke, K., Gneezy, A., & Bos, M. W. (2017). Journal of the Association for Consumer Research2(2), 140-154.
[link to article]

A study exploring the impact of lecture capture availability and lecture capture usage on student attendance and attainment

Lecture capture has become more common and in some cases may even be retrofitted into classrooms. Despite perceived benefits of lecture capture for students (e.g. option to revisit lecture content), previous research has reported mixed findings about whether they are truly beneficial to student learning. In this study, researchers examined 2 cohorts of students in an introductory course: before lecture capture was available (n=161) and after lecture capture was implemented (n=160). Findings suggest that lecture capture availability had a negative impact on student attendance, that lecture capture usage associated with attendance was “negligible," and that, overall, lecture capture is an insufficient replacement for attendance for the vast majority students. 

Edwards, M. R., & Clinton, M. E. (2018). Higher Education, 1-19.
[link to article]

Self-Regulated Learning of Principle-Based Concepts: Do Students Prefer Worked Examples, Faded Examples, or Problem Solving?

Some research has shown the benefit of using worked examples for learners before engaging in independent problem-solving. This article explores what self-regulated learners choose to use when learning a new skill (in this case, solving probability problems). In Experiment 1 (N=102), participants chose between a worked example and independent problem solving over a series of 12 trials. Overall, participants favored independent problem solving unless their past trial was incorrectly answered. In Experiment 2 (N=182) and 3 (N=136), participants had a third option: partially worked examples. Participants chose this option more frequently than the fully worked examples. Researchers concluded that, although most learners were more likely to choose some form of worked examples after incorrect answers. they are less likely to choose as their first practice worked examples, despite it being better for their learning. When partially worked examples are available, however, they are more frequently chosen than fully worked examples when participants have that option. 

Foster, N. L., K. A. Rawson, J. Dunlosky. 2018. Learning and Instruction, vol. 55, pp. 124-138.
[link to article]

The Negative Consequences of Threat: A Functional Magnetic Resonance Imaging Investigation of the Neural Mechanisms Underlying Women's Underperformance in Math.

Brain imaging experiments were performed using FMRI on 28 female undergraduate students in a laboratory setting. All participants were asked to perform math tasks, however some were told that "research has shown gender differences in math ability and performance", presenting a stereotype threat condition. FMRI scans revealed that the women who did not hear this statement used parts of their brain associated with mathematical learning, while the women who heard the message reinforcing gender stereotypes about math did not employ these parts of the brain. Instead, the parts that were activated were associated with social and emotional processing. This study provides a complementary perspective to other stereotype threat literature about women's math performance.

Krendl, A.C.; Richeson, J.A.; Kelley, W.M.; Heatherton, T.F. (2008). Psychological Science, 19(2). 168-175.
[link to article]

Anatomy of STEM teaching in North American Universities

A large study of over 2000 classes by more than 500 faculty across North America was conducted to characterize the current state of STEM teaching using the COPUS protocol. Despite the large body of evidence to support the benefits of active learning, investigators found that didactic lecture is still the most common instructional style (74.9 +- 27.8% of the total time observed). Smaller classes and those in rooms with flexible seating were more likely than large, fixed-seat lectures to use interactive or student-centered approaches, although didactic lecture was sill the predominant teaching method. The authors recommend that institutions revise tenure, promotion, and merit-recognition policies to support evidence-based teaching practices, and state that research-based guidelines to measure effective teaching need to continue to be funded and developed.

Stains, M., et al. (2018). Science359 (6383), 1468-1470.
[link to article]

Realistic Details in Visualizations Require Color Cues to Foster Retention

Previous research has shown that students retained less information from realistic representations of biological structures. Instead, simplified drawings have been promoted to enhance learning. Skulmowski and Rey conducted a study to test whether color coding a realistic representation of a fictional bone would enhance the learning of the bone’s structure. Participants (N= 108) were randomly assigned and tested on one of 4 representations of a fictional bone: a simplified bone representation with labeled names of structure, a simplified bone representation with labeled names of structure and color coding of different areas of the bone, a realistic bone representation with labeled names of structure, a realistic bone representation with labeled names of structure and color coding of different areas of the bone. The researchers found that the realistic rendering of the bone structure enhanced by color coding led to better retention of information while students’ retention scores lowered when color coding was used on the simplistic bone representation. The authors concluded that realistic visualizations require appropriate visual aids in order to be effective but can be more effective for retention than simplified drawings. 

Skulmowski, A. and G. Rey (2018). Computers & Education vol 122, pp. 23-31
[link to article]

A Bayesian network meta-analysis to synthesize the influence of contexts of scaffolding use on cognitive outcomes in STEM education

This meta-analysis looks at N=56 studies on the impact of computer-based scaffolding on student learning. The researchers looked at within subjects gains in pre-post analysis of learning. The reviewed studies span from K-12 to graduate level students. The researchers found that computer scaffolding showed strong effect across student populations particularly for college and graduate students and STEM disciplines. 

Belland, Brian R., Andrew E. Walker, and Nam Ju Kim (2017). SReview of Educational Research, vol. 87, no. 6, pp. 1042-1081.
[link to article]

Stereotype threat effects on learning from a cognitively demanding mathematics lesson

A study of predominantly African-American 5th grade students (N=135) examined the effects of stereotype threat on initial student learning and knowledge formation using a math lesson.  Students assigned to the “threat” condition were asked to give their race at the beginning of the lesson - led to believe the study was about how students ‘like them’ learn best - and students in the control condition were asked to record the date instead of their race.  Results showed that students in the threat condition retained less information, enjoyed the lesson less, and reported lower levels of motivation than students in the control condition.  This effect was especially pronounced for students with higher levels of baseline executive functioning.  These findings suggest that stereotype threat may play a key role in the initial development of achievement gaps in education, and that its effects impact students on a deeper level beyond just test-taking performance.

Lyons, E. M., Simms, N., Begolli, K. N., & Richland, L. E. (2017). Cognitive science
[link to article]

Changes in syllabus tone affect warmth (but not competence) ratings of both male and female instructors

This study aimed to show correlations between the "friendliness" of a syllabus and student perceptions of the instructor's approachability and competence by gender. 150 undergraduate students were given either "friendly" or " not friendly" syllabi from either a female, male, or gender-unspecified instructor. The results showed that students perceived the "friendly" syllabus instructors as "more approachable, more caring, and more motivating, but not any more or less competent than those receiving the not friendly syllabus," regardless of gender. The article also includes a detailed list of example phrases from both the friendly and unfriendly syllabi, providing examples that could be used when teaching instructors about syllabus tone.

Denton, A.W.; Veloso, J. (2017). Social Psychology of Education (20). 1-15.
[link to article]

Student learning with permissive and restrictive cell phone policies: A classroom experiment

Two sections of a class taught by the same instructor had either a restrictive cell phone policy or a permissive cell phone policy (N=31). Students in the restrictive section were told they could be removed from class for use of their cell phones, and students in the permissive section were allowed to use cell phones during class. Researchers found no significant difference in the students' learning but found a statistically significant difference in the ratings of the instructor. The restrictive section gave higher ratings to the instructor than the permissive section. This research shows that student attitudes toward the instructor may not be negatively affected by a restrictive cell phone policy. 

Lancaster, Alexander (2018). International Journal for the Scholarship of Teaching and Learning, vol. 12, no. 1
[link to article]

Humor in the classroom: The effects of integrated humor on student learning 

Some previous research has suggested that humor could positively affect student motivation and attention. In this lab-based study, researchers asked whether integrated humor used to teach course concepts would affect students’ recall. In Study 1, 87 students were divided into two conditions. Students in condition 1 received a written lesson on self-efficacy that employed a humorous example. Students in condition 2 received the same lesson but a serious example. Students in condition 2 performed better on a multiple-choice recall test. Study 2 replicated the findings for study 1 using a new student population (more upper-level students), a new lesson (on cohesion in communication), a new learning environment (two different universities), and a new recall test (multiple choice + short answer). This study suggests that using integrated humor to directly teach core course concepts may negatively affect students’ learning of this content because humorous examples “potentially depress students’ ability to retain and transfer the ideas being presented.”

Bolkan, S., Griffin, D. J., & Goodboy, A. K. (2018). Communication Education, 1–21.
[link to article]

Reverse the routine: Problem solving before instruction improves conceptual knowledge in undergraduate physics

Exploratory learning in STEM courses at the university level is a relatively under-researched area. In three studies, researchers tested the effects of completing a collaborative (or solo) learning activity prior to being lectured about that topic (explore-first condition) versus completing the same activity after having the lecture (instruct-first condition). Across two semesters, students in three separate sections of a physics course (N = 362) were randomly assigned to receive either explore-first or instruct-first treatment in groups or individually. After completing both the activity and the lecture (in either order), all students completed a quiz to assess their learning as well as an interest/enjoyment questionnaire. Results showed that students in the explore-first condition tended to struggle more with the activity itself, but performed better on conceptual questions on the quiz than students in the instruct-first condition. There was no difference in performance on procedural quiz questions. Furthermore, students in the explore-first condition reported higher levels of interest and enjoyment. Results also showed that there was no added benefit of exploring in a group over exploring individually, suggesting that the benefits of exploratory learning are not simply driven by working collaboratively with others.  

Weaver, J. P., Chastain, R. J., DeCaro, D. A., & DeCaro, M. S. (2018). Contemporary Educational Psychology52, 36-47.
[link to article]

Peer Mentor Program for the General Chemistry Laboratory Designed To Improve Undergraduate STEM Retention. 

Several hundred undergraduate students enrolled in introductory chemistry labs at a large state university were assigned an upper-class peer mentor to each laboratory section. Instead of grading student work or focusing on course content, the mentor's responsibility was to check in with lab groups to talk about adjusting to college, offer advice on being a successful STEM student, and discuss other relevant topics such as undergraduate research opportunities or pursuing graduate school. After four years of the program, the peer-mentored cohort graduation rate was substantially higher (∼30%) than the non-peer laboratories group, and ∼22% higher than the general student population. 

Damkaci, F.; Braun, T.F.; Gublo, K. (2017). Journal of Chemical Education, ASAP. 
[link to article]

The Dark Side of Interpolated Testing: Frequent Switching Between Retrieval and Encoding Impairs New Learning

In a laboratory experiment using undergraduate participants (N=376), researchers sought to replicate and extend prior work that showed how retrieval practice can impair new learning (see Davis and Chan, 2015). Experimenters had participants switch between retrieval practice trials for a set of previously learned items and trials for the learning of new items at varying frequencies to determine the effects of making that switch has on learning the new content.  Results showed that as the frequency of switching between retrieval practice and new learning increased, the successful learning of the new information decreased. This research suggests that information retrieval and information encoding rely on separate cognitive processes that may compete for the same mental resources, and contributes to our understanding of how the use of interpolated testing could potentially hinder the learning of new information.

Davis, S. D., Chan, J. C., & Wilford, M. M. (2017). Journal of Applied Research in Memory and Cognition.
[link to article]