Scaling up behavioral science interventions in online education.

Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over 2.5 y, with one-quarter million students, from nearly every country, across 247 online courses offered by Harvard, the Massachusetts Institute of Technology, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically preregistering new hypotheses in between waves of data collection, we identified individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We found minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order-of-magnitude. However, iterative scientific investigations can uncover what works where for whom.

Eliciting Learner Knowledge: Enabling Focused Practice through an Open-Source Online Tool.

Eliciting and interpreting students' ideas are essential skills in teaching, yet pre-service teachers (PSTs) rarely have adequate opportunities to develop these skills. In this study, we examine PSTs' patterns of discourse and perceived learning through engaging in an interactive digital simulation called Eliciting Learner Knowledge (ELK). ELK is a seven-minute, chat-based virtual role play between a PST playing a "teacher" and a PST playing a "student" where the goal is for the teacher to find out what the student knows about a topic. ELK is designed to be a practice space where pre-service and in-service teachers can learn strategies for effectively eliciting their students' knowledge. We review the implementation of ELK in eight teacher education courses in math or science methods at six different universities and assess (a) patterns of interaction during ELK and (b) PSTs' perceptions of ELK and their learning from the simulation. Our findings suggest that PSTs engage in effective practices such as eliciting and probing more often than less effective practices such as evaluating and telling. Results suggest that PSTs gain experience in practicing talk moves and having empathy for students' perspectives through using ELK.

The Civic Mission of MOOCs: Engagement across Political Differences in Online Forums.

Massive open online courses (MOOCs) attract diverse student bodies, and course forums could potentially be an opportunity for students with different political beliefs to engage with one another. We test whether this engagement actually takes place in two politically-themed MOOCs, on education policy and American government. We collect measures of students' political ideology, and then observe student behavior in the course discussion boards. Contrary to the common expectation that online spaces often become echo chambers or ideological silos, we find that students in these two political courses hold diverse political beliefs, participate equitably in forum discussions, directly engage (through replies and upvotes) with students holding opposing beliefs, and converge on a shared language rather than talk-ing past one another. Research that focuses on the civic mission of MOOCs helps ensure that open online learning engages the same breadth of purposes that higher education aspires to serve.

Democratizing education? Examining access and usage patterns in massive open online courses.

Massive open online courses (MOOCs) are often characterized as remedies to educational disparities related to social class. Using data from 68 MOOCs offered by Harvard and MIT between 2012 and 2014, we found that course participants from the United States tended to live in more-affluent and better-educated neighborhoods than the average U.S. resident. Among those who did register for courses, students with greater socioeconomic resources were more likely to earn a certificate. Furthermore, these differences in MOOC access and completion were larger for adolescents and young adults, the traditional ages where people find on-ramps into science, technology, engineering, and mathematics (STEM) coursework and careers. Our findings raise concerns that MOOCs and similar approaches to online learning can exacerbate rather than reduce disparities in educational outcomes related to socioeconomic status.