Empowering faculty to initiate STEM education transformation: Efficacy of a systems thinking approach.

Just a decade ago Vision and Change in Undergraduate Biology Education: A Call to Action was released, catalyzing several initiatives to transform undergraduate life sciences education. Among these was the Partnership for Undergraduate Life Sciences Education (PULSE), a national organization commissioned to increase the adoption of Vision and Change recommendations within academic life sciences departments. PULSE activities have been designed based on the recognition that life sciences departments and faculty are embedded within institutions of higher education which, similar to other large organizations, are complex systems composed of multiple, interconnected subsystems. The organizational change research suggests that effecting large-scale changes (e.g., undergraduate STEM education transformation) may be facilitated by applying systems thinking to change efforts. In this paper we introduce the approach of systems thinking as a professional development tool to empower individual STEM faculty to effect department-level transformation. We briefly describe a professional development experience designed to increase life sciences faculty members' understanding of systems thinking, present evidence that faculty applied a systems thinking approach to initiate department-level change, and discuss the degree to which transformation efforts were perceived to be successful. Though focused on faculty in the life sciences, our findings are broadly transferable to other efforts seeking to effect change in undergraduate STEM education.

Fidelity of Implementation as a Guiding Framework for Transitioning Research-Based Instructional Practices from On Site to Online.

In this Perspective, we contribute to the Journal of Microbiology & Biology Education special issue "Opportunities and Challenges of Online Instruction-Blurring the Lines Between Online and On-Site Teaching and Learning" by introducing a framework to guide instructors in transitioning from on-site to online instruction. The discipline-based education research community has produced a significant literature base demonstrating the effectiveness of various research-based instructional practices (RBIPs) that support student learning in face-to-face environments. However, little is known about how to transport these practices into the online environment. We introduce fidelity of implementation (FOI) as a framework for considering how to adapt RBIPs for online contexts while maintaining the integrity of the aspects of the practice that are thought to support student learning. We use our own experience responding to the global pandemic to provide rich examples of how FOI was used to anticipate challenges in transitioning online and describe how it was used to adapt an RBIP for online instruction.

Interpret with Caution: COPUS Instructional Styles May Not Differ in Terms of Practices That Support Student Learning.

There is a growing need for valid and reliable measures to monitor the efficacy of undergraduate science, technology, engineering, and mathematics (STEM) reform initiatives. The Classroom Observation Protocol for Undergraduate STEM (COPUS) is a widely used tool originally designed to measure the presence of overt instructor and student behaviors. It has subsequently been used to characterize instruction along a continuum from didactic to student centered, and more recently to categorize instruction into one of three styles. Initiatives focused on professional development often support instructors' progression from didactic to student-centered styles. There is a need to examine COPUS instructional styles in terms of behaviors that research has shown to improve student learning. Formative assessment is a research-based practice that involves behaviors accounted for by the COPUS (e.g., posing a question). We qualitatively compared the formative assessment behaviors in 16 biology class sessions categorized into each of the three COPUS styles. We were unable to detect differences in formative assessment behaviors between the COPUS styles. Caution should be taken when interpreting COPUS data to make inferences about the effects of reform efforts. This study underscores the need for additional measures to monitor national reform initiatives in undergraduate STEM.

Development of a Certification Exam to Assess Undergraduate Students' Proficiency in Biochemistry and Molecular Biology Core Concepts.

With support from the American Society for Biochemistry and Molecular Biology (ASBMB), a community of biochemistry and molecular biology (BMB) scientist-educators has developed and administered an assessment instrument designed to evaluate student competence across four core concept and skill areas fundamental to BMB. The four areas encompass energy and metabolism; information storage and transfer; macromolecular structure, function, and assembly; and skills including analytical and quantitative reasoning. First offered in 2014, the exam has now been administered to nearly 4000 students in ASBMB-accredited programs at more than 70 colleges and universities. Here, we describe the development and continued maturation of the exam program, including the organic role of faculty volunteers as drivers and stewards of all facets: content and format selection, question development, and scoring.

Fourteen Recommendations to Create a More Inclusive Environment for LGBTQ+ Individuals in Academic Biology.

Individuals who identify as lesbian, gay, bisexual, transgender, queer, and otherwise nonstraight and/or non-cisgender (LGBTQ+) have often not felt welcome or represented in the biology community. Additionally, biology can present unique challenges for LGBTQ+ students because of the relationship between certain biology topics and their LGBTQ+ identities. Currently, there is no centralized set of guidelines to make biology learning environments more inclusive for LGBTQ+ individuals. Rooted in prior literature and the collective expertise of the authors who identify as members and allies of the LGBTQ+ community, we present a set of actionable recommendations to help biologists, biology educators, and biology education researchers be more inclusive of individuals with LGBTQ+ identities. These recommendations are intended to increase awareness of LGBTQ+ identities and spark conversations about transforming biology learning spaces and the broader academic biology community to become more inclusive of LGBTQ+ individuals.

Can I Have Your Recipe? Using a Fidelity of Implementation (FOI) Framework to Identify the Key Ingredients of Formative Assessment for Learning.

For decades, formative assessment has been identified as a high-impact instructional practice that positively affects student learning. Education reform documents such as Vision and Change: A Call to Action expressly identify frequent, ongoing formative assessment and feedback as a key instructional practice in student-centered learning environments. Historically, effect sizes between 0.4 and 0.7 have been reported for formative assessment experiments. However, more recent meta-analyses have reported much lower effect sizes. It is unclear whether the variability in reported effects is due to formative assessment as an instructional practice in and of itself, differences in how formative assessment was enacted across studies, or other mitigating factors. We propose that application of a fidelity of implementation (FOI) framework to define the critical components of formative assessment will increase the validity of future impact studies. In this Essay, we apply core principles from the FOI literature to hypothesize about the critical components of formative assessment as a high-impact instructional practice. In doing so, we begin the iterative process through which further research can develop valid and reliable measures of the FOI of formative assessment. Such measures are necessary to empirically determine when, how, and under what conditions formative assessment supports student learning.

Investigation of a Stand-Alone Online Learning Module for Cellular Respiration Instruction.

With the recent rise of alternative instructional methodologies such as flipped classrooms and active learning, many core concepts are being introduced outside of the classroom prior to scheduled class meeting times. One popular means for external concept introduction in many undergraduate biology courses is the use of stand-alone online learning modules. Using a group of four large introductory biology course sections, we investigate the use of a stand-alone online learning module developed using animations from Virtual Cell Animation Collection as a resource for the introduction of cellular respiration concepts outside of the classroom. Results from four sections of introductory biology (n = 629) randomized to treatments show that students who interacted with the stand-alone online learning module had significantly higher normalized gain scores on a cellular respiration assessment than students who only attended a traditional lecture as a means of concept introduction (p < 0.001, d = 0.59). These findings suggest a superior ability to convey certain introductory cellular respiration topics in a stand-alone manner outside of the classroom than in a more traditional lecture-based classroom setting.

Visual Literacy in Bloom: Using Bloom's Taxonomy to Support Visual Learning Skills.

Vision and Change identifies science communication as one of the core competencies in undergraduate biology. Visual representations are an integral part of science communication, allowing ideas to be shared among and between scientists and the public. As such, development of scientific visual literacy should be a desired outcome of undergraduate instruction. We developed the Visualization Blooming Tool (VBT), an adaptation of Bloom's taxonomy specifically focused on visual representations, to aid instructors in designing instruction and assessments to target scientific visual literacy in undergraduate instruction. In this article, we identify the need for the VBT, describe its development, and provide concrete examples of its application to a curriculum redesign effort in undergraduate biochemistry.

Learning about Chemiosmosis and ATP Synthesis with Animations Outside of the Classroom.

Many undergraduate biology courses have begun to implement instructional strategies aimed at increasing student interaction with course material outside of the classroom. Two examples of such practices are introducing students to concepts as preparation prior to instruction, and as conceptual reinforcement after the instructional period. Using a three-group design, we investigate the impact of an animation developed as part of the Virtual Cell Animation Collection on the topic of concentration gradients and their role in the actions of ATP synthase as a means of pre-class preparation or post-class reinforcement compared with a no-intervention control group. Results from seven sections of introductory biology (n = 732) randomized to treatments over two semesters show that students who viewed animation as preparation (d = 0.44, p < 0.001) or as reinforcement (d = 0.53, p < 0.001) both outperformed students in the control group on a follow-up assessment. Direct comparison of the preparation and reinforcement treatments shows no significant difference in student outcomes between the two treatment groups (p = 0.87). Results suggest that while student interaction with animations on the topic of concentration gradients outside of the classroom may lead to greater learning outcomes than the control group, in the traditional lecture-based course the timing of such interactions may not be as important.

Efficacy of a Meiosis Learning Module Developed for the Virtual Cell Animation Collection.

Recent reports calling for change in undergraduate biology education have resulted in the redesign of many introductory biology courses. Reports on one common change to course structure, the active-learning environment, have placed an emphasis on student preparation, noting that the positive outcomes of active learning in the classroom depend greatly on how well the student prepares before class. As a possible preparatory resource, we test the efficacy of a learning module developed for the Virtual Cell Animation Collection. This module presents the concepts of meiosis in an interactive, dynamic environment that has previously been shown to facilitate learning in introductory biology students. Participants (n = 534) were enrolled in an introductory biology course and were presented the concepts of meiosis in one of two treatments: the interactive-learning module or a traditional lecture session. Analysis of student achievement shows that students who viewed the learning module as their only means of conceptual presentation scored significantly higher (d = 0.40, p < 0.001) than students who only attended a traditional lecture on the topic. Our results show the animation-based learning module effectively conveyed meiosis conceptual understanding, which suggests that it may facilitate student learning outside the classroom. Moreover, these results have implications for instructors seeking to expand their arsenal of tools for "flipping" undergraduate biology courses.

Lighten the Load: Scaffolding Visual Literacy in Biochemistry and Molecular Biology.

The development of scientific visual literacy has been identified as critical to the training of tomorrow's scientists and citizens alike. Within the context of the molecular life sciences in particular, visual representations frequently incorporate various components, such as discipline-specific graphical and diagrammatic features, varied levels of abstraction, and spatial arrangements of visual elements to convey information. Visual literacy is achieved when an individual understands the various ways in which a discipline uses these components to represent a particular way of knowing. Owing to the complex nature of visual representations, the activities through which visual literacy is developed have high cognitive load. Cognitive load can be reduced by first helping students to become fluent with the discrete components of visual representations before asking them to simultaneously integrate these components to extract the intended meaning of a representation. We present a taxonomy for characterizing one component of visual representations-the level of abstraction-as a first step in understanding the opportunities afforded students to develop fluency. Further, we demonstrate how our taxonomy can be used to analyze course assessments and spur discussions regarding the extent to which the development of visual literacy skills is supported by instruction within an undergraduate biochemistry curriculum.

Variation in external representations as part of the classroom lecture:An investigation of virtual cell animations in introductory photosynthesis instruction.

The use of external representations (ERs) to introduce concepts in undergraduate biology has become increasingly common. Two of the most prevalent are static images and dynamic animations. While previous studies comparing static images and dynamic animations have resulted in somewhat conflicting findings in regards to learning outcomes, the benefits of each have been shown individually. Using ERs developed by the Virtual Cell Animation project, we aim to further investigate student learning using different ERs as part of an introductory biology lecture. We focus our study on the topic of photosynthesis as reports have noted that students struggle with a number of basic photosynthesis concepts. Students (n = 167) in ten sections of introductory biology laboratory were introduced to photosynthesis concepts by instructional lectures differing only in the format of the embedded ERs. Normalized gain scores were calculated, showing that students who learned with dynamic animations outperformed students who learned from static images on the posttest. The results of this study provide possible instructional guidelines for those delivering photosynthesis instruction in the introductory biology classroom. © 2016 by The International Union of Biochemistry and Molecular Biology, 45(3):226-234, 2017.

Student-generated reading questions: diagnosing student thinking with diverse formative assessments.

Formative assessment has long been identified as a critical element to teaching for conceptual development in science. It is therefore important for university instructors to have an arsenal of formative assessment tools at their disposal which enable them to effectively uncover and diagnose all students' thinking, not just the most vocal or assertive. We illustrate the utility of one type of formative assessment prompt (reading question assignment) in producing high-quality evidence of student thinking (student-generated reading questions). Specifically, we characterized student assessment data using three distinct analytic frames to exemplify their effectiveness in diagnosing student learning in relationship to three sample learning outcomes. Our data will be useful for university faculty, particularly those engaged in teaching upper-level biochemistry courses and their prerequisites, as they provide an alternative mechanism for uncovering and diagnosing student understanding.

Commentary: PhDs in biochemistry education-5 years later.

In this commentary, the discussion of PhDs in biochemistry education research is expanded to explore a number of diverse pathways leading to a competitive research program in biochemistry education research.

Using assessments to investigate and compare the nature of learning in undergraduate science courses.

Assessments and student expectations can drive learning: students selectively study and learn the content and skills they believe critical to passing an exam in a given subject. Evaluating the nature of assessments in undergraduate science education can, therefore, provide substantial insight into student learning. We characterized and compared the cognitive skills routinely assessed by introductory biology and calculus-based physics sequences, using the cognitive domain of Bloom's taxonomy of educational objectives. Our results indicate that both introductory sequences overwhelmingly assess lower-order cognitive skills (e.g., knowledge recall, algorithmic problem solving), but the distribution of items across cognitive skill levels differs between introductory biology and physics, which reflects and may even reinforce student perceptions typical of those courses: biology is memorization, and physics is solving problems. We also probed the relationship between level of difficulty of exam questions, as measured by student performance and cognitive skill level as measured by Bloom's taxonomy. Our analyses of both disciplines do not indicate the presence of a strong relationship. Thus, regardless of discipline, more cognitively demanding tasks do not necessarily equate to increased difficulty. We recognize the limitations associated with this approach; however, we believe this research underscores the utility of evaluating the nature of our assessments.

Stereotyped: investigating gender in introductory science courses.

Research in science education has documented achievement gaps between men and women in math and physics that may reflect, in part, a response to perceived stereotype threat. Research efforts to reduce achievement gaps by mediating the impact of stereotype threat have found success with a short values-affirmation writing exercise. In biology and biochemistry, however, little attention has been paid to the performance of women in comparison with men or perceptions of stereotype threat, despite documentation of leaky pipelines into professional and academic careers. We used methodologies developed in physics education research and cognitive psychology to 1) investigate and compare the performance of women and men across three introductory science sequences (biology, biochemistry, physics), 2) document endorsement of stereotype threat in these science courses, and 3) investigate the utility of a values-affirmation writing task in reducing achievement gaps. In our study, analysis of final grades and normalized learning gains on content-specific concept inventories reveals no achievement gap in the courses sampled, little stereotype threat endorsement, and no impact of the values-affirmation writing task on student performance. These results underscore the context-dependent nature of achievement gaps and stereotype threat and highlight calls to replicate education research across a range of student populations.

Comparison of views of the nature of science between natural science and nonscience majors.

Science educators have the common goal of helping students develop scientific literacy, including understanding of the nature of science (NOS). University faculties are challenged with the need to develop informed NOS views in several major student subpopulations, including science majors and nonscience majors. Research into NOS views of undergraduates, particularly science majors, has been limited. In this study, NOS views of undergraduates in introductory environmental science and upper-level animal behavior courses were measured using Likert items and open-ended prompts. Analysis revealed similarities in students' views between the two courses; both populations held a mix of naïve, transitional, and moderately informed views. Comparison of pre- and postcourse mean scores revealed significant changes in NOS views only in select aspects of NOS. Student scores on sections addressing six aspects of NOS were significantly different in most cases, showing notably uninformed views of the distinctions between scientific theories and laws. Evidence-based insight into student NOS views can aid in reforming undergraduate science courses and will add to faculty and researcher understanding of the impressions of science held by undergraduates, helping educators improve scientific literacy in future scientists and diverse college graduates.