Building conceptual and methodological bridges between SSE's diversity, equity, and inclusion statement and educational actions in evolutionary biology.

The field of evolutionary biology must bridge the gap between its diversity, equity, and inclusion (DEI) commitments and data-driven educational actions in the nation's undergraduate classrooms and degree programs. In this article, we discuss the urgent need for the adoption of equity frameworks and why they are centrally important to data-driven DEI efforts in evolutionary biology. We describe why equity indicators (e.g., measures) must be anchored in and aligned with equity frameworks. We introduce a specific equity framework for learning (the enhanced educational debt framework) and illustrate how it may be leveraged to document, interpret, and improve outcomes in evolutionary biology. We apply the equity framework and associated indicators to >3,500 students' first college-level experience with evolutionary biology at a public, 4-year institution in the Northeastern United States to demonstrate how these conceptual tools and empirical perspectives may be used by faculty, departments, and degree programs to better understand their roles in mitigating or perpetuating inequities. We end by discussing how this framework may be applied to a range of evolution concepts and courses in the educational hierarchy and used to help evolutionary biologists better understand the extent to which a core aspect of SSE's diversity statement is being realized.

Enhancing data pipelines for forecasting student performance: integrating feature selection with cross-validation.

Educators seek to harness knowledge from educational corpora to improve student performance outcomes. Although prior studies have compared the efficacy of data mining methods (DMMs) in pipelines for forecasting student success, less work has focused on identifying a set of relevant features prior to model development and quantifying the stability of feature selection techniques. Pinpointing a subset of pertinent features can (1) reduce the number of variables that need to be managed by stakeholders, (2) make "black-box" algorithms more interpretable, and (3) provide greater guidance for faculty to implement targeted interventions. To that end, we introduce a methodology integrating feature selection with cross-validation and rank each feature on subsets of the training corpus. This modified pipeline was applied to forecast the performance of 3225 students in a baccalaureate science course using a set of 57 features, four DMMs, and four filter feature selection techniques. Correlation Attribute Evaluation (CAE) and Fisher's Scoring Algorithm (FSA) achieved significantly higher Area Under the Curve (AUC) values for logistic regression (LR) and elastic net regression (GLMNET), compared to when this pipeline step was omitted. Relief Attribute Evaluation (RAE) was highly unstable and produced models with the poorest prediction performance. Borda's method identified grade point average, number of credits taken, and performance on concept inventory assessments as the primary factors impacting predictions of student performance. We discuss the benefits of this approach when developing data pipelines for predictive modeling in undergraduate settings that are more interpretable and actionable for faculty and stakeholders. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s41239-021-00279-6.

Are Faculty Changing? How Reform Frameworks, Sampling Intensities, and Instrument Measures Impact Inferences about Student-Centered Teaching Practices.

Although recent studies have used the Classroom Observation Protocol for Undergraduate STEM (COPUS) to make claims about faculty reform, important questions remain: How should COPUS measures be situated within existing reform frameworks? Is there a universal sampling intensity that allows for valid inferences about the frequency of student-centered instruction within a semester or across semesters of a course? These questions were addressed using longitudinal COPUS observations (128 classes, three faculty, 4 years). COPUS behaviors were used to categorize classes into didactic, interactive lecture, or student-centered instructional styles. Sampling intensities (one to 11 classes) were simulated (1000 times) within a course and across semesters. The sampling intensities required for generating valid inferences about 1) the presence of student-centered instruction and 2) the proportion of instructional styles in a course and through time were calculated. Results indicated that the sampling intensity needed to characterize courses and instructors varied and was much higher than previously recommended for instructors with: 1) rare instances of student-centered classes, 2) variability in instructional style, and 3) longitudinal changes in instructional patterns. These conditions are common in early reform contexts. This study highlights the risks of broad, decontextualized sampling protocol recommendations and illustrates how reform frameworks, sampling intensities, and COPUS measures interact to impact inferences about faculty change.

Illuminating the complexities of conflict with evolution: validation of the scales of evolutionary conflict measure (SECM).

BACKGROUND: Although personal, familial, and community conflict with evolution have been documented in the literature, these scales require conceptualization as a construct and operationalization as a measure. The Scales of Conflict with Evolution Measure (SECM) instrument was developed in response to these needs. Using a construct validity framework, the content, internal structure, convergent, and substantive validity of the SECM were evaluated using Rasch analysis, Structural Equation Modeling (SEM), and follow up questioning. The conceptual utility of the instrument was explored by examining whether it added explanatory insights into evolution acceptance above and beyond religiosity, evolution knowledge, and background variables. RESULTS: A literature review and expert consultation indicated that construct of evolutionary conflict perception should (i) encompass the hierarchical nature of human social structures (personal, family, community) and (ii) probe conflict as it relates to human values, cultures, and beliefs. A three-dimensional construct was operationalized as a nine-item rating scale measure. Using Rasch analyses of SECM responses from a diverse sample of > 1000 students studying evolution, the instrument met criteria of robust measurement, including: fit to model expectations; three-dimensional structure; high reliability; good rating scale function; measurement invariance with time; and convergence with a similar construct. SEM showed that: (i) family and community conflict had unique causal contributions to personal conflict, with family showing a stronger and modest impact, and (ii) personal conflict had a significant and modest causal impact on evolution acceptance above and beyond the contributions of religiosity, evolution knowledge, and background variables. CONCLUSION: The SECM is an easy-to-administer instrument to measure conflict with evolution and is supported by several forms of validity evidence. The SECM has potential for facilitating measurement of evolutionary conflict in educational settings, thereby raising instructor awareness of conflict levels in students, promoting rigorous evaluations of educational interventions designed to reduce conflict, and fostering conceptual advances in the field of evolution education. Future work is needed to gather additional forms of validity evidence and to test current validity claims in additional participant samples. SECM measures should also be incorporated into more complex SEM models that treat evolution knowledge and religiosity as part of the structural paths to evolution acceptance. Such models could provide insights into the most worthwhile targets for the development of educational interventions to mitigate conflict at multiple scales.

A Faculty Professional Development Model That Improves Student Learning, Encourages Active-Learning Instructional Practices, and Works for Faculty at Multiple Institutions.

Helping faculty develop high-quality instruction that positively affects student learning can be complicated by time limitations, a lack of resources, and inexperience using student data to make iterative improvements. We describe a community of 16 faculty from five institutions who overcame these challenges and collaboratively designed, taught, iteratively revised, and published an instructional unit about the potential effect of mutations on DNA replication, transcription, and translation. The unit was taught to more than 2000 students in 18 courses, and student performance improved from preassessment to postassessment in every classroom. This increase occurred even though faculty varied in their instructional practices when they were teaching identical materials. We present information on how this faculty group was organized and facilitated, how members used student data to positively affect learning, and how they increased their use of active-learning instructional practices in the classroom as a result of participation. We also interviewed faculty to learn more about the most useful components of the process. We suggest that this professional development model can be used for geographically separated faculty who are interested in working together on a known conceptual difficulty to improve student learning and explore active-learning instructional practices.

What Motivates Biology Instructors to Engage and Persist in Teaching Professional Development?

We conducted a study of 19 biology instructors participating in small, local groups at six research-intensive universities connected to the Automated Analysis of Constructed Response (AACR) project (www.msu.edu/∼aacr). Our aim was to uncover participants' motivation to persist in a long-term teaching professional development effort, a topic that is understudied in discipline-based educational research. We interviewed each participant twice over a 2-year period and conducted qualitative analyses on the data, using expectancy-value theory as a framework for considering motivation. Our analyses revealed that motivation among instructors was high due to their enjoyment of the AACR groups. The high level of motivation is further explained by the fact that AACR groups facilitated instructor involvement with the larger AACR project. We also found that group dynamics encouraged persistence; instructors thought they might never talk with colleagues about teaching in the absence of AACR groups; and groups were perceived to have a low-enough time requirement to warrant sustained involvement. We conclude that instructors have persisted in AACR groups because the groups provided great value with limited cost. The characterization of instructor experiences described here can contribute to a better understanding of faculty needs in teaching professional development.

Examining Gender Differences in Written Assessment Tasks in Biology: A Case Study of Evolutionary Explanations.

Understanding sources of performance bias in science assessment provides important insights into whether science curricula and/or assessments are valid representations of student abilities. Research investigating assessment bias due to factors such as instrument structure, participant characteristics, and item types are well documented across a variety of disciplines. However, the relationships among these factors are unclear for tasks evaluating understanding through performance on scientific practices, such as explanation. Using item-response theory (Rasch analysis), we evaluated differences in performance by gender on a constructed-response (CR) assessment about natural selection (ACORNS). Three isomorphic item strands of the instrument were administered to a sample of undergraduate biology majors and nonmajors (Group 1: n = 662 [female = 51.6%]; G2: n = 184 [female = 55.9%]; G3: n = 642 [female = 55.1%]). Overall, our results identify relationships between item features and performance by gender; however, the effect is small in the majority of cases, suggesting that males and females tend to incorporate similar concepts into their CR explanations. These results highlight the importance of examining gender effects on performance in written assessment tasks in biology.

A critical analysis of assessment quality in genomics and bioinformatics education research.

The growing importance of genomics and bioinformatics methods and paradigms in biology has been accompanied by an explosion of new curricula and pedagogies. An important question to ask about these educational innovations is whether they are having a meaningful impact on students' knowledge, attitudes, or skills. Although assessments are necessary tools for answering this question, their outputs are dependent on their quality. Our study 1) reviews the central importance of reliability and construct validity evidence in the development and evaluation of science assessments and 2) examines the extent to which published assessments in genomics and bioinformatics education (GBE) have been developed using such evidence. We identified 95 GBE articles (out of 226) that contained claims of knowledge increases, affective changes, or skill acquisition. We found that 1) the purpose of most of these studies was to assess summative learning gains associated with curricular change at the undergraduate level, and 2) a minority (<10%) of studies provided any reliability or validity evidence, and only one study out of the 95 sampled mentioned both validity and reliability. Our findings raise concerns about the quality of evidence derived from these instruments. We end with recommendations for improving assessment quality in GBE.

Applying computerized-scoring models of written biological explanations across courses and colleges: prospects and limitations.

Our study explored the prospects and limitations of using machine-learning software to score introductory biology students' written explanations of evolutionary change. We investigated three research questions: 1) Do scoring models built using student responses at one university function effectively at another university? 2) How many human-scored student responses are needed to build scoring models suitable for cross-institutional application? 3) What factors limit computer-scoring efficacy, and how can these factors be mitigated? To answer these questions, two biology experts scored a corpus of 2556 short-answer explanations (from biology majors and nonmajors) at two universities for the presence or absence of five key concepts of evolution. Human- and computer-generated scores were compared using kappa agreement statistics. We found that machine-learning software was capable in most cases of accurately evaluating the degree of scientific sophistication in undergraduate majors' and nonmajors' written explanations of evolutionary change. In cases in which the software did not perform at the benchmark of "near-perfect" agreement (kappa > 0.80), we located the causes of poor performance and identified a series of strategies for their mitigation. Machine-learning software holds promise as an assessment tool for use in undergraduate biology education, but like most assessment tools, it is also characterized by limitations.

Understanding undergraduates' problem-solving processes.

Fostering effective problem-solving skills is one of the most longstanding and widely agreed upon goals of biology education. Nevertheless, undergraduate biology educators have yet to leverage many major findings about problem-solving processes from the educational and cognitive science research literatures. This article highlights key facets of problem-solving processes and introduces methodologies that may be used to reveal how undergraduate students perceive and represent biological problems. Overall, successful problem-solving entails a keen sensitivity to problem contexts, disciplined internal representation or modeling of the problem, and the principled management and deployment of cognitive resources. Context recognition tasks, problem representation practice, and cognitive resource management receive remarkably little emphasis in the biology curriculum, despite their central roles in problem-solving success.

Selective use of the primary literature transforms the classroom into a virtual laboratory.

CREATE (consider, read, elucidate hypotheses, analyze and interpret the data, and think of the next experiment) is a new method for teaching science and the nature of science through primary literature. CREATE uses a unique combination of novel pedagogical tools to guide undergraduates through analysis of journal articles, highlighting the evolution of scientific ideas by focusing on a module of four articles from the same laboratory. Students become fluent in the universal language of data analysis as they decipher the figures, interpret the findings, and propose and defend further experiments to test their own hypotheses about the system under study. At the end of the course students gain insight into the individual experiences of article authors by reading authors' responses to an e-mail questionnaire generated by CREATE students. Assessment data indicate that CREATE students gain in ability to read and critically analyze scientific data, as well as in their understanding of, and interest in, research and researchers. The CREATE approach demystifies the process of reading a scientific article and at the same time humanizes scientists. The positive response of students to this method suggests that it could make a significant contribution to retaining undergraduates as science majors.