Using the Intended-Enacted-Experienced Curriculum Model to Map the Vision and Change Core Competencies in Undergraduate Biology Programs and Courses.

One critical step in the challenging process of curricular reform is determining how closely a curriculum aligns with national recommendations. Here, we examine the alignment of teaching, assessment, and student experience in undergraduate biology courses with the Vision and Change core competency recommendations. We applied the intended-enacted-experienced curriculum model to obtain a more complete, multiperspective view of the curriculum. First, we developed and piloted the BioSkills Curriculum Survey with more than 100 biology instructors across five institutions. Using multilevel logistic regression modeling of the survey data, we found that instructors were equally likely to report teaching all competencies; however, they reported assessing some competencies more than others. After adding course characteristics to our model, we found that the likelihood of teaching certain competencies depended on course type. Next, we analyzed class materials and student perceptions of instruction in 10 biology courses in one department. Within this smaller sample, we found that instructors messaged a narrower range of competency learning outcomes on their syllabi than they reported teaching on the survey. Finally, modeling revealed that inclusion of an outcome on assessments, but not syllabi, increased the likelihood that students and their instructor agreed whether it was taught.

BioSkills Guide: Development and National Validation of a Tool for Interpreting the Vision and Change Core Competencies.

To excel in modern science, technology, engineering, and mathematics careers, biology majors need a range of transferable skills, yet competency development is often a relatively underdeveloped facet of the undergraduate curriculum. We have elaborated the Vision and Change core competency framework into a resource called the BioSkills Guide, a set of measurable learning outcomes that can be more readily implemented by faculty. Following an iterative review process including more than 200 educators, we gathered evidence of the BioSkills Guide's content validity using a national survey of more than 400 educators. Rates of respondent support were high (74.3-99.6%) across the 77 outcomes in the final draft. Our national sample during the development and validation phases included college biology educators representing more than 250 institutions, including 73 community colleges, and a range of course levels and biology subdisciplines. Comparison of the BioSkills Guide with other science competency frameworks reveals significant overlap but some gaps and ambiguities. These differences may reflect areas where understandings of competencies are still evolving in the undergraduate biology community, warranting future research. We envision the BioSkills Guide supporting a variety of applications in undergraduate biology, including backward design of individual lessons and courses, competency assessment development, and curriculum mapping and planning.

Resources for Teaching and Assessing the Vision and Change Biology Core Concepts.

The Vision and Change report called for the biology community to mobilize around teaching the core concepts of biology. This essay describes a collection of resources developed by several different groups that can be used to respond to the report's call to transform undergraduate education at both the individual course and departmental levels. First, we present two frameworks that help articulate the Vision and Change core concepts, the BioCore Guide and the Conceptual Elements (CE) Framework, which can be used in mapping the core concepts onto existing curricula and designing new curricula that teach the biology core concepts. Second, we describe how the BioCore Guide and the CE Framework can be used alongside the Partnership for Undergraduate Life Sciences Education curricular rubric as a way for departments to self-assess their teaching of the core concepts. Finally, we highlight three sets of instruments that can be used to directly assess student learning of the core concepts: the Biology Card Sorting Task, the Biology Core Concept Instruments, and the Biology-Measuring Achievement and Progression in Science instruments. Approaches to using these resources independently and synergistically are discussed.

Tools for Change: Measuring Student Conceptual Understanding Across Undergraduate Biology Programs Using Bio-MAPS Assessments.

Assessing learning across a biology major can help departments monitor achievement of broader program-level goals and identify opportunities for curricular improvement. However, biology departments have lacked suitable tools to measure learning at the program scale. To address this need, we developed four freely available assessments-called Biology-Measuring Achievement and Progression in Science or Bio-MAPS-for general biology, molecular biology, ecology/evolution, and physiology programs. When administered at multiple time points in a curriculum, these instruments can provide departments with information on how student conceptual understanding changes across a major and help guide curricular modifications to enhance learning.

Impact of automated response systems on in-class cell phone use.

Cell phones have long been known as a potential distraction from attention intensive activities such as studying and driving. Many, however, are developing the cell phone as a powerful tool to augment some of these same activities. Audience response systems (ARSs) are a type of teaching tool that allows educators to poll audience members in real time. Increasingly, cell phones are being integrated into ARSs to make them more versatile and affordable. As cell phones and other personal electronic devices (tablets, laptops) are becoming more common classroom learning tools, we sought to explore how student cell phone use is impacted by this change. Additionally, we studied how a student's seat location and how the time during a term impacts students' cell phone use. To measure student cell phone use in lecture, we observed introductory biology classes at the University of Washington and recorded when students' cell phones were visible. We found that students sitting in the back of the room showed an increased likelihood of having a cell phone out. However contrary to our expectations, students using personal device (cell phone) based polling technology were no more likely to be using cell phones during lecture than their peers using traditional ARSs. Our results suggest that the downsides to using cell phones as teaching tools may be limited. © 2019 The International Union of Biochemistry and Molecular Biology, 47(5):538-546, 2019.

GenBio-MAPS: A Programmatic Assessment to Measure Student Understanding of Vision and Change Core Concepts across General Biology Programs.

The Vision and Change report provides a nationally agreed upon framework of core concepts that undergraduate biology students should master by graduation. While identifying these concepts was an important first step, departments also need ways to measure the extent to which students understand these concepts. Here, we present the General Biology-Measuring Achievement and Progression in Science (GenBio-MAPS) assessment as a tool to measure student understanding of the core concepts at key time points in a biology degree program. Data from more than 5000 students at 20 institutions reveal that this instrument distinguishes students at different stages of the curriculum, with an upward trend of increased performance at later time points. Despite this trend, we identify several concepts that advanced students find challenging. Linear mixed-effects models reveal that gender, race/ethnicity, English-language status, and first-generation status predict overall performance and that different institutions show distinct performance profiles across time points. GenBio-MAPS represents the first programmatic assessment for general biology programs that spans the breadth of biology and aligns with the Vision and Change core concepts. This instrument provides a needed tool to help departments monitor student learning and guide curricular transformation centered on the teaching of core concepts.

Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates.

We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates.

A new assessment tool, Ecology and Evolution-Measuring Achievement and Progression in Science or EcoEvo-MAPS, measures student thinking in ecology and evolution during an undergraduate course of study. EcoEvo-MAPS targets foundational concepts in ecology and evolution and uses a novel approach that asks students to evaluate a series of predictions, conclusions, or interpretations as likely or unlikely to be true given a specific scenario. We collected evidence of validity and reliability for EcoEvo-MAPS through an iterative process of faculty review, student interviews, and analyses of assessment data from more than 3000 students at 34 associate's-, bachelor's-, master's-, and doctoral-granting institutions. The 63 likely/unlikely statements range in difficulty and target student understanding of key concepts aligned with the Vision and Change report. This assessment provides departments with a tool to measure student thinking at different time points in the curriculum and provides data that can be used to inform curricular and instructional modifications.

Student perception of group dynamics predicts individual performance: Comfort and equity matter.

Active learning in college classes and participation in the workforce frequently hinge on small group work. However, group dynamics vary, ranging from equitable collaboration to dysfunctional groups dominated by one individual. To explore how group dynamics impact student learning, we asked students in a large-enrollment university biology class to self-report their experience during in-class group work. Specifically, we asked students whether there was a friend in their group, whether they were comfortable in their group, and whether someone dominated their group. Surveys were administered after students participated in two different types of intentionally constructed group activities: 1) a loosely-structured activity wherein students worked together for an entire class period (termed the 'single-group' activity), or 2) a highly-structured 'jigsaw' activity wherein students first independently mastered different subtopics, then formed new groups to peer-teach their respective subtopics. We measured content mastery by the change in score on identical pre-/post-tests. We then investigated whether activity type or student demographics predicted the likelihood of reporting working with a dominator, being comfortable in their group, or working with a friend. We found that students who more strongly agreed that they worked with a dominator were 17.8% less likely to answer an additional question correct on the 8-question post-test. Similarly, when students were comfortable in their group, content mastery increased by 27.5%. Working with a friend was the single biggest predictor of student comfort, although working with a friend did not impact performance. Finally, we found that students were 67% less likely to agree that someone dominated their group during the jigsaw activities than during the single group activities. We conclude that group activities that rely on positive interdependence, and include turn-taking and have explicit prompts for students to explain their reasoning, such as our jigsaw, can help reduce the negative impact of inequitable groups.

ASPECT: A Survey to Assess Student Perspective of Engagement in an Active-Learning Classroom.

The primary measure used to determine relative effectiveness of in-class activities has been student performance on pre/posttests. However, in today's active-learning classrooms, learning is a social activity, requiring students to interact and learn from their peers. To develop effective active-learning exercises that engage students, it is important to gain a more holistic view of the student experience in an active-learning classroom. We have taken a mixed-methods approach to iteratively develop and validate a 16-item survey to measure multiple facets of the student experience during active-learning exercises. The instrument, which we call Assessing Student Perspective of Engagement in Class Tool (ASPECT), was administered to a large introductory biology class, and student responses were subjected to exploratory factor analysis. The 16 items loaded onto three factors that cumulatively explained 52% of the variation in student response: 1) value of activity, 2) personal effort, and 3) instructor contribution. ASPECT provides a rapid, easily administered means to measure student perception of engagement in an active-learning classroom. Gaining a better understanding of students' level of engagement will help inform instructor best practices and provide an additional measure for comprehensively assessing the impact of different active-learning strategies.

Males Under-Estimate Academic Performance of Their Female Peers in Undergraduate Biology Classrooms.

Women who start college in one of the natural or physical sciences leave in greater proportions than their male peers. The reasons for this difference are complex, and one possible contributing factor is the social environment women experience in the classroom. Using social network analysis, we explore how gender influences the confidence that college-level biology students have in each other's mastery of biology. Results reveal that males are more likely than females to be named by peers as being knowledgeable about the course content. This effect increases as the term progresses, and persists even after controlling for class performance and outspokenness. The bias in nominations is specifically due to males over-nominating their male peers relative to their performance. The over-nomination of male peers is commensurate with an overestimation of male grades by 0.57 points on a 4 point grade scale, indicating a strong male bias among males when assessing their classmates. Females, in contrast, nominated equitably based on student performance rather than gender, suggesting they lacked gender biases in filling out these surveys. These trends persist across eleven surveys taken in three different iterations of the same Biology course. In every class, the most renowned students are always male. This favoring of males by peers could influence student self-confidence, and thus persistence in this STEM discipline.

BioCore Guide: A Tool for Interpreting the Core Concepts of Vision and Change for Biology Majors.

Vision and Change in Undergraduate Biology Education outlined five core concepts intended to guide undergraduate biology education: 1) evolution; 2) structure and function; 3) information flow, exchange, and storage; 4) pathways and transformations of energy and matter; and 5) systems. We have taken these general recommendations and created a Vision and Change BioCore Guide-a set of general principles and specific statements that expand upon the core concepts, creating a framework that biology departments can use to align with the goals of Vision and Change. We used a grassroots approach to generate the BioCore Guide, beginning with faculty ideas as the basis for an iterative process that incorporated feedback from more than 240 biologists and biology educators at a diverse range of academic institutions throughout the United States. The final validation step in this process demonstrated strong national consensus, with more than 90% of respondents agreeing with the importance and scientific accuracy of the statements. It is our hope that the BioCore Guide will serve as an agent of change for biology departments as we move toward transforming undergraduate biology education.

Sequence and spacing of TATA box elements are critical for accurate initiation from the beta-phaseolin promoter.

The beta-phaseolin (phas) gene, which encodes one of the major seed storage proteins of P. vulgaris, is tightly regulated at the transcription level resulting in strict tissue-specific and spatial expression during embryonic development. The phas proximal promoter contains a complex arrangement of core promoter elements including three TATA boxes as well as several putative initiator elements. To delineate the respective contributions of the core promoter elements to transcription initiation we have performed site-directed mutagenesis of the phas promoter. In vivo expression studies were performed on transgenic Arabidopsis harboring phas promoter mutants driving expression of the beta-glucuronidase (gus) reporter gene. Quantitative assessment of GUS activity in seeds bearing the promoter mutants indicated that both sequence and spacing of the TATA elements influenced the efficiency of transcription. Substitution, insertion or deletion mutations had no effect on histochemical staining patterns indicating that strict spacing requirements are not essential for correct spatial expression of phas during embryogenesis. Further evaluation of the phas promoter by in vitro transcription analysis revealed the presence of multiple TATA-dependent transcription initiation start sites. The distance between TATA elements and transcription start sites was maintained in insertion and deletion mutants through the creation of novel initiation sites, indicating that positioning of the TATA elements rather than DNA sequence was the primary determinant of start site location. We conclude that, while dispensable for proper spatial distribution, the complex architecture of the phas promoter is required to ensure high levels of accurate phas transcription initiation in the developing embryo.