Undergraduate students' neurophysiological reasoning: what we learn from the attractive distractors students select.

The basis for mastering neurophysiology is understanding ion movement across cell membranes. The Electrochemical Gradients Assessment Device (EGAD) is a 17-item test assessing students' understanding of fundamental concepts of neurophysiology, e.g., electrochemical gradients and resistance, synaptic transmission, and stimulus strength. We collected responses to the EGAD from 534 students from seven institutions nationwide, before and after instruction. We determined the relative difficulty of neurophysiology topics and noted that students did better on "what" questions compared to "how" questions, particularly those integrating concentration gradient and electric forces to predict ion movement. We also found that, even after instruction, students selected one incorrect answer, at a rate greater than random chance for nine questions. We termed these incorrect answers attractive distractors. Most attractive distractors contained terms associated with concentration gradients, equilibrium, or anthropomorphic and teleological reasoning, and incorrect answers containing multiple terms were more attractive. We used χ2 analysis and alluvial diagrams to investigate how individual students moved or did not move between answer choices on the pre- and posttest. Interestingly, students selecting the attractive distractor on the pretest were just as likely as other incorrect students to move to the correct answer on the posttest. In contrast, of students incorrect on both the pre- and posttest, students who selected the attractive distractor on the pretest were more likely to stick with this answer on the posttest than students choosing other incorrect answers. Combining the EGAD results with alluvial diagrams can inform neurophysiology instruction to address points of student confusion.NEW & NOTEWORTHY Investigating students' alternative reasoning in neurophysiology, this research is the first to investigate how analyzing the most common incorrect answer can shed light on the concepts students struggle with when reasoning about neurophysiological problems, especially those dealing with both chemical and electrical driving forces to predict ion movement across cell membranes.

Oaks to arteries: the Physiology Core Concept of flow down gradients supports transfer of student reasoning.

The Physiology Core Concept of flow down gradients is a major concept in physiology, as pressure gradients are the key driving force for the bulk flow of fluids in biology. However, students struggle to understand that this principle is foundational to the mechanisms governing bulk flow across diverse physiological systems (e.g., blood flow, phloem sap flow). Our objective was to investigate whether bulk flow items that differ in scenario context (i.e., taxa, amount of scientific terminology, living or nonliving system) or in which aspect of the pressure gradient is kept constant (i.e., starting pressure or pressure gradient) influence undergraduate students' reasoning. Item scenario context did not impact the type of reasoning students used. However, students were more likely to use the Physiology Core Concept of "flow down [pressure] gradients" when the pressure gradient was kept constant and less likely to use this concept when the starting pressure was kept constant. We also investigated whether item scenario context or which aspect of the pressure gradient is kept constant impacted how consistent students were in the type of reasoning they used across two bulk flow items on the same homework. Most students were consistent across item scenario contexts (76%) and aspects of the pressure gradient kept constant (70%). Students who reasoned using "flow down gradients" on the first item were the most consistent (86, 89%), whereas students using "pressures indicate (but don't cause) flow" were the least consistent (43, 34%). Students who are less consistent know that pressure is somehow involved or indicates fluid flow but do not have a firm grasp of the concept of a pressure gradient as the driving force for fluid flow. These findings are the first empirical evidence to support the claim that using Physiology Core Concept reasoning supports transfer of knowledge across different physiological systems.NEW & NOTEWORTHY These findings are the first empirical evidence to support the claim that using Physiology Core Concept reasoning supports transfer of knowledge across different physiological systems.

A new assessment to monitor student performance in introductory neurophysiology: Electrochemical Gradients Assessment Device.

The basis for understanding neurophysiology is understanding ion movement across cell membranes. Students in introductory courses recognize ion concentration gradients as a driving force for ion movement but struggle to simultaneously account for electrical charge gradients. We developed a 17-multiple-choice item assessment of students' understanding of electrochemical gradients and resistance in neurophysiology, the Electrochemical Gradients Assessment Device (EGAD). We investigated the internal evidence validity of the assessment by analyzing item characteristic curves of score probability and student ability for each question, and a Wright map of student scores and ability. We used linear mixed-effect regression to test student performance and ability. Our assessment discriminated students with average ability (weighted likelihood estimate: -2 to 1.5 Θ); however, it was not as effective at discriminating students at the highest ability (weighted likelihood estimate: >2 Θ). We determined the assessment could capture changes in both assessment scores (model r2 = 0.51, P < 0.001, n = 444) and ability estimates (model r2 = 0.47, P < 0.001, n = 444) after a simulation-based laboratory and course instruction for 222 students. Differential item function analysis determined that each item on the assessment performed equitably for all students, regardless of gender, race/ethnicity, or economic status. Overall, we found that men scored higher (r2 = 0.51, P = 0.014, n = 444) and had higher ability scores (P = 0.003) on the EGAD assessment. Caucasian students of both genders were positively correlated with score (r2 = 0.51, P < 0.001, n = 444) and ability (r2 = 0.47, P < 0.001, n = 444). Based on the evidence gathered through our analyses, the scores obtained from the EGAD can distinguish between levels of content knowledge on neurophysiology principles for students in introductory physiology courses.

Active learning increases student performance in science, engineering, and mathematics.

To test the hypothesis that lecturing maximizes learning and course performance, we metaanalyzed 225 studies that reported data on examination scores or failure rates when comparing student performance in undergraduate science, technology, engineering, and mathematics (STEM) courses under traditional lecturing versus active learning. The effect sizes indicate that on average, student performance on examinations and concept inventories increased by 0.47 SDs under active learning (n = 158 studies), and that the odds ratio for failing was 1.95 under traditional lecturing (n = 67 studies). These results indicate that average examination scores improved by about 6% in active learning sections, and that students in classes with traditional lecturing were 1.5 times more likely to fail than were students in classes with active learning. Heterogeneity analyses indicated that both results hold across the STEM disciplines, that active learning increases scores on concept inventories more than on course examinations, and that active learning appears effective across all class sizes--although the greatest effects are in small (n ≤ 50) classes. Trim and fill analyses and fail-safe n calculations suggest that the results are not due to publication bias. The results also appear robust to variation in the methodological rigor of the included studies, based on the quality of controls over student quality and instructor identity. This is the largest and most comprehensive metaanalysis of undergraduate STEM education published to date. The results raise questions about the continued use of traditional lecturing as a control in research studies, and support active learning as the preferred, empirically validated teaching practice in regular classrooms.

Validating a conceptual framework for the core concept of "cell-cell communication".

We have created and validated a conceptual framework for the core physiology concept of "cell-cell communication." The conceptual framework is composed of 51 items arranged in a hierarchy that is, in some instances, four levels deep. We have validated it with input from faculty who teach at a wide variety of institutional types. All items making up the framework were deemed essential to moderately important. However, some of the main ideas were clearly judged to be more important than others. Furthermore, the lower in the hierarchy an item is, the less important it is thought to be. Finally, there was no significant difference in the ratings given by faculty at different types of institutions.

A conceptual framework for homeostasis: development and validation.

We have developed and validated a conceptual framework for understanding and teaching organismal homeostasis at the undergraduate level. The resulting homeostasis conceptual framework details critical components and constituent ideas underlying the concept of homeostasis. It has been validated by a broad range of physiology faculty members from community colleges, primarily undergraduate institutions, research universities, and medical schools. In online surveys, faculty members confirmed the relevance of each item in the framework for undergraduate physiology and rated the importance and difficulty of each. The homeostasis conceptual framework was constructed as a guide for teaching and learning of this critical core concept in physiology, and it also paves the way for the development of a concept inventory for homeostasis.

A physiologist's view of homeostasis.

Homeostasis is a core concept necessary for understanding the many regulatory mechanisms in physiology. Claude Bernard originally proposed the concept of the constancy of the "milieu interieur," but his discussion was rather abstract. Walter Cannon introduced the term "homeostasis" and expanded Bernard's notion of "constancy" of the internal environment in an explicit and concrete way. In the 1960s, homeostatic regulatory mechanisms in physiology began to be described as discrete processes following the application of engineering control system analysis to physiological systems. Unfortunately, many undergraduate texts continue to highlight abstract aspects of the concept rather than emphasizing a general model that can be specifically and comprehensively applied to all homeostatic mechanisms. As a result, students and instructors alike often fail to develop a clear, concise model with which to think about such systems. In this article, we present a standard model for homeostatic mechanisms to be used at the undergraduate level. We discuss common sources of confusion ("sticky points") that arise from inconsistencies in vocabulary and illustrations found in popular undergraduate texts. Finally, we propose a simplified model and vocabulary set for helping undergraduate students build effective mental models of homeostatic regulation in physiological systems.

What a Difference in Pressure Makes! A Framework Describing Undergraduate Students' Reasoning about Bulk Flow Down Pressure Gradients.

Pressure gradients serve as the key driving force for the bulk flow of fluids in biology (e.g., blood, air, phloem sap). However, students often struggle to understand the mechanism that causes these fluids to flow. To investigate student reasoning about bulk flow, we collected students' written responses to assessment items and interviewed students about their bulk flow ideas. From these data, we constructed a bulk flow pressure gradient reasoning framework that describes the different patterns in reasoning that students express about what causes fluids to flow and ordered those patterns into sequential levels from more informal ways of reasoning to more scientific, mechanistic ways of reasoning. We obtained validity evidence for this bulk flow pressure gradient reasoning framework by collecting and analyzing written responses from a national sample of undergraduate biology and allied health majors from 11 courses at five institutions. Instructors can use the bulk flow pressure gradient reasoning framework and assessment items to inform their instruction of this topic and formatively assess their students' progress toward more scientific, mechanistic ways of reasoning about this important physiological concept.

Evidence-based teaching practices correlate with increased exam performance in biology.

Evidence-based teaching practices are associated with improved student academic performance. However, these practices encompass a wide range of activities and determining which type, intensity or duration of activity is effective at improving student exam performance has been elusive. To address this shortcoming, we used a previously validated classroom observation tool, Practical Observation Rubric to Assess Active Learning (PORTAAL) to measure the presence, intensity, and duration of evidence-based teaching practices in a retrospective study of upper and lower division biology courses. We determined the cognitive challenge of exams by categorizing all exam questions obtained from the courses using Bloom's Taxonomy of Cognitive Domains. We used structural equation modeling to correlate the PORTAAL practices with exam performance while controlling for cognitive challenge of exams, students' GPA at start of the term, and students' demographic factors. Small group activities, randomly calling on students or groups to answer questions, explaining alternative answers, and total time students were thinking, working with others or answering questions had positive correlations with exam performance. On exams at higher Bloom's levels, students explaining the reasoning underlying their answers, students working alone, and receiving positive feedback from the instructor also correlated with increased exam performance. Our study is the first to demonstrate a correlation between the intensity or duration of evidence-based PORTAAL practices and student exam performance while controlling for Bloom's level of exams, as well as looking more specifically at which practices correlate with performance on exams at low and high Bloom's levels. This level of detail will provide valuable insights for faculty as they prioritize changes to their teaching. As we found that multiple PORTAAL practices had a positive association with exam performance, it may be encouraging for instructors to realize that there are many ways to benefit students' learning by incorporating these evidence-based teaching practices.

Design-Based Research: A Methodology to Extend and Enrich Biology Education Research.

Recent calls in biology education research (BER) have recommended that researchers leverage learning theories and methodologies from other disciplines to investigate the mechanisms by which students to develop sophisticated ideas. We suggest design-based research from the learning sciences is a compelling methodology for achieving this aim. Design-based research investigates the "learning ecologies" that move student thinking toward mastery. These "learning ecologies" are grounded in theories of learning, produce measurable changes in student learning, generate design principles that guide the development of instructional tools, and are enacted using extended, iterative teaching experiments. In this essay, we introduce readers to the key elements of design-based research, using our own research into student learning in undergraduate physiology as an example of design-based research in BER. Then, we discuss how design-based research can extend work already done in BER and foster interdisciplinary collaborations among cognitive and learning scientists, biology education researchers, and instructors. We also explore some of the challenges associated with this methodological approach.

Learning Progressions: An Empirically Grounded, Learner-Centered Framework to Guide Biology Instruction.

Vision and Change challenged biology instructors to develop evidence-based instructional approaches that were grounded in the core concepts and competencies of biology. This call for reform provides an opportunity for new educational tools to be incorporated into biology education. In this essay, we advocate for learning progressions as one such educational tool. First, we address what learning progressions are and how they leverage research from the cognitive and learning sciences to inform instructional practices. Next, we use a published learning progression about carbon cycling to illustrate how learning progressions describe the maturation of student thinking about a key topic. Then, we discuss how learning progressions can inform undergraduate biology instruction, citing three particular learning progressions that could guide instruction about a number of key topics taught in introductory biology courses. Finally, we describe some challenges associated with learning progressions in undergraduate biology and some recommendations for how to address these challenges.

Is formative assessment an effective way to improve learning? A Symposium at Experimental Biology 2008.

Formative assessment is designed to provide information about students' learning to help them and their teachers to identify deficiencies and misconceptions. It differs from summative assessment, which aims to rank students according to their achievements to determine which students pass or fail or to assign grades to students. This article reports on a symposium concerned with evidence for the effectiveness of formative assessment in improving learning. It was presented by the Teaching of Physiology Section of the American Physiological Society at the Experimental Biology Meeting of 2008.

Peer vs. Self-Grading of Practice Exams: Which Is Better?

Practice exams are a type of deliberate practice that have been shown to improve student course performance. Deliberate practice differs from other types of practice, because it is targeted, mentally challenging, can be repeated, and requires feedback. Providing frequent instructor feedback to students, particularly in large classes, can be prohibitive. A possible solution is to have students grade practice exams using an instructor-generated rubric, receiving points only for completion. Students can either grade their own or a peer's work. We investigated whether peer or self-grading had a differential impact on completion of practice exam assignments, performance on practice exams or course exams, or student grading accuracy. We also investigated whether student characteristics mattered. We found that 90% of students took all practice exams or only missed one and that there was no difference on practice or course exam performance between the peer and self-graders. However, in the peer-grading treatment, students with lower incoming grade point averages and students identified as economically or educationally disadvantaged were less accurate and more lenient graders than other students. As there is no clear benefit of peer grading over self-grading, we suggest that either format can solve the challenge instructors face in giving frequent personalized feedback to many students.

Development and Validation of the Homeostasis Concept Inventory.

We present the Homeostasis Concept Inventory (HCI), a 20-item multiple-choice instrument that assesses how well undergraduates understand this critical physiological concept. We used an iterative process to develop a set of questions based on elements in the Homeostasis Concept Framework. This process involved faculty experts and undergraduate students from associate's colleges, primarily undergraduate institutions, regional and research-intensive universities, and professional schools. Statistical results provided strong evidence for the validity and reliability of the HCI. We found that graduate students performed better than undergraduates, biology majors performed better than nonmajors, and students performed better after receiving instruction about homeostasis. We used differential item analysis to assess whether students from different genders, races/ethnicities, and English language status performed differently on individual items of the HCI. We found no evidence of differential item functioning, suggesting that the items do not incorporate cultural or gender biases that would impact students' performance on the test. Instructors can use the HCI to guide their teaching and student learning of homeostasis, a core concept of physiology.

Cognitive Difficulty and Format of Exams Predicts Gender and Socioeconomic Gaps in Exam Performance of Students in Introductory Biology Courses.

Recent reform efforts in undergraduate biology have recommended transforming course exams to test at more cognitively challenging levels, which may mean including more cognitively challenging and more constructed-response questions on assessments. However, changing the characteristics of exams could result in bias against historically underserved groups. In this study, we examined whether and to what extent the characteristics of instructor-generated tests impact the exam performance of male and female and middle/high- and low-socioeconomic status (SES) students enrolled in introductory biology courses. We collected exam scores for 4810 students from 87 unique exams taken across 3 yr of the introductory biology series at a large research university. We determined the median Bloom's level and the percentage of constructed-response questions for each exam. Despite controlling for prior academic ability in our models, we found that males and middle/high-SES students were disproportionately favored as the Bloom's level of exams increased. Additionally, middle/high-SES students were favored as the proportion of constructed-response questions on exams increased. Given that we controlled for prior academic ability, our findings do not likely reflect differences in academic ability level. We discuss possible explanations for our findings and how they might impact how we assess our students.

PORTAAL: A Classroom Observation Tool Assessing Evidence-Based Teaching Practices for Active Learning in Large Science, Technology, Engineering, and Mathematics Classes.

There is extensive evidence that active learning works better than a completely passive lecture. Despite this evidence, adoption of these evidence-based teaching practices remains low. In this paper, we offer one tool to help faculty members implement active learning. This tool identifies 21 readily implemented elements that have been shown to increase student outcomes related to achievement, logic development, or other relevant learning goals with college-age students. Thus, this tool both clarifies the research-supported elements of best practices for instructor implementation of active learning in the classroom setting and measures instructors' alignment with these practices. We describe how we reviewed the discipline-based education research literature to identify best practices in active learning for adult learners in the classroom and used these results to develop an observation tool (Practical Observation Rubric To Assess Active Learning, or PORTAAL) that documents the extent to which instructors incorporate these practices into their classrooms. We then use PORTAAL to explore the classroom practices of 25 introductory biology instructors who employ some form of active learning. Overall, PORTAAL documents how well aligned classrooms are with research-supported best practices for active learning and provides specific feedback and guidance to instructors to allow them to identify what they do well and what could be improved.

Caution, Student Experience May Vary: Social Identities Impact a Student's Experience in Peer Discussions.

In response to calls for implementing active learning in college-level science, technology, engineering, and mathematics courses, classrooms across the country are being transformed from instructor centered to student centered. In these active-learning classrooms, the dynamics among students becomes increasingly important for understanding student experiences. In this study, we focus on the role a student prefers to assume during peer discussions, and how this preferred role may vary given a student's social identities. In addition we explore whether three hypothesized barriers to participation may help explain participation difference in the classroom. These barriers are 1) students are excluded from the discussion by actions of their groupmates; 2) students are anxious about participating in peer discussion; and 3) students do not see value in peer discussions. Our results indicate that self-reported preferred roles in peer discussions can be predicted by student gender, race/ethnicity, and nationality. In addition, we found evidence for all three barriers, although some barriers were more salient for certain students than others. We encourage instructors to consider structuring their in-class activities in ways that promote equity, which may require more purposeful attention to alleviating the current differential student experiences with peer discussions.

Gender gaps in achievement and participation in multiple introductory biology classrooms.

Although gender gaps have been a major concern in male-dominated science, technology, engineering, and mathematics disciplines such as physics and engineering, the numerical dominance of female students in biology has supported the assumption that gender disparities do not exist at the undergraduate level in life sciences. Using data from 23 large introductory biology classes for majors, we examine two measures of gender disparity in biology: academic achievement and participation in whole-class discussions. We found that females consistently underperform on exams compared with males with similar overall college grade point averages. In addition, although females on average represent 60% of the students in these courses, their voices make up less than 40% of those heard responding to instructor-posed questions to the class, one of the most common ways of engaging students in large lectures. Based on these data, we propose that, despite numerical dominance of females, gender disparities remain an issue in introductory biology classrooms. For student retention and achievement in biology to be truly merit based, we need to develop strategies to equalize the opportunities for students of different genders to practice the skills they need to excel.

BioCore Guide: A Tool for Interpreting the Core Concepts of Vision and Change for Biology Majors.

Vision and Change in Undergraduate Biology Education outlined five core concepts intended to guide undergraduate biology education: 1) evolution; 2) structure and function; 3) information flow, exchange, and storage; 4) pathways and transformations of energy and matter; and 5) systems. We have taken these general recommendations and created a Vision and Change BioCore Guide-a set of general principles and specific statements that expand upon the core concepts, creating a framework that biology departments can use to align with the goals of Vision and Change. We used a grassroots approach to generate the BioCore Guide, beginning with faculty ideas as the basis for an iterative process that incorporated feedback from more than 240 biologists and biology educators at a diverse range of academic institutions throughout the United States. The final validation step in this process demonstrated strong national consensus, with more than 90% of respondents agreeing with the importance and scientific accuracy of the statements. It is our hope that the BioCore Guide will serve as an agent of change for biology departments as we move toward transforming undergraduate biology education.