The SkillsCenter: Creating scalable research opportunities for STEM students.

Undergraduate students generally need laboratory skills and experience to be accepted into a position within an academic lab or a company. However, those settings are traditionally where students would develop that necessary expertise. We developed a laboratory course paradigm to equip students with the skills they need to access future opportunities.

Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates.

We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

Relationships between Prediction Accuracy, Metacognitive Reflection, and Performance in Introductory Genetics Students.

Cognitive scientists have previously shown that students' perceptions of their learning and performance on assessments often do not match reality. This process of self-assessing performance is a component of metacognition, which also includes the practice of thinking about one's knowledge and identifying and implementing strategies to improve understanding. We used a mixed-methods approach to investigate the relationship between students' perceptions of their performance through grade predictions, their metacognitive reflections after receiving their grades, and their actual performance during a semester-long introductory genetics course. We found that, as a group, students do not display better predictive accuracy nor more metacognitive reflections over the semester. However, those who shift from overpredicting to matching or underpredicting also show improved performance. Higher performers are overall more likely to answer reflection questions than lower-performing peers. Although high-performing students are usually more metacognitive in their reflections, an increase in a student's frequency of metacognitive responses over time does not necessarily predict a grade increase. We illustrate several example trends in student reflections and suggest possible next steps for helping students implement better metacognitive regulation.

Successful Problem Solving in Genetics Varies Based on Question Content.

Problem solving is a critical skill in many disciplines but is often a challenge for students to learn. To examine the processes both students and experts undertake to solve constructed-response problems in genetics, we collected the written step-by-step procedures individuals used to solve problems in four different content areas. We developed a set of codes to describe each cognitive and metacognitive process and then used these codes to describe more than 1800 student and 149 expert answers. We found that students used some processes differently depending on the content of the question, but reasoning was consistently predictive of successful problem solving across all content areas. We also confirmed previous findings that the metacognitive processes of planning and checking were more common in expert answers than student answers. We provide suggestions for instructors on how to highlight key procedures based on each specific genetics content area that can help students learn the skill of problem solving.

Student Behaviors and Interactions Influence Group Discussions in an Introductory Biology Lab Setting.

Past research on group work has primarily focused on promoting change through implementation of interventions designed to increase performance. Recently, however, education researchers have called for more descriptive analyses of group interactions. Through detailed qualitative analysis of recorded discussions, we studied the natural interactions of students during group work in the context of a biology laboratory course. We analyzed multiple interactions of 30 different groups as well as data from each of the 91 individual participants to characterize the ways students engage in discussion and how group dynamics promote or prevent meaningful discussion. Using a set of codes describing 15 unique behaviors, we determined that the most common behavior seen in student dialogue was analyzing data, followed by recalling information and repeating ideas. We also classified students into one of 10 different roles for each discussion, determined by their most common behaviors. We found that, although students cooperated with one another by exchanging information, they less frequently fully collaborated to explain their conclusions through the exchange of reasoning. Within this context, these findings show that students working in groups generally choose specific roles during discussions and focus on data analysis rather than constructing logical reasoning chains to explain their conclusions.

Demystifying the Meaning of Active Learning in Postsecondary Biology Education.

Active learning is frequently used to describe teaching practices, but the term is not well-defined in the context of undergraduate biology education. To clarify this term, we explored how active learning is defined in the biology education literature (n = 148 articles) and community by surveying a national sample of biology education researchers and instructors (n = 105 individuals). Our objectives were to increase transparency and reproducibility of teaching practices and research findings in biology education. Findings showed the majority of the literature concerning active learning never defined the term, but the authors often provided examples of specific active-learning strategies. We categorized the available active-learning definitions and strategies obtained from the articles and survey responses to highlight central themes. Based on data from the BER literature and community, we provide a working definition of active learning and an Active-Learning Strategy Guide that defines 300+ active-learning strategies. These tools can help the community define, elaborate, and provide specificity when using the term active learning to characterize teaching practices.

Problem Solving in Genetics: Content Hints Can Help.

Problem solving is an integral part of doing science, yet it is challenging for students in many disciplines to learn. We explored student success in solving genetics problems in several genetics content areas using sets of three consecutive questions for each content area. To promote improvement, we provided students the choice to take a content-focused prompt, termed a "content hint," during either the second or third question within each content area. Overall, for students who answered the first question in a content area incorrectly, the content hints helped them solve additional content-matched problems. We also examined students' descriptions of their problem solving and found that students who improved following a hint typically used the hint content to accurately solve a problem. Students who did not improve upon receipt of the content hint demonstrated a variety of content-specific errors and omissions. Overall, ultimate success in the practice assignment (on the final question of each topic) predicted success on content-matched final exam questions, regardless of initial practice performance or initial genetics knowledge. Our findings suggest that some struggling students may have deficits in specific genetics content knowledge, which when addressed, allow the students to successfully solve challenging genetics problems.

Characterizing Students' Ideas about the Effects of a Mutation in a Noncoding Region of DNA.

Understanding student ideas in large-enrollment biology courses can be challenging, because easy-to-administer multiple-choice questions frequently do not fully capture the diversity of student ideas. As part of the Automated Analysis of Constructed Responses (AACR) project, we designed a question prompting students to describe the possible effects of a mutation in a noncoding region of DNA. We characterized answers from 1127 students enrolled in eight different large-enrollment introductory biology courses at three different institutions over five semesters and generated an analytic scoring system containing three categories of correct ideas and five categories of incorrect ideas. We iteratively developed a computer model for scoring student answers and tested the model before and after implementing an instructional activity designed to help a new set of students explore this concept. After completing a targeted activity and re-answering the question, students showed improvement from preassessment, with 64% of students in incorrect and 67% of students in partially incorrect (mixed) categories shifting to correct ideas only. This question, computer-scoring model, and instructional activity can now be reliably used by other instructors to better understand and characterize student ideas on the effects of mutations outside a gene-coding region.

GenBio-MAPS: A Programmatic Assessment to Measure Student Understanding of Vision and Change Core Concepts across General Biology Programs.

The Vision and Change report provides a nationally agreed upon framework of core concepts that undergraduate biology students should master by graduation. While identifying these concepts was an important first step, departments also need ways to measure the extent to which students understand these concepts. Here, we present the General Biology-Measuring Achievement and Progression in Science (GenBio-MAPS) assessment as a tool to measure student understanding of the core concepts at key time points in a biology degree program. Data from more than 5000 students at 20 institutions reveal that this instrument distinguishes students at different stages of the curriculum, with an upward trend of increased performance at later time points. Despite this trend, we identify several concepts that advanced students find challenging. Linear mixed-effects models reveal that gender, race/ethnicity, English-language status, and first-generation status predict overall performance and that different institutions show distinct performance profiles across time points. GenBio-MAPS represents the first programmatic assessment for general biology programs that spans the breadth of biology and aligns with the Vision and Change core concepts. This instrument provides a needed tool to help departments monitor student learning and guide curricular transformation centered on the teaching of core concepts.

Tools for Change: Measuring Student Conceptual Understanding Across Undergraduate Biology Programs Using Bio-MAPS Assessments.

Assessing learning across a biology major can help departments monitor achievement of broader program-level goals and identify opportunities for curricular improvement. However, biology departments have lacked suitable tools to measure learning at the program scale. To address this need, we developed four freely available assessments-called Biology-Measuring Achievement and Progression in Science or Bio-MAPS-for general biology, molecular biology, ecology/evolution, and physiology programs. When administered at multiple time points in a curriculum, these instruments can provide departments with information on how student conceptual understanding changes across a major and help guide curricular modifications to enhance learning.

Through the Eyes of Faculty: Using Personas as a Tool for Learner-Centered Professional Development.

College science instructors need continuous professional development (PD) to meet the call to evidence-based practice. New PD efforts need to focus on the nuanced blend of factors that influence instructors' teaching practices. We used persona methodology to describe the diversity among instructors who were participating in a long-term PD initiative. Persona methodology originates from ethnography. It takes data from product users and compiles those data in the form of fictional characters. Personas facilitate user-centered design. We identified four personas among our participants: Emma the Expert views herself as the subject-matter expert in the classroom and values her hard-earned excellence in lecturing. Ray the Relater relates to students and focuses on their points of view about innovative pedagogies. Carmen the Coach coaches her students by setting goals for them and helping them develop skill in scientific practices. Beth the Burdened owns the responsibility for her students' learning and feels overwhelmed that students still struggle despite her use of evidence-based practice. Each persona needs unique PD. We suggest ways that PD facilitators can use our personas as a reflection tool to determine how to approach the learners in their PD. We also suggest further avenues of research on learner-centered PD.

Peer Instruction.

Peer instruction, a form of active learning, is generally defined as an opportunity for peers to discuss ideas or to share answers to questions in an in-class environment, where they also have opportunities for further interactions with their instructor. When implementing peer instruction, instructors have many choices to make about group design, assignment format, and grading, among others. Ideally, these choices can be informed by research about the impact of these components of peer instruction on student learning. This essay describes an online, evidence-based teaching guide published by CBE-Life Sciences Education at http://lse.ascb.org/evidence-based-teaching-guides/peer-instruction . The guide provides condensed summaries of key research findings organized by teaching choices, summaries of and links to research articles and other resources, and actionable advice in the form of a checklist for instructors. In addition to describing key features of the guide, this essay also identifies areas in which further empirical studies are warranted.

Student, instructor, and observer agreement regarding frequencies of scientific teaching practices using the Measurement Instrument for Scientific Teaching-Observable (MISTO).

BACKGROUND: The Scientific Teaching (ST) pedagogical framework encompasses many of the best practices recommended in the literature and highlighted in national reports. Understanding the growth and impact of ST requires instruments to accurately measure the extent to which practitioners implement ST in their courses. Researchers have typically relied on students, instructors, or observers to document course teaching practices, but it remains unclear whether and how these perspectives differ from each other. To address this issue, we modified our previously published instrument to generate the Measurement Instrument for Scientific Teaching-Observable (MISTO), which can be completed by students, instructors, and observers, and we investigated the degree of similarity between these three perspectives across 70 undergraduate science courses at seven different institutions in the USA. RESULTS: We found that the full MISTO and Active Learning subcategory scores showed the highest correlations among the three perspectives, but the degree of correlation between perspectives varied for the other subcategories. Match scores between students and instructors were significantly higher than observer matches for the full MISTO and for the Active Learning, Inclusivity, and Responsiveness subcategories. CONCLUSIONS: We find that the level and type of agreement between perspectives varies across MISTO subcategories and that this variation likely stems from intrinsic differences in the course access and scoring decisions of the three perspectives. Building on this data, we recommend MISTO users consider their research goals, available resources, and potential artifacts that may arise when deciding which perspective best fits their needs in measuring classroom teaching practices.

A Faculty Professional Development Model That Improves Student Learning, Encourages Active-Learning Instructional Practices, and Works for Faculty at Multiple Institutions.

Helping faculty develop high-quality instruction that positively affects student learning can be complicated by time limitations, a lack of resources, and inexperience using student data to make iterative improvements. We describe a community of 16 faculty from five institutions who overcame these challenges and collaboratively designed, taught, iteratively revised, and published an instructional unit about the potential effect of mutations on DNA replication, transcription, and translation. The unit was taught to more than 2000 students in 18 courses, and student performance improved from preassessment to postassessment in every classroom. This increase occurred even though faculty varied in their instructional practices when they were teaching identical materials. We present information on how this faculty group was organized and facilitated, how members used student data to positively affect learning, and how they increased their use of active-learning instructional practices in the classroom as a result of participation. We also interviewed faculty to learn more about the most useful components of the process. We suggest that this professional development model can be used for geographically separated faculty who are interested in working together on a known conceptual difficulty to improve student learning and explore active-learning instructional practices.

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates.

A new assessment tool, Ecology and Evolution-Measuring Achievement and Progression in Science or EcoEvo-MAPS, measures student thinking in ecology and evolution during an undergraduate course of study. EcoEvo-MAPS targets foundational concepts in ecology and evolution and uses a novel approach that asks students to evaluate a series of predictions, conclusions, or interpretations as likely or unlikely to be true given a specific scenario. We collected evidence of validity and reliability for EcoEvo-MAPS through an iterative process of faculty review, student interviews, and analyses of assessment data from more than 3000 students at 34 associate's-, bachelor's-, master's-, and doctoral-granting institutions. The 63 likely/unlikely statements range in difficulty and target student understanding of key concepts aligned with the Vision and Change report. This assessment provides departments with a tool to measure student thinking at different time points in the curriculum and provides data that can be used to inform curricular and instructional modifications.

Measurement Instrument for Scientific Teaching (MIST): A Tool to Measure the Frequencies of Research-Based Teaching Practices in Undergraduate Science Courses.

The Scientific Teaching (ST) pedagogical framework provides various approaches for science instructors to teach in a way that more closely emulates how science is practiced by actively and inclusively engaging students in their own learning and by making instructional decisions based on student performance data. Fully understanding the impact of ST requires having mechanisms to quantify its implementation. While many useful instruments exist to document teaching practices, these instruments only partially align with the range of practices specified by ST, as described in a recently published taxonomy. Here, we describe the development, validation, and implementation of the Measurement Instrument for Scientific Teaching (MIST), a survey derived from the ST taxonomy and designed to gauge the frequencies of ST practices in undergraduate science courses. MIST showed acceptable validity and reliability based on results from 7767 students in 87 courses at nine institutions. We used factor analyses to identify eight subcategories of ST practices and used these categories to develop a short version of the instrument amenable to joint administration with other research instruments. We further discuss how MIST can be used by instructors, departments, researchers, and professional development programs to quantify and track changes in ST practices.

Using Pre-Assessment and In-Class Questions to Change Student Understanding of Molecular Movements.

Understanding how different types of molecules move through cell membranes is a fundamental part of cell biology. To identify and address student misconceptions surrounding molecular movement through cell membranes, we surveyed student understanding on this topic using pre-class questions, in-class clicker questions, and subsequent exam questions in a large introductory biology course. Common misconceptions identified in student responses to the pre-class assessment questions were used to generate distractors for clicker questions. Two-tier diagnostic clicker questions were used to probe incoming common student misconceptions (first tier) and their reasoning (second tier). Two subsequent lectures with assessment clicker questions were used to help students construct a new framework to understand molecular movement through cell membranes. Comparison of pre-assessment and post-assessment (exam) performance showed dramatic improvement in students' understanding of molecular movement: student answers to exam questions were 74.6% correct with correct reasoning while only 1.3% of the student answers were correct with correct reasoning on the pre-class assessment. Our results show that students' conceptual understanding of molecular movement through cell membranes progressively increases through discussions of a series of clicker questions and suggest that this clicker-based teaching strategy was highly effective in correcting common student misconceptions on this topic.

What Motivates Biology Instructors to Engage and Persist in Teaching Professional Development?

We conducted a study of 19 biology instructors participating in small, local groups at six research-intensive universities connected to the Automated Analysis of Constructed Response (AACR) project (www.msu.edu/∼aacr). Our aim was to uncover participants' motivation to persist in a long-term teaching professional development effort, a topic that is understudied in discipline-based educational research. We interviewed each participant twice over a 2-year period and conducted qualitative analyses on the data, using expectancy-value theory as a framework for considering motivation. Our analyses revealed that motivation among instructors was high due to their enjoyment of the AACR groups. The high level of motivation is further explained by the fact that AACR groups facilitated instructor involvement with the larger AACR project. We also found that group dynamics encouraged persistence; instructors thought they might never talk with colleagues about teaching in the absence of AACR groups; and groups were perceived to have a low-enough time requirement to warrant sustained involvement. We conclude that instructors have persisted in AACR groups because the groups provided great value with limited cost. The characterization of instructor experiences described here can contribute to a better understanding of faculty needs in teaching professional development.

Group Random Call Can Positively Affect Student In-Class Clicker Discussions.

Understanding how instructional techniques and classroom norms influence in-class student interactions has the potential to positively impact student learning. Many previous studies have shown that students benefit from discussing their ideas with one another in class. In this study of introductory biology students, we explored how using an in-class accountability system might affect the nature of clicker-question discussions. Clicker-question discussions in which student groups were asked to report their ideas voluntarily (volunteer call) were compared with discussions in which student groups were randomly selected to report their ideas (random call). We hypothesized that the higher-accountability condition (random call) would impress upon students the importance of their discussions and thus positively influence how they interacted. Our results suggest that a higher proportion of discussions in the random call condition contained exchanges of reasoning, some forms of questioning, and both on- and off-topic comments compared with discussion in the volunteer call condition. Although group random call does not impact student performance on clicker questions, the positive impact of this instructional approach on exchanges of reasoning and other features suggests it may encourage some types of student interactions that support learning.

Using Student Writing and Lexical Analysis to Reveal Student Thinking about the Role of Stop Codons in the Central Dogma.

Previous work has shown that students have persistent difficulties in understanding how central dogma processes can be affected by a stop codon mutation. To explore these difficulties, we modified two multiple-choice questions from the Genetics Concept Assessment into three open-ended questions that asked students to write about how a stop codon mutation potentially impacts replication, transcription, and translation. We then used computer-assisted lexical analysis combined with human scoring to categorize student responses. The lexical analysis models showed high agreement with human scoring, demonstrating that this approach can be successfully used to analyze large numbers of student written responses. The results of this analysis show that students' ideas about one process in the central dogma can affect their thinking about subsequent and previous processes, leading to mixed models of conceptual understanding.