Bringing Back the Core Concepts of Physiology in an Advanced Medical School Senior Elective.

Integrating physiology core concepts into the clinical years of medical education has been challenging despite efforts. This paper describes a fourth-year medical school elective, Advanced Physiology in Critical Care Medicine, that focused on integrating physiology core concepts in a case-based learning approach. The elective used interdisciplinary faculty in a virtual forum. Senior students were asked to generate mechanism of disease (MOD) maps, highlight the physiology core concepts associated with paper cases of critically ill patients, and discuss with faculty the relevance of the underlying basic science. The weekly footprint consisted of a student led session presenting MOD maps for three cases which examined aspects of core physiology concepts, and later in the same week, student presentation of order sets for the management of the cases. Students ended the 4-week elective with a mini-grand rounds presentation on a topic of their choice incorporating the core concept paradigm. Student perception data and faculty reflections of the elective course are included. Student data and faculty observations suggest they appreciate and can apply physiological core concepts to patient care. Faculty experience in the course suggests that this senior elective helped them better approach all pre-clinical teaching with the Core Concepts framework in mind.

Relationships between medical student wellness, self-efficacy, and academic performance during the "post-COVID" period.

This study was a part of a longitudinal study investigating the relationships between medical student wellness, self-efficacy, and performance. Eighty-two eligible students were asked to complete online surveys during their second (M2) and third (M3) years. Performance outcomes included scores on various summative assessments during the M.D. program. Wellness survey results indicated that the sample of 38 M2 and 28 M3 students were overall well and self-efficacious, and they broadly maintained similar wellness characteristics across their medical education despite COVID-19 disruptions. Twenty-three students completed both surveys, and a paired analysis for this subgroup showed modest increases in stress and burnout in the M3 year. Notable correlations were observed between self-efficacy for academic work and a whole range of wellness variables for M2 students. M2 academic performance was modestly correlated to self-efficacy (rs = 0.38, P = 0.02, n = 38) and student burnout (rs = -0.34, P = 0.04, n = 38). In contrast, for the M3 students there was little correlation between wellness, clinical self-efficacy, and clinical performance, with the only significant relationships observed to be between overall clinical self-efficacy and the strength of social networks (rs = 0.41, P = 0.03, n = 28) and between scores for postencounter notes during Objective Structure Clinical Examinations (OSCEs) and self-efficacy in evidence-based medicine (rs = 0.44, P = 0.02, n = 28). In conclusion, 1) students remained generally well throughout the post-COVID period, and 2) self-efficacy for academic work is a good predictor of student wellness and performance during the preclerkship period but not during clinical training.NEW & NOTEWORTHY This study followed a group of medical students through the "post-COVID" period to assess their wellness as they transitioned from basic sciences to clinical training. We found that their wellness and belief in their ability to succeed (self-efficacy) remained strong, showing their resiliency. We observed correlations between self-efficacy and their level of wellness and academic performance during basic science classes but not during clinical training.

Comparing Machine Learning Models and Human Raters When Ranking Medical Student Performance Evaluations.

Background: The Medical Student Performance Evaluation (MSPE), a narrative summary of each student's academic and professional performance in US medical school is long, making it challenging for residency programs evaluating large numbers of applicants. Objective: To create a rubric to assess MSPE narratives and to compare the ability of 3 commercially available machine learning models (MLMs) to rank MSPEs in order of positivity. Methods: Thirty out of a possible 120 MSPEs from the University of Central Florida class of 2020 were de-identified and subjected to manual scoring and ranking by a pair of faculty members using a new rubric based on the Accreditation Council for Graduate Medical Education competencies, and to global sentiment analysis by the MLMs. Correlation analysis was used to assess reliability and agreement between student rank orders produced by faculty and MLMs. Results: The intraclass correlation coefficient used to assess faculty interrater reliability was 0.864 (P<.001; 95% CI 0.715-0.935) for total rubric scores and ranged from 0.402 to 0.768 for isolated subscales; faculty rank orders were also highly correlated (rs=0.758; P<.001; 95% CI 0.539-0.881). The authors report good feasibility as the rubric was easy to use and added minimal time to reading MSPEs. The MLMs correctly reported a positive sentiment for all 30 MSPE narratives, but their rank orders produced no significant correlations between different MLMs, or when compared with faculty rankings. Conclusions: The rubric for manual grading provided reliable overall scoring and ranking of MSPEs. The MLMs accurately detected positive sentiment in the MSPEs but were unable to provide reliable rank ordering.

A metacognitive confidence calibration (MCC) tool to help medical students scaffold diagnostic reasoning in decision-making during high-fidelity patient simulations.

The purpose of this study was to 1) help novice students scaffold problem-solving and engage safely in the deliberate practice of diagnostic reasoning and medical decision-making in real time; 2) assess how accurately students gather and apply data in medical reasoning and treatment during high-fidelity patient simulations (HFPSs); 3) identify students' scientific misconceptions related to the case; 4) promote student metacognitive processing, self-assessment, and self-efficacy; and 5) facilitate the explicit calibration of student confidence in deliberate reasoning with patient outcomes. In a mixed-method design, a metacognitive calibration self-assessing (MCC) survey tool was applied to HFPS (n = 80, 20 teams of 6 medical students) and semistructured interviews were conducted with faculty (n = 5). When scored by faculty with a rubric, the mean student accuracy ranged from 23% to 74%, whereas their self-assessment of confidence ranged from 71% to 86%. This result revealed overconfidence bias in novice students regarding the correctness of their wrong responses. The most common misconception identified was inverting cause and effect: metabolic acidosis was pointed to as the cause of the patient's problems rather than a consequence of untreated diabetes mellitus. The most common treatment error was overtreatment, with unnecessary added medication. Interviews with faculty suggested that the MCC tool improved the team process by slowing students down, requiring them to think through their answers, and that overall the tool improved their critical thinking. This study demonstrated the feasibility of using a metacognitive confidence calibration tool to assist novice students in learning safely to make deliberate diagnostic reasoning and decisions on patient care in real time during complex simulations while observing objectively their levels of psychological confidence against patient outcomes.NEW & NOTEWORTHY This study demonstrates the feasibility of a metacognitive confidence calibration tool (MCC) to assess and promote novices in the learning of diagnostic reasoning and treatment decisions on patient care in real time during high-fidelity patient simulations while comparing confidence and accuracy data and identifying students' scientific misconceptions. Results revealed the presence of overconfidence bias, overtreatment, and the misconception of metabolic acidosis as the cause of the patient's problems rather than a consequence of untreated diabetes mellitus.

Understanding reasons for electing gap years between undergraduate education and medical school and the impact of gap years on the student experience of medical education.

Gap years have grown in popularity among students attempting to go to medical school in recent years. According to the Association of American Medical Colleges (AAMC), more than half of matriculated students now have taken time off between their undergraduate studies and medical school. Although many claim benefits to taking gap years, research on the impacts of these experiences on medical students is largely nonexistent. This study aimed to qualitatively analyze the gap year phenomenon and its impacts on medical students through semistructured interviews conducted at the University of Central Florida College of Medicine. Through these interviews, the research team collected in-depth responses that were thematically analyzed. Analysis revealed the following major themes regarding the impacts of gap years: 1) adaptability to change and failure, 2) professional identity formation and understanding of team role, 3) understanding the "real world," 4) refocusing of goals, 5) resilience and stress management, and 6) reinforcement of motivation.

Effects of voluntary attendance patterns on first-year medical students' wellness and academic performance during COVID-19.

There is increasing evidence that attendance is not a reliable predictor of academic performance, which invites the question of whether attendance may have alternative positive effects such as increased student wellness. While conducting an observational cohort study on the relationship of attendance to performance and wellness, the COVID-19 pandemic lockdown changed the ability of students to attend class. Matriculating medical students were recruited to participate at the beginning of their first year in August 2019. In-person attendance was collected prospectively until the lockdown began in March 2020. This was then correlated with performance on course final examinations and responses to two separate incidences of a survey on stress, burnout, social isolation, and loneliness. The first survey was deployed January through February 2020 (pre-COVID) and the second June 2020 (during COVID lockdown). Attendance declined across the year (76.0%-25.0%). There was a small positive correlation of attendance to performance in module 1 (r = 0.235, P = 0.035) with none in the remaining two modules (P = 0.870, P = 0.410). The high attenders at the time of the lockdown had no decline in their performance when attendance was no longer possible. Attendance negatively correlated with stress and loneliness but not with the remainder of the wellness metrics. There was no significant change in any of the wellness metrics between the pre-COVID and during COVID time points. While the data support the conclusion that attendance is not required to maintain performance in this population, these data suggest a small protective effect against stress and loneliness.

Using the physiology of normal aging as a capstone integration exercise in a medical physiology course.

As the Baby Boomer generation reaches old age, there has been a significant increase in the number of older adults needing healthcare over the past decade. The physiology of aging is therefore a highly relevant topic for the preclinical medical curriculum. I describe a new capstone unit on the physiology of aging, placed at the end a medical physiology course, to provide a vehicle for integration of prior learning about physiology of each individual body system. Students were provided with online self-study modules as preparation for a mandatory small group case-based learning activity. A detailed case of an elderly female patient being assessed for fall risk was provided. Students were required to document a "Review of Systems" predicting decreased system functions due to senescence and to prepare a group concept map illustrating how physiologic deficits contributed to fall risk in the patient. Students successfully completed the activity and reported generally good satisfaction with the experience. The activity was judged an effective tool for students to consolidate prior learning and to apply physiology to an important medical topic. The lesson also provided several opportunities for curriculum integration with cell biology, biochemistry, anatomy, and clinical skills components.

Patterns of medical student engagement in a second-year pathophysiology course: relationship to USMLE Step 1 performance.

Historically, attendance has been a marker of academic performance, but the current medical education literature has had mixed results. In addition, attendance is dropping in the preclinical curricula, whereas, at the same time, the focus on United States Medical Licensing Examination Step 1 performance is increasing. This present study is a mixed-method approach correlating student attendance and access to the formal curriculum in a second-year pathophysiology course to performance on Step 1. Additionally, survey and focus group data evaluated the usage and importance of both the formal curriculum and third-party resources. Out of 112 eligible students, 77 participated in the study. There was no correlation between attendance or access to the learning materials and Step 1 performance. There was a strong correlation between the performance on the final examination and that of Step 1 (r = 0.813; P < 0.001) and a moderate correlation between formative quiz (r = 0.321; P = 0.005) and individual readiness assessment test performance (r = 0.351; P = 0.002) and Step 1 performance. Survey and focus group data show that students place high importance on faculty-developed materials that they can use on their own, but not attendance. The third-party resources are highly used as an adjunct to the formal curriculum and to focus on Step 1 study. Attendance and access to the formal curriculum do not predict Step 1 performance, whereas performance on high- and low-stakes internal assessments do. Further study on how the lack of social interaction gained from attendance affects development of other competencies and the learning climate are warranted.

Relationship between classroom attendance and examination performance in a second-year medical pathophysiology class.

Studies completed with undergraduate populations have shown that attendance positively correlates with academic performance. A marked decline in classroom attendance within medical school has recently been noticed with the availability of video capture of lectures and other online material. This study compares these in the era of online material. It took place during the second-year Gastrointestinal and Renal Systems module. Attendance was mandatory at team-based learning and case-based learning sessions on new material and voluntary at lectures (29 sessions) and case-based learning on material previously covered (9 sessions). Attendance was recorded prospectively. All lectures were recorded, and all related files were available to students online. Performance was based on a 118 multiple-choice question final examination. Students voluntarily completed the Motivated Strategies for Learning Questionnaire (MSLQ). The study group consisted of 78 students (68% of 114 total) of whom 48 completed the MSLQ. Mean attendance was 24%, with 33% of students attending none of the nonmandatory sessions. The median score on the final exam for participants was 86.0 (range: 28.8). High levels of self-efficacy and the ability to self-regulate effort were predictive of low attendance. Attendance was positively predicted by an orientation toward peer learning and help seeking. There was no correlation between the percentage of classes attended and performance on the final exam. We conclude that different facets of self-regulated learning predict attendance, with highly confident students being the least likely to attend, and that attendance at in-class sessions is no longer a good marker for performance.

Developing a tool for observing group critical thinking skills in first-year medical students: a pilot study using physiology-based, high-fidelity patient simulations.

Most assessments of physiology in medical school use multiple choice tests that may not provide information about a student's critical thinking (CT) process. There are limited performance assessments, but high-fidelity patient simulations (HFPS) may be a feasible platform. The purpose of this pilot study was to determine whether a group's CT process could be observed over a series of HFPS. An instrument [Critical Thinking Skills Rating Instrument CTSRI)] was designed with the IDEAS framework. Fifteen groups of students participated in three HFPS that consisted of a basic knowledge quiz and introduction, HFPS session, and debriefing. HFPS were video recorded, and two raters reviewed and scored all HFPS encounters with the CTSRI independently. Interrater analysis suggested good reliability. There was a correlation between basic knowledge scores and three of the six observations on the CTSRI providing support for construct validity. The median CT ratings significantly increased for all observations between the groups' first and last simulation. However, there were still large percentages of video ratings that indicated students needed substantial prompting during the HFPS. The data from this pilot study suggest that it is feasible to observe CT skills in HFPS using the CTSRI. Based on the findings from this study, we strongly recommend that first-year medical students be competent in basic knowledge of the relevant physiology of the HFPS before participating, to minimize the risk of a poor learning experience.

Understanding factors affecting participation in online formative quizzes: an interview study.

A positive correlation between performance and participation in formative quizzes and final summative examinations has been reported many times. The goal of the present interview study was to construct a model to explain why students may elect not to engage with formative assessment opportunities. Our medical school's preclinical curriculum has an established policy of offering weekly online quizzes in all courses during the first 2 yr. Quizzes do not count for credit. Semistructured interviews were recorded and transcribed verbatim, and a formal thematic analysis was applied. A total of 16 in-depth interviews were conducted, with 8 female and 8 male participants. Thematic analysis revealed four major interacting themes that we propose to converge to account for nonparticipation in quizzes: 1) inadequate feedback, 2) curriculum organization and student mistrust, 3) time constraints, and 4) fear of judgment. We propose seven practice points to improve the effectiveness of formative assessment quizzes of medical knowledge.

Current Practices in Assessing Professionalism in United States and Canadian Allopathic Medical Students and Residents.

Professionalism is a critically important competency that must be evaluated in medical trainees but is a complex construct that is hard to assess. A systematic review was undertaken to give insight into the current best practices for assessment of professionalism in medical trainees and to identify new research priorities in the field. A search was conducted on PubMed for behavioral assessments of medical students and residents among the United States and Canadian allopathic schools in the last 15 years. An initial search yielded 594 results, 28 of which met our inclusion criteria. Our analysis indicated that there are robust generic definitions of the major attributes of medical professionalism. The most commonly used assessment tools are survey instruments that use Likert scales tied to attributes of professionalism. While significant progress has been made in this field in recent years, several opportunities for system-wide improvement were identified that require further research. These include a paucity of information about assessment reliability, the need for rater training, a need to better define competency in professionalism according to learner level (preclinical, clerkship, resident etc.) and ways to remediate lapses in professionalism. Student acceptance of assessment of professionalism may be increased if assessment tools are shifted to better incorporate feedback. Tackling the impact of the hidden curriculum in which students may observe lapses in professionalism by faculty and other health care providers is another priority for further study.

Best practices in summative assessment.

The goal of this review is to highlight key elements underpinning excellent high-stakes summative assessment. This guide is primarily aimed at faculty members with the responsibility of assigning student grades and is intended to be a practical tool to help throughout the process of planning, developing, and deploying tests as well as monitoring their effectiveness. After a brief overview of the criteria for high-quality assessment, the guide runs through best practices for aligning assessment with learning outcomes and compares common testing modalities. Next, the guide discusses the kind of validity evidence needed to support defensible grading of student performance. This review concentrates on how to measure the outcome of student learning; other reviews in this series will expand on the related concepts of formative testing and how to leverage testing for learning.

Team-based learning in large enrollment classes.

The goal of this review is to highlight the key elements needed to successfully deploy team-based learning (TBL) in any class, but especially in large enrolment classes, where smooth logistics are essential. The text is based on a lecture and workshop given at the American Physiological Society's Institute on Teaching and Learning in Madison, WI, in June 2016. After a short overview of the TBL method, its underpinning in learning theory, and a summary of current evidence for its effectiveness, we present two case studies from our own teaching practices in a new medical school. The first case study explores critical elements of design and planning for a TBL module, and the second explores best practices in classroom management. As medical educators in the fields of physiology, pediatrics, nephrology, and family medicine, we present the objective views of subject matter experts who adopted TBL as one teaching method rather than TBL experts or advocates per se. The review is aimed primarily at faculty contemplating using TBL for the first time who are interested in exploring the significant benefits and challenges of TBL.

Application of a utility analysis to evaluate a novel assessment tool for clinically oriented physiology and pharmacology.

Multiple-choice questions are a gold-standard tool in medical school for assessment of knowledge and are the mainstay of licensing examinations. However, multiple-choice questions items can be criticized for lacking the ability to test higher-order learning or integrative thinking across multiple disciplines. Our objective was to develop a novel assessment that would address understanding of pathophysiology and pharmacology, evaluate learning at the levels of application, evaluation and synthesis, and allow students to demonstrate clinical reasoning. The rubric assesses student writeups of clinical case problems. The method is based on the physician's traditional postencounter Subjective, Objective, Assessment and Plan note. Students were required to correctly identify subjective and objective findings in authentic clinical case problems, to ascribe pathophysiological as well as pharmacological mechanisms to these findings, and to justify a list of differential diagnoses. A utility analysis was undertaken to evaluate the new assessment tool by appraising its reliability, validity, feasibility, cost effectiveness, acceptability, and educational impact using a mixed-method approach. The Subjective, Objective, Assessment and Plan assessment tool scored highly in terms of validity and educational impact and had acceptable levels of statistical reliability but was limited in terms of acceptance, feasibility, and cost effectiveness due to high time demands on expert graders and workload concerns from students. We conclude by making suggestions for improving the tool and recommend deployment of the instrument for low-stakes summative assessment or formative assessment.

Learning theories 101: application to everyday teaching and scholarship.

Shifts in educational research, in how scholarship in higher education is defined, and in how funding is appropriated suggest that educators within basic science fields can benefit from increased understanding of learning theory and how it applies to classroom practice. This article uses a mock curriculum design scenario as a framework for the introduction of five major learning theories. Foundational constructs and principles from each theory and how they apply to the proposed curriculum designs are described. A summative table that includes basic principles, constructs, and classroom applications as well as the role of the teacher and learner is also provided for each theory.

Use of the NBME Comprehensive Basic Science Examination as a progress test in the preclerkship curriculum of a new medical school.

In the present study, we describe the innovative use of the National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination (CBSE) as a progress test during the preclerkship medical curriculum. The main aim of this study was to provide external validation of internally developed multiple-choice assessments in a new medical school. The CBSE is a practice exam for the United States Medical Licensing Examination (USMLE) Step 1 and is purchased directly from the NBME. We administered the CBSE five times during the first 2 yr of medical school. Student scores were compared with scores on newly created internal summative exams and to the USMLE Step 1. Significant correlations were observed between almost all our internal exams and CBSE scores over time as well as with USMLE Step 1 scores. The strength of correlations of internal exams to the CBSE and USMLE Step 1 broadly increased over time during the curriculum. Student scores on courses that have strong emphasis on physiology and pathophysiology correlated particularly well with USMLE Step 1 scores. Student progress, as measured by the CBSE, was found to be linear across time, and test performance fell behind the anticipated level by the end of the formal curriculum. These findings are discussed with respect to student learning behaviors. In conclusion, the CBSE was found to have good utility as a progress test and provided external validation of our new internally developed multiple-choice assessments. The data also provide performance benchmarks both for our future students to formatively assess their own progress and for other medical schools to compare learning progression patterns in different curricular models.

Faculty reflections on the process of building an integrated preclerkship curriculum: a new school perspective.

This is a reflective essay based on the experience of developing a structure and function module within a new integrated medical curriculum. Our hope is that the insights we gained during a 4-yr journey in a new medical school will be transferable to others engaged with curriculum development. Here, we present an interpretive analysis of our personal experiences together with some original research data and a synthesis of the literature. We will argue that a focus on teaching faculty is the key to successful curriculum integration and suggest an agenda for faculty development. Our essay begins by exploring what curriculum integration really means and what its purpose might be. Our case study explores the challenges of building a shared understanding among stakeholders and of negotiating learning outcomes and methods of teaching as well as the process of developing content and assessment. We feel that many of our experiences in the new medical school are applicable in other settings, such as curriculum reform in established schools and for developers of competency-based premedical curricula. We conclude with recommendations to assist other curriculum planners and teachers by offering some benefits of hindsight.