A Multi-institutional Study of the Feasibility and Reliability of the Implementation of Constructed Response Exam Questions.

PROBLEM: Some medical schools have incorporated constructed response short answer questions (CR-SAQs) into their assessment toolkits. Although CR-SAQs carry benefits for medical students and educators, the faculty perception that the amount of time required to create and score CR-SAQs is not feasible and concerns about reliable scoring may impede the use of this assessment type in medical education. INTERVENTION: Three US medical schools collaborated to write and score CR-SAQs based on a single vignette. Study participants included faculty question writers (N = 5) and three groups of scorers: faculty content experts (N = 7), faculty non-content experts (N = 6), and fourth-year medical students (N = 7). Structured interviews were performed with question writers and an online survey was administered to scorers to gather information about their process for creating and scoring CR-SAQs. A content analysis was performed on the qualitative data using Bowen's model of feasibility as a framework. To examine inter-rater reliability between the content expert and other scorers, a random selection of fifty student responses from each site were scored by each site's faculty content experts, faculty non-content experts, and student scorers. A holistic rubric (6-point Likert scale) was used by two schools and an analytic rubric (3-4 point checklist) was used by one school. Cohen's weighted kappa (κw) was used to evaluate inter-rater reliability. CONTEXT: This research study was implemented at three US medical schools that are nationally dispersed and have been administering CR-SAQ summative exams as part of their programs of assessment for at least five years. The study exam question was included in an end-of-course summative exam during the first year of medical school. IMPACT: Five question writers (100%) participated in the interviews and twelve scorers (60% response rate) completed the survey. Qualitative comments revealed three aspects of feasibility: practicality (time, institutional culture, teamwork), implementation (steps in the question writing and scoring process), and adaptation (feedback, rubric adjustment, continuous quality improvement). The scorers' described their experience in terms of the need for outside resources, concern about lack of expertise, and value gained through scoring. Inter-rater reliability between the faculty content expert and student scorers was fair/moderate (κw=.34-.53, holistic rubrics) or substantial (κw=.67-.76, analytic rubric), but much lower between faculty content and non-content experts (κw=.18-.29, holistic rubrics; κw=.59-.66, analytic rubric). LESSONS LEARNED: Our findings show that from the faculty perspective it is feasible to include CR-SAQs in summative exams and we provide practical information for medical educators creating and scoring CR-SAQs. We also learned that CR-SAQs can be reliably scored by faculty without content expertise or senior medical students using an analytic rubric, or by senior medical students using a holistic rubric, which provides options to alleviate the faculty burden associated with grading CR-SAQs.

Supporting Self-Directed Learning: Development of a Faculty Evaluation Scale.

Construct: We sought to assess medical education faculty members' ability to support students in the development of self-regulated learning skills. Background: It is broadly agreed that medical students should become self-directed lifelong learners to succeed as physicians. To that end, many instruments have been developed that measure student attributes of self-directed learning (SDL). By contrast, no such analogous valid instrument is available to assess faculty familiarity in pedagogical strategies to promote SDL among students. Approach: An item bank with 45 items was created and its content-related validity evaluated by a panel of twenty experts. The items were selected and validated in the framework of Item Response Theory (IRT). The unidimensionality of all items within four constructs was assessed by using modified parallel analysis and 2 parameter IRT model calibration. The final version of the Recognition of Learning-Oriented Teaching Strategies (RoLOTS) instrument was validated by using the IRT marginal reliability as well as 2PL model calibrations. Different Item Functioning (DIF) in student-educators and gender was examined. Findings: The final version of RoLOTS included 20 items, with five in each of four subdomains: building content knowledge; the emotional and motivational aspects of learning; leveraging the social nature of the learning process; and metacognitive processes that promote student regulation of the learning process. Construct validity and reliability of each of the four domains were well supported by the results. Significant DIF was not detected in student-educator and gender. Conclusions: The RoLOTS successfully evaluated whether a faculty member is familiar with pedagogical tools to promote medical students' self-directed learning, which can be used as a first step in needs-based professional development. Further research is needed to provide more validity and reliability evidence among groups with diverse SDL experience.

Third year medical students impersonalize and hedge when providing negative upward feedback to clinical faculty.

Medical students provide clinical teaching faculty with feedback on their skills as educators through anonymous surveys at the end of their clerkship rotation. Because faculty are in a position of power, students are hesitant to provide candid feedback. Our objective was to determine if medical students were willing to provide negative upward feedback to clinical faculty and describe how they conveyed their feedback. A qualitative analysis of third year medical students' open-ended comments from evaluations of six clerkships was performed using politeness theory as a conceptual framework. Students were asked to describe how the clerkship enhanced their learning and how it could be improved. Midway through the academic year, instructions to provide full names of faculty/residents was added. Overall, there were significantly more comments on what worked well than suggestions for improvement regarding faculty/residents. Instructing students to name-names increased the rate of naming from 35% to 75% for what worked well and from 13% to 39% for suggestions for improvement. Hedging language was included in 61% of suggestions for improvement, but only 2% of what worked well. Students described the variability of their experience, used passive language and qualified negative experiences with positive ones. Medical students may use linguistic strategies, such as impersonalizing and hedging, to mitigate the impact of negative upward feedback. Working towards a culture that supports upward feedback would allow students to feel more comfortable providing candid comments about their experience.

Changing Medical Education, Overnight: The Curricular Response to COVID-19 of Nine Medical Schools.

Issue: Calls to change medical education have been frequent, persistent, and generally limited to alterations in content or structural re-organization. Self-imposed barriers have prevented adoption of more radical pedagogical approaches, so recent predictions of the 'inevitability' of medical education transitioning to online delivery seemed unlikely. Then in March 2020 the COVID-19 pandemic forced medical schools to overcome established barriers overnight and make the most rapid curricular shift in medical education's history. We share the collated reports of nine medical schools and postulate how recent responses may influence future medical education. Evidence: While extraneous pandemic-related factors make it impossible to scientifically distinguish the impact of the curricular changes, some themes emerged. The rapid transition to online delivery was made possible by all schools having learning management systems and key electronic resources already blended into their curricula; we were closer to online delivery than anticipated. Student engagement with online delivery varied with different pedagogies used and the importance of social learning and interaction along with autonomy in learning were apparent. These are factors known to enhance online learning, and the student-centered modalities (e.g. problem-based learning) that included them appeared to be more engaging. Assumptions that the new online environment would be easily adopted and embraced by 'technophilic' students did not always hold true. Achieving true distance medical education will take longer than this 'overnight' response, but adhering to best practices for online education may open a new realm of possibilities. Implications: While this experience did not confirm that online medical education is really 'inevitable,' it revealed that it is possible. Thoughtfully blending more online components into a medical curriculum will allow us to take advantage of this environment's strengths such as efficiency and the ability to support asynchronous and autonomous learning that engage and foster intrinsic learning in our students. While maintaining aspects of social interaction, online learning could enhance pre-clinical medical education by allowing integration and collaboration among classes of medical students, other health professionals, and even between medical schools. What remains to be seen is whether COVID-19 provided the experience, vision and courage for medical education to change, or whether the old barriers will rise again when the pandemic is over.

Using the "Hallmarks of Cancer" as a framework for medical students and clinicians to understand oncogenesis.

Two landmark reviews in 2000 and 2011, describing the "Hallmarks of Cancer", provided a new and valuable framework for understanding the process of oncogenesis as a progressive accumulation of characteristics, each characteristic essential for a tumor to become a clinically relevant, metastatic neoplasia. The process of oncogenesis is conceptually important for physicians, both for clinical reasons, and for their engagement in oncological research. However, these reviews are written for specialists in the field, which presents barriers for novice learners. Therefore, to allow students, and also clinicians external to the oncological field, to access this valuable framework for understanding oncogenesis, we have created a condensed summary of the original reviews. Our institutions use a "flipped" approach to the large-group components of our preclinical education. We have successfully used our Hallmarks of Cancer summary as the prework for sessions on oncogenesis for five years at one institution, and nine years at the other, typically at the end of cancer blocks within integrated, multidisciplinary courses. We report here survey results indicating learners strongly appreciate the summary as both preparation material for participation in relevant flipped classroom sessions, and as a general review of oncogenesis. This condensed summary of the original Hallmarks of Cancer reviews makes many of the key concepts of oncogenesis available to medical students in their preclinical years, as well as to physicians outside the field of oncology.

Trusting early learners with critical professional activities through emergency medical technician certification.

BACKGROUND: Two dominant themes face medical education: developing integrated curricula and improving the undergraduate medical education (UME) to graduate medical education (GME) transition. An innovative solution to both of these challenges at the Zucker School of Medicine has been the application of the cognitive apprenticeship framework in requiring emergency medical technician (EMT) certification during the first course in medical school as the core on which to build an integrated curriculum and provide entrustable clinical skills. METHODS: Beginning with the Class of 2011, student feedback about the short-term impact of the experience was collected annually. In addition, perceptions of near graduates and alumni were surveyed in 2017 to explore the long-term impact of the experience. Theme analysis was conducted via inductive coding. RESULTS: Both first-year and more experienced learners report the value of the EMT curriculum as an integrated component of the first course of medical school. Reported positive long-term impacts included the first-hand observation of social determinants of health and interprofessionalism. Negative comments by early learners focused on course logistics, whereas older learners recalled the variability of clinical experiences during ambulance runs. CONCLUSIONS: The integration of the EMT curriculum as a core component of the first course serves multiple purposes: 1) it provides the foundation of a spiral learning approach; 2) it contextualizes the basic sciences within clinical practice; 3) it provides opportunities for students to engage in authentic clinical activities under the guidance of mentors; 4) it introduces students to the interdisciplinary nature of medicine; and 5) it serves as the first entrustable professional activity (EPA) for our students.

Evaluation and refinement of Self-Directed Learning Readiness Scale for medical students.

PURPOSE: This study evaluated the underlying subdomain structure of the Self-Directed Learning Readiness Scale (SDLRS) for medical students and refined the instrument to measure the subdomains to provide evidence for construct validity. Developing self-directed learners is a well-recognized goal amongst medical educators. The SDLRS has been frequently used, however, lack of construct validity makes it difficult to interpret results. METHODS: To identify the valid subdomains of the SDLRS, items were calibrated with the graded response model (GRM) and results were used to construct a 30-item short form. Short-form validity was evaluated by examining the correspondence between the total scores from the short form and the original instrument for individual students. RESULTS: A five-subdomain model explained the SDLRS item response data reasonably well. These included: (1) initiative and independence in learning, (2) self-concept as an effective learner, (3) openness to learning opportunity, (4) love of learning, and (5) acceptance for one's own learning. The unidimensional GRM for each subdomain fits the data better than multi-dimensional models. The total scores from the refined short form and the original form were correlated at 0.98 and the mean difference was 1.33, providing evidence for validation. Nearly 91% of 179 respondents were accurately classified within the low, average, and high readiness groups. CONCLUSION: Sufficient evidence was obtained for the validity and reliability of the refined 30-item short-form targeting five subdomains to measure medical students' readiness to engage in self-directed learning.

Multi-tier regulation of the streptomycete morphogenetic peptide SapB.

Streptomyces coelicolor is a morphologically complex bacterium requiring the secretion of surface-active proteins to progress through its life cycle. SapB represents an important class of these biosurfactants, as illustrated by its ability to restore aerial hyphae formation when applied exogenously to developmental mutants. However, such aerial hyphae fail to sporulate, exemplifying the need to co-ordinate the timing of SapB production with other developmental events. SapB has an unusual lantibiotic structure. Its structural gene, ramS, is only 38 nucleotides downstream of the gene encoding its putative modification enzyme, RamC. Transient, co-ordinated expression of the operon was thought to be controlled by the response regulator RamR. However, we show that ramS is transcribed throughout the cell cycle with a dual expression profile dissimilar to the tightly controlled ramC expression. Surprisingly, post-translational modification relies on prior membrane localization of the precursor peptide, RamS, as demonstrated by the absence of RamS modification in S. coelicolor hyphae treated with the Bacillus subtilis lipoprotein surfactin. Our results demonstrate that interspecies interaction can also be mediated by interference of post-translational events. Further, temporal and spatial regulation of irreversible post-translational modification of a surface-active morphogenetic peptide suggests a new model for the control of key developmental events.

The Impact of Changing Step 1 to Pass/Fail Reporting on Anxiety, Learning Approaches, and Curiosity.

Purpose: Given the significance of the US Medical Licensing Exam (USMLE) Step 1 score moving from a 3-digit value to pass/fail, the authors investigated the impact of the change on students' anxiety, approach to learning, and curiosity. Method: Two cohorts of pre-clerkship medical students at three medical schools completed a composite of four instruments: the State-Trait Anxiety Inventory, the revised two-factor Study Process Questionnaire, the Interest/Deprivation Type Epistemic Curiosity Scale, and the Short Grit Scale prior to taking the last 3-digit scored Step 1 in 2021 or taking the first pass/fail scored Step 1 in 2022. Responses of 3-digit and pass/fail exam takers were compared (Mann-Whitney U) and multiple regression path analysis was performed to determine the factors that significantly impacted learning strategies. Results: There was no difference between 3-digit (n = 86) and pass/fail exam takers (n = 154) in anxiety (STA-I scores, 50 vs. 49, p = 0.85), shallow learning strategies (22 vs. 23, p = 0.84), or interest curiosity scores (median scores 15 vs. 15, p = 0.07). However, pass/fail exam takers had lower deprivation curiosity scores (median 12 vs. 11, p = 0.03) and showed a decline in deep learning strategies (30 vs. 27, p = 0.0012). Path analysis indicated the decline in deep learning strategies was due to the change in exam scoring (ß = - 2.0428, p < 0.05). Conclusions: Counter to the stated hypothesis and intentions, the initial impact of the change to pass/fail grading for USMLE Step 1 failed to reduce learner anxiety, and reduced curiosity and deep learning strategies. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-023-01878-w.

A Generalizable Approach to Predicting Performance on USMLE Step 2 CK.

Introduction: The elimination of the USMLE Step 1 three-digit score has created a deficit in standardized performance metrics for undergraduate medical educators and residency program directors. It is likely that there will be greater emphasis on USMLE Step 2 CK, an exam found to be associated with later clinical performance in residents and physicians. Because many previous models relied on Step 1 scores to predict student performance on Step 2 CK, we developed a model using other metrics. Materials and Methods: Assessment data for 228 students in three cohorts (classes of 2018, 2019, and 2020) were collected, including the Medical College Admission Test (MCAT), NBME Customized Assessment Service (CAS) exams and NBME Subject exams. A linear regression model was conducted to predict Step 2 CK scores at five time-points: at the end of years one and two and at three trimester intervals in year three. An additional cohort (class of 2021) was used to validate the model. Results: Significant models were found at 5 time-points in the curriculum and increased in predictability as students progressed: end of year 1 (adj R2 = 0.29), end of year 2 (adj R2 = 0.34), clerkship trimester 1 (adj R2 = 0.52), clerkship trimester 2 (adj R2 = 0.58), clerkship trimester 3 (adj R2 = 0.62). Including Step 1 scores did not significantly improve the final model. Using metrics from the class of 2021, the model predicted Step 2 CK performance within a mean square error (MSE) of 8.3 points (SD = 6.8) at the end of year 1 increasing predictability incrementally to within a mean of 5.4 points (SD = 4.1) by the end of year 3. Conclusion: This model is highly generalizable and enables medical educators to predict student performance on Step 2 CK in the absence of Step 1 quantitative data as early as the end of the first year of medical education with increasingly stronger predictions as students progressed through the clerkship year.

Exploring the impact of postponing core clerkships on future performance.

Despite the many clerkship models of medical education, all can be considered a form of experiential learning. Experiential learning is a complex pedagogical approach involving the development of cognitive skills in an environment with a unique culture with multiple stakeholders, which may impact learner motivation, confidence, and other noncognitive drivers of success. Students may delay the transition to the clerkship year for myriad reasons, and the intricate nature of experiential learning suggested this may impact student performance. This retrospective, observational study investigated the impact of clerkship postponement by measuring subsequent clerkship performance. Pre-clerkship and third-year clerkship performance were analyzed for three cohorts of students (classes of 2018, 2019, and 2020, N = 274) where students had the option to delay the start of their clerkship year. A mixed analysis of variance (ANOVA) and paired t-tests were conducted to compare academic performance over time among students who did and did not delay. Across three cohorts of students, 12% delayed the start of the clerkship year (N = 33). Regardless of prior academic performance, these students experienced a significant reduction in clerkship grades compared to their non-delaying peers. Delaying the start of the clerkship year may have negative durable effects on future academic performance. This information should be kept in mind for student advisement.

Innovation in Leadership Development in Undergraduate Medical Education.

In response to the need for physician leaders, the Donald and Barbara Zucker School of Medicine at Hofstra/Northwell developed the Klar Leadership Development and Innovation Management program. This novel program leverages its partnership with a large Northeast health system to longitudinally provide students with leadership fundamentals and mentored experiences.

Supporting Self-Directed Learning: A National Needs Analysis.

INTRODUCTION: It is broadly agreed that physicians must be self-directed learners, and many studies measuring medical student readiness and capacity for self-directed learning (SDL) have been reported. However, less attention has been paid to faculty tasked with guiding students to develop SDL skills. In an effort to determine whether faculty feel equipped to foster SDL, a needs assessment was developed. METHODS: The faculty needs assessment for fostering SDL (FNA-fSDL) was constructed and distributed nationally to medical educators. Faculty perceptions of the value of SDL, administration support, and faculty development were queried. Items also measured faculty recognition of cognitive, affective, and metacognitive strategies shown to cultivate SDL. Descriptive statistical analysis and frequency comparison tests to identify potential group differences related to work responsibility, classroom/clinical setting, and level of learner served were performed. RESULTS: Among the 359 respondents representing diverse areas of medical education, there was overwhelming agreement that physicians in training should be self-directed learners and that faculty require training in educational approaches promoting SDL. Faculty were more likely to recognize strategies supporting development of metacognitive skills and least likely to identify those related to cognitive aspects of learning. Fewer than half of faculty had received training in fostering SDL at their home institutions or at conferences. DISCUSSION: While there is strong support for the development of SDL among learners, these results suggest most faculty do not feel sufficiently trained to accomplish this goal. The FNA-fSDL can be used by institutions to assess their own faculty.

Pandemics Past and Present: A Guided Inquiry Approach.

BACKGROUND: COVID-19 exposed undergraduate medical education curricular gaps in exploring historical pandemics, how to critically consume scientific literature and square it with the lay press, and how to grapple with emerging ethical issues. In addition, as medical students were dismissed from clinical environments, their capacity to build community and promote professional identity formation was compromised. METHODS: A synchronous, online course entitled Life Cycle of a Pandemic was developed using a modified guided inquiry approach. Students met daily for 2 weeks in groups of 15 to 18 with a process facilitator. During the first week, students reported on lessons learned from past pandemics; in the second week, students discussed ethical concerns surrounding COVID-19 clinical trials, heard from physicians who provided patient care in the HIV and COVID-19 pandemics, and concluded with an opportunity for reflection. Following the course, students were asked to complete an anonymous, voluntary survey to assess their perceptions of the course. RESULTS: With a response rate of 69%, an overwhelming majority of students agreed or strongly agreed that learning about historical pandemics helped them understand COVID-19 (72, 99%). The course successfully helped students understand current and potential COVID-19 management strategies as 66 (90%) agreed or strongly agreed they developed a better understanding of nonpharmacological interventions and new pharmacological treatments. Students also gained insight into the experiences of healthcare providers who cared for patients with HIV and COVID-19. Qualitative analysis of the open-ended comments yielded 5 main themes: critical appraisal of resources, responsibility of the physician, humanism, knowledge related to pandemics, and learning from history. CONCLUSIONS: The onset of the COVID-19 crisis illustrated curricular gaps that could be remedied by introducing the history and biology of pandemics earlier in the curriculum. It was also apparent that learners need more practice in critically reviewing literature and comparing scientific literature with lay press. The flexible format of the course promotes the development of future iterations that could cover evolving topics related to COVID-19. The course could also be repurposed for a graduate or continuing medical education audience.

Patients don't come with multiple choice options: essay-based assessment in UME.

Curricular revision efforts have resulted in learner-centered programs that value content integration and active learning. Yet, less attention has been placed on assessment methods that are learner-centered and promote assessment for learning. The use of context rich short answer question (CR-SAQ) exams in the preclinical years of medical school was evaluated to determine if this format aligns with the criteria for assessment for learning. Medical students and preclinical faculty members were sent a survey comprised of closed and open-ended questions about their experience using CR-SAQ exams. Data were analyzed using a mixed-method design. Open-ended responses were evaluated using thematic analysis within the framework of criteria for assessment for learning. A total of 274 students (94%) and 24 faculty (75%) completed the survey. Fifty four percent of students reported preferring a CR-SAQ exam format over multiple choice questions (MCQ) format. Quantitative data and qualitative comments by students supported that CR-SAQ exams aligned with criteria for assessment for learning, including acceptability, authenticity, educational effect, and the cueing effect. Student concerns included preparation for USMLE Step 1 exam, as well as the validity and reproducibility of CR-SAQ assessments. Faculty largely agreed with the benefits of the CR-SAQ, but were concerned about feasibility, acceptability and reproducibility. The CR-SAQ exam format assessment strategy supports assessment for learning in an undergraduate medical education setting. Both benefits and drawbacks of this method are presented, however students and faculty describe a broader impact that this assessment method has on their development as a physician.

Applying the Hedgehog Concept to Transform Undergraduate Medical Education.

As the U.S. health care system changes and physician responsibilities shift, medical educators must reconsider how best to prepare medical school graduates for the future practice of medicine. Thoughtful reexamination of the goals of undergraduate medical education (UME) and the roles of educators, medical students, and physicians is warranted to ensure that they align with evolving health care environments and delivery systems. In this Invited Commentary, the authors apply Jim Collins's "hedgehog concept" from Good to Great-a business-world framework designed to transform companies-to UME. The hedgehog concept is defined by the intersection of an organization's passion, area of expertise, and economic and resource engines. Focusing on this single concept can guide key decisions, reject what does not align conceptually, and drive overall organizational success. The authors use the hedgehog concept to frame the programmatic development of the Donald and Barbara Zucker School of Medicine at Hofstra/Northwell (Zucker SOM), a millennial medical school, as an organization with the passion to develop innovative UME curricula by challenging the status quo; the drive to be the best at leveraging health system resources to train graduates to excel in systems-based care; and the economic and resource engine of faculty time, financial and infrastructure support, and reputation building. The success of this approach is assessed at Zucker SOM through student and graduate outcomes data. The authors suggest that this hedgehog concept is generalizable to other UME programs whose leaders seek to transform medical education to meet 21st-century workforce and health care delivery needs.

Modeling integration: co-teaching basic and clinical sciences medicine in the classroom.

PURPOSE: Calls for revision in undergraduate medical education frequently cite the importance of integrating basic and clinical sciences and the use of active pedagogies. One under-appreciated approach to accomplishing both is interactive co-teaching, defined as two instructors with complementary expertise engaging students and each other instead of lecturing. This study sought to determine if interactive co-teaching helped students integrate and learn basic and clinical sciences, as well as to explore potential advantages and barriers to co-teaching. METHODS: The comparative success of solo- and co-teaching in a microbiology/infectious disease course was determined by surveying student perceptions at the end of the course and examination scores for questions based on either solo- or co-taught content. The advantages and barriers to co-teaching were explored by thematic analysis of student responses to open-ended survey questions. RESULTS: Results suggest that co-teaching supported content integration as a significant majority of students (92%, n=112) reported they understood the connection between basic and clinical sciences better when content was co-taught. In addition, a plurality of students indicated that co-teaching provided a better overall learning experience (81%, n=99), was more engaging (74%, n=90), and made it easier to apply content (74%, n=90). These positive perceptions were reflected in better exam outcomes for materials covered in co-taught over solo-taught sessions. CONCLUSION: Results suggest students value co-teaching as a means to integrate basic and clinical sciences. However, interactive co-teaching pedagogies require careful planning and collaboration among faculty. Co-teaching requires the commitment of both faculty members to this pedagogy.

Formative Assessment in an Integrated Curriculum: Identifying At-Risk Students for Poor Performance on USMLE Step 1 Using NBME Custom Exam Questions.

PURPOSE: The Hofstra Northwell School of Medicine (HNSOM) uses an essay-based assessment system. Recognizing the emphasis graduate medical education places on the United States Medical Licensing Examination (USMLE) Step exams, the authors developed a method to predict students at risk for lower performance on USMLE Step 1. METHOD: Beginning with the inaugural class (2015), HNSOM administered National Board of Medical Examiners (NBME) Customized Assessment Service (CAS) examinations as formative assessment at the end of each integrated course in the first two years of medical school. Using preadmission data, the first two courses in the educational program, and NBME score deviation from the national test takers' mean, a statistical model was built to predict students who scored below the Step 1 national mean. RESULTS: A regression equation using the highest Medical College Admission Test (MCAT) score and NBME score deviation predicted student Step 1 scores. The MCAT alone accounted for 21% of the variance. Adding the NBME score deviation from the first and second courses increased the variance to 40% and 50%, respectively. Adding NBME exams from later courses increased the variance to 52% and 64% by the end of years one and two, respectively. Cross-validation demonstrated the model successfully predicted 63% of at-risk students by the end of the fifth month of medical school. CONCLUSIONS: The model identified students at risk for lower performance on Step 1 using the NBME CAS. This model is applicable to schools reforming their curriculum delivery and assessment programs toward an integrated model.

Contextualizing the relevance of basic sciences: small-group simulation with debrief for first- and second-year medical students in an integrated curriculum.

AIM: There has been a call for increased integration of basic and clinical sciences during preclinical years of undergraduate medical education. Despite the recognition that clinical simulation is an effective pedagogical tool, little has been reported on its use to demonstrate the relevance of basic science principles to the practice of clinical medicine. We hypothesized that simulation with an integrated science and clinical debrief used with early learners would illustrate the importance of basic science principles in clinical diagnosis and management of patients. METHODS: Small groups of first- and second-year medical students were engaged in a high-fidelity simulation followed by a comprehensive debrief facilitated by a basic scientist and clinician. Surveys including anchored and open-ended questions were distributed at the conclusion of each experience. RESULTS: The majority of the students agreed that simulation followed by an integrated debrief illustrated the clinical relevance of basic sciences (mean ± standard deviation: 93.8% ± 2.9% of first-year medical students; 96.7% ± 3.5% of second-year medical students) and its importance in patient care (92.8% of first-year medical students; 90.4% of second-year medical students). In a thematic analysis of open-ended responses, students felt that these experiences provided opportunities for direct application of scientific knowledge to diagnosis and treatment, improving student knowledge, simulating real-world experience, and developing clinical reasoning, all of which specifically helped them understand the clinical relevance of basic sciences. CONCLUSION: Small-group simulation followed by a debrief that integrates basic and clinical sciences is an effective means of demonstrating the relationship between scientific fundamentals and patient care for early learners. As more medical schools embrace integrated curricula and seek opportunities for integration, our model is a novel approach that can be utilized.