Understanding Academic Self-Concept and Performance in Medical Education.

PROBLEM: Medical students' academic self-concept (ASC) is an important factor in better understanding noncognitive mediators of performance in medical school. However, research is limited on ASC in medical students across multiple phases of undergraduate medical education curriculum. This pilot study explored the relationship between ASC and academic performance across different phases of a U.S. medical school curriculum, specifically at the end of the second (preclinical) and third (clinical) years. APPROACH: Medical students across 2 cohorts at Virginia Commonwealth University School of Medicine, Richmond, Virginia, were surveyed using an ASC confidence subscale in 2019. Multiple linear regression analysis was conducted using medical student ASC scores in preclinical (n = 190) and clinical (n = 149) phases and performance data. Clinical performance was calculated through a weighted mean of clerkship grades based on the number of weeks for each clerkship. OUTCOMES: Preclinical performance was related to ASC, gender, and performance after year 1. ASC scores varied significantly by gender in the preclinical cohort ( P < .01), with men reporting higher ASC than women (mean [SD], 2.94 [0.41] vs 2.78 [0.38]). Significant gender differences in performance were found at the end of year 3 ( P < .01), with women performing more favorably compared with men (mean [SD], 94.1 [59.04] vs 124.24 [64.54]). The relationship between ASC and performance at the end of year 2 suggested students with higher ASC perform better during their preclinical phase. NEXT STEPS: This pilot study supports future scholarship in 2 areas: (1) identification and assessment of additional factors that influence the relationship between ASC and academic performance across the entire undergraduate medical education curriculum and (2) development and implementation of evidence-based interventions to support student ASC and performance and enhance the learning environment. Analyzing longitudinal trends across multiple cohorts will drive evidence-based interventions at learner and programmatic levels.

Measuring the Master Adaptive Learner: Development and Internal Structure Validity Evidence for a New Instrument.

BACKGROUND: The master adaptive learner (MAL) uses self-regulated learning skills to develop adaptive, efficient, and accurate skills in practice. Given rapid changes in healthcare, it is essential that medical students develop into MALs. There is a need for an instrument that can capture MAL behaviors and characteristics. The objective of this study was to develop an instrument for measuring the MAL process in medical students and evaluate its psychometric properties. METHODS: As part of curriculum evaluation, 818 students completed previously developed instruments with validity evidence including the Self-Regulated Learning Perception Scale, Brief Resilience Scale, Goal Orientation Scale, and Jefferson Scale of Physician Lifelong Learning. The authors performed exploratory factor analysis to examine underlying relationships between items. Items with high factor loadings were retained. Cronbach's alpha was computed. In parallel, the multi-institutional research team rated the same items to provide content validity evidence of the items to MAL model. RESULTS: The original 67 items were reduced to 28 items loading onto four factors: Planning, Learning, Resilience, and Motivation. Each subscale included the following number of items and Cronbach's alpha: Planning (10 items, alpha = 0.88), Learning (6 items, alpha = 0.81), Resilience (6 items, alpha = 0.89), and Motivation (6 items, alpha = 0.81). The findings from the factor analyses aligned with the research team ratings of linkage to the components of MAL. CONCLUSION: These findings serve as a starting point for future work measuring master adaptive learning to identify and support learners. To fully measure the MAL construct, additional items may need to be developed.

Examining Bloom's Taxonomy in Multiple Choice Questions: Students' Approach to Questions.

BACKGROUND: Analytic thinking skills are important to the development of physicians. Therefore, educators and licensing boards utilize multiple-choice questions (MCQs) to assess these knowledge and skills. MCQs are written under two assumptions: that they can be written as higher or lower order according to Bloom's taxonomy, and students will perceive questions to be the same taxonomical level as intended. This study seeks to understand the students' approach to questions by analyzing differences in students' perception of the Bloom's level of MCQs in relation to their knowledge and confidence. METHODS: A total of 137 students responded to practice endocrine MCQs. Participants indicated the answer to the question, their interpretation of it as higher or lower order, and the degree of confidence in their response to the question. RESULTS: Although there was no significant association between students' average performance on the content and their question classification (higher or lower), individual students who were less confident in their answer were more than five times as likely (OR = 5.49) to identify a question as higher order than their more confident peers. Students who responded incorrectly to the MCQ were 4 times as likely to identify a question as higher order than their peers who responded correctly. CONCLUSIONS: The results suggest that higher performing, more confident students rely on identifying patterns (even if the question was intended to be higher order). In contrast, less confident students engage in higher-order, analytic thinking even if the question is intended to be lower order. Better understanding of the processes through which students interpret MCQs will help us to better understand the development of clinical reasoning skills.

Generalizability of the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) Scale to Assess Medical Student Performance on Core EPAs in the Workplace: Findings From One Institution.

PURPOSE: Assessment of the Core Entrustable Professional Activities for Entering Residency (Core EPAs) requires direct observation of learners in the workplace to support entrustment decisions. The purpose of this study was to examine the internal structure validity evidence of the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) scale when used to assess medical student performance in the Core EPAs across clinical clerkships. METHOD: During the 2018-2019 academic year, the Virginia Commonwealth University School of Medicine implemented a mobile-friendly, student-initiated workplace-based assessment (WBA) system to provide formative feedback for the Core EPAs across all clinical clerkships. Students were required to request a specified number of Core EPA assessments in each clerkship. A modified O-SCORE scale (1 = "I had to do" to 4 = "I needed to be in room just in case") was used to rate learner performance. Generalizability theory was applied to assess the generalizability (or reliability) of the assessments. Decision studies were then conducted to determine the number of assessments needed to achieve a reasonable reliability. RESULTS: A total of 10,680 WBAs were completed on 220 medical students. The majority of ratings were completed on EPA 1 (history and physical) (n = 3,129; 29%) and EPA 6 (oral presentation) (n = 2,830; 26%). Mean scores were similar (3.5-3.6 out of 4) across EPAs. Variance due to the student ranged from 3.5% to 8%, with the majority of the variation due to the rater (29.6%-50.3%) and other unexplained factors. A range of 25 to 63 assessments were required to achieve reasonable reliability (Phi > 0.70). CONCLUSIONS: The O-SCORE demonstrated modest reliability when used across clerkships. These findings highlight specific challenges for implementing WBAs for the Core EPAs including the process for requesting WBAs, rater training, and application of the O-SCORE scale in medical student assessment.

Video streaming: pushing a swimming pool through a straw.

If someone asked you to move all the water from one swimming pool to another, you would try to use the biggest hose and the most pressure you could find to make the transfer relatively quick and easy. Moving a video file from one computer to another is very similar to moving all that water: there's usually a tremendous amount of data (lots and lots of water) and very little bandwidth (the size of the hose).