Evaluating Performance and Autonomy Levels of Previous Preliminary Surgery Interns.

INTRODUCTION: Surgery residents who complete a nondesignated preliminary year have an additional year of training compared to those who begin as categorical residents. While this additional year is sometimes perceived negatively, these more experienced residents may outperform traditional categorical (TC) residents in their first year of training. METHODS: Operative assessment ratings were recorded for first year categorical general surgery residents in the United States between 2015 and 2023 using the Society for Improving Medical and Professional Learning assessment platform. Residents were categorized based on the completion of a nondesignated preliminary year ("Previous Prelim" [PP]) or not ("Traditional Categorical"). Ratings were analyzed using generalized mixed effects models. Performance and autonomy outcomes were dichotomized: "less experience" or "more experience" and "no autonomy" or "some autonomy", respectively. Fixed effects included academic month and case complexity, while random effects included resident, faculty, program, and procedure. RESULTS: A total of 34,353 evaluations from 86 general surgery programs were collected. Of these, 829 were evaluations from PP residents. Faculty ratings of PP versus TC revealed no differences in adjusted probabilities of achieving a "more experience" rating (82% versus 76%, P = 0.26) but a higher adjusted probability of achieving a "some autonomy" rating (88% versus 80%, P = 0.04) for PP compared to TC. Analysis of resident self-reported ratings revealed higher adjusted probabilities of a "more experience" rating (77% versus 50%, P = 0.01) and "some autonomy" rating (87% versus 73%; P = 0.02) for PP compared to TC. CONCLUSIONS: First year general surgery residents who previously completed a preliminary year have similar operative performance faculty ratings when compared to their peers.

Performance of ChatGPT on American Board of Surgery In-Training Examination Preparation Questions.

INTRODUCTION: Chat Generative Pretrained Transformer (ChatGPT) is a large language model capable of generating human-like text. This study sought to evaluate ChatGPT's performance on Surgical Council on Resident Education (SCORE) self-assessment questions. METHODS: General surgery multiple choice questions were randomly selected from the SCORE question bank. ChatGPT (GPT-3.5, April-May 2023) evaluated questions and responses were recorded. RESULTS: ChatGPT correctly answered 123 of 200 questions (62%). ChatGPT scored lowest on biliary (2/8 questions correct, 25%), surgical critical care (3/10, 30%), general abdomen (1/3, 33%), and pancreas (1/3, 33%) topics. ChatGPT scored higher on biostatistics (4/4 correct, 100%), fluid/electrolytes/acid-base (4/4, 100%), and small intestine (8/9, 89%) questions. ChatGPT answered questions with thorough and structured support for its answers. It scored 56% on ethics questions and provided coherent explanations regarding end-of-life discussions, communication with coworkers and patients, and informed consent. For many questions answered incorrectly, ChatGPT provided cogent, yet factually incorrect descriptions, including anatomy and steps of operations. In two instances, it gave a correct explanation but chose the wrong answer. It did not answer two questions, stating it needed additional information to determine the next best step in treatment. CONCLUSIONS: ChatGPT answered 62% of SCORE questions correctly. It performed better at questions requiring standard recall but struggled with higher-level questions that required complex clinical decision making, despite providing detailed responses behind its rationale. Due to its mediocre performance on this question set and sometimes confidently-worded, yet factually inaccurate responses, caution should be used when interpreting ChatGPT's answers to general surgery questions.

American Board of Surgery In-Training Exam Performance Predicted by Question Bank Use While Unassociated With Other Learning Strategies.

INTRODUCTION: There is no consensus regarding optimal curricula to teach cognitive elements of general surgery. The American Board of Surgery In-Training Exam (ABSITE) aims to measure trainees' progress in attaining this knowledge. Resources like question banks (QBs), Surgical Council on Resident Education (SCORE) curriculum, and didactic conferences have mixed findings related to ABSITE performance and are often evaluated in isolation. This study characterized relationships between multiple learning methods and ABSITE performance to elucidate the relative educational value of learning strategies. METHODS: Use and score of QB, SCORE use, didactic conference attendance, and ABSITE percentile score were collected at an academic general surgery residency program from 2017 to 2022. QB data were available in the years 2017-2018 and 2021-2022 during institutional subscription to the same platform. Given differences in risk of qualifying exam failure, groups of ≤30th and >30th percentile were analyzed. Linear quantile mixed regressions and generalized linear mixed models determined factors associated with ABSITE performance. RESULTS: Linear quantile mixed regressions revealed a relationship between ABSITE performance and QB questions completed (1.5 percentile per 100 questions, P < 0.001) and QB score (1.2 percentile per 1% score, P < 0.001), but not with SCORE use and didactic attendance. Performers >30th percentile had a significantly higher QB score. CONCLUSIONS: Use and score of QB had a significant relationship with ABSITE performance, while SCORE use and didactic attendance did not. Performers >30th percentile completed a median 1094 QB questions annually with a score of 65%. Results emphasize success of QB use as an active learning strategy, while passive learning methods warrant further evaluation.

Effects of homicide timing on test scores: Quasi-experimental evidence from two cities in Colombia.

This study examines the effect of homicides around schools on the standardized test scores of fifth and ninth graders (N = 4729; Mage = 12.71 years, SDage = 2.13) using a quasi-experimental design in two Colombian cities. Exposure to homicides occurring within 7 days of the test and within 500 m of the school decreases test scores by 0.10 SD. Effects show a greater sensitivity to timing than distance, becoming null as the time to the testing date increases but remaining consistent across larger radii. Since students in the study are on average exposed to 12.1 homicides per year, even short-lived learning losses can accumulate to impair learning for substantial portions of the school year. Findings are discussed, considering previous empirical work.

From admissions to licensure: education data associations from a multi-centre undergraduate medical education collaboration.

This paper reports the findings of a Canada based multi-institutional study designed to investigate the relationships between admissions criteria, in-program assessments, and performance on licensing exams. The study's objective is to provide valuable insights for improving educational practices across different institutions. Data were gathered from six medical schools: McMaster University, the Northern Ontario School of Medicine University, Queen's University, University of Ottawa, University of Toronto, and Western University. The dataset includes graduates who undertook the Medical Council of Canada Qualifying Examination Part 1 (MCCQE1) between 2015 and 2017. The data were categorized into five distinct sections: demographic information as well as four matrices: admissions, course performance, objective structured clinical examination (OSCE), and clerkship performance. Common and unique variables were identified through an extensive consensus-building process. Hierarchical linear regression and a manual stepwise variable selection approach were used for analysis. Analyses were performed on data set encompassing graduates of all six medical schools as well as on individual data sets from each school. For the combined data set the final model estimated 32% of the variance in performance on licensing exams, highlighting variables such as Age at Admission, Sex, Biomedical Knowledge, the first post-clerkship OSCE, and a clerkship theta score. Individual school analysis explained 41-60% of the variance in MCCQE1 outcomes, with comparable variables to the analysis from of the combined data set identified as significant independent variables. Therefore, strongly emphasising the need for variety of high-quality assessment on the educational continuum. This study underscores the importance of sharing data to enable educational insights. This study also had its challenges when it came to the access and aggregation of data. As such we advocate for the establishment of a common framework for multi-institutional educational research, facilitating studies and evaluations across diverse institutions. This study demonstrates the scientific potential of collaborative data analysis in enhancing educational outcomes. It offers a deeper understanding of the factors influencing performance on licensure exams and emphasizes the need for addressing data gaps to advance multi-institutional research for educational improvements.

Ultrasound Incorporation in Gross Anatomy Labs in a Master of Medical Sciences Program: A Mixed-Methods Analysis of Student Performance and Perception.

OBJECTIVES: Teaching ultrasound imaging is on the rise in undergraduate medical anatomy education. However, there is little research exploring the use of ultrasound in preparatory graduate programs. The purpose of this study is to identify the effects of ultrasound imaging inclusion in a graduate gross anatomy course. METHODS: Master of Medical Sciences students were enrolled in a prosection-based anatomy course that included pinned cadaver stations and an ultrasound station. Using ultrasound, teaching assistants imaged volunteers demonstrating anatomical structures students previously learned at cadaver stations. Students answered one ultrasound image question on each practical exam and were asked to participate in a pre- and post-course survey. Student practical and lecture exam scores and final course grades from the 2022 cohort were compared to a historical control cohort from 2021 via statistical analysis, including a survey administered to the 2022 cohort. RESULTS: Two hundred students from the 2021 cohort and 164 students from the 2022 cohort participated in this study. Students in the 2022 cohort had significantly higher scores in 1 of the 5 practical exams (P < .05, d = .289), and 2 of the 5 written exams (P < .05, d = .207), (P < .05, d = .311). Survey data revealed increased (P < .05, d = 1.203) learning outcome achievement from pre-survey to post-survey in the intervention cohort. Students who correctly answered the ultrasound question performed significantly better on practical's 3 (P < .05) and 4 (P < .05) than those who missed the ultrasound question. CONCLUSIONS: These findings suggest that ultrasound imaging in a cadaver lab is beneficial to graduate students' learning and understanding of gross anatomy.

Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis.

BACKGROUND: Over the past 2 years, researchers have used various medical licensing examinations to test whether ChatGPT (OpenAI) possesses accurate medical knowledge. The performance of each version of ChatGPT on the medical licensing examination in multiple environments showed remarkable differences. At this stage, there is still a lack of a comprehensive understanding of the variability in ChatGPT's performance on different medical licensing examinations. OBJECTIVE: In this study, we reviewed all studies on ChatGPT performance in medical licensing examinations up to March 2024. This review aims to contribute to the evolving discourse on artificial intelligence (AI) in medical education by providing a comprehensive analysis of the performance of ChatGPT in various environments. The insights gained from this systematic review will guide educators, policymakers, and technical experts to effectively and judiciously use AI in medical education. METHODS: We searched the literature published between January 1, 2022, and March 29, 2024, by searching query strings in Web of Science, PubMed, and Scopus. Two authors screened the literature according to the inclusion and exclusion criteria, extracted data, and independently assessed the quality of the literature concerning Quality Assessment of Diagnostic Accuracy Studies-2. We conducted both qualitative and quantitative analyses. RESULTS: A total of 45 studies on the performance of different versions of ChatGPT in medical licensing examinations were included in this study. GPT-4 achieved an overall accuracy rate of 81% (95% CI 78-84; P<.01), significantly surpassing the 58% (95% CI 53-63; P<.01) accuracy rate of GPT-3.5. GPT-4 passed the medical examinations in 26 of 29 cases, outperforming the average scores of medical students in 13 of 17 cases. Translating the examination questions into English improved GPT-3.5's performance but did not affect GPT-4. GPT-3.5 showed no difference in performance between examinations from English-speaking and non-English-speaking countries (P=.72), but GPT-4 performed better on examinations from English-speaking countries significantly (P=.02). Any type of prompt could significantly improve GPT-3.5's (P=.03) and GPT-4's (P<.01) performance. GPT-3.5 performed better on short-text questions than on long-text questions. The difficulty of the questions affected the performance of GPT-3.5 and GPT-4. In image-based multiple-choice questions (MCQs), ChatGPT's accuracy rate ranges from 13.1% to 100%. ChatGPT performed significantly worse on open-ended questions than on MCQs. CONCLUSIONS: GPT-4 demonstrates considerable potential for future use in medical education. However, due to its insufficient accuracy, inconsistent performance, and the challenges posed by differing medical policies and knowledge across countries, GPT-4 is not yet suitable for use in medical education. TRIAL REGISTRATION: PROSPERO CRD42024506687; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=506687.

Approaches to Item-Level Data with Cross-Classified Structure: An Illustration with Student Evaluation of Teaching.

Student evaluation of teaching (SET) questionnaires are ubiquitously applied in higher education institutions in North America for both formative and summative purposes. Data collected from SET questionnaires are usually item-level data with cross-classified structure, which are characterized by multivariate categorical outcomes (i.e., multiple Likert-type items in the questionnaires) and cross-classified structure (i.e., non-nested students and instructors). Recently, a new approach, namely the cross-classified IRT model, was proposed for appropriately handling SET data. To inform researchers in higher education, in this article, the cross-classified IRT model, along with three existing approaches applied in SET studies, including the cross-classified random effects model (CCREM), the multilevel item response theory (MLIRT) model, and a two-step integrated strategy, was reviewed. The strengths and weaknesses of each of the four approaches were also discussed. Additionally, the new and existing approaches were compared through an empirical data analysis and a preliminary simulation study. This article concluded by providing general suggestions to researchers for analyzing SET data and discussing limitations and future research directions.

[Practical Year in Public Health: 10 Years of Experience at the Public Health Department in Frankfurt]. / Das Praktische Jahr im Fach Öffentliches Gesundheitswesen: Ergebnisse aus 10 Jahre Erfahrung am Gesundheitsamt Frankfurt am Main.

BACKGROUND: Since the winter semester 2013/14, the Public Health Department in Frankfurt has been the first to offer an internship in public health to medical students in their practical year in Germany. In this study, we analyzed how students evaluated their practical experience and the logbook within clinical training at the Public Health Department. METHODS: A rotation procedure was set up and a logbook with subject-specific learning objectives was developed. After the internship, students participated in an evaluation using a standardized questionnaire. The results of the last 10 years were evaluated quantitatively. RESULTS: The evaluation results of 15 medical students of the last 10 years showed a very high level of internship satisfaction. There was a high acceptance of the logbook regarding structure, learning objectives and skills. CONCLUSION: The internship at the Frankfurt Public Health Department provides the opportunity to be involved in daily activities of public health practice, gives an overview of learning objectives, and underlines good public health training.

Residency program factors affecting ACVS board examination pass rates from 2017 to 2021: A survey of ACVS small animal surgery residency programs.

OBJECTIVE: To identify small animal surgery (SAS) residency program factors associated with board examination pass rates for the American College of Veterinary Surgeons (ACVS) board certification examinations. STUDY DESIGN: Online survey. SAMPLE POPULATION: Program directors of ACVS-registered SAS residency programs. METHODS: An online survey was distributed to eligible 2021 ACVS SAS residency program directors. Respondents provided program information and Phase I and Phase II board examination pass rate data from 2017 to 2021. Programs were grouped based on overall combined board examination pass rates as "excellent" (≥90%) versus "adequate" (<90%). Data were analyzed using recursive partitioning. RESULTS: Responses from 36 ACVS SAS residency programs (18 private practice [PP], 18 academic programs [AP]) were included. Pass rates were considered "excellent" for 17/36 programs (11/17 AP, 6/17 PP). The strongest indicator of "excellent" programs was most mentors participating in rounds >75% of the time. Of those, the second indicator of "excellent" programs was a soft tissue caseload of <7 surgeries/resident/week. If the soft tissue caseload was >7 surgeries/resident/week, a high orthopedic caseload (>9 surgeries/resident/week) was indicative of "excellent" programs. If most mentors were not participating in rounds >75% of the time, APs were more likely to be "excellent." CONCLUSION: SAS APs may have higher ACVS board examination pass rates. Mentor involvement, caseload, and residency practice type may influence board examination pass rates. CLINICAL IMPACT: ACVS SAS residency programs' board examination pass rates may improve by optimizing mentor involvement and caseload regardless of practice type.

Assessment of pressure injury knowledge using PZ-PUKT questionnaire among Indian nurses and the evaluation of impact of an educational intervention on the knowledge scores: A quasi-experimental study.

AIM: To determine the knowledge of pressure injury among Indian nurses using PZ-PUKT questionnaire and to evaluate the effect of an educational intervention on knowledge scores. MATERIALS AND METHODS: A Quasi-experimental study design was used to test the Pressure injury knowledge of 273 nurses in a tertiary care teaching hospital. The Pieper Zulkowski Pressure Ulcer Knowledge Test questionnaire was given as a pre-test prior to education session and re-administered after the activity was completed. RESULTS: The mean score of the pre-test was 48.58 ± 6.75 (C·I 47.8-49.4) and post-test 54.14 ± 7.69 (C·I 53.3-55.1), which showed a statistically significant improvement. In the pre-test, nurses had highest score in the prevention subset while wound subset had the greatest improvement in the post-test. Female participants had a better understanding about Pressure injuries when compared to males. Also, the Assistant Nursing Superintendents and ward in Charge nurses had better knowledge as compared with other nurses. CONCLUSIONS: The knowledge of pressure injury among nurses is limited. Knowledge deficits should be identified and targeted educational interventions should be administered to all the nurses irrespective of their educational level and work experience. Wound certification courses should be instituted so that it gives the nurses a better opportunity to learn about pressure injuries at a certified level. All nurses should undergo periodic training in this ever-evolving field so as to provide the best care to their patients.

A novel algorithm-driven hybrid simulation learning method to improve acquisition of endotracheal intubation skills: a randomized controlled study.

BACKGROUND: Simulation-based training is a clinical skill learning method that can replicate real-life situations in an interactive manner. In our study, we compared a novel hybrid learning method with conventional simulation learning in the teaching of endotracheal intubation. METHODS: One hundred medical students and residents were randomly divided into two groups and were taught endotracheal intubation. The first group of subjects (control group) studied in the conventional way via lectures and classic simulation-based training sessions. The second group (experimental group) used the hybrid learning method where the teaching process consisted of distance learning and small group peer-to-peer simulation training sessions with remote supervision by the instructors. After the teaching process, endotracheal intubation (ETI) procedures were performed on real patients under the supervision of an anesthesiologist in an operating theater. Each step of the procedure was evaluated by a standardized assessment form (checklist) for both groups. RESULTS: Thirty-four subjects constituted the control group and 43 were in the experimental group. The hybrid group (88%) showed significantly better ETI performance in the operating theater compared with the control group (52%). Further, all hybrid group subjects (100%) followed the correct sequence of actions, while in the control group only 32% followed proper sequencing. CONCLUSIONS: We conclude that our novel algorithm-driven hybrid simulation learning method improves acquisition of endotracheal intubation with a high degree of acceptability and satisfaction by the learners' as compared with classic simulation-based training.

Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom.

We compared students' self-reported perception of learning with their actual learning under controlled conditions in large-enrollment introductory college physics courses taught using 1) active instruction (following best practices in the discipline) and 2) passive instruction (lectures by experienced and highly rated instructors). Both groups received identical class content and handouts, students were randomly assigned, and the instructor made no effort to persuade students of the benefit of either method. Students in active classrooms learned more (as would be expected based on prior research), but their perception of learning, while positive, was lower than that of their peers in passive environments. This suggests that attempts to evaluate instruction based on students' perceptions of learning could inadvertently promote inferior (passive) pedagogical methods. For instance, a superstar lecturer could create such a positive feeling of learning that students would choose those lectures over active learning. Most importantly, these results suggest that when students experience the increased cognitive effort associated with active learning, they initially take that effort to signify poorer learning. That disconnect may have a detrimental effect on students' motivation, engagement, and ability to self-regulate their own learning. Although students can, on their own, discover the increased value of being actively engaged during a semester-long course, their learning may be impaired during the initial part of the course. We discuss strategies that instructors can use, early in the semester, to improve students' response to being actively engaged in the classroom.

Analysis of Performance Trends from 2010-2019 on the American Board of Internal Medicine Nephrology Certifying Exam.

BACKGROUND: The pass rate on the American Board of Internal Medicine (ABIM) nephrology certifying exam has declined and is among the lowest of all internal medicine (IM) subspecialties. In recent years, there have also been fewer applicants for the nephrology fellowship match. METHODS: This retrospective observational study assessed how changes between 2010 and 2019 in characteristics of 4094 graduates of US ACGME-accredited nephrology fellowship programs taking the ABIM nephrology certifying exam for the first time, and how characteristics of their fellowship programs were associated with exam performance. The primary outcome measure was performance on the nephrology certifying exam. Fellowship program pass rates over the decade were also studied. RESULTS: Lower IM certifying exam score, older age, female sex, international medical graduate (IMG) status, and having trained at a smaller nephrology fellowship program were associated with poorer nephrology certifying exam performance. The mean IM certifying exam percentile score among those who subsequently took the nephrology certifying exam decreased from 56.7 (SD, 27.9) to 46.1 (SD, 28.7) from 2010 to 2019. When examining individuals with comparable IM certifying exam performance, IMGs performed less well than United States medical graduates (USMGs) on the nephrology certifying exam. In 2019, only 57% of nephrology fellowship programs had aggregate 3-year certifying exam pass rates ≥80% among their graduates. CONCLUSIONS: Changes in IM certifying exam performance, certain trainee demographics, and poorer performance among those from smaller fellowship programs explain much of the decline in nephrology certifying exam performance. IM certifying exam performance was the dominant determinant.

General Surgery Residency Application Evaluation in a USMLE Step 1 Pass/Fail World: A Retrospective Comparison.

Introduction The United States Medical Licensure Exam (USMLE) Step 1 has been used as both a licensing exam and a way for residency programs to evaluate applicants. It has had significant impact upon the match process over time. With the 2020 decision to make the exam pass/fail due to its unclear validity as an evaluation for future physician performance, programs will go through the match without the Step 1 score. We set out to better understand the effects of the exam score on our selection process, with the hypothesis that without the step 1 score, the ranking of our applicants would be significantly altered. Methods We performed a retrospective analysis of applications to a single General Surgery residency program with 4 categorial residents per year at a physician led, academic, tertiary care medical center from 2017-2020. Important applicant factors including USMLE Step 1 and 2, AOA status, science grades, clerkship scores, audition rotations, volunteer activities, research activities, letters of recommendation, and personal statements were given points and evaluated through our equation, the sum of which was used to create a rank list and offer interviews. The standard deviation of scores was calculated with and without Step 1, and the distribution of scores compared. The range and average of applicants' change in point scores were examined. Results The applications of 653 students were reviewed. After removal of USMLE step 1 points, 40% of all applicants decreased in rank, 35% remained the same, and 24% increased. Specifically, 18.8% of the top third dropped to the middle third, and 11.7% of the bottom third jumped to the middle third, while the middle third changed little (0.2% dropped and 0.9% jumped out of middle third). The points given for USMLE step 1 created a wider distribution of scores with a negative skewness, suggesting there were more applicants below the mean than above. After removing those points, applicants' scores had a narrower distribution and skewness closer to 0, showing fewer upper outliers and more applicants near the mean. Conclusions The USMLE Step 1 score significantly affected the evaluation of applicants, and the removal of it from the recruitment criteria tightened applicant rankings. The elimination of the USMLE Step 1 score in the assessment of applicants will allow for its replacement with variables that better reflect the core values of residency programs.

2020 family medicine postgraduate examinations at The University of the West Indies: successes and challenges in the time of COVID-19 pandemic.

Little has been published regarding postgraduate assessments during the COVID-19 pandemic. There is an urgent need to graduate well-trained specialists including family physicians who play a key role in patient care. The successes and challenges encountered in mounting qualifying 2020 Family Medicine examinations during the COVID-19 pandemic at the University of the West Indies are described in this paper. Human resource, planning, use of technology and virtual environments are discussed, which enabled successful examinations at this multicampus regional site.

Knowledge towards COVID-19 among healthcare students in the central region of Saudi Arabia: a cross-sectional observational study.

STUDY PURPOSE: Coronavirus disease 2019 (COVID-19) is an infectious illness of high public concern. Healthcare students are directly or indirectly exposed to this disease. This study aimed to evaluate the knowledge of healthcare students in the central region of Saudi Arabia. STUDY DESIGN: A cross-sectional survey was planned to collect information from healthcare students living in the central region of Saudi Arabia. A questionnaire was formulated to evaluate the healthcare student's knowledge towards COVID-19 pandemic. The collected data were analysed by using frequencies of correct knowledge answers. RESULTS: A total of 612 students were analysed for the study. The overall knowledge score of healthcare students was 65.7%. The regression analysis showed a significant association between academic year and knowledge score p<0.001. CONCLUSIONS: Our analysis demonstrates satisfactory student's knowledge but lacks awareness about background and mode of transmission of this disease; therefore, there is a strong need for further education and training programmes.

More Than One-third of Orthopaedic Applicants Are in the Top 10%: The Standardized Letter of Recommendation and Evaluation of Orthopaedic Resident Applicants.

BACKGROUND: The American Orthopaedic Association (AOA) released the standardized letter of recommendation (SLOR) form to provide standardized information to evaluators of orthopaedic residency applicants. The SLOR associates numerical data to an applicant's letter of recommendation. However, it remains unclear whether the new letter form effectively distinguishes among orthopaedic applicants, for whom letters are perceived to suffer from "grade inflation." In addition, it is unknown whether letters from more experienced faculty members differ in important ways from those written by less experienced faculty. QUESTIONS/PURPOSES: (1) What proportion of SLOR recipients were rated in the top 10th percentile and top one-third of the applicant pool? (2) Did letters from program leaders (program directors and department chairs) demonstrate lower aggregate SLOR scores compared with letters written by other faculty members? (3) Did letters from away rotation program leaders demonstrate lower aggregate SLOR scores compared with letters written by faculty at the applicant's home institution? METHODS: This was a retrospective, single institution study examining 559 applications from the 2018 orthopaedic match. Inclusion criteria were all applications submitted to this residency. Exclusion criteria included all letters without an associated SLOR. In all, 1852 letters were received; of these, 26% (476) were excluded, and 74% (1376) were analyzed for SLOR data. We excluded 12% (169 of 1376) of letters that did not include a final summative score. Program leaders were defined as orthopaedic chairs and program directors. Away rotation letters were defined as letters written by faculty during an applicant's away rotation. Our study questions were answered accounting for each subcategory on the SLOR (scale 1-10) and the final ranking (scale 1-5) to form an aggregated score from the SLOR form for each letter. All SLOR questions were included in the creation of these scores. Correlations between program leaders and other faculty letter writers were assessed using a chi-square test. We considered a 1-point difference on 5-point scales to be a clinically important difference and a 2-point difference on 10-point scales to be clinically important. RESULTS: We found that 36% (437 of 1207) of the letters we reviewed indicated the candidate was in the top 10th percentile of all applicants evaluated, and 51% (619 of 1207) of the letters we reviewed indicated the candidate was in the top one-third of all applicants evaluated. We found no clinically important difference between program leaders and other faculty members in terms of summative scores on the SLOR (1.9 ± 0.7 versus 1.7 ± 0.7, mean difference -0.2 [95% CI -0.3 to 0.1]; p < 0.001). We also found no clinically important difference between home program letter writers and away program letter writers in terms of the mean summative scores (1.9 ± 0.7 versus 1.7 ± 0.7, mean difference 0.2; p < 0.001). CONCLUSION: In light of these discoveries, programs should examine the data obtained from SLOR forms carefully. SLOR scores skew very positively, which may benefit weaker applicants and harm stronger applicants. Program leaders give summative scores that do not differ substantially from junior faculty, suggesting there is no important difference in grade inflation between these faculty types, and as such, there is no strong need to adjust scores by faculty level. Likewise, away rotation letter writers' summative scores were not substantially different from those of home institution letters writers, indicating that there is no need to adjust scores between these groups either. Based on these findings, we should interpret letters with the understanding that overall there is substantial grade inflation. However, while weight used to be given to letters written by senior faculty members and those obtained on away rotations, we should now examine them equally, rather than trying to adjust them for overly high or low scores. LEVEL OF EVIDENCE: Level III, therapeutic study.

An online educational model in andrology for student training in the art of scientific writing in the COVID-19 pandemic.

In 2020, the COVID-19 pandemic led to the suspension of the annual Summer Internship at the American Center for Reproductive Medicine (ACRM). To transit it into an online format, an inaugural 6-week 2020 ACRM Online Mentorship Program was developed focusing on five core pillars of andrology research: scientific writing, scientific methodology, plagiarism understanding, soft skills development and mentee basic andrology knowledge. This study aims to determine mentee developmental outcomes based on student surveys and discuss these within the context of the relevant teaching and learning methodology. The mentorship was structured around scientific writing projects established by the team using a student-centred approach, with one-on-one expert mentorship through weekly formative assessments. Furthermore, weekly online meetings were conducted, including expert lectures, formative assessments and social engagement. Data were collected through final assessments and mentee surveys on mentorship outcomes. Results show that mentees (n = 28) reported a significant (p < .0001) improvement in all criteria related to the five core pillars. These results illustrate that the aims of the online mentorship program were achieved through a unique and adaptive online educational model and that our model has demonstrated its effectiveness as an innovative structured educational experience through the COVID-19 crisis.

Factors Associated with Acceptance of Repeat MCAT Test Takers into a West Virginian Allopathic Medical School.

OBJECTIVES: A paucity of information exists to advise medical school applicants who have had to retake the Medical College Admission Test (MCAT) to achieve a competitive score. To better advise repeat test takers from West Virginia and other Appalachian and southern areas, MCAT data from West Virginia applicants were analyzed and compared with national data. METHODS: In the application cycles of 2017-2020, the following factors were analyzed in relation to medical school acceptance in West Virginia applicants: MCAT scores, the number of test-taking attempts, biology-chemistry-physics-math grade point average, time between test-taking attempts, and academic major. MCAT data from medical school applicants from West Virginia who took the test more than once also were compared with national data. RESULTS: Of the total repeat test takers from West Virginia (N = 285) in the study timeframe, 57 (20%) were ultimately accepted into medical school. Factors associated with medical school acceptance were as follows: first MCAT test score (odds ratio [OR] 1.3, 95% confidence level [CL] 1.2-1.4, P < 0.001), change in MCAT test score (OR 1.2, 95% CL 1.1-1.3, P = 0.0015), and biology-chemistry-physics-math grade point average (OR 15.1, 95% CL 4.2-54.8, P < 0.0001). The highest benefit for improved scores occurred between the first and second attempts. The highest point gain occurred when the first MCAT score was in the range of 477 to 487 (<1st-12th percentile); this finding was not found in the national data. CONCLUSIONS: Although the study was limited to West Virginia medical school applicants, this information could prove useful in advising premedical applicants from other Appalachian and southern US areas.