The Associations Between United States Medical Licensing Examination Performance and Outcomes of Patient Care.

PURPOSE: The United States Medical Licensing Examination (USMLE) comprises a series of assessments required for the licensure of U.S. MD-trained graduates as well as those who are trained internationally. Demonstration of a relationship between these examinations and outcomes of care is desirable for a process seeking to provide patients with safe and effective health care. METHOD: This was a retrospective cohort study of 196,881 hospitalizations in Pennsylvania over a 3-year period (January 1, 2017 to December 31, 2019) for 5 primary diagnoses: heart failure, acute myocardial infarction, stroke, pneumonia, or chronic obstructive pulmonary disease. The 1,765 attending physicians for these hospitalizations self-identified as family physicians or general internists. A converted score based on USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores was available, and the outcome measures were in-hospital mortality and log length of stay (LOS). The research team controlled for characteristics of patients, hospitals, and physicians. RESULTS: For in-hospital mortality, the adjusted odds ratio was 0.94 (95% confidence interval [CI] = 0.90, 0.99; P < .02). Each standard deviation increase in the converted score was associated with a 5.51% reduction in the odds of in-hospital mortality. For log LOS, the adjusted estimate was 0.99 (95% CI = 0.98, 0.99; P < .001). Each standard deviation increase in the converted score was associated with a 1.34% reduction in log LOS. CONCLUSIONS: Better provider USMLE performance was associated with lower in-hospital mortality and shorter log LOS for patients, although the magnitude of the latter is unlikely to be of practical significance. These findings add to the body of evidence that examines the validity of the USMLE licensure program.

Perception of Medical Student Mistreatment: Does Specialty Matter?

PURPOSE: Medical student mistreatment is pervasive, yet whether all physicians have a shared understanding of the problem is unclear. The authors presented professionally designed trigger videos to physicians from 6 different specialties to determine if they perceive mistreatment and its severity similarly. METHOD: From October 2016 to August 2018, resident and attending physicians from 10 U.S. medical schools viewed 5 trigger videos showing behaviors that could be perceived as mistreatment. They completed a survey exploring their perceptions. The authors compared perceptions of mistreatment across specialties and, for each scenario, evaluated the relationship between specialty and perception of mistreatment. RESULTS: Six-hundred fifty resident and attending physicians participated. There were statistically significant differences in perception of mistreatment across specialties for 3 of the 5 scenarios: aggressive questioning (range, 74.1%-91.2%), negative feedback (range, 25.4%-63.7%), and assignment of inappropriate tasks (range, 5.5%-25.5%) (P ≤ .001, for all). After adjusting for gender, race, professional role, and prior mistreatment, physicians in surgery viewed 3 scenarios (aggressive questioning, negative feedback, and inappropriate tasks) as less likely to represent mistreatment compared with internal medicine physicians. Physicians from obstetrics-gynecology and "other" specialties perceived less mistreatment in 2 scenarios (aggressive questioning and negative feedback), while family physicians perceived more mistreatment in 1 scenario (negative feedback) compared with internal medicine physicians. The mean severity of perceived mistreatment on a 1 to 7 scale (7 most serious) also varied statistically significantly across the specialties for 3 scenarios: aggressive questioning (range, 4.4-5.4; P < .001), ethnic insensitivity (range, 5.1-6.1; P = .001), and sexual harassment (range, 5.5-6.3; P = .004). CONCLUSIONS: Specialty was associated with differences in the perception of mistreatment and rating of its severity. Further investigation is needed to understand why these perceptions of mistreatment vary among specialties and how to address these differences.

The Flipped Classroom: An Innovative Approach to Medical Education in Ophthalmology.

OBJECTIVE: The aim of the study is to report the creation of a flipped ophthalmology course and preclinical medical student perceptions and knowledge gains before and after a flipped ophthalmology course. DESIGN: The form of the study discussed is an observational study. SUBJECTS: The subjects involved in the study are second-year (U.S.) United States medical students at the University of Miami, Miller School of Medicine (n = 401). METHODS: Second-year medical students participated in a 1-week "flipped classroom" ophthalmology course geared toward primary care providers at the University of Miami, Miller School of Medicine. Eleven hours of traditional classroom lectures were condensed into 4.5 hours of short videos with self-assessment quizzes, small group discussions, and a large group case-based discussion. Fifty-seven short videos (<9 minutes) focused on major ophthalmology topics and common conditions were viewed by the students at their leisure. Students completed a pre- and post-course evaluation on their perceptions and opinions of the flipped classroom approach. Final exam scores in the flipped classroom cohort were compared with the final exam scores in the traditional didactic format used in years prior. MAIN OUTCOME MEASURES: The main outcome measures include: student final exam performance; student satisfaction, opinions, and perceptions. RESULTS: Over the course of 2 years, 401 second-year U.S. medical students participated in the flipped classroom ophthalmology course. The majority of students enjoyed the flipped classroom experience (75.3%) and expressed interest in using the approach for future lessons (74.6%). The flipped classroom videos were preferred to live lectures (61.2%). Over 90% of students stated the self-assessment quizzes were useful, 79% reported that the small group discussions were an effective way to apply knowledge, and 76% cited the large group case-based discussion as useful. Pre-course knowledge assessment scores averaged 48%. Final examination scores in the flipped group (average ± standard deviation [SD] = 92.1% ± 6.1) were comparable to that of the traditional group when evaluating identical questions (average ± SD = 91.7% ± 5.54), p = 0.34. CONCLUSION: The flipped classroom approach proved to be a well-received and successful approach to preclinical medical education for ophthalmology. This was achieved using 35% less course time than our traditional course. This innovative approach has potential for expansion to other medical schools, medical education abroad, and for other medical school modules.

The Fallacy of Teaching and the Illusion of Learning: Improving Articulation of Basic Science in the Medical School Curriculum.

Despite calls from educators to re-engineer how faculty deliver medical student curricula with integrated basic science concepts, this content is still frequently disarticulated from other curricular components. We renewed our curriculum using evidence-based pedagogical and cognitive learning strategies to interleave basic science across the 4-year curriculum.

Teaching health advocacy to medical students: a comparison study.

CONTEXT: Many encourage service learning and health advocacy training in medical student education, but related evaluation is limited. OBJECTIVES: To assess (1) impact of a required community health advocacy training for medical students on student attitudes, knowledge, and skills; (2) student characteristics associated with higher advocacy knowledge and skills; and (3) perspectives of community-based organizations (CBOs). DESIGN: Cross-sectional surveys. SETTING: University of Miami Miller School of Medicine (UMMSM) Regional Medical Campus and main campus. PARTICIPANTS: Medical students at both UMMSM campuses. INTERVENTION: Required community health advocacy training for first- and second-year students including classroom experiences and hands-on project in partnership with a CBO. MAIN OUTCOME MEASURES: Student characteristics, health advocacy-related attitudes, self-reported and objective knowledge, and skills. Scores were compared between campuses, with multivariable modeling adjusting for individual student characteristics. Community-based organization perspectives were assessed via separate surveys. RESULTS: Ninety-eight (77%) regional campus students (intervention group) and 139 (30%) main campus students (comparison group) completed surveys. Versus the comparison group, the intervention group reported greater: mean knowledge of community health needs: 34.6 versus 31.1 (range: 11-44, P < .01), knowledge about CBOs: 3.0 versus 2.7 (range 1-4, P < .01) and knowledge of community resources: 5.4 versus 2.3 (range, 0-11, P < .01), and mean skill scores: 12.7 versus 10.5 (score range: 4-16, P < .01), following the intervention. Using adjusted analysis across both groups, female gender was associated with higher attitudes score. High level of previous community involvement was associated with higher attitude and skill scores. Higher self-reported educational debt was associated with higher skill scores. Community-based organization perspectives included high satisfaction and a desire to influence the training of future physicians. CONCLUSIONS: Medical student advocacy training in partnership with community-based organizations could be beneficial in improving student advocacy knowledge and skills in addressing community health issues and in developing sustainable community partnerships.

Decreasing the ceiling effect in assessing meeting quality at an academic professional meeting.

BACKGROUND: The psychometric properties of evaluations at academic meetings have not been well studied. PURPOSE: To explore the ceiling effect in the evaluation of quality of a professional meeting and whether a change in the scale labels would decrease the ceiling effect. METHODS: Cross-sectional study at two national meetings (2009-2010), attendees completed the evaluation on paper forms or online (5-point Likert scale). RESULTS: Of 1,064 evaluations, the mean session ratings was higher among respondents to the paper version in 2009 (4.2; 95% confidence interval [CI], 4.1 to 4.3) as compared to online responders in 2009 (3.0; 95% CI, 2.9 to 3.1) or online responders in 2010 (3.0; 95% CI, 2.9 to 3.1)(p < 0.001). CONCLUSION: A ceiling effect was present in the evaluation of an academic meeting. A change in the evaluation scale labels decreased the ceiling effect and increased evaluation variability.

An online spaced-education game to teach and assess medical students: a multi-institutional prospective trial.

PURPOSE: To investigate whether a spaced-education (SE) game can be an effective means of teaching core content to medical students and a reliable and valid method of assessing their knowledge. METHOD: This nine-month trial (2008-2009) enrolled students from three U.S. medical schools. The SE game consisted of 100 validated multiple-choice questions-explanations in preclinical/clinical domains. Students were e-mailed two questions daily. Adaptive game mechanics re-sent questions in three or six weeks if answered, respectively, incorrectly or correctly. Questions expired if not answered on time (appointment dynamic). Students retired questions by answering each correctly twice consecutively (progression dynamic). Posting of relative performance fostered competition. Main outcome measures were baseline and completion scores. RESULTS: Seven-hundred thirty-one students enrolled. Median baseline score was 53% (interquartile range [IQR] 16) and varied significantly by year (P<.001, dmax=2.08), school (P<.001, dmax=0.75), and gender (P<.001, d=0.38). Median completion score was 93% (IQR 12) and varied significantly by year (P=.001, dmax=1.12), school (P<.001, dmax=0.34), and age (P=.019, dmax=0.43). Scores did not differ significantly between years 3 and 4. Seventy percent of enrollees (513/731) requested to participate in future SE games. CONCLUSIONS: An SE game is an effective and well-accepted means of teaching core content and a reliable and valid method to assess student knowledge. SE games may be valuable tools to identify and remediate students who could benefit from additional educational support.

Analysis and publication rates of Clerkship Directors in Internal Medicine (CDIM) annual meeting abstracts 1995-2005.

BACKGROUND: Abstracts presented at meetings may be a reflection of the meeting's quality. SUMMARY: The goal is to determine purpose, content areas, research design, and subsequent publication rates of abstracts presented at Clerkship Directors in Internal Medicine's annual meetings. Abstracts presented in 1995-2005 were analyzed. A total of 201 abstracts were analyzed and coded. The purpose of the majority of the studies was description (155, 77%), 44 (22%) were justification studies, and 2 (1%) were clarification studies. In all, 109 (54%) assessed the relationship between teaching and student performance. Seven (4%) were studies about how students learn, study, solve problems, obtain medical knowledge, and think critically. Nineteen (10%) were studies about students' noncognitive skills, professionalism, interpersonal skills, and well-being studies. In all, 29 (14%) were measurement studies assessing reliability or validity of assessments. Twenty (10%) focused on career or faculty development pertaining to either faculty or residents. Research design of most studies was descriptive or pre-experimental (175, 87%), some were quasi-experimental (13, 7%), and two thirds (135, 66%) were not published as full articles in peer-reviewed journals. and there was an upward trend in publication rates. CONCLUSIONS: These findings may serve as an indicator of the quality of the educational meeting. It may further stimulate efforts to develop programs to help program attendees achieve scholarly publications for work presented and serve as a benchmark against which future meetings may be judged.

Competency-based education and training in internal medicine.

Recent efforts to improve medical education include adopting a new framework based on 6 broad competencies defined by the Accreditation Council for Graduate Medical Education. In this article, the Alliance for Academic Internal Medicine Education Redesign Task Force II examines the advantages and challenges of a competency-based educational framework for medical residents. Efforts to refine specific competencies by developing detailed milestones are described, and examples of training program initiatives using a competency-based approach are presented. Meeting the challenges of a competency-based framework and supporting these educational innovations require a robust faculty development program. Challenges to competency-based education include teaching and evaluating the competencies related to practice-based learning and improvement and systems-based practice, as well as implementing a flexible time frame to achieve competencies. However, the Alliance for Academic Internal Medicine Education Redesign Task Force II does not favor reducing internal medicine training to less than 36 months as part of competency-based education. Rather, the 36-month time frame should allow for remediation to address deficiencies in achieving competencies and for diverse enrichment experiences in such areas as quality of care and practice improvement for residents who have demonstrated skills in all required competencies.

Readdressing the need for consensus in preclinical education.

PURPOSE: Clinical skills are often taught in preclinical courses such as "Introduction to Clinical Medicine" (ICM). We sought to define current national course objectives, roles of standardized and real patients, and methods of assessment in ICM courses, and to readdress the need and desire for establishing consensus. METHODS: In April 2005, the Clerkship Directors in Internal Medicine (CDIM) organization conducted its annual, voluntary, and confidential survey. The survey contained a section (24 questions) on preclinical education teaching methods and evaluations. RESULTS: Wide variations exist in the teaching and assessment methods used by ICM courses in 4-year medical schools. Respondents indicated a desire for a consensus statement concerning ICM course objectives in addition to guidance on case development, use of standardized patients, and student assessments. CONCLUSION: Our findings suggest that consensus remains elusive for preclinical education teaching objectives, but would be embraced by ICM course directors.

Training and assessment of ECG interpretation skills: results from the 2005 CDIM survey.

BACKGROUND: Despite published consensus-based statements on assessment of ECG interpretation skills, studies and curricula regarding the training needed to obtain basic ECG interpretation skills are lacking. These consensus statements have focused on attaining competency in ECG interpretation during postgraduate training; however, recommendations regarding assessment of competency in the undergraduate curriculum are not discussed. PURPOSE: The purpose is to describe the current methods of teaching and assessing ECG interpretation skills across institutions on the 3rd-year internal medicine (IM) clerkship. METHOD: In 2005, the Clerkship Directors in Internal Medicine surveyed its institutional members. Twelve questions on the survey dealt with ECG interpretation. Descriptive statistics, chi-square, and Mann-Whitney U were used for analysis. RESULTS: Eighty-eight of 109 members (81%) responded to the survey. Overall, 89% of institutional respondents feel that ECG interpretation is an important clinical skill for medical students with 92% indicating that instruction occurs on the IM clerkship. Lectures (75%) and teaching rounds (44%) were the most cited methods of instruction. Most schools spend 1 to 6 hr during the IM clerkship on formal ECG instruction. Over 63% indicated that ECG interpretation skills are assessed during the clerkship. The most common assessment methods were written exam (40%) and OSCE (23%). CONCLUSIONS: Objective data regarding attainment and assessment of basic ECG interpretation competency in the undergraduate curriculum are lacking; our report provides preliminary descriptive data regarding ECG teaching and assessment on the 3rd-year IM clerkship. Further studies are needed to determine the ideal method of instruction and evaluation of this important clinical skill.

Procedural and interpretive skills of medical students: experiences and attitudes of fourth-year students.

BACKGROUND: Recent data do not exist regarding fourth-year medical students' performance of and attitudes toward procedural and interpretive skills, and how these differ from third-year students'. METHOD: Cross-sectional survey conducted in February 2006 of 122 fourth-year students from seven U.S. medical schools, compared with their responses in summer 2005. Students estimated their cumulative performance of 22 skills and reported self-confidence and perceived importance using a five-point Likert-type scale. RESULTS: The response rate was 79% (96/122). A majority reported never having performed cardioversion, thoracentesis, cardiopulmonary resuscitation, blood culture, purified protein derivative placement, or paracentesis. One fifth of students had never performed peripheral intravenous catheter insertion, phlebotomy, or arterial blood sampling. Students reported increased cumulative performance of 17 skills, increased self-confidence in five skills, and decreased perceived importance in three skills (two-sided P < .05). CONCLUSIONS: A majority of fourth-year medical students still have never performed important procedures, and a substantial minority have not performed basic procedures.

Complementary and alternative medicine use by primary care patients with chronic pain.

OBJECTIVES: To describe the characteristics and attitudes toward complementary and alternative medicine (CAM) use among primary care patients with chronic pain disorders and to determine if CAM use is associated with better pain control. DESIGN: Cross-sectional survey. SUBJECTS: Four hundred sixty-three patients suffering from chronic, nonmalignant pain receiving primary care at 12 U.S. academic medical centers. OUTCOME MEASURE: Self-reported current CAM usage by patients with chronic pain disorders. RESULTS: The survey had an 81% response rate. Fifty-two percent reported current use of CAM for relief of chronic pain. Of the patients that used CAM, 54% agreed that nontraditional remedies helped their pain and 14% indicated that their individual alternative remedy entirely relieved their pain. Vitamin and mineral supplements were the most frequently used CAM modalities. There was no association between reported use of CAM and pain severity, functional status, or perceived self-efficacy. Patients who reported having at least a high school education (odds ratio [OR] 1.1, 95% confidence interval [CI] 1.02-1.19, P = 0.016) and high levels of satisfaction with their health care (OR 1.47, 95% CI 1.13-1.91, P = 0.004) were significantly more likely to report using CAM. CONCLUSIONS: Complementary and alternative therapies were popular among patients with chronic pain disorders surveyed in academic primary care settings. When asked to choose between traditional therapies or CAM, most patients still preferred traditional therapies for pain relief. We found no association between reported CAM usage and pain severity, functional status, or self-efficacy.

Changes in clinical skills education resulting from the introduction of the USMLE step 2 clinical skills (CS) examination.

BACKGROUND: Step 2 Clinical Skills (CS) was recently introduced into the United States Medical Licensing Examination (USMLE) to ensure that successful candidates for licensure possess the clinical skills to provide safe and effective patient care. AIMS: To explore if medical schools had changed the objectives, content, or emphasis in their pre-clinical curriculum in response to its implementation. METHODS: In April 2005, the Clerkship Directors in Internal Medicine (CDIM) sent an electronic survey to a single member from each medical school with a CDIM member. The survey instrument included 26 pre-clinical curricular questions with nine questions specifically addressing changes in response to implementation of the Step 2 CS. RESULTS: Forty-five percent of respondents reported changes to the Introduction to Clinical Medicine (ICM) course objectives while 39% and 40% reported changes in content and emphasis. Seventy-four percent felt their students were adequately prepared for the Step 2 CS and 18% were unsure. CONCLUSIONS: Over a third of medical schools are implementing changes to the objectives, content, and/or emphasis of their curriculum, at least partially in response to the institution of Step 2 CS.

Evaluation, grading, and use of the RIME vocabulary on internal medicine clerkships: results of a national survey and comparison to other clinical clerkships.

BACKGROUND: Evaluation methods within and across clerkships are rapidly evolving, including greater emphasis or frameworks for descriptive evaluation and direct observation of competence. PURPOSE: The purpose of this study is to describe current evaluation methods, use of the Reporter-Interpreter-Manager/Educator (RIME) framework, and grade assignment by internal medicine clerkship directors. METHODS: In 2005, the Clerkship Directors in Internal Medicine surveyed its 109 institutional members. Topics included evaluation methods and grade contribution, use of evaluation sessions and/or RIME, and grade assignment (criterion referenced or normative). RESULTS: Response rate was 81% (88/109). The evaluation methods were as follows: teachers' evaluations, 93% (64% of grade); National Board of Medical Examiners subject examination, 81% (25% of grade); faculty written exam, 34% (14% of grade); objective structured clinical examinations, 32% (12% of grade); direct observation, 22% (7% of grade). RIME is used by 42% of respondents. Many clerkship directors (43%) meet with teachers to discuss student performance. Criterion-referenced grading is used by 59%, and normative grading is used by 27%. Unsatisfactory grades are given for examination failures (72%), unprofessional behavior (49%), poor clinical performance (42%), and failure to meet requirements (18%). CONCLUSIONS: Internal medicine clerkship directors emphasize description and observation of students. RIME and discussions with teachers are becoming commonplace.

Training and assessment of CXR/basic radiology interpretation skills: results from the 2005 CDIM Survey.

BACKGROUND: Despite published literature demonstrating deficiencies in chest radiograph (CXR)/basic radiology interpretation skills of 4th-year medical students, studies and subsequent curricula regarding the training needed to obtain these skills are lacking. Terms such as clinical exposure and radiology teaching have been used to describe the experience for these basic interpretive skills, but best practice methods of delivery, let alone common methods, have yet to be defined. PURPOSE: The objective is to describe the current methods of teaching and assessing CXR/basic radiology interpretation skills across institutions on the 3rd-year internal medicine (IM) clerkship. METHODS: In 2005, the Clerkship Directors in Internal Medicine (CDIM), an international organization representing U.S. and Canadian medical schools, surveyed its institutional members. Twelve questions on the survey dealt with X-ray interpretation. RESULTS: Eighty-eight of 109 members (81%) responded to the survey. Overall, 81% of respondents felt that CXR interpretation is an important clinical skill for medical students. Seventy-six percent indicated that instruction in these skills occurs on the IM clerkship. The most cited methods of instruction were lectures (56%) and teaching rounds (48%). Most schools spent on average of 2 to 4 hr during the IM clerkship on formal radiology instruction. Only 33% indicated that radiology interpretation skills are assessed during the clerkship. The most common assessment methods were written examination (19%) and OSCE (19%). CONCLUSION: Substantive data regarding attainment and assessment of CXR/basic radiology interpretation skills in the undergraduate curriculum are lacking. Our study provides preliminary descriptive data regarding CXR instruction and assessment on the 3rd-year IM clerkship.