Assessing clinical competence: a multitrait-multimethod matrix construct validity study.

Education in Doctor of Medicine programs has moved towards an emphasis on clinical competency, with entrustable professional activities providing a framework of learning objectives and outcomes to be assessed within the clinical environment. While the identification and structured definition of objectives and outcomes have evolved, many methods employed to assess clerkship students' clinical skills remain relatively unchanged. There is a paucity of medical education research applying advanced statistical design and analytic techniques to investigate the validity of clinical skills assessment. One robust statistical method, multitrait-multimethod matrix analysis, can be applied to investigate construct validity across multiple assessment instruments and settings. Four traits were operationalized to represent the construct of critical clinical skills (professionalism, data gathering, data synthesis, and data delivery). The traits were assessed using three methods (direct observations by faculty coaches, clinical workplace-based evaluations, and objective structured clinical examination type clinical practice examinations). The four traits and three methods were intercorrelated for the multitrait-multimethod matrix analysis. The results indicated reliability values in the adequate to good range across the three methods with the majority of the validity coefficients demonstrating statistical significance. The clearest evidence for convergent and divergent validity was with the professionalism trait. The correlations on the same method/different traits analyses indicated substantial method effect; particularly on clinical workplace-based assessments. The multitrait-multimethod matrix approach, currently underutilized in medical education, could be employed to explore validity evidence of complex constructs such as clinical skills. These results can inform faculty development programs to improve the reliability and validity of assessments within the clinical environment.

The Impact of Internal Medicine Clerkship Characteristics and NBME Subject Exams on USMLE Step 2 Clinical Knowledge Exam Performance.

BACKGROUND: Residency program directors will likely emphasize the United States Medical Licensing Exam (USMLE) Step 2 clinical knowledge (CK) exam more during residency application given the recent USMLE Step 1 transition to pass/fail scoring. We examined how internal medicine clerkship characteristics and NBME subject exam scores affect USMLE Step 2 CK performance. DESIGN: The authors used univariable and multivariable generalized estimating equations to determine associations between Step 2 CK performance and internal medicine clerkship characteristics and NBME subject exams. The sample had 21,280 examinees' first Step 2 CK scores for analysis. RESULTS: On multivariable analysis, Step 1 performance (standardized ß = 0.45, p < .001) and NBME medicine subject exam performance (standardized ß = 0.40, p < .001) accounted for approximately 60% of the variance in Step 2 CK performance. Students who completed the internal medicine clerkship last in the academic year scored lower on Step 2 CK (Mdiff = -3.17 p < .001). Students who had a criterion score for passing the NBME medicine subject exam scored higher on Step 2 CK (Mdiff = 1.10, p = .03). There was no association between Step 2 CK performance and other internal medicine clerkship characteristics (all p > 0.05) nor with the total NBME subject exams completed (ß=0.05, p = .78). CONCLUSION: Despite similarities between NBME subject exams and Step 2 CK, the authors did not identify improved Step 2 CK performance for students who had more NBME subject exams. The lack of association of Step 2 CK performance with many internal medicine clerkship characteristics and more NBME subject exams has implications for future clerkship structure and summative assessment. The improved Step 2 CK performance in students that completed their internal medicine clerkship earlier warrants further study given the anticipated increase in emphasis on Step 2 CK.

Harm Reduction in Male Patients Actively Using Anabolic Androgenic Steroids (AAS) and Performance-Enhancing Drugs (PEDs): a Review.

Anabolic androgenic steroid (AAS) and performance-enhancing drug (PED) use is a prevalent medical issue, especially among men, with an estimated 2.9-4 million Americans using AAS in their lifetime. Prior studies of AAS use reveal an association with polycythemia, dyslipidemia, infertility, hypertension, left ventricular hypertrophy, and multiple behavioral disorders. AAS withdrawal syndrome, a state of depression, anhedonia, and sexual dysfunction after discontinuing AAS use, is a common barrier to successful cessation. Clinical resources for these patients and training of physicians on management of the patient using AAS are limited. Many men are hesitant to seek traditional medical care due to fear of judgment and lack of confidence in physician knowledge base regarding AAS. While proposed approaches to weaning patients off AAS are published, guidance on harm reduction for actively using patients remains sparse. Medical education regarding the management of AAS use disorder is paramount to improving care of this currently underserved patient population. Management of these patients must be non-judgmental and focus on patient education, harm reduction, and support for cessation. The approach to harm reduction should be guided by the specific AAS/PEDs used.

Impact of an Interprofessional Collaborative Quality Improvement Initiative to Decrease Inappropriate Thyroid Function Testing.

Background: Published data show that thyroid function laboratory tests are often ordered inappropriately in the acute care setting, which leads to unnecessary costs and inappropriate therapy decisions. Pilot data at our institution indicated that approximately two-thirds of the thyroid-stimulating hormone (TSH) laboratories were unnecessary, correlating to a potential cost avoidance of more than $20,000 annually. The purpose of this study was to improve the appropriateness of thyroid function test ordering with a multipronged initiative. Methodology: This controlled, single-center, before and after study included inpatients or emergency department (ED) patients at Wake Forest Baptist Medical Center who were at least 18 years of age and had a TSH level ordered during the study period. Patients with a history of thyroid cancer were excluded. The initiative included an electronic ordering intervention, direct education of providers (medical residents, attendings, and clinical pharmacists), and distribution of pocket information cards with appropriate ordering criteria. The primary outcome was the number and percentage of inappropriate TSH tests ordered before and after implementing the 3 interventions. Secondary outcomes included cost savings, inappropriate changes in thyroid therapy based on improperly ordered tests, and the number of free T4 lab tests ordered on patients with a TSH within the therapeutic range. Results: All 3 interventions were implemented, except for education of ED residents and faculty, who chose to forgo the direct education component. Inappropriate ordering of TSH levels decreased from 63 to 50 (13% reduction, P = .062) after implementation. Inappropriate TSH ordering decreased across all services, except in the ED. Inappropriate Free T4 orders decreased from 191 to 133 (30% reduction, P = .01). There were no therapy changes based on inappropriate TSH orders. Extrapolated annual cost savings were approximately $6,000. Conclusion: This multipronged interprofessional collaborative quality improvement initiative was associated with a nonstatistically significant reduction in inappropriate TSH orders, statistically significant reduction in inappropriate free T4 orders, and cost savings. There was a reduction in inappropriate ordering across all services except the ED, which may have been due the ED not participating in the direct education component of the initiative.

Association between glycemic control, age, and outcomes among intensively treated patients with acute myeloid leukemia.

PURPOSE: To investigate the impact of hyperglycemia and glycemic variability during intensive acute myeloid leukemia therapy (AML) on outcomes by age. METHODS: Retrospective study of 262 consecutive patients with newly diagnosed AML hospitalized for intensive induction. Hyperglycemia was assessed by mean blood glucose (BG) (mg/dL) during hospitalization and glycemic variability was determined by the standard deviation (SD) of mean BG. Outcomes were complete remission ± incomplete count recovery (CR + CRi), and overall survival (OS). We used logistic regression to evaluate CR + CRi, and Cox proportional hazard models for OS, stratified by age (< 60 vs ≥ 60 years). RESULTS: Older patients (N = 138, median age 70) had higher baseline comorbidity (CCI > 1 60.1% vs 25.8%) and prevalence of diabetes (20.3% vs 7.3%) compared to younger (N = 124, median age 47). The mean ± SD number of BG values obtained per patient during hospitalization was 61 ± 71. The mean (± SD) glucose (mg/dL) during hospitalization was 121.7 (25.9) in older patients (≥ 60 years) versus 111.6 (16.4) in younger. In older patients, higher mean glucose and greater glycemic variability were associated with lower odds of remission (OR 0.80, 95% CI 0.69-0.93 and OR 0.73, 95% CI 0.61-0.88 respectively, per 10-unit increase) and higher mortality rates (HR 1.13, 95% CI 1.05-1.21 and HR 1.17, 95% CI 1.09-1.26, respectively, per 10-unit increase) in multivariate analyses. CONCLUSIONS: Our observations that hyperglycemia and increased glycemic variability were associated with lower remission rates and increased mortality in older patients suggest glycemic control may be a potentially modifiable factor to improve AML outcomes.

Stress-Dosed Glucocorticoids and Mineralocorticoids Before Intensive Endurance Exercise in Primary Adrenal Insufficiency.

Patients with primary adrenal insufficiency (PAI) require increased doses of glucocorticoids and mineralocorticoids during stressors, such as surgery, trauma, and sepsis. Although current guidelines exist for dose adjustments in these situations, there is no accepted dosing regimen for patients with PAI participating in intensive endurance exercise. Given the extensive physiologic stress of events, such as marathons, triathlons, and similar events, it is likely that a "stress-dose" of adrenal replacement therapy will not only prevent adrenal crisis, but also improve performance. A 50-year-old male endurance athlete with known PAI reported severe fatigue, nausea, and malaise after competing in prior marathons and intensive endurance exercise. After supplementing with glucocorticoids and mineralocorticoids before competition, he experienced decreased symptoms and improved performance. To better care for these patients, further studies should be conducted to provide safe and effective glucocorticoid and mineralocorticoid dose adjustments before intensive endurance exercise.

Preclinical medical student observations associated with later professionalism concerns.

PURPOSE: Professionalism is a core physician competency and identifying students at risk for poor professional development early in their careers may allow for mentoring. This study identified indicators in the preclinical years associated with later professionalism concerns. METHODS: A retrospective analysis of observable indicators in the preclinical and clinical years was conducted using two classes of students (n = 226). Relationships between five potential indicators of poor professionalism in the preclinical years and observations related to professional concerns in the clinical years were analyzed. RESULTS: Fifty-three medical students were identified with at least one preclinical indicator and one professionalism concern during the clinical years. Two observable preclinical indicators were significantly correlated with unprofessional conduct during the clinical years: Three or more absences from attendance-required sessions (odds ratio 4.47; p=.006) and negative peer assessment (odds ratio 3.35; p=.049). CONCLUSIONS: We identified two significant observable preclinical indicators associated with later professionalism concerns: excessive absences and negative peer assessments. Early recognition of students at risk for future professionalism struggles would provide an opportunity for proactive professional development prior to the clinical years, when students' permanent records may be affected. Peer assessment, coupled with attention to frequent absences, may be a method to provide early recognition.

Multidisciplinary Endocrine Tumor Board: Assessment of the Patient Experience.

Research surrounding tumor boards has focused on patient outcomes and care coordination. Little is known about the patient experience with tumor boards. This survey examined aspects of the patient experience for patients presented at our multidisciplinary endocrine tumor board (ETB). A 15-item survey was distributed via the online patient portal to patients over the age of 18 whose case had been discussed at our ETB over an 18-month period. Descriptive statistics were reported, and a Fisher's exact test was used to examine relationships between variables. A total of 47 patients completed the survey (46%). A majority (72%) report their provider explained what the ETB is, and 77% report being informed their case would be discussed. Most patients were satisfied their case was being discussed (72%). A number of patients did report moderate or severe anxiety knowing their case was being discussed (15%). Sixty-four percent of patients report the ETB recommendations were clearly explained; however, satisfaction with the recommendations was slightly lower (53%). Despite the somewhat low satisfaction with the recommendations, 75% of patients felt more confident in their treatment plan knowing their case was discussed. Finally, if given the chance, 66% responded that they would have been interested in participating in their own ETB discussion. This study provides some insight into the patient experience surrounding tumor board discussions. Overall, patients are satisfied when their case is discussed at ETB. Patients can also experience anxiety about these discussions, and many patients desire to be present for their own discussions.

Primary Adrenal Insufficiency due to Cryptococcus With Persistent Adrenal Enlargement and Insufficiency.

Background/Objective: Infiltrative fungal infections are an unusual cause of primary adrenal insufficiency (AI). Our objective is to present a long-term follow-up of a patient with AI due to cryptococcal adrenalitis. Case Report: A 47-year-old woman presented in January 2004, with 50-lb weight loss, nausea, emesis, and headache with diplopia. During the 6 months prior to her presentation the patient had multiple admissions for evaluation of recurrent nausea and emesis. Prior to the most recent of these admissions, the patient developed a headache; evaluation of her cerebrospinal fluid revealed the presence of Cryptococcus, and she was treated with a 2-week course of amphotericin B. Physical examination demonstrated a temperature of 101.1 °F, heart rate of 110 bpm, and blood pressure of 94/65 mm Hg. She appeared ill and was underweight with dry mucous membranes and photophobia. Laboratory tests revealed random cortisol of 0.5 µg per dL. CT imaging showed bilateral adrenal gland enlargement and fine needle aspiration of the adrenal gland revealed encapsulated budding yeast. Stress dose intravenous glucocorticoids were administered and switched to oral hydrocortisone and fludrocortisone because the patient clinically improved with a second course of amphotericin B. Further evaluation in 2017 revealed persistently enlarged adrenal glands, positive cryptococcus antigen, and low IgG levels. Discussion: Our literature review noted few publications of AI caused by disseminated cryptococcus with no long-term follow-up of these cases beyond a 1- to 4-year time frame. Conclusion: Patients with AI due to disseminated fungal infection need long-term follow-up to assess for resolution of adrenal enlargement and evaluation of immunocompromised status.

Writing Is Thinking: Implementation and Evaluation of an Internal Medicine Residency Clinical Reasoning and Documentation Curriculum.

With increasingly complicated patients and faster throughput, time for thorough critical thinking and thoughtful clinical documentation is limited, especially in the training environment. Advocating for the value of clinical documentation as a robust opportunity for critical thinking, we describe the implementation and evaluation of a clinical reasoning and documentation curriculum for internal medicine residents. Our curriculum employed facilitated discussion, practical application, and a resident-as-teacher model. Resident surveys showed improved perceptions of the clinical and educational value of clinical documentation. Residents reported increased feedback to interns about their documentation and more appreciation of documentation as a venue for critical thinking. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-022-01570-5.

Prematriculation Healthcare Employment Predicts Success in Clerkship Environment.

PURPOSE: The average age of the matriculating medical student is increasing as more students take time between college and medical school. Increasing numbers of students are employed in the healthcare field during these gap years. Studies have explored the relationship between matriculation age and medical school performance with conflicting findings. The impact of prior healthcare employment (PHE) on future clerkship performance has not been explored. We hypothesize that medical school performance metrics would be higher for students with PHE than their peers. METHODS: A retrospective review of four medical school classes at a single institution was conducted. Each student's admission application was examined to identify students with at least 6 months paid employment in a clinical healthcare position (i.e., pre-matriculation direct patient interaction, PHE cohort). Multiple medical school performance metrics were obtained for each student. RESULTS: Of the 434 included students, 49 were PHE (11.29%) and percent of students with PHE trended up over time. MCAT scores, USMLE Step 1, and Step 2 CK scores were not different for PHE and non-PHE medical students. PHE students had significantly higher NBME subject exams, clinical clerkship scores, and cumulative year 3 performance. CONCLUSIONS: Students who matriculate to medical school with prior healthcare employment outperform their peers in the clinical environment, possibly due to acquisition of knowledge or skills from their prior employment. These findings support students seeking paid healthcare experiences prior to medical school and have implications for pre-medicine advising, admissions, and medical school curricula.

Does Clerkship Rotation Sequence Affect Performance on National Board of Medical Examiners (NBME) Clinical Subject Examinations and United States Medical Licensing Examination (USMLE) Step 2 Clinical Knowledge (CK) Examination?

PURPOSE: To investigate whether starting the clerkship year in family medicine (FM), internal medicine (IM), pediatrics, or surgery influences NBME shelf and USMLE Step 2 CK examination performance. METHODS: USMLE Step 1, Step 2 CK, and shelf examination scores for FM, IM, pediatrics, and surgery were collected. Sequences were selected on the following assignment criteria: rotation 1 (either FM or IM), rotation 5 (pediatrics), rotation 8 (surgery), rotation 1 (either pediatrics or surgery), and rotation 8 (IM). Multivariate analysis of covariance, with Step 1 as the covariate, was used to investigate rotation sequence on examination performance. RESULTS: Wilks's statistic found no statistically significant effect of rotation sequence (starting the clerkship year in FM or IM) on the pediatrics, surgery, and Step 2 CK examinations (Λ = .95, F[3,51] = .93, p ≤ .432). Wilk's statistic for the covariate (Step 1) was statistically significant (Λ = .488, F[3,51] = 17.827, p ≤ .001), indicating the two groups differ on Step 1 performance. Wilk's statistic found no statistically significant effect of rotation sequence (starting the clerkship year in pediatrics or surgery) on the IM and Step 2 CK results (Λ = .925, F[2,75] = 3.036, p ≤ .054). Wilk's statistic for the covariate (Step 1) was statistically significant (Λ = .309, F[2,75] = 83.915, p ≤ .001) indicating that the two groups differ on Step 1 performance. CONCLUSION: Starting the clerkship year in FM, IM, pediatrics, or surgery does not influence subsequent performance on shelf examinations or on Step 2 CK.

Musculoskeletal Basics: The Shoulder and the Knee Workshop for Primary Care Residents.

Introduction: Although musculoskeletal (MSK) complaints are very common in ambulatory clinics, internal medicine residents report low confidence in performing MSK examinations and intra-articular steroid injections. Our goal is to strengthen residents' MSK competence by using visual, auditory, and kinesthetic educational modalities during an academic half-day session. Methods: Our intervention was a workshop/academic half-day session that included multimodal educational materials on common shoulder and knee MSK complaints. The intervention featured a PowerPoint presentation highlighting three areas of MSK education: history taking, physical examination, and procedural skills. The curriculum contained a novel interactive charades game in which competitors demonstrated joint exams, an engaging anatomy quiz, and the performance of knee and subacromial bursae injections using interactive models and educational videos. Results: The effectiveness of our session was measured using a pre- and postsurvey. In our 2017 resident survey to determine the level of comfort in performing knee and shoulder physical examination and intra-articular injections, the majority of respondents perceived themselves as inadequately trained. After the session, confidence in performing knee and shoulder examinations and in performing knee and subacromial bursae injections increased significantly. In addition, 100% of the residents who attended and evaluated the session either agreed or strongly agreed that the experience improved MSK knowledge and employed effective teaching strategies. Discussion: The workshop/academic half-day session positively enhanced residents' perception of their knowledge of MSK medicine and their ability to perform joint injections.

3 for the Price of 1: Teaching Chest Pain Risk Stratification in a Multidisciplinary, Problem-based Learning Workshop.

INTRODUCTION: Chest pain is a common chief complaint among patients presenting to health systems and often leads to complex and intensive evaluations. While these patients are often cared for by a multidisciplinary team (primary care, emergency medicine, and cardiology), medical students usually learn about the care of these patients in a fragmented, single-specialty paradigm. The present and future care of patients with chest pain is multidisciplinary, and the education of medical students on the subject should be as well. Our objective was to evaluate the effectiveness of a multidisciplinary, problem-based learning workshop to teach third-year medical students about risk assessment for patients presenting with chest pain, specifically focusing on acute coronary syndromes. METHODS: To create an educational experience consistent with multidisciplinary team-based care, we designed a multidisciplinary, problem-based learning workshop to provide medical students with an understanding of how patients with chest pain are cared for in a systems-based manner to improve outcomes. Participants included third-year medical students (n=219) at a single, tertiary care, academic medical center. Knowledge acquisition was tested in a pre-/post-retention test study design. RESULTS: Following the workshop, students achieved a 19.7% (95% confidence interval [CI] [17.3-22.2%]) absolute increase in scores on post-testing as compared to pre-testing. In addition, students maintained an 11.1% (95% CI [7.2-15.0%]) increase on a retention test vs. the pre-test. CONCLUSION: A multidisciplinary, problem-based learning workshop is an effective method of producing lasting gains in student knowledge about chest pain risk stratification.