A Guide for Increasing Scholarship for Medical Educators.

Disseminating scholarly work as a clinician educator is critical to furthering new knowledge in medical education, creating an evidence base for new practices, and increasing the likelihood of promotion. Knowing how to initiate scholarship and develop habits to support it, however, may not be clear. This perspective is designed to help readers choose and narrow their focus of scholarly interest, garner mentors, find potential project funding, and identify outside support through involvement with national organizations, collaborators, and faculty development programs. By incorporating these suggestions into their daily work, educators can find ways to connect their clinical and educational interests and make their daily work count toward scholarship.

Crowdsourcing for assessment items to support adaptive learning.

PURPOSE: Adaptive learning requires frequent and valid assessments for learners to track progress against their goals. This study determined if multiple-choice questions (MCQs) "crowdsourced" from medical learners could meet the standards of many large-scale testing programs. METHODS: Users of a medical education app (Osmosis.org, Baltimore, MD) volunteered to submit case-based MCQs. Eleven volunteers were selected to submit MCQs targeted to second year medical students. Two hundred MCQs were subjected to duplicate review by a panel of internal medicine faculty who rated each item for relevance, content accuracy, and quality of response option explanations. A sample of 121 items was pretested on clinical subject exams completed by a national sample of U.S. medical students. RESULTS: Seventy-eight percent of the 200 MCQs met faculty reviewer standards based on relevance, accuracy, and quality of explanations. Of the 121 pretested MCQs, 50% met acceptable statistical criteria. The most common reasons for exclusion were that the item was too easy or had a low discrimination index. CONCLUSIONS: Crowdsourcing can efficiently yield high-quality assessment items that meet rigorous judgmental and statistical criteria. Similar models may be adopted by students and educators to augment item pools that support adaptive learning.

The SGIM TEACH Program: A Curriculum for Teachers of Clinical Medicine.

BACKGROUND: Demand for faculty with teaching expertise is increasing as medical education is becoming well established as a career pathway. Junior faculty may be expected to take on teaching responsibilities with minimal training in teaching skills. AIM: To address the faculty development needs of junior clinician-educators with teaching responsibilities and those changing their career focus to include teaching. SETTING: Sessions at two Society of General Internal Medicine (SGIM) annual meetings combined with local coaching and online learning during the intervening year. PARTICIPANTS: Eighty-nine faculty scholars in four consecutive annual cohorts from 2013 to 2016. PROGRAM DESCRIPTION: Scholars participate in a full-day core teaching course as well as selective workshops at the annual meetings. Between meetings they receive direct observation and feedback on their teaching from a local coach and participate in an online discussion group. PROGRAM EVALUATION: Sessions were evaluated using a post-session survey. Overall content rating was 4.48 (out of 5). Eighty-nine percent of participants completed all requirements. Of these, 100% agreed that they had gained valuable knowledge and skills. DISCUSSION: The TEACH certificate program provides inexperienced faculty teachers an opportunity to develop core skills. Satisfaction is high. Future research should focus on the impact that this and similar programs have on teaching skills.

Testing and evaluation: the present and future of the assessment of medical professionals.

The aim of this review is to highlight recent and potential future enhancements to the United States Licensing Examination (USMLE) program. The USMLE program is co-owned by the National Board of Medical Examiners (NBME) and the Federation of State Medical Boards. The USMLE includes four examinations: Step 1, Step 2 Clinical Knowledge, Step 2 Clinical Skills, and Step 3; every graduate of Liaison Committee on Medical Education-accredited allopathic medical schools and all international medical graduates must pass this examination series to practice medicine in the United States. From 2006 to 2009, the program underwent an indepth review resulting in five accepted recommendations. These recommendations have been the primary driver for many of the recent enhancements, such as an increased emphasis on foundational science and changes in the clinical skills examination, including more advanced communication skills assessment. These recommendations will continue to inform future changes such as access to references (e.g., a map of metabolic pathways) or decision-making tools for use during the examination. The NBME also provides assessment services globally to medical schools, students, residency programs, and residents. In 2015, >550,000 assessments were provided through the subject examination program, NBME self-assessment services, and customized assessment services.

Medical Knowledge Assessment by Hematology and Medical Oncology In-Training Examinations Are Better Than Program Director Assessments at Predicting Subspecialty Certification Examination Performance.

The Accreditation Council for Graduate Medical Education's Next Accreditation System requires training programs to demonstrate that fellows are achieving competence in medical knowledge (MK), as part of a global assessment of clinical competency. Passing American Board of Internal Medicine (ABIM) certification examinations is recognized as a metric of MK competency. This study examines several in-training MK assessment approaches and their ability to predict performance on the ABIM Hematology or Medical Oncology Certification Examinations. Results of a Hematology In-Service Examination (ISE) and an Oncology In-Training Examination (ITE), program director (PD) ratings, demographic variables, United States Medical Licensing Examination (USMLE), and ABIM Internal Medicine (IM) Certification Examination were compared. Stepwise multiple regression and logistic regression analyses evaluated these assessment approaches as predictors of performance on the Hematology or Medical Oncology Certification Examinations. Hematology ISE scores were the strongest predictor of Hematology Certification Examination scores (ß = 0.41) (passing odds ratio [OR], 1.012; 95 % confidence interval [CI], 1.008-1.015), and the Oncology ITE scores were the strongest predictor of Medical Oncology Certification Examination scores (ß = 0.45) (passing OR, 1.013; 95 % CI, 1.011-1.016). PD rating of MK was the weakest predictor of Medical Oncology Certification Examination scores (ß = 0.07) and was not significantly predictive of Hematology Certification Examination scores. Hematology and Oncology ITEs are better predictors of certification examination performance than PD ratings of MK, reinforcing the effectiveness of ITEs for competency-based assessment of MK.

The relationship between performance on the Infectious Diseases In-Training and Certification Examinations.

BACKGROUND: The Infectious Diseases Society of America In-Training Examination (IDSA ITE) is a feedback tool used to help fellows track their knowledge acquisition during fellowship training. We determined whether the scores on the IDSA ITE and from other major medical knowledge assessments predict performance on the American Board of Internal Medicine (ABIM) Infectious Disease Certification Examination. METHODS: The sample was 1021 second-year fellows who took the IDSA ITE and ABIM Infectious Disease Certification Examination from 2008 to 2012. Multiple regression analysis was used to determine if ABIM Infectious Disease Certification Examination scores were predicted by IDSA ITE scores, prior United States Medical Licensing Examination (USMLE) scores, ABIM Internal Medicine Certification Examination scores, fellowship director ratings of medical knowledge, and demographic variables. Logistic regression was used to evaluate if these same assessments predicted a passing outcome on the certification examination. RESULTS: IDSA ITE scores were the strongest predictor of ABIM Infectious Disease Certification Examination scores (ß = .319), followed by prior ABIM Internal Medicine Certification Examination scores (ß = .258), USMLE Step 1 scores (ß = .202), USMLE Step 3 scores (ß = .130), and fellowship directors' medical knowledge ratings (ß = .063). IDSA ITE scores were also a significant predictor of passing the Infectious Disease Certification Examination (odds ratio, 1.017 [95% confidence interval, 1.013-1.021]). CONCLUSIONS: The significant relationship between the IDSA ITE score and performance on the ABIM Infectious Disease Certification Examination supports the use of the ITE as a valid feedback tool in fellowship training.

The impact of postgraduate training on USMLE® step 3® and its computer-based case simulation component.

BACKGROUND: The United States Medical Licensing Examination® (USMLE®) Step 3® examination is a computer-based examination composed of multiple choice questions (MCQ) and computer-based case simulations (CCS). The CCS portion of Step 3 is unique in that examinees are exposed to interactive patient-care simulations. OBJECTIVE: The purpose of the following study is to investigate whether the type and length of examinees' postgraduate training impacts performance on the CCS component of Step 3, consistent with previous research on overall Step 3 performance. DESIGN: Retrospective cohort study PARTICIPANTS: Medical school graduates from U.S. and Canadian institutions completing Step 3 for the first time between March 2007 and December 2009 (n = 40,588). METHODS: Post-graduate training was classified as either broadly focused for general areas of medicine (e.g. pediatrics) or narrowly focused for specific areas of medicine (e.g. radiology). A three-way between-subjects MANOVA was utilized to test for main and interaction effects on Step 3 and CCS scores between the demographic characteristics of the sample and type of residency. Additionally, to examine the impact of postgraduate training, CCS scores were regressed on Step 1 and Step 2 Clinical Knowledge (CK) scores. Residuals from the resulting regressions were plotted. RESULTS: There was a significant difference in CCS scores between broadly focused (µ = 216, σ = 17) and narrowly focused (µ=211, σ = 16) residencies (p < 0.001). Examinees in broadly focused residencies performed better overall and as length of training increased, compared to examinees in narrowly focused residencies. Predictors of Step 1 and Step 2 CK explained 55% of overall Step 3 variability and 9% of CCS score variability. CONCLUSIONS: Factors influencing performance on the CCS component may be similar to those affecting Step 3 overall. Findings are supportive of the validity of the Step 3 program and may be useful to program directors and residents in considering readiness to take this examination.

Gender differences in examinee performance on the Step 2 Clinical Skills data gathering (DG) and patient note (PN) components.

Multiple studies examining the relationship between physician gender and performance on examinations have found consistent significant gender differences, but relatively little information is available related to any gender effect on interviewing and written communication skills. The United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (CS) examination is a multi-station examination where examinees (physicians in training) interact with, and are rated by, standardized patients (SPs) portraying cases in an ambulatory setting. Data from a recent complete year (2009) were analyzed via a series of hierarchical linear models to examine the impact of examinee gender on performance on the data gathering (DG) and patient note (PN) components of this examination. Results from both components show that not only do women have higher scores on average, but women continue to perform significantly better than men when other examinee and case variables are taken into account. Generally, the effect sizes are moderate, reflecting an approximately 2% score advantage by encounter. The advantage for female examinees increased for encounters that did not require a physical examination (for the DG component only) and for encounters that involved a Women's Health issue (for both components). The gender of the SP did not have an impact on the examinee gender effect for DG, indicating a desirable lack of interaction between examinee and SP gender. The implications of the findings, especially with respect to the validity of the use of the examination outcomes, are discussed.

Colorectal cancer screening: patients' and physicians' perspectives on decision-making factors.

While patient barriers to colorectal cancer (CRC) screening have been identified, how well this knowledge is utilized during the patient-physician interaction is not fully understood. This study aims to assess among primary care physicians the degree of consensus between perceived and actual patients' CRC screening decision-making influential factors. During 2004-2006, 30 patients were interviewed to identify factors influencing screening decisions and 66 physicians were interviewed to understand what factors they thought were important to patients. The factors were categorized using the PRECEDE-PROCEED framework, and perspectives were compared. The researchers found little consensus on CRC screening decision-making influential factors between family practitioners, general internists, and patients. The recommendations to reach consensus are provided on the individual (e.g., updating the contents of a physician's screening recommendation to proactively address patients' decision-making needs) and population (e.g., providing cross-cultural training to medical students enabling them to better understand their patients) levels.

Which Internal Medicine Clerkship Characteristics Are Associated With Students' Performance on the NBME Medicine Subject Exam? A Multi-Institutional Analysis.

PURPOSE: To identify which internal medicine clerkship characteristics may relate to NBME Medicine Subject Examination scores, given the growing trend toward earlier clerkship start dates. METHOD: The authors used linear mixed effects models (univariable and multivariable) to determine associations between medicine exam performance and clerkship characteristics (longitudinal status, clerkship length, academic start month, ambulatory clinical experience, presence of a study day, involvement in a combined clerkship, preclinical curriculum type, medicine exam timing). Additional covariates included number of NBME clinical subject exams used, number of didactic hours, use of a criterion score for passing the medicine exam, whether medicine exam performance was used to designate clerkship honors, and United States Medical Licensing Examination Step 1 performance. The sample included 24,542 examinees from 62 medical schools spanning 3 academic years (2011-2014). RESULTS: The multivariable analysis found no significant association between clerkship length and medicine exam performance (all pairwise P > .05). However, a small number of examinees beginning their academic term in January scored marginally lower than those starting in July (P < .001). Conversely, examinees scored higher on the medicine exam later in the academic year (all pairwise P < .001). Examinees from schools that used a criterion score for passing the medicine exam also scored higher than those at schools that did not (P < .05). Step 1 performance remained positively associated with medicine exam performance even after controlling for all other variables in the model (P < .001). CONCLUSIONS: In this sample, the authors found no association between many clerkship variables and medicine exam performance. Instead, Step 1 performance was the most powerful predictor of medicine exam performance. These findings suggest that medicine exam performance reflects the overall medical knowledge students accrue during their education rather than any specific internal medicine clerkship characteristics.

A complementary and alternative medicine workshop using standardized patients improves knowledge and clinical skills of medical students.

OBJECTIVES: As the use of complementary and alternative medicine (CAM) has increased in the general population, so has the interest in CAM education among medical students and medical educators. The purpose of this study is to determine the impact of a CAM workshop using standardized patients (SP) on knowledge and clinical skills of third-year medical students. DESIGN: A 4-hour CAM workshop was developed as part of a new curriculum for a required third-year 4-week primary care internal medicine clerkship. The CAM workshop and 3 other novel workshops were randomized for delivery to half of the rotational groups. The CAM workshop incorporates 4 SP cases representing different clinical challenges. All students in every rotation group are assigned CAM readings. At the end of the rotation, all students take a 100-item written exam (7 CAM items) and 9-station SP exam (1 CAM station) including a post-SP encounter open-ended written exercise. Scores on the written exam CAM items, CAM SP checklist, and CAM open-ended written exercise of workshop participants and nonparticipants were analyzed with simple means, standard deviations, and multiple regression approaches. RESULTS: The CAM workshop was delivered to 12 of the 24 rotation groups during the 2004-2005 and 2005-2006 academic years. Ninety-two students participated in the workshop, and 94 did not. Workshop participants performed significantly better than nonparticipants on the CAM-specific SP checklist items (58 vs 36.6%, P<.0001), post-SP encounter written exercise (76.9 vs 63.3%, P<.0001), and 7 CAM written exam items (84.8 vs 76.3%, P<.0001). CONCLUSIONS: Students participating in a 4-hour SP workshop exhibit superior CAM knowledge as assessed by SP checklist, open-ended exercises, and multiple choice items. It appears that practice with SPs assists in acqusition and application of CAM knowledge and deferential counseling skills.

Modeling the decision to undergo colorectal cancer screening: insights on patient preventive decision making.

BACKGROUND: Little is known about how patients decide whether or not to undergo colorectal cancer screening. Although low screening rates evidence the outcome of these decisions, the processes patients use to balance benefits, risks, and costs with their own values and preferences are unclear. To increase screening rates, and ultimately save lives, it is important for providers to be aware of how patients make screening decisions. OBJECTIVES: The purpose of this study was to identify patterns of patient colorectal cancer screening decisions that might be supported by health care providers. POPULATION: In this study, we focused on people from Central Kentucky--a region with historically low screening rates. METHOD: We interviewed patients using a semi-structured format. Three members of the research team independently analyzed each interview transcript for factors that influenced the decision, and a pictorial representation of each decision process, based on Kurt Lewin's theory of decision making, was constructed for each participant. The individual decision processes were compared with identify patterns. RESULTS: Seventeen women and 13 men made up the sample. We discerned 7 decision patterns. CONCLUSIONS: This research documents 7 patterns and identifies common driving and restraining forces.

Sexual history inquiry and HIV counseling: improving clinical skills and medical knowledge through an interactive workshop utilizing standardized patients.

BACKGROUND: Sexual history and HIV counseling (SHHIVC) are essential clinical skills. Our project's purpose was to evaluate a standardized patient educational intervention teaching third-year medical students SHHIVC. METHODS: A four-hour standardized patient workshop was delivered to one-half of the class each of three consecutive years at one medical school. Approximately 3.5 weeks later, all students engaged in a standardized patient examination including one station assessing SHHIVC, answered an open-ended written exercise following the standardized patient encounter, and completed a written examination including sexual history and HIV-related questions. RESULTS: Workshop participants scored higher than non-participants on SHHIVC items on the standardized patient station (P < .0001), written exam (P < .0001), and open-ended written exercise after the standardized patient encounter (P = .024). CONCLUSIONS: Our SHHIVC curriculum was associated with students demonstrating better clinical skills on a SHHIVC standardized patient examination station and more SHHIVC knowledge on two measures of medical knowledge than students not participating in the SHHIVC educational intervention.

Performance on the Nephrology In-Training Examination and ABIM Nephrology Certification Examination Outcomes.

BACKGROUND AND OBJECTIVES: Medical specialty and subspecialty fellowship programs administer subject-specific in-training examinations to provide feedback about level of medical knowledge to fellows preparing for subsequent board certification. This study evaluated the association between the American Society of Nephrology In-Training Examination and the American Board of Internal Medicine Nephrology Certification Examination in terms of scores and passing status. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: The study included 1684 nephrology fellows who completed the American Society of Nephrology In-Training Examination in their second year of fellowship training between 2009 and 2014. Regression analysis examined the association between In-Training Examination and first-time Nephrology Certification Examination scores as well as passing status relative to other standardized assessments. RESULTS: This cohort included primarily men (62%) and international medical school graduates (62%), and fellows had an average age of 32 years old at the time of first completing the Nephrology Certification Examination. An overwhelming majority (89%) passed the Nephrology Certification on their first attempt. In-Training Examination scores showed the strongest association with first-time Nephrology Certification Examination scores, accounting for approximately 50% of the total explained variance in the model. Each SD increase in In-Training Examination scores was associated with a difference of 30 U (95% confidence interval, 27 to 33) in certification performance. In-Training Examination scores also were significantly associated with passing status on the Nephrology Certification Examination on the first attempt (odds ratio, 3.46 per SD difference in the In-Training Examination; 95% confidence interval, 2.68 to 4.54). An In-Training Examination threshold of 375, approximately 1 SD below the mean, yielded a positive predictive value of 0.92 and a negative predictive value of 0.50. CONCLUSIONS: American Society of Nephrology In-Training Examination performance is significantly associated with American Board of Internal Medicine Nephrology Certification Examination score and passing status.

Using decision tree models to depict primary care physicians CRC screening decision heuristics.

OBJECTIVE: The purpose of this study was to identify decision heuristics utilized by primary care physicians in formulating colorectal cancer screening recommendations. DESIGN: Qualitative research using in-depth semi-structured interviews. PARTICIPANTS: We interviewed 66 primary care internists and family physicians evenly drawn from academic and community practices. A majority of physicians were male, and almost all were white, non-Hispanic. APPROACH: Three researchers independently reviewed each transcript to determine the physician's decision criteria and developed decision trees. Final trees were developed by consensus. The constant comparative methodology was used to define the categories. RESULTS: Physicians were found to use 1 of 4 heuristics ("age 50," "age 50, if family history, then earlier," "age 50, if family history, then screen at age 40," or "age 50, if family history, then adjust relative to reference case") for the timing recommendation and 5 heuristics ["fecal occult blood test" (FOBT), "colonoscopy," "if not colonoscopy, then...," "FOBT and another test," and "a choice between options"] for the type decision. No connection was found between timing and screening type heuristics. CONCLUSIONS: We found evidence of heuristic use. Further research is needed to determine the potential impact on quality of care.

Physician colorectal cancer screening recommendations: an examination based on informed decision making.

OBJECTIVE: The purpose of this research was to examine the content of physicians' colorectal cancer screening recommendations. More specifically, using the framework of informed decision making synthesized by Braddock and colleagues, we conducted a qualitative study of the content of recommendations to describe how physicians are currently presenting this information to patients. METHODS: We conducted semi-structured interviews with 65 primary care physicians. We analyzed responses to a question designed to elicit how the physicians typically communicate their recommendation. RESULTS: Almost all of the physicians (98.5%) addressed the "nature of decision" element. A majority of physicians discussed "uncertainties associated with the decision" (67.7%). Fewer physicians covered "the patient's role in decision making" (33.8%), "risks and benefits" (16.9%), "alternatives" (10.8%), "assessment of patient understanding" (6.2%), or "exploration of patient's preferences" (1.5%). CONCLUSION: We propose that the content of the colorectal screening recommendation is a critical determinant to whether a patient undergoes screening. Our examination of physician recommendations yielded mixed results, and the deficiencies identified opportunities for improvement. PRACTICE IMPLICATIONS: We suggest primary care physicians clarify that screening is meant for those who are asymptotic, present tangible and intangible benefits and risks, as well as make a primary recommendation, and, if needed, a "compromise" recommendation, in order to increase screening utilization.