Education curriculum assessment for teaching electrocardiography: Rationale and design for the prospective, international, randomized controlled EDUCATE trial.

BACKGROUND: Electrocardiogram (ECG) interpretation training is a fundamental component of medical education across disciplines. However, the skill of interpreting ECGs is not universal among medical graduates, and numerous barriers and challenges exist in medical training and clinical practice. An evidence-based and widely accessible learning solution is needed. DESIGN: The EDUcation Curriculum Assessment for Teaching Electrocardiography (EDUCATE) Trial is a prospective, international, investigator-initiated, open-label, randomized controlled trial designed to determine the efficacy of self-directed and active-learning approaches of a web-based educational platform for improving ECG interpretation proficiency. Target enrollment is 1000 medical professionals from a variety of medical disciplines and training levels. Participants will complete a pre-intervention baseline survey and an ECG interpretation proficiency test. After completion, participants will be randomized into one of four groups in a 1:1:1:1 fashion: (i) an online, question-based learning resource, (ii) an online, lecture-based learning resource, (iii) an online, hybrid question- and lecture-based learning resource, or (iv) a control group with no ECG learning resources. The primary endpoint will be the change in overall ECG interpretation performance according to pre- and post-intervention tests, and it will be measured within and compared between medical professional groups. Secondary endpoints will include changes in ECG interpretation time, self-reported confidence, and interpretation accuracy for specific ECG findings. CONCLUSIONS: The EDUCATE Trial is a pioneering initiative aiming to establish a practical, widely available, evidence-based solution to enhance ECG interpretation proficiency among medical professionals. Through its innovative study design, it tackles the currently unaddressed challenges of ECG interpretation education in the modern era. The trial seeks to pinpoint performance gaps across medical professions, compare the effectiveness of different web-based ECG content delivery methods, and create initial evidence for competency-based standards. If successful, the EDUCATE Trial will represent a significant stride towards data-driven solutions for improving ECG interpretation skills in the medical community.

Validity of a cardiology fellow performance assessment: reliability and associations with standardized examinations and awards.

BACKGROUND: Most work on the validity of clinical assessments for measuring learner performance in graduate medical education has occurred at the residency level. Minimal research exists on the validity of clinical assessments for measuring learner performance in advanced subspecialties. We sought to determine validity characteristics of cardiology fellows' assessment scores during subspecialty training, which represents the largest subspecialty of internal medicine. Validity evidence included item content, internal consistency reliability, and associations between faculty-of-fellow clinical assessments and other pertinent variables. METHODS: This was a retrospective validation study exploring the domains of content, internal structure, and relations to other variables validity evidence for scores on faculty-of-fellow clinical assessments that include the 10-item Mayo Cardiology Fellows Assessment (MCFA-10). Participants included 7 cardiology fellowship classes. The MCFA-10 item content included questions previously validated in the assessment of internal medicine residents. Internal structure evidence was assessed through Cronbach's α. The outcome for relations to other variables evidence was overall mean of faculty-of-fellow assessment score (scale 1-5). Independent variables included common measures of fellow performance. FINDINGS: Participants included 65 cardiology fellows. The overall mean ± standard deviation faculty-of-fellow assessment score was 4.07 ± 0.18. Content evidence for the MCFA-10 scores was based on published literature and core competencies. Cronbach's α was 0.98, suggesting high internal consistency reliability and offering evidence for internal structure validity. In multivariable analysis to provide relations to other variables evidence, mean assessment scores were independently associated with in-training examination scores (beta = 0.088 per 10-point increase; p = 0.05) and receiving a departmental or institutional award (beta = 0.152; p = 0.001). Assessment scores were not associated with educational conference attendance, compliance with completion of required evaluations, faculty appointment upon completion of training, or performance on the board certification exam. R2 for the multivariable model was 0.25. CONCLUSIONS: These findings provide sound validity evidence establishing item content, internal consistency reliability, and associations with other variables for faculty-of-fellow clinical assessment scores that include MCFA-10 items during cardiology fellowship. Relations to other variables evidence included associations of assessment scores with performance on the in-training examination and receipt of competitive awards. These data support the utility of the MCFA-10 as a measure of performance during cardiology training and could serve as the foundation for future research on the assessment of subspecialty learners.

Autonomy and professional identity formation in residency training: A qualitative study.

CONTEXT: Professional identity formation (PIF) involves the development of professional values, actions and aspirations and is central to medical education. Current understanding of PIF is informed by psychological and sociological theories. One ideal of medicine is responsibility to the patient; the development of this ideal is related to learner autonomy. The purpose of this study was to further theorise the relationships between the individual and contextual aspects of PIF through exploring the associations amongst autonomy, making decisions and responsibility for patients during residency training. METHODS: Using constructivist grounded theory, we conducted 23 semi-structured interviews with internal medicine residents at an academic medical centre. Interview transcripts were de-identified and processed through open coding and analytic memo writing. During data collection and analysis, we identified social cognitive theory (SCT), specifically reciprocal determinism, or the triadic and reciprocal relationship between context, person and behaviour, as a useful theoretical lens through which to illuminate the relationship between autonomy and PIF. Using SCT to guide analysis, we organised themes, identified relationships amongst themes, and refined them through group discussion and constant comparison with new data. RESULTS: Residents discussed three main themes: autonomy; making decisions, and responsibility for patient care. Autonomy allowed residents to feel personally responsible for patient care, and build confidence and trust. Autonomy allowed residents to feel engaged in making 'real decisions' for patient care. By displaying confidence in their decision making, residents were granted more autonomy. Lack of autonomy led to disengagement and lack of ownership over patient care. CONCLUSIONS: This study highlighted the role of autonomy in the PIF of learners. Exploring the tension between autonomy and supervision through the lens of SCT highlighted the interconnectedness of context, behaviour and identity in PIF. To ensure optimal learning environments for PIF, educators should weigh the need for autonomy against the demands for supervision and patient safety.

Effect of Pause Procedures on Participant Reflection and Commitment-to-Change in Continuing Medical Education.

Problem: Conferences are the most common form of continuing medical education (CME), but their effect on clinician practice is inconsistent. Reflection is a critical step in the process of practice change among clinicians and may lead to improved outcomes following conference-based CME. However, reflection requires time to process newly-learned material. Adequate time for reflection may be noticeably absent during many conference presentations. Intervention: The pause procedure is a 90-second 'pause' during a 30-minute presentation so learners can review and discuss content. The goal of the pause procedure is to stimulate learners' active engagement with newly learned material which will, in turn, promote learner reflection. Context: Fifty-six presentations at two hospital medicine CME conferences were assigned to the pause procedure or control. Study outcomes provided by conference participants were validated reflection scores and commitment-to-change (CTC) statements for each presentation. A post-hoc survey of the intervention group was conducted to assess presenters' experiences with the pause procedure. Impact: A total of 527 conference participants completed presentation evaluations (response rate 72.7%). Presentations incorporating the pause procedure failed to lead higher participant reflection scores (percentage 'top box' score; intervention: 39.2% vs. control: 41.7%, p = 0.40) or participant CTC rates (median [IQR]; intervention: 4.64 [3.04, 10.57] vs. control: 8.16 [5.28, 17.12], p = 0.13) than control presentations. However, the majority of presenters (16 out of 17 survey respondents) had never before used the intervention and little active engagement among learners was noted during the pause procedure. Lessons Learned: Adding the pause procedure to CME presentations did not lead to greater reflection or CTC among clinician learners. However, presenters had limited experience with the intervention, which may have reduced their fidelity to the educational principles of the pause procedure. Faculty development may be necessary when planning a new educational intervention that is to be implemented by conference presenters.

Characteristics of internal medicine residents who successfully match into cardiology fellowships.

BACKGROUND: The unique traits of residents who matriculate into subspecialty fellowships are poorly understood. We sought to identify characteristics of internal medicine (IM) residents who match into cardiovascular (CV) fellowships. METHODS: We conducted a retrospective cohort study of 8 classes of IM residents who matriculated into residency from 2007 to 2014. The primary outcome was successful match to a CV fellowship within 1 year of completing IM residency. Independent variables included residents' licensing exam scores, research publications, medical school reputation, Alpha Omega Alpha (AOA) membership, declaration of intent to pursue CV in the residency application personal statement, clinical evaluation scores, mini-clinical evaluation exercise scores, in-training examination (ITE) performance, and exposure to CV during residency. RESULTS: Of the 339 included residents (59% male; mean age 27) from 120 medical schools, 73 (22%) matched to CV fellowship. At the time of residency application, 104 (31%) had ≥1 publication, 38 (11%) declared intention to pursue CV in their residency application personal statement, and 104 (31%) were members of AOA. Prior to fellowship application, 111 (33%) completed a CV elective rotation. At the completion of residency training, 108 (32%) had ≥3 publications. In an adjusted logistic regression analysis, declaration of intention to pursue CV (OR 6.4, 99% CI 1.7-23.4; p < 0.001), completion of a CV elective (OR 7.3, 99% CI 2.8-19.0; p < 0.001), score on the CV portion of the PGY-2 ITE (OR 1.05, 99% CI 1.02-1.08; p < 0.001), and publication of ≥3 manuscripts (OR 4.7, 99% CI 1.1-20.5; p = 0.007) were positively associated with matching to a CV fellowship. Overall PGY-2 ITE score was negatively associated (OR 0.93, 99% CI 0.90-0.97; p < 0.001) with matching to a CV fellowship. CONCLUSIONS: Residents' matriculation into CV fellowships was associated with declaration of CV career intent, completion of a CV elective rotation, CV medical knowledge, and research publications during residency. These findings may be useful when advising residents about pursuing careers in CV. They may also help residents understand factors associated with a successful match to a CV fellowship. The negative association between matching into CV fellowship and overall ITE score may indicate excessive subspecialty focus during IM residency.

The relationship between learner engagement and teaching effectiveness: a novel assessment of student engagement in continuing medical education.

BACKGROUND: Continuing medical education (CME) often uses passive educational models including lectures. However, numerous studies have questioned the effectiveness of these less engaging educational strategies. Studies outside of CME suggest that engaged learning is associated with improved educational outcomes. However, measuring participants' engagement can be challenging. We developed and determined the validity evidence for a novel instrument to assess learner engagement in CME. METHODS: We conducted a cross-sectional validation study at a large, didactic-style CME conference. Content validity evidence was established through review of literature and previously published engagement scales and conceptual frameworks on engagement, along with an iterative process involving experts in the field, to develop an eight-item Learner Engagement Instrument (LEI). Response process validity was established by vetting LEI items on item clarity and perceived meaning prior to implementation, as well as using a well-developed online platform with clear instructions. Internal structure validity evidence was based on factor analysis and calculating internal consistency reliability. Relations to other variables validity evidence was determined by examining associations between LEI and previously validated CME Teaching Effectiveness (CMETE) instrument scores. Following each presentation, all participants were invited to complete the LEI and the CMETE. RESULTS: 51 out of 206 participants completed the LEI and CMETE (response rate 25%) Correlations between the LEI and the CMETE overall scores were strong (r = 0.80). Internal consistency reliability for the LEI was excellent (Cronbach's alpha = 0.96). To support validity to internal structure, a factor analysis was performed and revealed a two dimensional instrument consisting of internal and external engagement domains. The internal consistency reliabilities were 0.96 for the internal engagement domain and 0.95 for the external engagement domain. CONCLUSION: Engagement, as measured by the LEI, is strongly related to teaching effectiveness. The LEI is supported by robust validity evidence including content, response process, internal structure, and relations to other variables. Given the relationship between learner engagement and teaching effectiveness, identifying more engaging and interactive methods for teaching in CME is recommended.

Qualitative Research Methods in Medical Education.

Qualitative research was originally developed within the social sciences. Medical education is a field that comprises multiple disciplines, including the social sciences, and utilizes qualitative research to gain a broader understanding of key phenomena within the field. Many clinician educators are unfamiliar with qualitative research. This article provides a primer for clinician educators who want to appraise or conduct qualitative research in medical education. This article discusses a definition and the philosophical underpinnings for qualitative research. Using the Standards for Reporting Qualitative Research as a guide, this article provides a step-wise approach for conducting and evaluating qualitative research in medical education. This review will enable the reader to understand when to utilize qualitative research in medical education and how to interpret reports using qualitative approaches.

Quantitative Research Methods in Medical Education.

There has been a dramatic growth of scholarly articles in medical education in recent years. Evaluating medical education research requires specific orientation to issues related to format and content. Our goal is to review the quantitative aspects of research in medical education so that clinicians may understand these articles with respect to framing the study, recognizing methodologic issues, and utilizing instruments for evaluating the quality of medical education research. This review can be used both as a tool when appraising medical education research articles and as a primer for clinicians interested in pursuing scholarship in medical education.

A cross-sectional study of learning styles among continuing medical education participants.

PURPOSE: Experiential learning has been suggested as a framework for planning continuing medical education (CME). We aimed to (1) determine participants' learning styles at traditional CME courses and (2) explore associations between learning styles and participant characteristics. MATERIALS AND METHODS: Cross-sectional study of all participants (n = 393) at two Mayo Clinic CME courses who completed the Kolb Learning Style Inventory and provided demographic data. RESULTS: A total of 393 participants returned 241 surveys (response rate, 61.3%). Among the 143 participants (36.4%) who supplied complete demographic and Kolb data, Kolb learning styles included diverging (45; 31.5%), assimilating (56; 39.2%), converging (8; 5.6%), and accommodating (34; 23.8%). Associations existed between learning style and gender (p = 0.02). For most men, learning styles were diverging (23 of 63; 36.5%) and assimilating (30 of 63; 47.6%); for most women, diverging (22 of 80; 27.5%), assimilating (26 of 80; 32.5%), and accommodating (26 of 80; 32.5%). CONCLUSIONS: Internal medicine and psychiatry CME participants had diverse learning styles. Female participants had more variation in their learning styles than men. Teaching techniques must vary to appeal to all learners. The experiential learning theory sequentially moves a learner from Why? to What? to How? to If? to accommodate learning styles.

Validation of a clinical rotation evaluation for physician assistant students.

BACKGROUND: We conducted a prospective validation study to develop a physician assistant (PA) clinical rotation evaluation (PACRE) instrument. The specific aims of this study were to 1) develop a tool to evaluate PA clinical rotations, and 2) explore associations between validated rotation evaluation scores and characteristics of the students and rotations. METHODS: The PACRE was administered to rotating PA students at our institution in 2016. Factor analysis, internal consistency reliability, and associations between PACRE scores and student or rotation characteristics were determined. RESULTS: Of 206 PACRE instruments sent, 124 were returned (60.2% response). Factor analysis supported a unidimensional model with a mean (SD) score of 4.31 (0.57) on a 5-point scale. Internal consistency reliability was excellent (Cronbach α=0.95). PACRE scores were associated with students' gender (P = .01) and rotation specialty (P = .006) and correlated with students' perception of being prepared (r = 0.32; P < .001) and value of the rotation (r = 0.57; P < .001). CONCLUSIONS: This is the first validated instrument to evaluate PA rotation experiences. Application of the PACRE questionnaire could inform rotation directors about ways to improve clinical experiences. The findings of this study suggest that PA students must be adequately prepared to have a successful experience on their rotations. PA programs should consider offering transition courses like those offered in many medical schools to prepare their students for clinical experiences. Future research should explore whether additional rotation characteristics and educational outcomes are associated with PACRE scores.

Validation of a Teaching Effectiveness Assessment in Psychiatry Continuing Medical Education.

OBJECTIVE: Little is known about factors associated with effective continuing medical education (CME) in psychiatry. The authors aimed to validate a method to assess psychiatry CME teaching effectiveness and to determine associations between teaching effectiveness scores and characteristics of presentations, presenters, and participants. METHODS: This cross-sectional study was conducted at the Mayo Clinic Psychiatry Clinical Reviews and Psychiatry in Medical Settings. Presentations were evaluated using an eight-item CME teaching effectiveness instrument, its content based on previously published instruments. Factor analysis, internal consistency and interrater reliabilities, and temporal stability reliability were calculated. Associations were determined between teaching effectiveness scores and characteristics of presentations, presenters, and participants. RESULTS: In total, 364 participants returned 246 completed surveys (response rate, 67.6%). Factor analysis revealed a unidimensional model of psychiatry CME teaching effectiveness. Cronbach α for the instrument was excellent at 0.94. Item mean score (SD) ranged from 4.33 (0.92) to 4.71 (0.59) on a 5-point scale. Overall interrater reliability was 0.84 (95% CI, 0.75-0.91), and temporal stability was 0.89 (95% CI, 0.77-0.97). No associations were found between teaching effectiveness scores and characteristics of presentations, presenters, and participants. CONCLUSIONS: This study provides a new, validated measure of CME teaching effectiveness that could be used to improve psychiatry CME. In contrast to prior research in other medical specialties, CME teaching effectiveness scores were not associated with use of case-based or interactive presentations. This outcome suggests the need for distinctive considerations regarding psychiatry CME; a singular approach to CME teaching may not apply to all medical specialties.

Publication Rates of Abstracts Presented at the Society of General Internal Medicine Annual Meeting.

BACKGROUND: Abstracts accepted at scientific meetings are often not subsequently published. Data on publication rates are largely from subspecialty and surgical studies. OBJECTIVE: The aims of this study were to 1) determine publication rates of abstracts presented at a general internal medicine meeting; 2) describe research activity among academic general internists; 3) identify factors associated with publication and with the impact factor of the journal of publication; and 4) evaluate for publication bias. DESIGN: Retrospective cohort study. PARTICIPANTS: All scientific abstracts presented at the Society of General Internal Medicine 2009 Annual Meeting. MAIN MEASURES: Publication rates were determined by searching for full-text publications in MEDLINE. Data were abstracted regarding authors' institution, research topic category, number of study sites, sample size, study design, statistical significance (p value and confidence interval) in abstract and publication, journal of publication, publication date, and journal impact factor. KEY RESULTS: Of the 578 abstracts analyzed, 274 (47.4%) were subsequently published as a full article in a peer-reviewed journal indexed in MEDLINE. In a multivariable model adjusting for institution site, research topic, number of study sites, study design, sample size, and abstract results, publication rates for academic general internists were highest in the areas of medical education (52.5%, OR 5.05, 95% CI 1.57-17.25, reference group Veterans Affairs (VA)-based research, publication rate 36.7%), mental health/substance use (67.7%, OR 4.16, 95% CI 1.39-13.06), and aging/geriatrics/end of life (65.7%, OR 3.31, 95% CI 1.15-9.94, p = 0.01 across topics). Publication rates were higher for multicenter studies than single-institution studies (52.4% vs. 40.4%, OR 1.66, 95% CI 1.10-2.52, p = 0.04 across categories). Randomized controlled trials had higher publication rates than other study designs (66.7% vs. 45.9%, OR 2.72, 95% CI 1.30-5.94, p = 0.03 across study designs). Studies with positive results did not predict higher publication rates than negative studies (OR 0.89, 95% CI 0.6-1.31, p = 0.21). CONCLUSIONS: This study demonstrated that 47.4% of abstracts presented at a general internal medicine national conference were subsequently published in a peer-reviewed journal indexed in MEDLINE.

Use of the pause procedure in continuing medical education: A randomized controlled intervention study.

During lectures, a pause procedure (the presenter pauses so students can discuss content) can improve educational outcomes. We aimed to determine whether (1) continuing medical education (CME) presentations with a pause procedure were evaluated more favorably and (2) a pause procedure improved recall. In this randomized controlled intervention study of all participants (N = 214) at the Mayo Clinic Internal Medicine Board Review course, 48 lectures were randomly assigned to an intervention (pause procedure) or control (traditional lecture) group. The pause procedure was a 1-min pause at the middle and end of the presentation. Study outcomes were (1) presentation evaluation instrument scores and (2) number of recalled items per lecture. A total of 214 participants returned 145 surveys (response rate, 68%). Mean presentation evaluation scores were significantly higher for pause procedure than for traditional presentations (70.9% vs 65.8%; 95%CI for the difference, 3.5-6.7; p < .0001). Mean number of rapid recall items was higher for pause procedure presentations (0.68 vs 0.59; 95%CI for the difference, 0.02-0.14; p = .01). In a traditional CME course, presentations with a pause procedure had higher evaluation scores and more content was recalled. The pause procedure could arm CME presenters with an easy technique to improve educational content delivery.

Associations between teaching effectiveness and participant self-reflection in continuing medical education.

Effective medical educators can engage learners through self-reflection. However, little is known about the relationships between teaching effectiveness and self-reflection in continuing medical education (CME). We aimed to determine associations between presenter teaching effectiveness and participant self-reflection in conference-based CME. This cross-sectional study evaluated presenters and participants at a national CME course. Participants provided CME teaching effectiveness (CMETE) ratings and self-reflection scores for each presentation. Overall CMETE and CME self-reflection scores (five-point Likert scale with one as strongly disagree and five as strongly agree) were averaged for each presentation. Correlations were measured among self-reflection, CMETE, and presentation characteristics. In total, 624 participants returned 430 evaluations (response, 68.9%) for the 38 presentations. Correlation between CMETE and self-reflection was medium (Pearson correlation, 0.3-0.5) or large (0.5-1.0) for most presentations (n = 33, 86.9%). Higher mean (SD) CME reflection scores were associated with clinical cases (3.66 [0.12] vs. 3.48 [0.14]; p = 0.003) and audience response (3.66 [0.12] vs. 3.51 [0.14]; p = 0.005). To our knowledge, this is the first study to show a relationship between teaching effectiveness and participant self-reflection in conference-based CME. Presenters should consider using clinical cases and audience response systems to increase teaching effectiveness and promote self-reflection among CME learners.

A model of self-directed learning in internal medicine residency: a qualitative study using grounded theory.

BACKGROUND: Existing theories of self-directed learning (SDL) have emphasized the importance of process, personal, and contextual factors. Previous medical education research has largely focused on the process of SDL. We explored the experience with and perception of SDL among internal medicine residents to gain understanding of the personal and contextual factors of SDL in graduate medical education. METHODS: Using a constructivist grounded theory approach, we conducted 7 focus group interviews with 46 internal medicine residents at an academic medical center. We processed the data by using open coding and writing analytic memos. Team members organized open codes to create axial codes, which were applied to all transcripts. Guided by a previous model of SDL, we developed a theoretical model that was revised through constant comparison with new data as they were collected, and we refined the theory until it had adequate explanatory power and was appropriately grounded in the experiences of residents. RESULTS: We developed a theoretical model of SDL to explain the process, personal, and contextual factors affecting SDL during residency training. The process of SDL began with a trigger that uncovered a knowledge gap. Residents progressed to formulating learning objectives, using resources, applying knowledge, and evaluating learning. Personal factors included motivations, individual characteristics, and the change in approach to SDL over time. Contextual factors included the need for external guidance, the influence of residency program structure and culture, and the presence of contextual barriers. CONCLUSIONS: We developed a theoretical model of SDL in medical education that can be used to promote and assess resident SDL through understanding the process, person, and context of SDL.

Conference presentation to publication: a retrospective study evaluating quality of abstracts and journal articles in medical education research.

BACKGROUND: There is little evidence regarding the comparative quality of abstracts and articles in medical education research. The Medical Education Research Study Quality Instrument (MERSQI), which was developed to evaluate the quality of reporting in medical education, has strong validity evidence for content, internal structure, and relationships to other variables. We used the MERSQI to compare the quality of reporting for conference abstracts, journal abstracts, and published articles. METHODS: This is a retrospective study of all 46 medical education research abstracts submitted to the Society of General Internal Medicine 2009 Annual Meeting that were subsequently published in a peer-reviewed journal. We compared MERSQI scores of the abstracts with scores for their corresponding published journal abstracts and articles. Comparisons were performed using the signed rank test. RESULTS: Overall MERSQI scores increased significantly for published articles compared with conference abstracts (11.33 vs 9.67; P < .001) and journal abstracts (11.33 vs 9.96; P < .001). Regarding MERSQI subscales, published articles had higher MERSQI scores than conference abstracts in the domains of sampling (1.59 vs 1.34; P = .006), data analysis (3.00 vs 2.43; P < .001), and validity of evaluation instrument (1.04 vs 0.28; P < .001). Published articles also had higher MERSQI scores than journal abstracts in the domains of data analysis (3.00 vs 2.70; P = .004) and validity of evaluation instrument (1.04 vs 0.26; P < .001). CONCLUSIONS: To our knowledge, this is the first study to compare the quality of medical education abstracts and journal articles using the MERSQI. Overall, the quality of articles was greater than that of abstracts. However, there were no significant differences between abstracts and articles for the domains of study design and outcomes, which indicates that these MERSQI elements may be applicable to abstracts. Findings also suggest that abstract quality is generally preserved from original presentation to publication.

E-learning in graduate medical education: survey of residency program directors.

BACKGROUND: E-learning-the use of Internet technologies to enhance knowledge and performance-has become a widely accepted instructional approach. Little is known about the current use of e-learning in postgraduate medical education. To determine utilization of e-learning by United States internal medicine residency programs, program director (PD) perceptions of e-learning, and associations between e-learning use and residency program characteristics. METHODS: We conducted a national survey in collaboration with the Association of Program Directors in Internal Medicine of all United States internal medicine residency programs. RESULTS: Of the 368 PDs, 214 (58.2%) completed the e-learning survey. Use of synchronous e-learning at least sometimes, somewhat often, or very often was reported by 85 (39.7%); 153 programs (71.5%) use asynchronous e-learning at least sometimes, somewhat often, or very often. Most programs (168; 79%) do not have a budget to integrate e-learning. Mean (SD) scores for the PD perceptions of e-learning ranged from 3.01 (0.94) to 3.86 (0.72) on a 5-point scale. The odds of synchronous e-learning use were higher in programs with a budget for its implementation (odds ratio, 3.0 [95% CI, 1.04-8.7]; P = .04). CONCLUSIONS: Residency programs could be better resourced to integrate e-learning technologies. Asynchronous e-learning was used more than synchronous, which may be to accommodate busy resident schedules and duty-hour restrictions. PD perceptions of e-learning are relatively moderate and future research should determine whether PD reluctance to adopt e-learning is based on unawareness of the evidence, perceptions that e-learning is expensive, or judgments about value versus effectiveness.

Rating the Quality of Entrustable Professional Activities: Content Validation and Associations with the Clinical Context.

BACKGROUND: Entrustable professional activities (EPAs) have been developed to assess resident physicians with respect to Accreditation Council for Graduate Medical Education (ACGME) competencies and milestones. Although the feasibility of using EPAs has been reported, we are unaware of previous validation studies on EPAs and potential associations between EPA quality scores and characteristics of educational programs. OBJECTIVES: Our aim was to validate an instrument for assessing the quality of EPAs for assessment of internal medicine residents, and to examine associations between EPA quality scores and features of rotations. DESIGN: This was a prospective content validation study to design an instrument to measure the quality of EPAs that were written for assessing internal medicine residents. PARTICIPANTS: Residency leadership at Mayo Clinic, Rochester participated in this study. This included the Program Director, Associate program directors and individual rotation directors. INTERVENTIONS: The authors reviewed salient literature. Items were developed to reflect domains of EPAs useful for assessment. The instrument underwent further testing and refinement. Each participating rotation director created EPAs that they felt would be meaningful to assess learner performance in their area. These 229 EPAs were then assessed with the QUEPA instrument to rate the quality of each EPA. MAIN MEASURES: Performance characteristics of the QUEPA are reported. Quality ratings of EPAs were compared to the primary ACGME competency, inpatient versus outpatient setting and specialty type. KEY RESULTS: QUEPA tool scores demonstrated excellent reliability (ICC range 0.72 to 0.94). Higher ratings were given to inpatient versus outpatient (3.88, 3.66; p = 0.03) focused EPAs. Medical knowledge EPAs scored significantly lower than EPAs assessing other competencies (3.34, 4.00; p < 0.0001). CONCLUSIONS: The QUEPA tool is supported by good validity evidence and may help in rating the quality of EPAs developed by individual programs. Programs should take care when writing EPAs for the outpatient setting or to assess medical knowledge, as these tended to be rated lower.

Functional neuroimaging correlates of thinking flexibility and knowledge structure in memory: Exploring the relationships between clinical reasoning and diagnostic thinking.

BACKGROUND: Diagnostic reasoning involves the thinking steps up to and including arrival at a diagnosis. Dual process theory posits that a physician's thinking is based on both non-analytic or fast, subconscious thinking and analytic thinking that is slower, more conscious, effortful and characterized by comparing and contrasting alternatives. Expertise in clinical reasoning may relate to the two dimensions measured by the diagnostic thinking inventory (DTI): memory structure and flexibility in thinking. AIM: Explored the functional magnetic resonance imaging (fMRI) correlates of these two aspects of the DTI: memory structure and flexibility of thinking. METHODS: Participants answered and reflected upon multiple-choice questions (MCQs) during fMRI. A DTI was completed shortly after the scan. The brain processes associated with the two dimensions of the DTI were correlated with fMRI phases - assessing flexibility in thinking during analytical clinical reasoning, memory structure during non-analytical clinical reasoning and the total DTI during both non-analytical and analytical reasoning in experienced physicians. RESULTS: Each DTI component was associated with distinct functional neuroanatomic activation patterns, particularly in the prefrontal cortex. CONCLUSION: Our findings support diagnostic thinking conceptual models and indicate mechanisms through which cognitive demands may induce functional adaptation within the prefrontal cortex. This provides additional objective validity evidence for the use of the DTI in medical education and practice settings.

Associations between resident physicians' publications and clinical performance during residency training.

BACKGROUND: Both research and clinical medicine requires similar attributes of efficiency, diligence and effective teamwork. Furthermore, residents must succeed at scholarship and patient care to be competitive for fellowship training. It is unknown whether research productivity among residents is related to broad measures of clinical achievement. Our goal was to examine associations between the quantity of internal medicine residents' publications and validated measures of their knowledge, skills and multi-source evaluations of performance. METHODS: This was a longitudinal study of 308 residents graduating from Mayo Clinic from 2006 to 2012. We identified peer-reviewed articles in Ovid MEDLINE between July of each resident's match year and the end of their graduation. Outcomes included American Board of Internal Medicine (ABIM) certification examination scores, mini clinical examination (mini-CEX) scores, and validated assessments of clinical performance by resident-peers, faculty and non-physicians. Performance assessments were averaged to form an overall score ranging from 1 to 5. Associations between quantity of resident publications--and ABIM, mini-CEX and performance assessment scores--were determined using multivariate linear regression. RESULTS: The residents published 642 papers, of which 443 (69.0%) were research papers, 198 (30.8%) were case reports, and 380 (59.2%) were first-authored. On adjusted analysis, multi-source clinical performance evaluations were significantly associated (beta; 99% CI; p-value) with the numbers of research articles (0.012; 0.001-0.024; 0.007), and overall publications (0.012; 0.002-0.022; 0.002). CONCLUSIONS: To our knowledge, this is the first study to demonstrate that scholarly productivity based on journal publication is associated with clinical performance during residency training. Our findings suggest that residents who invest substantial efforts in research are not compromised in their abilities to learn medicine and care for patients.