Rating the Quality of Entrustable Professional Activities: Content Validation and Associations with the Clinical Context.

BACKGROUND: Entrustable professional activities (EPAs) have been developed to assess resident physicians with respect to Accreditation Council for Graduate Medical Education (ACGME) competencies and milestones. Although the feasibility of using EPAs has been reported, we are unaware of previous validation studies on EPAs and potential associations between EPA quality scores and characteristics of educational programs. OBJECTIVES: Our aim was to validate an instrument for assessing the quality of EPAs for assessment of internal medicine residents, and to examine associations between EPA quality scores and features of rotations. DESIGN: This was a prospective content validation study to design an instrument to measure the quality of EPAs that were written for assessing internal medicine residents. PARTICIPANTS: Residency leadership at Mayo Clinic, Rochester participated in this study. This included the Program Director, Associate program directors and individual rotation directors. INTERVENTIONS: The authors reviewed salient literature. Items were developed to reflect domains of EPAs useful for assessment. The instrument underwent further testing and refinement. Each participating rotation director created EPAs that they felt would be meaningful to assess learner performance in their area. These 229 EPAs were then assessed with the QUEPA instrument to rate the quality of each EPA. MAIN MEASURES: Performance characteristics of the QUEPA are reported. Quality ratings of EPAs were compared to the primary ACGME competency, inpatient versus outpatient setting and specialty type. KEY RESULTS: QUEPA tool scores demonstrated excellent reliability (ICC range 0.72 to 0.94). Higher ratings were given to inpatient versus outpatient (3.88, 3.66; p = 0.03) focused EPAs. Medical knowledge EPAs scored significantly lower than EPAs assessing other competencies (3.34, 4.00; p < 0.0001). CONCLUSIONS: The QUEPA tool is supported by good validity evidence and may help in rating the quality of EPAs developed by individual programs. Programs should take care when writing EPAs for the outpatient setting or to assess medical knowledge, as these tended to be rated lower.

Enhancing motivation with the "virtual" supervisory role: a randomized trial.

BACKGROUND: We aimed to explore the influence of a motivationally-enhanced instructional design on motivation to learn and knowledge, hypothesizing that outcomes would be higher for the enhanced instructional format. METHODS: Medicine residents completed four online learning modules on primary care topics. Using a crossover design, learners were randomized to receive two standard and two motivationally-enhanced learning modules. Both formats had self-assessment questions, but the enhanced format questions were framed to place learners in a supervisory/teaching role. Learners received a baseline motivation questionnaire, a short motivation survey before and after each module, and a knowledge posttest. RESULTS: One hundred twenty seven residents were randomized. 123 residents (97%) completed at least one knowledge posttest and 119 (94%) completed all four posttests. Across all modules, a one-point increase in the pretest short motivation survey was associated with a 2.1-point increase in posttest knowledge. The change in motivation was significantly higher for the motivationally enhanced format (standard mean change -0.01, enhanced mean change +0.09, difference = 0.10, CI 0.001 to 0.19; p = 0.048). Mean posttest knowledge score was similar (standard mean 72.8, enhanced mean 73.0, difference = 0.2, CI -1.9 to 2.1; p = 0.90). CONCLUSIONS: The motivationally enhanced instructional format improved motivation more than the standard format, but impact on knowledge scores was small and not statistically significant. Learners with higher pre-intervention motivation scored better on post-intervention knowledge tests, suggesting that motivation may prove a viable target for future instructional enhancements.

An evaluation of internal medicine residency continuity clinic redesign to a 50/50 outpatient-inpatient model.

BACKGROUND: There have been recent calls for improved internal medicine outpatient training, yet assessment of clinical and educational variables within existing models is lacking. OBJECTIVE: To assess the impact of clinic redesign from a traditional weekly clinic model to a 50/50 outpatient-inpatient model on clinical and educational outcomes. DESIGN: Pre-intervention and post-intervention study intervals, comparing the 2009-2010 and 2010-2011 academic years. PARTICIPANTS: Ninety-six residents in a Primary Care Internal Medicine site of a large academic internal medicine residency program who provide care for > 13,000 patients. INTERVENTION: Continuity clinic redesign from a traditional weekly clinic model to a 50/50 model characterized by 50 % outpatient and 50 % inpatient experiences scheduled in alternating 1 month blocks, with twice weekly continuity clinic during outpatient months and no clinic during inpatient months. MAIN MEASURES: 1) Clinical outcomes (panel size, patient visits, adherence with chronic disease and preventive service guidelines, continuity of care, patient satisfaction, and perceived safety/teamwork in clinic); 2) Educational outcomes (attendance at teaching conference, resident and faculty satisfaction, faculty assessment of resident clinic performance, and residents' perceived preparedness for outpatient management). RESULTS: Redesign was associated with increased mean panel size (120 vs. 137.6; p ≤ 0.001), decreased continuity of care (63 % vs. 48 % from provider perspective; 61 % vs. 51 % from patient perspective; p ≤ 0.001 for both; team continuity was preserved), decreased missed appointments (12.5 % vs. 10.9 %; p ≤ 0.01), improved perceived safety and teamwork (3.6 vs. 4.1 on 5-point scale; p ≤ 0.001), improved mean teaching conference attendance (57.1 vs. 64.4; p ≤ 0.001), improved resident clinic performance (3.6 vs. 3.9 on 5-point scale; p ≤ 0.001), and little change in other outcomes. CONCLUSION: Although this model requires further study in other settings, these results suggest that a 50/50 model may allow residents to manage more patients while enhancing the climate of teamwork and safety in the continuity clinic, compared to traditional models. Future work should explore ways to preserve continuity of care within this model.

Group assessments of resident physicians improve reliability and decrease halo error.

BACKGROUND: Individual faculty assessments of resident competency are complicated by inconsistent application of standards, lack of reliability, and the "halo" effect. OBJECTIVE: We determined whether the addition of faculty group assessments of residents in an ambulatory clinic, compared with individual faculty-of-resident assessments alone, have better reliability and reduced halo effects. DESIGN: This prospective, longitudinal study was performed in the outpatient continuity clinics of a large internal medicine residency program. MAIN MEASURES: Faculty-on-resident and group faculty-on-resident assessment scores were used for comparison. KEY RESULTS: Overall mean scores were significantly higher for group than individual assessments (3.92 ± 0.51 vs. 3.83 ± 0.38, p = 0.0001). Overall inter-rater reliability increased when combining group and individual assessments compared to individual assessments alone (intraclass correlation coefficient, 95% CI = 0.828, 0.785-0.866 vs. 0.749, 0.686-0.804). Inter-item correlations were less for group (0.49) than individual (0.68) assessments. CONCLUSIONS: This study demonstrates improved inter-rater reliability and reduced range restriction (halo effect) of resident assessment across multiple performance domains by adding the group assessment method to traditional individual faculty-on-resident assessment. This feasible model could help graduate medical education programs achieve more reliable and discriminating resident assessments.

The Motivated Strategies for Learning Questionnaire: score validity among medicine residents.

CONTEXT: The Motivated Strategies for Learning Questionnaire (MSLQ) purports to measure motivation using the expectancy-value model. Although it is widely used in other fields, this instrument has received little study in health professions education. OBJECTIVES: The purpose of this study was to evaluate the validity of MSLQ scores. METHODS: We conducted a validity study evaluating the relationships of MSLQ scores to other variables and their internal structure (reliability and factor analysis). Participants included 210 internal medicine and family medicine residents participating in a web-based course on ambulatory medicine at an academic medical centre. Measurements included pre-course MSLQ scores, pre- and post-module motivation surveys, post-module knowledge test and post-module Instructional Materials Motivation Survey (IMMS) scores. RESULTS: Internal consistency was universally high for all MSLQ items together (Cronbach's α = 0.93) and for each domain (α ≥ 0.67). Total MSLQ scores showed statistically significant positive associations with post-test knowledge scores. For example, a 1-point rise in total MSLQ score was associated with a 4.4% increase in post-test scores (ß = 4.4; p < 0.0001). Total MSLQ scores showed moderately strong, statistically significant associations with several other measures of effort, motivation and satisfaction. Scores on MSLQ domains demonstrated associations that generally aligned with our hypotheses. Self-efficacy and control of learning belief scores demonstrated the strongest domain-specific relationships with knowledge scores (ß = 2.9 for both). Confirmatory factor analysis showed a borderline model fit. Follow-up exploratory factor analysis revealed the scores of five factors (self-efficacy, intrinsic interest, test anxiety, extrinsic goals, attribution) demonstrated psychometric and predictive properties similar to those of the original scales. CONCLUSIONS: Scores on the MSLQ are reliable and predict meaningful outcomes. However, the factor structure suggests a simplified model might better fit the empiric data. Future research might consider how assessing and responding to motivation could enhance learning.

Relationship between critical reflection and quality improvement proposal scores in resident doctors.

OBJECTIVES: transformative learning theory supports the idea that reflection on quality improvement (QI) opportunities and the ability to develop successful QI projects may be fundamentally linked. We used validated methods to explore associations between resident doctors' reflections on QI opportunities and the quality of their QI project proposals. METHODS: eighty-six residents completed written reflections on practice improvement opportunities and developed QI proposals. Two faculty members assessed residents' reflections using the 18-item Mayo Evaluation of Reflection on Improvement Tool (MERIT), and assessed residents' QI proposals using the seven-item Quality Improvement Project Assessment Tool (QIPAT-7). Both instruments have been validated in previous work. Associations between MERIT and QIPAT-7 scores were determined. Internal consistency reliabilities of QIPAT-7 and MERIT scores were calculated. RESULTS: there were no significant associations between MERIT overall and domain scores, and QIPAT-7 overall and item scores. The internal consistency of MERIT and QIPAT-7 item groups were acceptable (Cronbach's α 0.76-0.94). CONCLUSIONS: the lack of association between MERIT and QIPAT-7 scores indicates a distinction between resident doctors' skills at reflection on QI opportunities and their abilities to develop QI projects. These findings suggest that practice-based reflection and QI project development are separate constructs, and that skilful reflection may not predict the ability to design meaningful QI initiatives. Future QI curricula should consider teaching and assessing QI reflection and project development as distinct components.

Validation of a method to measure resident doctors' reflections on quality improvement.

OBJECTIVES: Resident reflection on the clinical learning environment is prerequisite to identifying quality improvement (QI) opportunities and demonstrating competence in practice-based learning. However, residents' abilities to reflect on QI opportunities are unknown. Therefore, we developed and determined the validity of the Mayo Evaluation of Reflection on Improvement Tool (MERIT) for assessing resident reflection on QI opportunities. METHODS: The content of MERIT, which consists of 18 items structured on 4-point scales, was based on existing literature and input from national experts. Using MERIT, six faculty members rated 50 resident reflections. Factor analysis was used to examine the dimensionality of MERIT instrument scores. Inter-rater and internal consistency reliabilities were calculated. RESULTS: Factor analysis revealed three factors (eigenvalue; number of items): Reflection on Personal Characteristics of QI (8.5; 7); Reflection on System Characteristics of QI (1.9; 6), and Problem of Merit (1.5; 5). Inter-rater reliability was very good (intraclass correlation coefficient range: 0.73-0.89). Internal consistency reliability was excellent (Cronbach's alpha 0.93 overall and 0.83-0.91 for factors). Item mean scores were highest for Problem of Merit (3.29) and lowest for Reflection on System Characteristics of QI (1.99). CONCLUSIONS: Validity evidence supports MERIT as a meaningful measure of resident reflection on QI opportunities. Our findings suggest that dimensions of resident reflection on QI opportunities may include personal, system and Problem of Merit factors. Additionally, residents may be more effective at reflecting on 'problems of merit' than personal and systems factors.

Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial.

BACKGROUND: Mini-CEX scores assess resident competence. Rater training might improve mini-CEX score interrater reliability, but evidence is lacking. OBJECTIVE: Evaluate a rater training workshop using interrater reliability and accuracy. DESIGN: Randomized trial (immediate versus delayed workshop) and single-group pre/post study (randomized groups combined). SETTING: Academic medical center. PARTICIPANTS: Fifty-two internal medicine clinic preceptors (31 randomized and 21 additional workshop attendees). INTERVENTION: The workshop included rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion. Delayed group received no intervention until after posttest. MEASUREMENTS: Mini-CEX ratings at baseline (just before workshop for workshop group), and four weeks later using videotaped resident-patient encounters; mini-CEX ratings of live resident-patient encounters one year preceding and one year following the workshop; rater confidence using mini-CEX. RESULTS: Among 31 randomized participants, interrater reliabilities in the delayed group (baseline intraclass correlation coefficient [ICC] 0.43, follow-up 0.53) and workshop group (baseline 0.40, follow-up 0.43) were not significantly different (p = 0.19). Mean ratings were similar at baseline (delayed 4.9 [95% confidence interval 4.6-5.2], workshop 4.8 [4.5-5.1]) and follow-up (delayed 5.4 [5.0-5.7], workshop 5.3 [5.0-5.6]; p = 0.88 for interaction). For the entire cohort, rater confidence (1 = not confident, 6 = very confident) improved from mean (SD) 3.8 (1.4) to 4.4 (1.0), p = 0.018. Interrater reliability for ratings of live encounters (entire cohort) was higher after the workshop (ICC 0.34) than before (ICC 0.18) but the standard error of measurement was similar for both periods. CONCLUSIONS: Rater training did not improve interrater reliability or accuracy of mini-CEX scores. CLINICAL TRIALS REGISTRATION: clinicaltrials.gov identifier NCT00667940

Alternative approaches to ambulatory training: internal medicine residents' and program directors' perspectives.

BACKGROUND: Internal medicine ambulatory training redesign, including recommendations to increase ambulatory training, is a focus of national discussion. Residents' and program directors' perceptions about ambulatory training models are unknown. OBJECTIVE: To describe internal medicine residents' and program directors' perceptions regarding ambulatory training duration, alternative ambulatory training models, and factors important for ambulatory education. DESIGN: National cohort study. PARTICIPANTS: Internal medicine residents (N = 14,941) and program directors (N = 222) who completed the 2007 Internal Medicine In-Training Examination (IM-ITE) Residents Questionnaire or Program Directors Survey, representing 389 US residency programs. RESULTS: A total of 58.4% of program directors and 43.7% of residents preferred one-third or more training time in outpatient settings. Resident preferences for one-third or more outpatient training increased with higher levels of training (48.3% PGY3), female sex (52.7%), primary care program enrollment (64.8%), and anticipated outpatient-focused career, such as geriatrics. Most program directors (77.3%) and residents (58.4%) preferred training models containing weekly clinic. Although residents and program directors reported problems with competing inpatient-outpatient responsibilities (74.9% and 88.1%, respectively) and felt that absence of conflict with inpatient responsibilities is important for good outpatient training (69.4% and 74.2%, respectively), only 41.6% of residents and 22.7% of program directors supported models eliminating ambulatory sessions during inpatient rotations. CONCLUSIONS: Residents' and program directors' preferences for outpatient training differ from recommendations for increased ambulatory training. Discordance was observed between reported problems with conflicting inpatient-outpatient responsibilities and preferences for models maintaining longitudinal clinic during inpatient rotations. Further study regarding benefits and barriers of ambulatory redesign is needed.

The association of transient ischemic attack symptoms with memory impairment among elderly participants of the Third US National Health and Nutrition Examination Survey.

BACKGROUND AND PURPOSE: Stroke is a well-known risk factor for vascular dementia. However, the association of transient ischemic attacks with cognitive impairment is less well-established. METHODS: Records from Third National Health and Nutrition Examination Survey were abstracted for demographic and medical information for participants with an age >or=60 years who reported being free of stroke. Five self-reported symptoms (weakness, numbness, loss of vision, inability to speak, and severe dizziness) were used as surrogates representing transient ischemic attacks. Information on conventional risk factors for vascular dementia was also obtained. Multivariable logistic regression was used to examine risk factors for memory impairment. RESULTS: 4617 participants were included with a sample-weighted prevalence of memory impairment of 6.6% (1417 participants). The final multivariable analysis revealed a significant association between transient weakness and memory impairment (odds ratio 1.52, 95% CI 1.11-2.07). The other 4 transient ischemic attacks symptoms were not significantly associated with memory impairment in the final model. Systolic blood pressure >140 was most strongly associated with prevalent memory impairment (odds ratio, 9.78, 95% CI 1.49-64.3). Other associated risk factors included non-white race, male gender, age, education

Lack of interaction between sensing-intuitive learning styles and problem-first versus information-first instruction: a randomized crossover trial.

BACKGROUND: Adaptation to learning styles has been proposed to enhance learning. OBJECTIVE: We hypothesized that learners with sensing learning style would perform better using a problem-first instructional method while intuitive learners would do better using an information-first method. DESIGN: Randomized, controlled, crossover trial. SETTING: Resident ambulatory clinics. PARTICIPANTS: 123 internal medicine residents. INTERVENTIONS: Four Web-based modules in ambulatory internal medicine were developed in both "didactic" (information first, followed by patient problem and questions) and "problem" (case and questions first, followed by information) format. MEASUREMENTS: Knowledge posttest, format preference, learning style (Index of Learning Styles). RESULTS: Knowledge scores were similar between the didactic (mean +/- standard error, 83.0 +/- 0.8) and problem (82.3 +/- 0.8) formats (p = .42; 95% confidence interval [CI] for difference, -2.3 to 0.9). There was no difference between formats in regression slopes of knowledge scores on sensing-intuitive scores (p = .63) or in analysis of knowledge scores by styles classification (sensing 82.5 +/- 1.0, intermediate 83.7 +/- 1.2, intuitive 81.0 +/- 1.5; p = .37 for main effect, p = .59 for interaction with format). Format preference was neutral (3.2 +/- 0.2 [1 strongly prefers didactic, 6 strongly prefers problem], p = .12), and there was no association between learning styles and preference (p = .44). Formats were similar in time to complete modules (43.7 +/- 2.2 vs 43.2 +/- 2.2 minutes, p = .72). CONCLUSIONS: Starting instruction with a problem (versus employing problems later on) may not improve learning outcomes. Sensing and intuitive learners perform similarly following problem-first and didactic-first instruction. Results may apply to other instructional media.

Patient preferences for and satisfaction with methods of communicating test results in a primary care practice.

BACKGROUND: Appropriate and timely communication of test results is an important element of high-quality health care. Patients' preferences regarding and satisfaction with test result notification methods in a primary care practice were evaluated. METHODS: Some 1,458 consecutive patients were surveyed for whom routine blood tests were performed in the primary care internal medicine division at the Mayo Clinic Rochester (Minnesota) between January and March 2006. RESULTS: Among 888 respondents, test result notification occurred by telephone call (43%), return visit (35%), letter (3%), e-mail (0.1%), or a combination of methods (19%). Most (60%) telephone calls were handled by nurses. Patient preferences for notification method were telephone call (55%), return visit (20%), letter (19%), e-mail (5%), and automated answering mechanism (1%). Among patients reporting preference for telephone call, 67% wanted a call from a physician or nurse practitioner. Overall, 44% of patients received results by their preferred method; patients who did not were more likely to be dissatisfied with the communication method than those who did (10% vs. 5%, p = 0.01). A majority of patients were at least somewhat anxious to learn their test results, and patients greatly valued timeliness in test-result notification. DISCUSSION: The results describe primary care patients preferences for communication from their providers. Disparities exist between current practice and patient preferences in this important care delivery process. A telephone call from a physician or nurse practitioner was used to deliver test results for fewer than half of the patients who preferred to receive their results by this method. Future work should explore reimbursement of patient-preferred options and assess ways to improve resource-conscious test result communication methods.

Adapting web-based instruction to residents' knowledge improves learning efficiency: a randomized controlled trial.

BACKGROUND: Increased clinical demands and decreased available time accentuate the need for efficient learning in postgraduate medical training. Adapting Web-based learning (WBL) to learners' prior knowledge may improve efficiency. OBJECTIVE: We hypothesized that time spent learning would be shorter and test scores not adversely affected for residents who used a WBL intervention that adapted to prior knowledge. DESIGN: Randomized, crossover trial. SETTING: Academic internal medicine residency program continuity clinic. PARTICIPANTS: 122 internal medicine residents. INTERVENTIONS: Four WBL modules on ambulatory medicine were developed in standard and adaptive formats. The adaptive format allowed learners who correctly answered case-based questions to skip the corresponding content. MEASUREMENTS AND MAIN RESULTS: The measurements were knowledge posttest, time spent on modules, and format preference. One hundred twenty-two residents completed at least 1 module, and 111 completed all 4. Knowledge scores were similar between the adaptive format (mean +/- standard error of the mean, 76.2 +/- 0.9) and standard (77.2 +/- 0.9, 95% confidence interval [CI] for difference -3.0 to 1.0, P = .34). However, time spent was lower for the adaptive format (29.3 minutes [CI 26.0 to 33.0] per module) than for the standard (35.6 [31.6 to 40.3]), an 18% decrease in time (CI 9 to 26%, P = .0003). Seventy-two of 96 respondents (75%) preferred the adaptive format. CONCLUSIONS: Adapting WBL to learners' prior knowledge can reduce learning time without adversely affecting knowledge scores, suggesting greater learning efficiency. In an era where reduced duty hours and growing clinical demands on trainees and faculty limit the time available for learning, such efficiencies will be increasingly important. For clinical trial registration, see http://www.clinicaltrials.gov NCT00466453 ( http://www.clinicaltrials.gov/ct/show/NCT00466453?order=1 ).

Introducing resident doctors to complexity in ambulatory medicine.

OBJECTIVES: Medical education instructional methods typically imply one 'best' management approach. Our objectives were to develop and evaluate an intervention to enhance residents' appreciation for the diversity of acceptable approaches when managing complex patients. METHODS: A total of 124 internal medicine residents enrolled in a randomised, crossover trial. Residents completed four web-based modules in ambulatory medicine during continuity clinic. For each module we developed three 'complex cases'. Cases were intended to be complex (numerous variables, including psychosocial and economic barriers) and to suggest multiple acceptable management strategies. Several experienced faculty members described how they would manage each case. Residents reviewed each case, answered the same questions, and compared their responses with expert responses. Participants were randomly assigned to complete two modules with, and two modules without complex cases. RESULTS: A total of 76 residents completed 279 complex cases. Residents agreed that complex cases enhanced their appreciation for the diversity of 'correct' options (mean +/- standard error of the mean 4.6 +/- 0.2 [1 = strongly disagree, 6 = strongly agree]; P < 0.001). Mean preference score was neutral (3.4 +/- 0.2 [1 = strongly favour no cases, 6 = strongly favour cases]; P = 0.72). Knowledge post-test scores were similar between modules with (76.0 +/- 0.9) and without (77.8 +/- 0.9) complex cases (95% confidence interval for difference - 4.0 to 0.3; P = 0.09). Resident comments suggested that lack of time and cognitive overload impeded learning. CONCLUSIONS: Residents felt complex cases made a valuable contribution to their learning, although preference was neutral and knowledge scores were not affected. Methods to facilitate trainee comfort in managing medically complex patients should be further explored.

Immunization education for internal medicine residents: A cluster-randomized controlled trial.

PURPOSE: The aims of this study are to evaluate the impact of a novel immunization curriculum based on the Preferred Cognitive Styles and Decision Making Model (PCSDM) on internal medicine (IM) resident continuity clinic patient panel immunization rates, as well as resident immunization knowledge, attitudes, and practices (KAP). METHODS: A cluster-randomized controlled trial was performed among 143 IM residents at Mayo Clinic to evaluate the PCSDM curriculum plus fact-based immunization curriculum (intervention) compared to fact-based immunization curriculum alone (control) on the outcomes of resident continuity clinic patient panel immunization rates for influenza, pneumococcal, tetanus, pertussis, and zoster vaccines. Pre-study and post-study immunization KAP surveys were administered to IM residents. RESULTS: Ninety-nine residents participated in the study. Eighty-two residents completed pre-study and post-study surveys. Influenza and pertussis immunization rates improved for both intervention and control groups. There was no significant difference in immunization rate improvement between the groups. Influenza immunization rates improved significantly by 33.4% and 32.3% in the intervention and control groups, respectively. The odds of receiving influenza immunization at the end of the study relative to pre-study for the entire study cohort was 4.6 (p < 0.0001). The odds of having received pertussis immunization at the end of the study relative to pre-study for the entire study cohort was 1.2 (p = 0.0002). Both groups had significant improvements in immunization knowledge. The intervention group had significant improvements in multiple domains that assessed confidence in counseling patients on immunizations. CONCLUSIONS: Fact-based immunization education was useful in improving IM resident immunization rates for influenza and pertussis. The PCSDM immunization curriculum did not lead to increases in immunization rates compared with the fact-based curriculum, but it did significantly increase resident confidence in communicating with patients about vaccines.

Validation of a method for assessing resident physicians' quality improvement proposals.

BACKGROUND: Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. OBJECTIVE: We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability. DESIGN: QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised. PARTICIPANTS: Seven raters used the instrument to assess 45 resident QI proposals. MEASUREMENTS: Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. RESULTS: QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high. CONCLUSIONS: This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

Use of a registry-generated audit, feedback, and patient reminder intervention in an internal medicine resident clinic--a randomized trial.

BACKGROUND: Disease registries, audit and feedback, and clinical reminders have been reported to improve care processes. OBJECTIVE: To assess the effects of a registry-generated audit, feedback, and patient reminder intervention on diabetes care. DESIGN: Randomized controlled trial conducted in a resident continuity clinic during the 2003-2004 academic year. PARTICIPANTS: Seventy-eight categorical Internal Medicine residents caring for 483 diabetic patients participated. Residents randomized to the intervention (n = 39) received instruction on diabetes registry use; quarterly performance audit, feedback, and written reports identifying patients needing care; and had letters sent quarterly to patients needing hemoglobin A1c or cholesterol testing. Residents randomized to the control group (n = 39) received usual clinic education. MEASUREMENTS: Hemoglobin A1c and lipid monitoring, and the achievement of intermediate clinical outcomes (hemoglobin A1c <7.0%, LDL cholesterol <100 mg/dL, and blood pressure <130/85 mmHg) were assessed. RESULTS: Patients cared for by residents in the intervention group had higher adherence to guideline recommendations for hemoglobin A1c testing (61.5% vs 48.1%, p = .01) and LDL testing (75.8% vs 64.1%, p = .02). Intermediate clinical outcomes were not different between groups. CONCLUSIONS: Use of a registry-generated audit, feedback, and patient reminder intervention in a resident continuity clinic modestly improved diabetes care processes, but did not influence intermediate clinical outcomes.

Internal medicine resident perceptions of optimal training duration.

PURPOSE: To describe internal medicine residents' opinions regarding the optimal duration of internal medicine residency training, and to assess whether these opinions are associated with specific career interests. METHOD: A national cohort study was conducted during the 2005 Internal Medicine In-Training Examination (IM-ITE), which involved 382 of 388 (98.5%) U.S. internal medicine programs. A sample of 14,579 residents enrolled in three-year categorical or primary care training programs in the United States reported their opinions regarding optimal residency training duration on the IM-ITE 2005 Residents Questionnaire. Reported optimal training duration was assessed by postgraduate training year, sex, medical school location, program type, and reported career plan. RESULTS: Among the residents surveyed, 78.1% reported a three-year optimal length of internal medicine residency training, 15.3% preferred a two-year training duration, and 6.7% preferred a four-year duration. Residents planning careers in general medicine, hospital medicine, and subspecialty fields all preferred a three-year training duration (83.8%, 82.6%, and 75.9%, respectively). Residents planning subspecialty careers were more likely than those planning general or hospital medicine careers to prefer a two-year program (18.7% versus 7.4% and 8.3%). Residents planning generalist or hospitalist careers were more likely to favor a four-year program (8.9% and 9.1%, respectively) compared with residents planning subspecialty careers (5.4%). CONCLUSIONS: Most internal medicine residents endorse a three-year optimal duration of internal medicine residency training. This perspective should be considered in further national discussions regarding the optimal duration of internal medicine training.