The effects of gender and age on evaluation of trainees and faculty in gastroenterology.

OBJECTIVES: We studied whether differences exist in evaluation scores of faculty and trainees in gastroenterology (GI) based on the gender of the evaluator or evaluatee, or the evaluator-evaluatee gender pairing. METHODS: We examined evaluations of faculty and trainees (GI fellows and internal medicine residents rotating on GI services), using mixed linear models to assess effects of the four possible evaluator-evaluatee gender pairings. Potential confounding variables were adjusted for, and random effects were used to account for repeated assessments. RESULTS: For internal medicine (IM) residents, no difference in evaluation scores based on gender was found. Resident age was negatively associated with performance rating, while percentage correct on the in-training examination (ITE) was positively associated. For GI fellows, the interaction between evaluator and evaluatee gender was significant. Fellow age and international medical graduate (IMG) status were negatively associated with performance rating, while ITE percentage correct was positively associated. For faculty, no difference was found in evaluation scores by IM residents based on the gender of the evaluated faculty or the evaluating resident, although the interaction between the evaluator and the evaluatee gender was significant. Gender had a significant marginal effect on faculty scores by GI fellows, with female faculty receiving lower scores. The interaction between evaluator and evaluatee gender was also significant for evaluations by fellows. Faculty age was negatively associated with performance rating. DISCUSSION: Gender, age, and ITE performance are associated with evaluation scores of GI trainees and faculty at our institution. The interaction of evaluator and evaluatee gender appears to play a more critical role in evaluation scoring than the gender of the evaluatee or evaluator in isolation.

The impact of adding 1 month of intensive care unit training in a categorical internal medicine residency program.

OBJECTIVE: To determine the impact of adding a month of critical care training in the postgraduate year (PGY)-2 on the critical care skills of PGY-3 residents. DESIGN: Prospective, nonrandomized. SETTING: The internal medicine (IM) residency program of a tertiary care medical center. STUDY SUBJECTS: The study subjects included the 2005/2006 and 2006/2007 academic year IM residents. INTERVENTIONS: The 2005/2006 IM residents (control group, n = 48) had 1 month of critical care training (internship year) before their 1-month PGY-3 rotations. The 2006/2007 residents (intervention group, n = 47) had an additional 1-month rotation in a multispecialty intensive care unit (ICU) during their second year. MEASUREMENTS AND MAIN RESULTS: At the beginning of their last ICU month rotation, the intervention group's self-assessment (1-5 Likert scale) of their skills in internal jugular venous catheterization (3.4 vs. 2.4, p < 0.001) and management of severe sepsis (4.0 vs. 2.4, p < 0.001) and acute lung injury (3.3 vs. 2.6, p < 0.001) was higher than that of the control group. However, the observed success rates of endotracheal intubation (55.4% vs. 54.9%, p = 0.953) and central venous catheterization (78.1% vs. 80.8%, p = 0.488) were similar between the two groups. No difference was noted in the complication rates for endotracheal intubation or central venous catheterization between the control and intervention groups. End of ICU rotation examination results, attending evaluations, and the observed application of evidence-based practice in the management of severe sepsis were similar between the two groups. CONCLUSIONS: Increasing IM residents' experience in the ICU resulted in modest, transient improvement of their perceived clinical skills in critical care procedures and management of severe sepsis and acute lung injury. However, no statistically significant and sustained improvement was noted in the observed cognitive or clinical skills.

Experience with a medical education research group at the Mayo Clinic.

BACKGROUND: Clinician-educators are encouraged to engage in scholarly activity despite numerous barriers to conducting educational research. We found no reports of formal educational research groups among general internists. AIM: To evaluate a Scholarship in Medical Education Group (SIMEG) for general internists. METHODS: The setting is a Division of General Internal Medicine at a large academic medical center. In 2005 twenty-two faculty members formed SIMEG to collaborate on educational research and award protected days for scholarly education projects. Through monthly dialogue and literature review, standards for critiquing educational research were created. RESULTS: A total of 63 protected days were awarded to successful applicants. SIMEG members' curriculum vitae were monitored over two years. From 2005 to 2006, SIMEG members' presentations (15 to 29; p=0.125), peer-reviewed publications (8 to 27; p=0.016) and collaborative projects (19 to 42; p=0.047) increased. CONCLUSIONS: Our educational research group enjoyed improving scholarly productivity over two years. Keys to success were supportive leadership, a culture of collaboration, creating a useful framework for critiquing research, and involving medical learners as co-investigators. Since faculty members are pressured to teach and care for patients while maintaining scholarly productivity, groups like SIMEG should become increasingly valued.

Determining reliability of clinical assessment scores in real time.

BACKGROUND: Assessment score reliability is usually based on a single analysis. However, reliability is an essential component of validity and assessment validation and revision is a never-ending cycle. For ongoing assessments over extended time frames, real-time reliability computations may alert users to possible changes in the learning environment that are revealed by variations in reliability over time. PURPOSE: To develop software that calculates the reliability of clinical assessments in real time. METHODS: Over 2,400 assessment forms were analyzed. We developed software that calculates reliability in real time. Software accuracy was verified by comparing data from our software with a standard method. Factor analysis determined scale dimensionality. RESULTS: Correlation between our software and a standard method was excellent (ICC for kappas = 0.97; Cronbach's alphas differed by < 0.03). Cronbach's alpha ranged from 0.94 to 0.97 and weighted kappa ranged from 0.08 to 0.40. Factor analysis confirmed 3 teaching domains. CONCLUSIONS: We describe an accurate method for calculating reliability in real time. The benefit of real time computation is that it provides a mechanism for detecting possible changes (related to curriculum, teachers, and students) in the learning environment indicated by changes in reliability over time. This technique will enable investigators to monitor and detect changes in the reliability of assessment scores and, with future study, isolate aspects of the learning environment that impact on reliability.

Behaviors of highly professional resident physicians.

CONTEXT: Unprofessional behaviors in medical school predict high stakes consequences for practicing physicians, yet little is known about specific behaviors associated with professionalism during residency. OBJECTIVE: To identify behaviors that distinguish highly professional residents from their peers. DESIGN, SETTING, AND PARTICIPANTS: Comparative study of 148 first-year internal medicine residents at Mayo Clinic from July 1, 2004, through June 30, 2007. MAIN OUTCOME MEASURES: Professionalism as determined by multiple observation-based assessments by peers, senior residents, faculty, medical students, and nonphysician professionals over 1 year. Highly professional residents were defined as those who received a total professionalism score at the 80th percentile or higher of observation-based assessments on a 5-point scale (1, needs improvement; 5, exceptional). They were compared with residents who received professionalism scores below the 80th percentile according to In-Training Examination (ITE) scores, Mini-Clinical Evaluation Exercise (mini-CEX) scores, conscientious behaviors (percentage of completed evaluations and conference attendance), and receipt of a warning or probation from the residency program. RESULTS: The median total professionalism score among highly professional residents was 4.39 (interquartile range [IQR], 4.32-4.44) vs 4.07 (IQR, 3.91-4.17) among comparison residents. Highly professional residents achieved higher median scores on the ITE (65.5; IQR, 60.5-73.0 vs 63.0; IQR, 59.0-67.0; P = .03) and on the mini-CEX (3.95; IQR, 3.63-4.20 vs 3.69; IQR, 3.36-3.90; P = .002), and they completed a greater percentage of required evaluations (95.6%; IQR, 88.1%-99.0% vs 86.1%; IQR, 70.6%-95.0%; P < .001) compared with residents with lower professionalism scores. In multivariate analysis, a professionalism score in the top 20% of residents was independently associated with ITE scores (odds ratio [OR] per 1-point increase, 1.07; 95% confidence interval [CI], 1.01-1.14; P = .046), mini-CEX scores (OR, 4.64; 95% CI, 1.23-17.48; P = .02), and completion of evaluations (OR, 1.07; 95% CI, 1.01-1.13; P = .02). Six of the 8 residents who received a warning or probation had total professionalism scores in the bottom 20% of residents. CONCLUSION: Observation-based assessments of professionalism were associated with residents' knowledge, clinical skills, and conscientious behaviors.

Clinician attitudes toward biostatistics.

OBJECTIVE: To assess clinician attitudes toward biostatistics at an academic medical center. PARTICIPANTS AND METHODS: We conducted a cross-sectional survey of medical students, internal medicine resident physicians, and internal medicine teaching faculty at Mayo Clinic College of Medicine in Rochester, MN, in April 2005. RESULTS: Of 468 eligible participants, 301 (64.3 percent) responded to the survey. A total of 87.3 percent of respondents (262/300) believed it would benefit their career to better understand biostatistics, but only 17.6 percent (53/301) believed their training in biostatistics was adequate for their needs. A total of 23.3 percent of respondents (70/ 300) agreed they could identify when correct statistical methods had been applied in a study, 28.0 percent (84/300) agreed they could design their own research projects with confidence, and 14.6 percent (44/301) agreed they could conduct their own statistical analyses with confidence. Respondents with the highest self-reported level of statistical education and research experience were more likely to report these skills (all, P less than .001). A total of 92.7 percent of respondents (279/301) believed biostatistics is an important part of evidence-based medicine (EBM), and 88.0 percent (265/301) believed EBM is important for clinical practice. However, biostatistics was not evaluated as being as important as many other areas of study within medicine.

A strategy to reduce interruptions at hospital morning report.

BACKGROUND: The learning atmosphere of hospital morning report is frequently interrupted by nonurgent beeper pages to participating residents. INTERVENTION: During 176 consecutive internal medicine morning reports, 1 of the 4 chief residents at a single academic medical center collected and answered house staff pages during the 30-minute conference. RESULTS: A total of 884 pages were answered during this period. Of these, 743 could wait until after the end of morning report, resulting in more than 4 interruptions averted per session. Over half of the pages were from nursing personnel. There was no significant decrease in the total number of pages or nurse pages during the observation period. DISCUSSION: In summary, our simple system of holding intern pages resulted in a dramatic reduction in interruptions during hospital morning report.

Musculoskeletal injection.

Patients commonly present to primary care physicians with musculoskeletal symptoms. Clinicians certified in internal medicine must be knowledgeable about the diagnosis and management of musculoskeletal diseases, yet they often receive inadequate postgraduate training on this topic. The musculoskeletal problems most frequently encountered in our busy injection practice involve, in decreasing order, the knees, trochanteric bursae, and glenohumeral joints. This article reviews the clinical presentations of these problems. It also discusses musculoskeletal injections for these problems in terms of medications, indications, injection technique, and supporting evidence from the literature. Experience with joint injection and the pharmacological principles described in this article should allow primary care physicians to become comfortable and proficient with musculoskeletal injections.

Changing morning report: evaluation of a transition to an interactive mixed-learner format in an internal medicine residency program.

BACKGROUND: Morning report is an important conference for medical education, but direct comparisons of approaches to morning report have only rarely been reported. DESCRIPTION: During the 2003 to 2004 academic year, we conducted both our traditional single-learner-level morning report and a new highly interactive multiple-learner-level format. Attendees were rotating students, residents, and faculty on the inpatient general medicine services. EVALUATION: We conducted anonymous surveys of participants to record their evaluations of the morning report formats. We received evaluations from 293 (60%) of 490 students, residents, and faculty. Students, 1st-year residents, and faculty preferred the mixed-learner model significantly more than did senior residents. Overall, more than 80% of participants rated the interactive multilevel format as good or very good when asked about content, discussion quality, level, and usefulness. CONCLUSIONS: An interactive morning report involving learners across multiple levels was well-received and has several educational benefits, with a greater emphasis on collaborative case discussion and active learning. Education leaders should remain open to experimenting with even the most well-established institutional traditions as they continuously reevaluate the effectiveness of teaching conferences.