RESUMO
Background: How to evaluate clinical educators is an important question in faculty development. The issue of who are best placed to evaluate their performance is also critical. However, the whos and the hows of clinical educator evaluation may differ culturally. This study aims to understand what comprises suitable evaluation criteria, alongside who is best placed to undertake the evaluation of clinical educators in medicine within an East Asian culture: specifically Taiwan. Methods: An 84-item web-based questionnaire was created based on a literature review and medical educational experts' opinions focusing on potential raters (i.e., who) and domains (i.e., what) for evaluating clinical educators. Using purposive sampling, we sent 500 questionnaires to clinical educators, residents, Post-Graduate Year Trainees (PGYs), Year-4~6/Year-7 medical students (M4~6/M7) and nurses. Results: We received 258 respondents with 52% response rate. All groups, except nurses, chose "teaching ability" as the most important domain. This contrasts with research from Western contexts that highlights role modeling, leadership and enthusiasm. The clinical educators and nurses have the same choices of the top five items in the "personal qualities" domain, but different choices in "assessment ability" and "curriculum planning" domains. The best fit rater groups for evaluating clinical educators were educators themselves and PGYs. Conclusions: There may well be specific suitable domains and populations for evaluating clinical educators' competence in East Asian culture contexts. Further research in these contexts is required to examine the reach of these findings.
RESUMO
STUDY OBJECTIVE: To assess professionalism in anesthesiology residents, it is important to obtain evaluations from people with whom they interact on daily basis. The purpose of this study was to evaluate the effect of a Multisource feedback (MSF) on resident's professional behavior and to assess the effect of faculty feedback on resident performance. DESIGN: This study was a two-group randomized clinical trial. SETTING: Residents were recruited from Cleveland Clinic Children's Hospital. PATIENTS: Participants included twenty eight residents doing a two-month rotation in Pediatric Anesthesia. INTERVENTIONS: Multisource feedback questionnaires were developed and then validated using face and content validity. Residents were randomly assigned to a feedback group or a control group. Both groups received the MSF evaluation. Only the group assigned to feedback had a 'coaching meeting' every month creating strategies for improvement. MEASUREMENTS: MSF questionnaires were validated using a face validation and expert content validity. The effect of MSF on a professionalism questionnaire was assessed using analysis of covariance and linear mixed effects regression models. MAIN RESULTS: Observed test-retest agreement was greater than 0.90 for all items, with more than half of kappa statistics greater than 0.50. Cronbach's alpha was 0.71.The MSF increased the self-assessment score with an estimated effect of 0.21 (95% CI 0.06, 0.37), P=.015. There was no detected effect on patient family evaluation, with mean difference (CI) in change from baseline of 0.03 (-0.15, 0.21), P=.77, faculty evaluation, 0.21 (-0.02, 0.44), P=.08, or coworker evaluation 0.13 (-0.11, 0.37). CONCLUSIONS: Our new multi-source feedback questionnaire to assess professionalism had good reliability and internal consistency. Using our validated questionnaire we assessed the effect of a monthly feedback to improve professionalism in anesthesia residents. While we did see improvement in anesthesiology residents' self-assessment, we did not see a similar effect on patient family, faculty or coworker evaluations.