Evaluation of clinical teaching quality in competency-based residency training in Lithuania.

BACKGROUND AND AIM: In 2013, all residency programs at the Lithuanian University of Health Sciences were renewed into the competency-based medical education curriculum (CBME). In 2015, we implemented the validated EFFECT questionnaire together with the EFFECT-System for quality assessment of clinical teaching in residency training. The aim of this study was to investigate the influence of characteristics of the resident (year of training) and clinical teacher (gender, age, and type of academic position) on teaching quality, as well as to assess areas for teaching quality improvement. MATERIALS AND METHODS: Residents from 7 different residency study programs filled out 333 EFFECT questionnaires evaluating 146 clinical teachers. We received 143 self-evaluations of clinical teachers using the same questionnaire. Items were scored on a 6-point Likert scale. Main outcome measures were residents' mean overall (MOS), mean subdomain (MSS) and clinical teachers' self-evaluation scores. The overall comparisons of MOS and MSS across study groups and subgroups were done using Student's t test and ANOVA for trend. The intraclass correlation coefficient (ICC) was calculated in order to see how residents' evaluations match with self-evaluations for every particular teacher. To indicate areas for quality improvement items were analyzed subtracting their mean score from the respective (sub)domain score. RESULTS: MOS for domains of "role modeling", "task allocation", "feedback", "teaching methodology" and "assessment" valued by residents were significantly higher than those valued by teachers (P<0.01). Teachers who filled out self-evaluation questionnaires were rated significantly higher by residents in role modeling subdomains (P<0.05). Male teachers in (sub)domains "role modeling: CanMEDS roles and reflection", "task allocation", "planning" and "personal support" were rated significantly higher than the female teachers (P<0.05). Teachers aged 40 years or younger were rated higher (P<0.01). Residents ratings by type of teachers' academic position almost in all (sub)domains differed significantly (P<0.05). No correlation observed between MOS of a particular teacher and MOS as rated by residents (ICC=0.055, P=0.399). The main areas for improvement were "feedback" and "assessment". CONCLUSIONS: Resident evaluations of clinical teachers are influenced by teachers' age, gender, year of residency training, type of teachers' academic position and whether or not a clinical teacher performed self-evaluation. Development of CBME should be focused on the continuous evaluation of quality, clinical teachers educational support and the implementation of e-portfolio.

Validation of the EFFECT questionnaire for competence-based clinical teaching in residency training in Lithuania.

BACKGROUND AND AIM: In 2013, all residency programs at the Lithuanian University of Health Sciences were renewed into a competency-based medical education curriculum. To assess the quality of clinical teaching in residency training, we chose the EFFECT (evaluation and feedback for effective clinical teaching) questionnaire designed and validated at the Radboud University Medical Centre in the Netherlands. The aim of this study was to validate the EFFECT questionnaire for quality assessment of clinical teaching in residency training. MATERIALS AND METHODS: The research was conducted as an online survey using the questionnaire containing 58 items in 7 domains. The questionnaire was double-translated into Lithuanian. It was sent to 182 residents of 7 residency programs (anesthesiology reanimathology, cardiology, dermatovenerology, emergency medicine, neurology, obstetrics and gynecology, physical medicine and rehabilitation). Overall, 333 questionnaires about 146 clinical teachers were filled in. To determine the item characteristics and internal consistency (Cronbach's α), the item and reliability analyses were performed. Furthermore, confirmatory factor analysis (CFI) was performed using a model for maximum-likelihood estimation. RESULTS: Cronbach's α within different domains ranged between 0.91 and 0.97 and was comparable with the original version of the questionnaire. Confirmatory factor analysis demonstrated satisfactory model-fit with CFI of 0.841 and absolute model-fit RMSEA of 0.098. CONCLUSIONS: The results suggest that the Lithuanian version of the EFFECT maintains its original validity and may serve as a valid instrument for quality assessment of clinical teaching in competency-based residency training in Lithuania.