Added value of assessing medical students' reflective writings in communication skills training: a longitudinal study in four academic centres.

OBJECTIVES: This study describes the development and implementation of a model to assess students' communication skills highlighting the use of reflective writing. We aimed to evaluate the usefulness of the students' reflections in the assessment of communication skills. DESIGN: Third-year and fourth-year medical students enrolled in an elective course on clinical communication skills development were assessed using different assessment methods. SETTING AND PARTICIPANTS: The communication skills course was offered at four universities (three in Brazil and one in Portugal) and included 69 students. OUTCOME MEASURES: The students were assessed by a Multiple-Choice Questionnaire (MCQ), an objective structured clinical examination (OSCE) and reflective writing narratives. The Cronbach's alpha, dimensionality and the person's correlation were applied to evaluate the reliability of the assessment methods and their correlations. Reflective witting was assessed by applying the Reflection Evaluation for Enhanced Competencies Tool Rubric (Reflect Score (RS)) to measure reflections' depth, and the Thematic Score (TS) to map and grade reflections' themes. RESULTS: The Cronbach alpha for the MCQ, OSCE global score, TS and RS were, respectively, 0.697, 0.633, 0.784 and 0.850. The interobserver correlation for the TS and RS were, respectively, 0.907 and 0.816. The assessment of reflection using the TS was significantly correlated with the MCQ (r=0.412; p=0.019), OSCE (0.439; p=0.012) and RS (0.410; p=0.020). The RS did not correlate with the MCQ and OSCE. CONCLUSIONS: Assessing reflection through mapping the themes and analysing the depth of reflective writing expands the assessment of communication skills. While the assessment of reflective themes is related to the cognitive and behavioural domains of learning, the reflective depth seems to be a specific competence, not correlated with other assessment methods-possibly a metacognitive domain.

Use of portfolios in teaching communication skills and professionalism for Portuguese-speaking medical students.

OBJECTIVES: This study aimed to analyse the effect of a portfolio with three activities fostering students' reflection, self-efficacy and teaching of communication skills and professionalism. METHODS: A cross-sectional study was applied with a sample of third- and fourth-year medical students in one Portuguese and three Brazilian universities. A three-activity portfolio (course evaluation and learning, self-efficacy activity and free reflective writing) was used during a two-month course on communication skills and professionalism. The 69 students enrolled in the course were invited to complete the three-activity portfolio via Likert-type questionnaires, open-ended questions and narrative. Content and lexical analysis and the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT) were used for assessing the qualitative data. The questionnaires were evaluated using principal components analysis and Cronbach's α. Pearson's correlation was applied to portfolio activities. RESULTS: Of the 69 participants, 85.5% completed at least one activity. Reflecting on what they learned in the communication module, the students did not mention professionalism themes. In the self-efficacy activity on communication, 25% of the fragments were related to professionalism themes. There was a negative correlation between students' self-efficacy and the REFLECT rubric score (r(19)=-0.744; p< 0.0001). CONCLUSIONS: Teachers must consider the activity's influence on the reflections when assessing the portfolio. This model of a three-activity portfolio provided diverse ways of encouraging and assessing reflections, supporting teaching improvement and adaptation, evaluating students' self-efficacy and showing that students' higher reflective capacity may promote feelings of low effectiveness.