Changing estimates of leadership ability before a programme: retrospective self-assessments and response-shift bias.

BACKGROUND: Most evaluations of clinical leadership development programmes rely on self-assessments. Self-assessments are vulnerable to response-shift bias. Using retrospective then-tests may help to avoid this bias.In this study, we investigate whether post-programme then-tests (retrospective self-assessments) are more sensitive to change in clinical leadership development programme participants than traditional pre-programme pre-tests when paired with post-test self-assessments. METHODS: 17 healthcare professionals participated in an 8-month single-centre multidisciplinary leadership development programme. Participants completed prospective pre-test, retrospective then-test and traditional post-test self-assessments using the Primary Colours Questionnaire (PCQ) and Medical Leadership Competency Framework Self-Assessment Tool (MLCFQ). Pre-post pairs and then-post pairs were analysed for changes using Wilcoxon signed-rank tests and compared with a parallel multimethod evaluation organised by Kirkpatrick levels. RESULTS: A greater number of significant changes were detected using then-test pairs than pre-test pairs for both the PCQ (11 of 12 vs 4 of 12 items) and MLCFQ (7 of 7 vs 3 of 7 domains). The multimethods data showed positive outcomes at all Kirkpatrick levels. CONCLUSIONS: In ideal circumstances, both pre-test and then-test evaluations should be conducted. We cautiously suggest that if only one post-programme evaluation can be conducted, then-tests may be appropriate means of detecting change.

Next Generation Leaders Programme: A Multi-Methods Evaluation of a Leadership Development Programme for Biomedical Researchers.

Background: Biomedical scientists have become de facto leaders for their research teams. Theories of expert leadership suggest that the specialist knowledge and credibility these researcher-leaders bring to their roles can lead to improved performance. Formal leadership development for biomedical researchers remains uncommon, and it is unclear whether existing leadership development programmes achieve improved individual and organisational outcomes. Our study evaluates the effectiveness of a single centre leadership development programme for biomedical researchers using a mixed-methods approach. Methods: 26 biomedical researchers participated in an 8 month single centre multidisciplinary leadership development programme. Participants completed prospective pre-test, retrospective then-test and traditional post-test self-assessments using the Primary Colours Questionnaire (PCQ) and Medical Leadership Competency Framework Self-Assessment Tool (MLCFQ). Pre-post pairs and then-post pairs were analysed for changes using Wilcoxon signed-rank tests and compared with a parallel mixed-methods evaluation organised by Kirkpatrick levels. Results: There were significant increases in 3/7 domains and 1/5 tasks of leadership in the PCQ, in both pre-post and then-post paired assessments. There were statistically significant but small increases in 2/7 domains of the MLCFQ. The mixed-methods data showed positive outcomes at all Kirkpatrick levels. Participants said the programme was relevant, interesting and well-organised, with 63% reporting increased confidence and motivation. Participants had a significant change in behaviour, spending 1-2 hours per week on group projects, which were successfully implemented locally. 42% of participants expected these projects to continue beyond the programme. Discussion: This study demonstrates a local leadership programme can have positive impact within a biomedical research centre despite time and financial constraints. We encourage future studies to utilise a mixed-methods approach to evaluating the impact of leadership development programmes.