Evaluation of a leadership development impact assessment toolkit: a comparative case study of experts' perspectives in three Canadian provinces.

ABSTRACT

PURPOSE: This paper aims to explore users' perceptions of whether the Leadership Development Impact Assessment (LDI) Toolkit is valid, reliable, simple to use and cost-effective as a guide to its quality improvement. DESIGN/METHODOLOGY/APPROACH: The Canadian Health Leadership Network codesigned and codeveloped the LDI Toolkit as a theory-driven and evidence-informed resource that aims to assist health-care organizational development practitioners to evaluate various programs at five levels of impact reaction, learning, application, impact and return on investment (ROI) and intangible benefits. A comparative evaluative case study was conducted using online questionnaires and semistructured telephone interviews with three health organizations where robust leadership development programs were in place. A total of seven leadership consultants and specialists participated from three Canadian provinces. Data were analyzed sequentially in two stages involving descriptive statistical analysis augmented with a qualitative content analysis of key themes. FINDINGS: Users perceived the toolkit as cost-effective in terms of direct costs, indirect costs and intangibles; they found it easy-to-use in terms of clarity, logic and structure, ease of navigation with a coherent layout; and they assessed the sources of the evidence-informed tools and guides as appropriate. Users rated the toolkit highly on their perceptions of its validity and reliability. The analysis also informed the refinement of the toolkit. ORIGINALITY/VALUE The refined LDI Toolkit is a comprehensive online collection of various tools to support health organizations to evaluate the leadership development investments effectively and efficiently at five impact levels including ROI.