Construct validity and reliability of structured assessment of endoVascular expertise in a simulated setting.

OBJECTIVES: To study the construct validity and reliability of a novel endovascular global rating scale, Structured Assessment of endoVascular Expertise (SAVE). DESIGN: A Clinical, experimental study. MATERIALS: Twenty physicians with endovascular experiences ranging from complete novices to highly experienced operators performed a video-recorded simulated contra-lateral iliac-artery-stenting procedure. The virtual-patient case was a novel technically challenging procedure presenting the distal arteries below the knee. METHODS: Three experts assessed the performances blinded to operator identity. Validity was analysed by correlating experience with performance results. Reliability was analysed according to generalisability theory. RESULTS: The mean score on the 29 items of the SAVE scale correlated well with clinical experience (R = 0.84, P < 0.01) and was found discriminative even among the more experienced participants having performed up to 500 endovascular procedures in total. Only the most experienced participants (>5000 procedures) obtained maximum scores. The inter-rater reliability was high (G = 0.94 and G = 0.95). The procedure time (median 69 min, range 32-86) correlated moderately with clinical experience (R = -0.53, P < 0.05), whereas the fluoroscopy time and amount of contrast fluid did not correlate. CONCLUSIONS: The construct validity and reliability of assessment with the SAVE scale was high when applied to performances in a simulation setting with advanced realism. No ceiling effect was present in the assessment situation.

Capturing the essence of developing endovascular expertise for the construction of a global assessment instrument.

OBJECTIVES: To explore what characterises the development of endovascular expertise and to construct a novel global assessment instrument. DESIGN: Literature review and an experimental study. MATERIALS AND METHODS: The literature was searched for information regarding available global rating scales (GRSs); scientific societies' official statements on endovascular competence; and task analyses of endovascular procedures. In the experimental study, clinicians performed a video-recorded simulated iliac-artery stenting procedure. Subsequently, by using the method of retrospective verbalisation, the clinicians were interviewed while watching their performance on video commenting on key issues of the construct. Data from all sources were analysed, categorised and synthesised into a novel rating scale. RESULTS: Available GRSs primarily included technical aspects of performance, whereas the competence statements, task analyses and clinicians' perceptions added a range of non-technical aspects. The novel rating scale SAVE (Structured Assessment of endoVascular Expertise) differs from prior scales by including issues of pre-planning; prediction of challenges; preparation of tools; management of imaging presentation; distinction of technical skills into external and internal control according to operator focus of visual attention; adaptation of strategy; clinical decision making; use of assistant; complications; inter-personal skills; and post-procedural planning. CONCLUSIONS: The essence of developing endovascular expertise goes far beyond mere technical aspects.