Certification of Basic Skills in Endovascular Aortic Repair Through a Modular Simulation Course With Real Time Performance Assessment.

OBJECTIVE: Endovascular aortic repair (EVAR) is being used increasingly for the treatment of infrarenal abdominal aortic aneurysms. Improvement in educational strategies is required to teach future vascular surgeons EVAR skills, but a comprehensive, pre-defined e-learning and simulation curriculum remains to be developed and tested. EndoVascular Aortic Repair Assessment of Technical Expertise (EVARATE), an assessment tool for simulation based education (SBE) in EVAR, has previously been designed to assess EVAR skills, and a pass limit defining mastery level has been set. However, EVARATE was developed for anonymous video ratings in a research setting, and its feasibility for real time ratings in a standardised SBE programme in EVAR is unproven. This study aimed to test the effect of a newly developed simulation based modular course in EVAR. In addition, the applicability of EVARATE for real time performance assessments was investigated. METHODS: The European Society of Vascular Surgery (ESVS) and Copenhagen Certification Programme in EVAR (ENHANCE-EVAR) was tested in a prospective cohort study. ENHANCE-EVAR is a modular SBE programme in EVAR consisting of e-learning and hands-on SBE. Participants were rated with the EVARATE tool by experienced EVAR surgeons. RESULTS: Twenty-four physicians completed the study. The mean improvement in EVARATE score during the course was +11.8 (95% confidence interval 9.8 - 13.7) points (p < .001). Twenty-two participants (92%) passed with a mean number of 2.8 ± 0.7 test attempts to reach the pass limit. Cronbach's alpha coefficient was 0.91, corresponding to excellent reliability of the EVARATE scale. Differences between instructors' EVARATE ratings were insignificant (p = .16), with a maximum variation between instructors of ± 1.3 points. CONCLUSION: ENHANCE-EVAR, a comprehensive certifying EVAR course, was proven to be effective. EndoVascular Aortic Repair Assessment of Technical Expertise (EVARATE) is a trustworthy tool for assessing performance within an authentic educational setting, enabling real time feedback.

Massed Training is Logistically Superior to Distributed Training in Acquiring Basic Endovascular Skills.

OBJECTIVE: A "PROficiency based StePwise Endovascular Curricular Training" (PROSPECT) has proven its superiority over traditional training in a randomised controlled trial to acquire basic endovascular skills outside theatre, but real life adherence is low. This study aimed to compare the original distributed training format, where trainees learn at their own pace, with a massed training format offering the same content within a limited time span while exempt from clinical duties. Secondly, long term skills retention was evaluated. METHODS: A multicentre, prospective study allocated participants to the distributed D-PROSPECT or to a massed, compact version (C-PROSPECT) based on logistics such as travel time, participant and instructor availability. A multiple choice question (MCQ) test (max. score 20) tested cognitive skills. Technical skills were assessed using a global rating scale (GRS) (max. score 55), examiner's checklist (max. score 85), and validated simulator metrics. Data were collected pre- and post-programme and at three, six, and 12 months after programme completion. RESULTS: Over four years and in two countries, D-PROSPECT was implemented in two centres and C-PROSPECT in three. A total of 22 participants completed D-PROSPECT with a 41% dropout rate, and 21 completed C-PROSPECT with 0% dropout rate. All participants showed significant improvement for all performance parameters after programme completion: MCQ test (median 14.5 vs. 18; p < .001), GRS (median 20 vs. 41; p < .001), examiner's checklist (median 49 vs. 78.5; p < .001), and simulation metrics (p < .001). Scores of C- or D-PROSPECT participants were not significantly different. No significant differences were seen between groups during the retention period. CONCLUSION: PROSPECT significantly improves the quality of simulated endovascular performances using a massed or distributed training format. A massed training format of PROSPECT may be preferred to decrease dropout during standardised training to obtain basic endovascular skills in existing surgical curricula.

Facilitators and Barriers to Implementation of Simulation Based Education in Vascular Surgery in Europe.

OBJECTIVE: This study explored the status and availability of simulation based education (SBE) for learning vascular surgical procedures identified in the 2019 General Needs Assessment in vascular surgery in Europe (GNA-2019) and identified facilitators and barriers to SBE implementation in vascular surgery. METHODS: A three round iterative survey was distributed via the European Society for Vascular Surgery and the Union Européenne des Médecins Spécialistes. Members from leading committees and organisations within the European vascular surgical community were invited to participate as key opinion leaders (KOLs). Three online survey rounds explored demographics, SBE availability, and facilitators and barriers to SBE implementation. RESULTS: Overall, 147 KOLs (target population 338) accepted invitation to round 1, representing 30 European countries. The dropout rates for rounds 2 and 3 were 29% and 40%, respectively. Most respondents (88%) were senior, consultant level or higher. No mandatory SBE training was required in their department before training on patients, according to 84% of the KOLs. There was high consensus on the need for structured SBE (87%) and mandatory SBE (81%). SBE is available for the top three prioritised procedures in GNA-2019 (basic open skills, basic endovascular skills, and vascular imaging interpretation) in 24, 23, and 20 of the 30 represented European countries, respectively. The highest ranking facilitators were structured SBE programmes, availability of simulation equipment locally and regionally, good quality simulators, and having a dedicated person running the SBE. The highest ranked barriers were lack of structured SBE curriculum, equipment costs, lack of SBE culture, no or limited dedicated time for faculty to teach in SBE, and clinical work overload. CONCLUSION: Based largely on the opinions of KOLs in vascular surgery in Europe, this study revealed that SBE is needed in vascular surgery training and that systematic and structured programmes are required to ensure successful implementation.

Simulation-Based Education of Endovascular Scrub Nurses Reduces Stress and Improves Team Performance.

INTRODUCTION: Endovascular procedures have become commonplace in vascular surgery. This development calls for new training strategies for future specialists. Most simulation-based education (SBE) programs have a monodisciplinary focus on physicians, although successful surgery is a multidisciplinary team effort. Mental stress impairs the learning process and surgical performance and heart rate variability (HRV) can be measured as a proxy for both mental and physical stress. This study aims to assess how SBE of endovascular scrub nurses affects team performance and HRV during endovascular aneurysm repair (EVAR). MATERIALS AND METHODS: Prospective interventional study in which EVAR-inexperienced scrub nurses followed a focused SBE EVAR program. During real-life EVAR procedures, HRV was continuously recorded with a wireless electrocardiogram patch and multidisciplinary team performance was assessed with the Imperial College Error CAPture (ICECAP) tool, before and after the SBE program, allowing each scrub nurse to serve as their own control. Eight scrub nurses with experience in lower limb endovascular procedures, but not EVAR, were invited to participate. RESULTS: Seven participants completed the study. In five of seven scrub nurses, HRV-derived stress levels during real-time EVAR procedures were lower after SBE compared to before SBE. Mean HRV increased from 24 msec to 35 msec (P < 0.001), indicating stress level reduction. Before SBE, the mean number of errors/hour was 7.3 (standard deviation ± 1.8) compared to 3.6 (standard deviation ± 2.7) after SBE. Most errors were categorized as technical (58 %) and communicative (23 %). CONCLUSIONS: SBE of scrub nurses may improve team performance and may lower mental stress during EVAR procedures. In this small study, we suggest using mental stress, as evaluated with HRV, and multidisciplinary team performance, as evaluated with ICECAP, to assess SBE effectiveness in real-case EVAR procedures. This SBE program and live ICECAP observations and electrocardiogram patches was well-accepted by scrub nurses and the entire team.

International Implementation of a PROficiency based StePwise Endovascular Curricular Training (PROSPECT) in Daily Practice.

OBJECTIVE: A randomised controlled trial (RCT) showed superior real life endovascular performance of trainees after PROficiency based StePwise Endovascular Curricular Training (PROSPECT) compared with traditional training. This study aimed to determine whether PROSPECT can be implemented routinely in surgical training (registry) and to identify the associated effects, facilitators, and barriers to implementation. METHODS: This was a multicentre prospective registry to evaluate real life PROSPECT implementation effects. Cognitive ability was tested with a Multiple Choice Question test (MCQ) (max. score 20 points), while technical skills were assessed with a Global Rating Scale (GRS) (max. score 55 points), Examiner's Checklist (max, score 85 points), and simulator metrics. A cross sectional anonymous survey for both trainees and faculty evaluated the programme's implementation. RESULTS: The PROSPECT registry was implemented in five centres in four countries over a period of three years. Only 17 of 48 starting trainees completed PROSPECT, resulting in a dropout rate of 65%. Participants who completed the programme showed significant improvement in the GRS (median 26 vs. 44, p = .018) and Examiner's Checklist (median 53.5 vs. 80, p = .028) after the programme. The survey was completed by 13/17 faculty members (76%) and 38/57 trainees who participated either in the initial PROSPECT RCT or the registry (67%). PROSPECT was perceived to provide more endovascular training opportunities, but non-availability of logistical/technical support, lack of training time within the working schedule, and little trainee motivation hindered broad implementation. CONCLUSION: PROSPECT has a significant effect on performance, and trainees and faculty agree that PROSPECT is a valuable addition to surgical training. However, external and internal factors pose significant barriers to integration of this simulation based programme into daily practice leading to a high dropout rate. To integrate PROSPECT into contemporary surgical training, it should be a prerequisite to treating real patients and protected training time should be provided, combined with accessible support.

Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool.

BACKGROUND: The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard. METHODS: Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups' method was used to set a pass/fail standard. RESULTS: Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach's alpha =0.94) and high inter-rater reliability (Cronbach's alpha =0.91). The total item score significantly correlated with the global score (rs=0.86, P<0.001). Participants' first performance correlated to second performance (test-retest reliability) with a Pearson's r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points. CONCLUSIONS: The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.

Effect of revascularization on lower extremity muscle function in combined type 2 diabetes and critical limb threatening ischemia.

BACKGROUND: Critical limb-threatening ischemia (CLTI) and type 2 diabetes (T2D) frequently co-exist and often with less favorable outcome after revascularization. The objective was to evaluate the effects of revascularization on muscle function, perfusion and mitochondrial respiration in patients with combined CLTI and T2D. METHODS: A prospective translational observational study. Two groups of patients facing unilateral peripheral revascularization was included: Patients suffering from combined disease with CLTI+T2D (N.=14) and patients suffering from CLTI (N.=15). During pedal exercise testing, calf muscle perfusion was monitored with near-infrared spectroscopy (NIRS) and leg arterial volume flow in the common femoral artery with duplex ultrasound. Calf muscle biopsy and subsequent assessment of mitochondrial respiratory capacity on isolated muscle fibers was performed. Tests was performed before and six weeks after revascularization. RESULTS: After revascularization, patients CLTI+T2D improved in muscle force from 8.48 kg (CI: 7.49-9.46) to 13.11 kg (CI: 11.58-14.63), (P<0.001). Conversely, muscle force in patients suffering from non-diabetic CLTI decreased from 10.03 kg (CI: 9.1-10.96) to 9.73 kg (CI: 8.77-10.69), (P=0.042). Muscle oxygenation during exercise improved more in the CLTI+T2D group (6.36 µM/kg/s [CI: 5.71-7.01] compared to 2.11 µM/kg/s [CI:1.38-2.83] in the CLTI group; P=0.002). No improvement or difference between groups regarding mitochondrial function was detected. CONCLUSIONS: Patients with combined CLTI+T2D, had an unsuspected better effect of revascularization compared to patients with non-diabetic CLTI, in terms of increased muscle force and improved muscle perfusion. Further studies are needed to elucidate the apparent interaction of the CLTI and T2D syndromes.

Learning Curves and Competences of Vascular Trainees Performing Open Aortic Repair in a Simulation-Based Environment.

BACKGROUND: The aim of this study was to analyze learning curves and competency gains of novice vascular trainees when performing open aortic repair in a simulation-based environment. METHODS: This was a prospective study of 16 vascular trainees performing infrarenal open aortic repair on an inanimate abdominal aortic aneurysm simulator with pulsatile pressure and flow. Each participant performed 4 procedures as a primary surgeon while getting structured feedback by a supervising experienced vascular surgeon. All sessions were video recorded and were anonymously and independently assessed by 3 rater-trained experts on an online platform using the newly validated open abdominal aortic aneurysm repair of technical expertise assessment tool. All supervisor interferences and procedure time was noted. RESULTS: Reliability between raters was excellent (intraclass correlation coefficient = 0.92). Participants' mean scores almost doubled during the course between the first (13.4, 95% confidence interval [CI], 6.8-20) and fourth session (29.8, 95% CI, 26.3-33.3) with a mean difference of 14.6 (P < 0.001). Supervisor interference also decreased significantly from mean 3.0 (95% CI, 1.5-3.6) in the first to 0.7 (95% CI, 0.4-1.0) in the fourth session (P = 0.004). Procedure time decreased with a mean of 24 minutes: from 81 min (95% CI, 71.8-90.3) to 57 min (95% CI, 51.1-63.2, P < 0.001). There was a significant negative correlation between procedure time and the Open Abdominal Aortic Aneurysm Repair of Technical Expertise score (Pearson's r = -0.72, P < 0.01). Only half of the participants passed the pass/fail score of 27.7 points during the course. CONCLUSIONS: Novice vascular trainees achieve skills and competencies in open aortic repair in a simulated setting with dedicated supervision and feedback and can become ready for supervised surgery on real patients. Learning rates are individual, and it is important to construct training programs with emphasis on proficiency and not merely attending a course.

Simulation Based Training and Assessment in Open Vascular Surgery: A Systematic Review.

OBJECTIVE: The aim of this study was to systematically review the literature and give evidence based recommendations for future initiatives for simulation based training (SBT) and assessment in open vascular surgery. DATA SOURCES: PubMed, Embase, and the Cochrane Library. REVIEW METHODS: A systematic review of PubMed, Embase, and the Cochrane Library was performed, with the last search on 31 March 2020, to identify studies describing SBT and assessment in open vascular surgery. Kirkpatrick's levels for efficacy of training were evaluated. Validity evidence for assessment tools was evaluated according to the recommended contemporary framework by Messick. RESULTS: Of 2 844 studies, 51 were included for data extraction. A high degree of heterogeneity in reporting standards and varying types of simulation was found. Vascular anastomosis was the most frequently simulated technical skill (43%). Assessment was mostly carried out using the Objective Structured Assessment of Technical Skills (55%). Validity evidence for assessment tools was found using outdated frameworks, and only one study used Messick's framework. Self directed training is valuable, the low trainer to trainee ratio is important to maximise efficiency, and experienced vascular surgeons are the most effective trainers. CONCLUSION: Carefully designed and structured SBT is effective and can improve technical skills, especially in less experienced trainees. However, the supporting evidence lacks homogeneity in the reporting standards and types of simulations. Pass/fail standards that support proficiency based learning and studies investigating skills transfer should be the focus in future studies. Validity evidence of assessment tools needs to be addressed using contemporary frameworks.

European Society for Vascular Surgery (ESVS) Certification of Theoretical and Practical Competences in Basic Vascular Ultrasound: Validity Investigation of the Assessment Tools.

OBJECTIVE: The aim of this study was to gather validity evidence for the Assessment of basic Vascular Ultrasound Expertise (AVAUSE) tool, and to establish a pass/fail score for each component, to support decisions for certification. METHODS: A cross sectional validation study performed during the European Society for Vascular Surgery's annual meeting. Validity evidence was sought for the theoretical test and two practical tests based on Messick's framework. The participants were vascular surgeons, vascular surgical trainees, sonographers, and nurses with varying experience levels. Five vascular ultrasound experts developed the theoretical and two practical test components of the AVAUSE tool for each test component. Two stations were set up for carotid examinations and two for superficial venous incompetence (SVI) examinations. Eight raters were assigned in pairs to each station. Three methods were used to set pass/fail scores: contrasting groups' method; rater consensus; and extended Angoff. RESULTS: Nineteen participants were enrolled. Acceptable internal consistency reliability (Cronbach's alpha) for the AVAUSE theoretical (0.93), carotid (0.84), and SVI (0.65) practical test were shown. In the carotid examination, inter-rater reliability (IRR) for the two rater pairs was good: 0.68 and 0.78, respectively. The carotid scores correlated significantly with years of experience (Pearson's r = 0.56, p = .013) but not with number of examinations in the last five years. For SVI, IRR was excellent at 0.81 and 0.87. SVI performance scores did not correlate with years of experience and number of examinations. The pass/fail score set by the contrasting groups' method was 29 points out of 50. The rater set pass/fail scores were 3.0 points for both carotid and SVI examinations and were used to determine successful participants. Ten of 19 participants passed the tests and were certified. CONCLUSION: Validity evidence was sought and established for the AVAUSE comprehensive tool, including pass/fail standards. AVAUSE can be used to assess competences in basic vascular ultrasound, allowing operators to progress towards independent practice.

Ensuring Competency in Open Aortic Aneurysm Repair - Development and Validation of a New Assessment Tool.

OBJECTIVE: The aims of this study were to develop a procedure specific assessment tool for open abdominal aortic aneurysm (AAA) repair, gather validity evidence for the tool and establish a pass/fail standard. METHODS: Validity was studied based on the contemporary framework by Messick. Three vascular surgeons experienced in open AAA repair and an expert in assessment and validation within medical education developed the OPEn aortic aneurysm Repair Assessment of Technical Expertise (OPERATE) tool. Vascular surgeons with varying experiences performed open AAA repair in a standardised simulation based setting. All procedures were video recorded with the faces anonymised and scored independently by three experts in a mutual blinded setup. The Angoff standard setting method was used to establish a credible pass/fail score. RESULTS: Sixteen novices and nine experienced open vascular surgeons were enrolled. The OPERATE tool achieved high internal consistency (Cronbach's alpha .92) and inter-rater reliability (Cronbach's alpha .95) and was able to differentiate novices and experienced surgeons with mean scores (higher score is better) of 13.4 ± 12 and 25.6 ± 6, respectively (p = .01). The pass/fail score was set high (27.7). One novice passed the test while six experienced surgeons failed. CONCLUSION: Validity evidence was established for the newly developed OPERATE tool and was able to differentiate between novices and experienced surgeons providing a good argument that this tool can be used for both formative and summative assessment in a simulation based environment. The high pass/fail score emphasises the need for novices to train in a simulation based environment up to a certain level of competency before apprenticeship training in the clinical environment under the tutelage of a supervisor. Familiarisation with the simulation equipment must be ensured before performance is assessed as reflected by the low scores in the experienced group's first attempt.