Operator competence in fetoscopic laser surgery for twin-twin transfusion syndrome: validation of a procedure-specific evaluation tool.

ABSTRACT

OBJECTIVES: Fetoscopic laser surgery for twin-twin transfusion syndrome is a procedure for which no objective tools exist to assess technical skills. To ensure that future fetal surgeons reach competence prior to performing the procedure unsupervised, we developed a performance assessment tool. The aim of this study was to validate this assessment tool for reliability and construct validity. METHODS: We made use of a procedure-specific evaluation instrument containing all essential steps of the fetoscopic laser procedure, which was previously created using Delphi methodology. Eleven experts and 13 novices from three fetal medicine centers performed the procedure on the same simulator. Two independent observers assessed each surgery using the instrument (maximum score 52). Interobserver reliability was assessed using Spearman correlation. We compared the performance of novices and experts to assess construct validity. RESULTS: The interobserver reliability was high (Rs = 0.974, P < 0.001). Checklist scores for experts and novices were significantly different; the median score for novices was 28/52 (54%), whereas that for experts was 47.5/52 (91%) (P < 0.001). The procedure time and fetoscopy time were significantly shorter (P < 0.001) for experts. Residual anastomoses were found in 1/11 (9%) procedures performed by experts and in 9/13 (69%) procedures performed by novices (P = 0.005). Multivariable analysis showed that the checklist score, independent of age and gender, predicted competence. CONCLUSIONS: The procedure-specific assessment tool for fetoscopic laser surgery shows good interobserver reliability and discriminates experts from novices. This instrument may therefore be a useful tool in the training curriculum for fetal surgeons. Further intervention studies with reassessment before and after training may increase the construct validity of the tool. Copyright © 2015 ISUOG. Published by John Wiley & Sons Ltd.