Direct observation of procedural skills (DOPS) assessment in diagnostic gastroscopy: nationwide evidence of validity and competency development during training.

BACKGROUND: Validated competency assessment tools and the data supporting milestone development during gastroscopy training are lacking. We aimed to assess the validity of the formative direct observation of procedural skills (DOPS) assessment tool in diagnostic gastroscopy and study competency development using DOPS. METHODS: This was a prospective multicentre (N = 275) analysis of formative gastroscopy DOPS assessments. Internal structure validity was tested using exploratory factor analysis and reliability estimated using generalisability theory. Item and global DOPS scores were stratified by lifetime procedure count to define learning curves, using a threshold determined from receiver operator characteristics (ROC) analysis. Multivariable binary logistic regression analysis was performed to identify independent predictors of DOPS competence. RESULTS: In total, 10086 DOPS were submitted for 987 trainees. Exploratory factor analysis identified three distinct item groupings, representing 'pre-procedure', 'technical', and 'post-procedure non-technical' skills. From generalisability analyses, sources of variance in overall DOPS scores included trainee ability (31%), assessor stringency (8%), assessor subjectivity (18%), and trainee case-to-case variation (43%). The combination of three assessments from three assessors was sufficient to achieve the reliability threshold of 0.70. On ROC analysis, a mean score of 3.9 provided optimal sensitivity and specificity for determining competency. This threshold was attained in the order of 'pre-procedure' (100-124 procedures), 'technical' (150-174 procedures), 'post-procedure non-technical' skills (200-224 procedures), and global competency (225-249 procedures). Higher lifetime procedure count, DOPS count, surgical trainees and assessors, higher trainee seniority, and lower case difficulty were significant multivariable predictors of DOPS competence. CONCLUSION: This study establishes milestones for competency acquisition during gastroscopy training and provides validity and reliability evidence to support gastroscopy DOPS as a competency assessment tool.

Changes in scoring of Direct Observation of Procedural Skills (DOPS) forms and the impact on competence assessment.

BACKGROUND: Direct Observation of Procedural Skills (DOPS) is an established competence assessment tool in endoscopy. In July 2016, the DOPS scoring format changed from a performance-based scale to a supervision-based scale. We aimed to evaluate the impact of changes to the DOPS scale format on the distribution of scores in novice trainees and on competence assessment. METHODS: We performed a prospective, multicenter (n = 276), observational study of formative DOPS assessments in endoscopy trainees with ≤ 100 lifetime procedures. DOPS were submitted in the 6-months before July 2016 (old scale) and after (new scale) for gastroscopy (n = 2998), sigmoidoscopy (n = 1310), colonoscopy (n = 3280), and polypectomy (n = 631). Scores for old and new DOPS were aligned to a 4-point scale and compared. RESULTS: 8219 DOPS (43 % new and 57 % old) submitted for 1300 trainees were analyzed. Compared with old DOPS, the use of the new DOPS was associated with greater utilization of the lowest score (2.4 % vs. 0.9 %; P < 0.001), broader range of scores, and a reduction in competent scores (60.8 % vs. 86.9 %; P < 0.001). The reduction in competent scores was evident on subgroup analysis across all procedure types (P < 0.001) and for each quartile of endoscopy experience. The new DOPS was superior in characterizing the endoscopy learning curve by demonstrating progression of competent scores across quartiles of procedural experience. CONCLUSIONS: Endoscopy assessors applied a greater range of scores using the new DOPS scale based on degree of supervision in two cohorts of trainees matched for experience. Our study provides construct validity evidence in support of the new scale format.

The Efficacy of a Newly Designed, Easy-to-Manufacture Training Simulator for Endoscopic Biopsy of the Stomach.

BACKGROUND/AIMS: We developed a new endoscopic biopsy training simulator and determined its efficacy for improving the endoscopic biopsy skills of beginners. METHODS: This biopsy simulator, which presents seven biopsy sites, was constructed using readily available materials. We enrolled 40 participants: 14 residents, 11 first-year clinical fellows, 10 second-year clinical fellows, and five staff members. We recorded the simulation completion time for all participants, and then simulator performance was assessed via a questionnaire using the 7-point Likert scale. RESULTS: The mean times for completing the five trials were 417.7±138.8, 145.2±31.5, 112.7±21.9, and 90.5±20.0 seconds for the residents, first-year clinical fellows, second-year clinical fellows, and staff members, respectively. Endoscopists with less experience reported that they found this simulator more useful for improving their biopsy technique (6.8±0.4 in the resident group and 5.7±1.0 in the first-year clinical fellow group). The realism score of the simulator for endoscopic handling was 6.4±0.5 in the staff group. CONCLUSIONS: This new, easy-to-manufacture endoscopic biopsy simulator is useful for biopsy training for beginner endoscopists and shows good efficacy and realism.

The learning curve for endoscopic submucosal dissection in an established experimental setting.

INTRODUCTION: Endoscopic submucosal dissection (ESD) has become a standard therapy for early gastric neoplasia, particularly in Asian countries. From a safety and efficacy standpoint, simulation training may empower the endoscopist to be able to learn the basic tenets of ESD in a safe, controlled and supervised setting before attempting first in humans. METHODS AND PROCEDURES: This study was designed as a prospective ex vivo study. Ex vivo porcine organs were utilized in the EASIE-R endoscopic simulator. A total of 150 artificial lesions, each 2 × 2 cm in size, were created in fresh ex vivo porcine stomachs at six different anatomical sites (fundus anterior and posterior, body anterior and posterior, antrum anterior and posterior). Three examiners (2 beginners, 1 expert) participated in this study. All parameters (procedure time, specimen size, en-bloc resection status, perforation) were recorded by an independent observer for each procedure. RESULTS: All 150 lesions were successfully resected using the ESD technique by the three endoscopists. After 30 ESD cases, the two novices performed ESD with a 100% en-bloc resection rate and without perforation. For the procedures performed by the novices, the total procedure time and perforation rate in the last 30 cases were significantly lower than during the first 30 cases (p < 0.05). CONCLUSIONS: Our study suggests that performing 30 ESD resections in an ex vivo simulator leads to a significant improvement in safety and efficiency of performing the ESD technique.