Computer-aided quality assessment of endoscopist competence during colonoscopy: A systematic review.

BACKGROUND AND AIMS: Endoscopists' competence can vary widely, as shown in the variation in adenoma detection rate (ADR). Computer-aided quality assessment (CAQ) can automatically assess performance during individual procedures. This review aims to identify and describe different CAQ systems for colonoscopy. METHODS: A systematic review of the literature was done using MEDLINE, EMBASE, and SCOPUS based on three blocks of terms according to the inclusion criteria: Colonoscopy, Competence assessment, and Automatic evaluation. Articles were systematically reviewed by two reviewers, first by abstract and then in full text. The methodological quality was assessed using the Medical Education Research Study Quality Instrument (MERSQI). RESULTS: 12,575 studies were identified, 6,831 remained after removal of duplicates, and 6,806 did not pass the eligibility criteria and were excluded, leaving thirteen studies for final analysis. Five categories of CAQ systems were identified: Withdrawal speedometer (seven studies), Scope movement analysis (three studies), Effective withdrawal time (one study), Fold examination quality (one study), and Visual gaze pattern (one study). The withdrawal speedometer was the only CAQ system that tested its feedback by examining changes in ADR. Three studies observed an improvement in ADR, and two studies did not. The methodological quality of the studies was high (mean MERSQI 15.2 points, maximum 18 points). CONCLUSIONS: Thirteen studies developed or tested CAQ systems, most frequently by correlating it to ADR. Only five studies tested feedback by implementing the CAQ system. A meta-analysis was impossible due to the heterogeneous study designs, and more studies are warranted.

Artificial Intelligence Improves Novices' Bronchoscopy Performance: A Randomized Controlled Trial in a Simulated Setting.

BACKGROUND: Navigating through the bronchial tree and visualizing all bronchial segments is the initial step toward learning flexible bronchoscopy. A novel bronchial segment identification system based on artificial intelligence (AI) has been developed to help guide trainees toward more effective training. RESEARCH QUESTION: Does feedback from an AI-based automatic bronchial segment identification system improve novice bronchoscopists' end-of-training performance? STUDY DESIGN AND METHODS: The study was conducted as a randomized controlled trial in a standardized simulated setting. Novices without former bronchoscopy experience practiced on a mannequin. The feedback group (n = 10) received feedback from the AI, and the control group (n = 10) trained according to written instructions. Each participant decided when to end training and proceed to performing a full bronchoscopy without any aids. RESULTS: The feedback group performed significantly better on all three outcome measures (median difference, P value): diagnostic completeness (3.5 segments, P < .001), structured progress (13.5 correct progressions, P < .001), and procedure time (-214 seconds, P = .002). INTERPRETATION: Training guided by this novel AI makes novices perform more complete, more systematic, and faster bronchoscopies. Future studies should examine its use in a clinical setting and its effects on more advanced learners.

Systematic Endobronchial Ultrasound - The Six Landmarks Approach.

Lung cancer is the leading cause of cancer mortality globally. To ensure the correct diagnosis and staging in relation to treatment options, it is crucial to obtain valid biopsies from suspected tumors and mediastinal lymph nodes and accurate identification of the mediastinal lymph nodes regarding the Tumor-Node-Metastasis (TNM)-classification. Flexible bronchoscopy combined with endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is essential in the workup and diagnosis of patients suspected of lung cancer. EBUS-TBNA from mediastinal lymph nodes is a technically difficult procedure and has been identified as one of the most important procedures that should be integrated into a simulation-based training program for invasive pulmonologists. More specific guidelines that govern training in EBUS-TBNA are needed to meet this demand. We hereby propose a systematic, stepwise approach with specific attention to six landmarks that support the endoscopist when navigating through the bronchial maze. The stepwise approach relying on the six landmarks is used in the EBUS-certified training program offered by the European Respiratory Society (ERS).

Systematic Bronchoscopy: the Four Landmarks Approach.

Flexible bronchoscopy is a technically difficult procedure and has been identified as the most important procedure that should be integrated into a simulation-based training program for pulmonologists. However, more specific guidelines that govern bronchoscopy training are needed to meet this demand. To ensure patients a competent examination, we propose a systematic, stepwise approach, splitting the procedure into four "landmarks" to support novice endoscopists navigating the bronchial maze. The procedure can be evaluated based on three established outcome measures to ensure a thorough and effective inspection of the bronchial tree: diagnostic completeness, structured progress, and procedure time. The stepwise approach relying on the four landmarks is used at all simulation centers in Denmark and is being implemented in the Netherlands. To provide instant feedback to novice bronchoscopists when training and to relieve time constraints from consultants, we suggest that future studies should implement artificial intelligence as a feedback and certification tool when training new bronchoscopists.

When Pulmonologists Are Novice to Navigational Bronchoscopy, What Predicts Diagnostic Yield?

Predicting factors of diagnostic yield in electromagnetic navigation bronchoscopy (ENB) have been explored in a number of previous studies based on data from experienced operators. However, little is known about predicting factors when the procedure is carried out by operators in the beginning of their learning curve. We here aim to identify the role of operators' experience as well as lesion- and procedure characteristics on diagnostic yield of ENB procedures in the hands of novice ENB operators. Four operators from three centers without prior ENB experience were enrolled. The outcome of consecutive ENB procedures was assessed and classified as either diagnostic or non-diagnostic and predicting factors of diagnostic yield were assessed. A total of 215 procedures were assessed. A total of 122 (57%) of the ENB procedures resulted in diagnostic biopsies. Diagnostic ENB procedures were associated with a minor yet significant difference in tumor size compared to non-diagnostic/inconclusive ENB procedures (28 mm vs. 24 mm; p = 0.03). Diagnostic ENB procedures were associated with visible lesions at either fluoroscopy (p = 0.003) or radial endobronchial ultrasound (rEBUS), (p = 0.001). In the logistic regression model, lesion visibility on fluoroscopy, but none of operator experience, the presence of a bronchus sign, lesion size, or location nor visibility on rEBUS significantly impacted the diagnostic yield. In novice ENB operators, lesion visibility on fluoroscopy was the only factor found to increase the chance of obtaining a diagnostic sample.

Automatic and Objective Assessment of Motor Skills Performance in Flexible Bronchoscopy.

BACKGROUND: Motor skills have been identified as a useful measure to evaluate competency in bronchoscopy. However, no automatic assessment system of motor skills with a clear pass/fail criterion in flexible bronchoscopy exists. OBJECTIVES: The objective of the study was to develop an objective and automatic measure of motor skills in bronchoscopy and set a pass/fail criterion. METHODS: Participants conducted 3 bronchoscopies each in a simulated setting. They were equipped with a Myo Armband that measured lower arm movements through an inertial measurement unit, and hand and finger motions through electromyography sensors. These measures were composed into an objective and automatic composite score of motor skills, the motor bronchoscopy skills score (MoBSS). RESULTS: Twelve novices, eleven intermediates, and ten expert bronchoscopy operators participated, resulting in 99 procedures available for assessment. MoBSS was correlated with a higher diagnostic completeness (Pearson's correlation, r = 0.43, p < 0.001) and a lower procedure time (Pearson's correlation, r = -0.90, p < 0.001). MoBSS was able to differentiate operator performance based on the experience level (one-way ANOVA, p < 0.001). Using the contrasting groups' method, a passing score of -0.08 MoBSS was defined that failed 30/36 (83%) novice, 5/33 (15%) intermediate, and 1/30 (3%) expert procedures. CONCLUSIONS: MoBSS can be used as an automatic and unbiased assessment tool for motor skills performance in flexible bronchoscopy. MoBSS has the potential to generate automatic feedback to help guide trainees toward expert performance.

Using structured progress to measure competence in flexible bronchoscopy.

BACKGROUND: Flexible bronchoscopy is a core invasive procedure in pulmonary medicine and training in the procedure is mandatory. Diagnostic completeness and procedure time have been identified as useful measures of competence. No outcome measures have been developed regarding navigational path in bronchoscopy to assess whether the bronchial segments have been identified in an arbitrary or structured order. We investigated whether a new outcome measure for structured progression could be used to assess competency in flexible bronchoscopy. METHODS: The study was designed as a prospective comparative study. Twelve novices, eleven intermediates, and ten expert bronchoscopy operators completed three full bronchoscopies in a simulated setting on a phantom. The following outcome measures were collected through a checklist evaluation by a trained rater: Diagnostic Completeness as amount of visualized bronchial segments, Structured Progress between the bronchial segments in ascending order, and average intersegmental time (AIT). RESULTS: The ability to follow a structured ascending path through the bronchial tree correlated with a higher amount of identified bronchial segments (Pearson's correlation, r=0.62, P<0.001) and a lower AIT (Pearson's correlation, r=-0.52, P<0.001). CONCLUSIONS: Operators should advance through the bronchial tree in a structured ascending order to ensure systematic progress with the highest level of diagnostic yield and the lowest procedure time. Structured progression is a useful measure to evaluate competency in flexible bronchoscopy.

Simulation-Based Mastery Learning of Flexible Bronchoscopy: Deciding Factors for Completion.

BACKGROUND: Several studies have shown the beneficial effects of mastery learning of a simulation-based course, but not all trainees complete it. OBJECTIVES: The purpose of this study was to find deciding factors for the completion of a simulation-based mastery learning course with distributive practice in flexible bronchoscopy. METHODS: Seventy-seven trainees who signed up for the course were invited to a survey for deciding factors of completing the course. Sixty-two (81%) trainees answered the survey. RESULTS: Male trainees were more likely to complete the course. The most important factor for completion was clinical relevance, and the most important factor for not completing the course was being "too busy." CONCLUSION: Several deciding factors for completing the course were identified. Successful simulation-based mastery learning courses should be clinically relevant, and the trainees should be provided protected time to complete the training. The instructional design should also be adapted systematically for male and female trainees to achieve the necessary competencies.