Impact of study design on adenoma detection in the evaluation of artificial intelligence-aided colonoscopy: a systematic review and meta-analysis.

BACKGROUND AND AIMS: Randomized controlled trials (RCTs) have reported that artificial intelligence (AI) improves endoscopic polyp detection. Different methodologies-namely, parallel and tandem designs-have been used to evaluate the efficacy of AI-assisted colonoscopy in RCTs. Systematic reviews and meta-analyses have reported a pooled effect that includes both study designs. However, it is unclear whether there are inconsistencies in the reported results of these 2 designs. Here, we aimed to determine whether study characteristics moderate between-trial differences in outcomes when evaluating the effectiveness of AI-assisted polyp detection. METHODS: A systematic search of Ovid MEDLINE, Embase, Cochrane Central, Web of Science, and IEEE Xplore was performed through March 1, 2023, for RCTs comparing AI-assisted colonoscopy with routine high-definition colonoscopy in polyp detection. The primary outcome of interest was the impact of study type on the adenoma detection rate (ADR). Secondary outcomes included the impact of the study type on adenomas per colonoscopy and withdrawal time, as well as the impact of geographic location, AI system, and endoscopist experience on ADR. Pooled event analysis was performed using a random-effects model. RESULTS: Twenty-four RCTs involving 17,413 colonoscopies (AI assisted: 8680; non-AI assisted: 8733) were included. AI-assisted colonoscopy improved overall ADR (risk ratio [RR], 1.24; 95% confidence interval [CI], 1.17-1.31; I2 = 53%; P < .001). Tandem studies collectively demonstrated improved ADR in AI-aided colonoscopies (RR, 1.18; 95% CI, 1.08-1.30; I2 = 0%; P < .001), as did parallel studies (RR, 1.26; 95% CI, 1.17-1.35; I2 = 62%; P < .001), with no statistical subgroup difference between study design. Both tandem and parallel study designs revealed improvement in adenomas per colonoscopy in AI-aided colonoscopies, but this improvement was more marked among tandem studies (P < .001). AI assistance significantly increased withdrawal times for parallel (P = .002), but not tandem, studies. ADR improvement was more marked among studies conducted in Asia compared to Europe and North America in a subgroup analysis (P = .007). Type of AI system used or endoscopist experience did not affect overall improvement in ADR. CONCLUSIONS: Either parallel or tandem study design can capture the improvement in ADR resulting from the use of AI-assisted polyp detection systems. Tandem studies powered to detect differences in endoscopic performance through paired comparison may be a resource-efficient method of evaluating new AI-assisted technologies.

Development and initial validation of a video-based peroral endoscopic myotomy assessment tool.

BACKGROUND AND AIMS: Video analysis has emerged as a potential strategy for performance assessment and improvement. We aimed to develop a video-based skill assessment tool for peroral endoscopic myotomy (POEM). METHODS: POEM was deconstructed into basic procedural components through video analysis by an expert panel. A modified Delphi approach and 2 validation exercises were conducted to refine the POEM assessment tool (POEMAT). Twelve assessors used the final POEMAT version to grade 10 videos. Fully crossed generalizability (G) studies investigated the contributions of assessors, endoscopists' performance, and technical elements to reliability. G coefficients below .5 were considered unreliable, between .5 and .7 as modestly reliable, and above .7 as indicative of satisfactory reliability. RESULTS: After task deconstruction, discussions, and the modified Delphi process, the final POEMAT comprised 9 technical elements. G analysis showed low variance for endoscopist performance (.8%-24.9%) and high interrater variability (range, 63.2%-90.1%). The G score was moderately reliable (≥.60) for "submucosal tunneling" and "myotomy" and satisfactorily reliable (≥.70) for "active hemostasis" and "mucosal closure." CONCLUSIONS: We developed and established initial content and response process validity evidence for the POEMAT. Future steps include appraisal of the tool using a wider range of POEM videos to establish and improve the discriminative validity of this tool.

Colonoscopy competence assessment tools: a systematic review of validity evidence.

BACKGROUND: Assessment tools are essential for endoscopy training, being required to support feedback provision, optimize learner capabilities, and document competence. We aimed to evaluate the strength of validity evidence that supports the available colonoscopy direct observation assessment tools using the unified framework of validity. METHODS: We systematically searched five databases for studies investigating colonoscopy direct observation assessment tools from inception until 8 April 2020. We extracted data outlining validity evidence (content, response process, internal structure, relations to other variables, and consequences) from the five sources and graded the degree of evidence, with a maximum score of 15. We assessed educational utility using an Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI). RESULTS: From 10 841 records, we identified 27 studies representing 13 assessment tools (10 adult, 2 pediatric, 1 both). All tools assessed technical skills, while 10 each assessed cognitive and integrative skills. Validity evidence scores ranged from 1-15. The Assessment of Competency in Endoscopy (ACE) tool, the Direct Observation of Procedural Skills (DOPS) tool, and the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) had the strongest validity evidence, with scores of 13, 15, and 14, respectively. Most tools were easy to use and interpret, and required minimal resources. MERSQI scores ranged from 9.5-11.5 (maximum score 14.5). CONCLUSIONS: The ACE, DOPS, and GiECAT have strong validity evidence compared with other assessments. Future studies should identify barriers to widespread implementation and report on the use of these tools in credentialing examinations.

Characterizing the Development of Colonoscopy Competence Using Assessment Data.

Assessment of endoscopist competence is an increasingly important component of colonoscopy quality assurance. In this study from the Joint Advisory Group on Gastrointestinal Endoscopy, validity evidence is provided for the use of the Direct Observation of Procedural Skills assessment tool in the formative setting during training. In this national UK dataset, overall colonoscopy competence was typically achieved after 200-249 procedures, although certain complex procedural skills ("proactive problem solving" and "loop management") had not reached the threshold for competence even after 300 procedures. These data will help inform the development and/or refinement of certification policies and practices in jurisdictions around the world.

Clinical, endoscopic, and histologic characteristics of lymphocytic esophagitis: a systematic review.

OBJECTIVE: Lymphocytic esophagitis (LyE) is a novel, yet poorly described, clinicopathologic entity. The aim of this systematic review was to characterize the demographic, clinical, endoscopic, and histologic features of LyE in observational studies of adult and pediatric patients. DESIGN: We searched the Embase, MEDLINE, and SCOPUS databases for relevant studies in 2018. Two authors reviewed and extracted data from studies that met the inclusion and exclusion criteria. RESULTS: We identified 20 studies for analysis of demographic, clinical, and endoscopic features of LyE. The mean age ranged from 9 to 67 years. When pooled, there were 231 (52.7%) patients with LyE that were female. The most common presenting symptom was dysphagia reported in 191 (48.8%) patients. On endoscopy, most patients with LyE tended to have abnormal findings (69.0%), which included erosive esophagitis, multiple esophageal rings, linear furrows, and narrow-caliber esophagus. In the 31 studies used to assess the histologic definition, the cut-off number of intraepithelial lymphocytes (IELs) was reported in 16 (51.6%) studies, peripapillary IEL specification in 18 (58.1%) studies, and presence of spongiosis in 6 (19.4%) studies. CONCLUSION: We identified a spectrum of demographic, clinical, and endoscopic findings characteristic of patients with LyE. A consensus on the diagnostic criteria of LyE is required.

The Characteristics of Physicians Who are Re-Disciplined by Medical Boards: A Retrospective Cohort Study.

BACKGROUND: Physician misconduct adversely affects patient safety and is therefore of societal importance. Little work has specifically examined re-disciplined physicians. A study was conducted to compare the characteristics of re-disciplined to first-time disciplined physicians. METHODS: A retrospective review of Canadian physicians disciplined by medical boards between 2000 and 2015 was conducted. Physicians were divided into those disciplined once and those disciplined more than once. Differences in demographics, transgressions, and penalties were evaluated. RESULTS: There were 938 disciplinary events for 810 disciplined physicians with 1 in 8 (n = 101, 12.5%) being re-disciplined. Re-disciplined physicians had up to six disciplinary events in the study period and 4 (4.0%) had events in more than one jurisdiction. Among those re-disciplined, 94 (93.1%) were male, 34 (33.7%) were international medical graduates, and 88 (87.1%) practiced family medicine (n = 59, 58.4%), psychiatry (n = 11, 10.9%), surgery (n = 9, 8.9%), or obstetrics/gynecology (n = 9, 8.9%). The proportion of obstetrician/gynecologists was higher among re-disciplined physicians (8.9% vs. 4.2%, p = 0.048). Re-disciplined physicians had more mental illness (1.7% vs. 0.1%, p = 0.01), unlicensed activity (19.2% vs. 7.2%, p <0.01), and less sexual misconduct (20.1% vs. 27.9%, p = 0.02). License suspension occurred more frequently among those re-disciplined (56.8% vs. 48.0%, p = 0.02) as did license restriction (38.4% vs. 26.7%, p <0.01). License revocation was not different between cohorts (10.9% vs. 13.5%, p = 0.36). CONCLUSION: Re-discipline is not uncommon and underscores the need for better identification of at-risk individuals and optimization of remediation and penalties. The distribution of transgression argues for a national disciplinary database that could improve communication between jurisdictional medical boards.

A hot spot for systemic lupus erythematosus, but not for psoriatic arthritis, identified by spatial analysis suggests an interaction between ethnicity and place of residence.

OBJECTIVE: To describe the spatial distribution of incident cases of systemic lupus erythematosus (SLE) using geographic information systems (GIS). METHODS: Spatial analyses were carried out on 890 SLE patients and 541 psoriatic arthritis (PsA) patients (controls). Age- and sex-adjusted rates for SLE/PsA for each census tract were calculated using denominator population values from the Canadian census. Spatial variations in relative risk were estimated by modeling risk as the product of a time effect, an age effect, and a spatially autocorrelated risk surface to identify hot spots. Patients within the detected hot spot were compared to those outside the hot spot to identify explanatory factors. RESULTS: SLE patients were predominantly female (87.75%) and the incidence rate was highest among those 15-19 years of age (2.4 cases/100,000 person-years). In an SLE hot spot containing 59 patients, 100% of the patients were female and 49.1% (n = 29) were Caucasian, while outside of the hot spot, 86.9% (n = 722) of the patients were female and 68.4% (n = 568) were Caucasian. The proportion of cases of Chinese ethnicity was significantly greater within the hot spot. An interaction was found between Chinese ethnicity and residence within the hot spot, with the risk of SLE to the Chinese population found to be twice the risk to the non-Chinese population. CONCLUSION: GIS was used to map SLE cases and a hot spot was identified after adjustment for age and sex. Ethnicity by itself did not confer an increased risk of SLE, but the interaction of ethnicity with location of residence significantly increased the risk of SLE.

Unified Magnifying Endoscopic Classification (UMEC) of Gastrointestinal Lesions: A North American Validation Study.

Background and study aim: Magnifying endoscopy enables the diagnosis of advanced neoplasia throughout the gastrointestinal tract. The unified magnifying endoscopic classification (UMEC) framework unifies optical diagnosis criteria in the esophagus, stomach, and colon, dividing lesions into three categories: non-neoplastic, intramucosal neoplasia, and deep submucosal invasive cancer. This study aims to ascertain the performance of North American endoscopists when using the UMEC. Methods: In this retrospective cohort study, five North American endoscopists without prior training in magnifying endoscopy independently diagnosed images of gastrointestinal tract lesions using UMEC. All endoscopists were blinded to endoscopic findings and histopathological diagnosis. Using histopathology as the gold standard, the endoscopists' diagnostic performances using UMEC were evaluated. Results: A total of 299 lesions (77 esophagus, 92 stomach, and 130 colon) were assessed. For esophageal squamous cell carcinoma, the sensitivity, specificity, and accuracy ranged from 65.2% (95%CI: 50.9-77.9) to 87.0% (95%CI: 75.3-94.6), 77.4% (95%CI: 60.9-89.6) to 96.8% (95%CI: 86.8-99.8), and 75.3% to 87.0%, respectively. For gastric adenocarcinoma, the sensitivity, specificity, and accuracy ranged from 94.9% (95%CI: 85.0-99.1) to 100%, 52.9% (95%CI: 39.4-66.2) to 92.2% (95%CI: 82.7-97.5), and 73.3% to 93.3%. For colorectal adenocarcinoma, the sensitivity, specificity, and accuracy ranged from 76.2% (95%CI: 62.0-87.3) to 83.3% (95%CI: 70.3-92.5), 89.7% (95%CI: 82.1-94.9) to 97.7% (95%CI: 93.1-99.6), and 86.8% to 90.7%. Intraclass correlation coefficients indicated good to excellent reliability. Conclusion: UMEC is a simple classification that may be used to introduce endoscopists to magnifying narrow-band imaging and optical diagnosis, yielding satisfactory diagnostic accuracy.

How Clinical Supervisors Conceptualize Procedural Entrustment: An Interview-Based Study of Entrustment Decision Making in Endoscopic Training.

PURPOSE: Entrustment is central to assessment in competency-based medical education (CBME). To date, little research has addressed how clinical supervisors conceptualize entrustment, including factors they consider in making entrustment decisions. The aim of this study was to characterize supervisors' decision making related to procedural entrustment, using gastrointestinal endoscopy as a test case. METHOD: Using methods from constructivist grounded theory, the authors interviewed 29 endoscopy supervisors in the United States and Canada across multiple specialties (adult and pediatric gastroenterology, surgery, and family medicine). Semistructured interviews, conducted between April and November 2019, focused on how supervisors conceptualize procedural entrustment, how they make entrustment decisions, and what factors they consider. Transcripts were analyzed using constant comparison to generate an explanatory framework and themes. RESULTS: Three themes were identified from the analysis of interview transcripts: (1) entrustment occurs in varying degrees and fluctuates over time; (2) entrustment decisions can transfer within and across procedural and nonprocedural contexts; (3a) persistent static factors (e.g., supervisor competence, institutional culture, legal considerations) influence entrustment decisions, as do (3b) fluctuating, situated dynamic factors (e.g., trainee skills, patient acuity, time constraints), which tend to change from one training encounter to the next. CONCLUSIONS: In the process of making procedural entrustment decisions, clinical supervisors appear to synthesize multiple dynamic factors against a background of static factors, culminating in a decision of whether to entrust. Entrustment decisions appear to fluctuate over time, and assessors may transfer decisions about specific trainees across settings. Understanding which factors supervisors perceive as influencing their decision making has the potential to inform faculty development, as well as competency committees seeking to aggregate faculty judgments about trainee unsupervised practice. Those leading CBME programs may wish to invest in optimizing the observed static factors, such that these foundational factors are tuned to facilitate trainee learning and achievement of entrustment.

Variable or Fixed? Exploring Entrustment Decision Making in Workplace- and Simulation-Based Assessments.

PURPOSE: Many models of competency-based medical education (CBME) emphasize assessing entrustable professional activities (EPAs). Despite the centrality of EPAs, researchers have not compared rater entrustment decisions for the same EPA across workplace- and simulation-based assessments. This study aimed to explore rater entrustment decision making across these 2 assessment settings. METHOD: An interview-based study using a constructivist grounded theory approach was conducted. Gastroenterology faculty at the University of Toronto and the University of Calgary completed EPA assessments of trainees' endoscopic polypectomy performance in both workplace and simulation settings between November 2019 and January 2021. After each assessment, raters were interviewed to explore how and why they made entrustment decisions within and across settings. Transcribed interview data were coded iteratively using constant comparison to generate themes. RESULTS: Analysis of 20 interviews with 10 raters found that participants (1) held multiple meanings of entrustment and expressed variability in how they justified their entrustment decisions and scoring, (2) held personal caveats for making entrustment decisions "comfortably" (i.e., authenticity, task-related variability, opportunity to assess trainee responses to adverse events, and the opportunity to observe multiple performances over time), (3) experienced cognitive tensions between formative and summative purposes when assessing EPAs, and (4) experienced relative freedom when using simulation to formatively assess EPAs but constraint when using only simulation-based assessments for entrustment decision making. CONCLUSIONS: Participants spoke about and defined entrustment variably, which appeared to produce variability in how they judged entrustment across participants and within and across assessment settings. These rater idiosyncrasies suggest that programs implementing CBME must consider how such variability affects the aggregation of EPA assessments, especially those collected in different settings. Program leaders might also consider how to fulfill raters' criteria for comfortably making entrustment decisions by ensuring clear definitions and purposes when designing and integrating workplace- and simulation-based assessments.

Paediatric to Adult Transition of Care in IBD: Understanding the Current Standard of Care Among Canadian Adult Academic Gastroenterologists.

BACKGROUND: The optimal form of health care delivery for paediatric to adult inflammatory bowel disease transition of care is unknown. The primary purpose of this study was to establish current standard of care across Canada among adult gastroenterologists. METHODS: Adult gastroenterologists interested in transition care were identified. Twenty-five anonymous surveys and 17 semistructured interviews representing 9 adult gastroenterology centers across 6 provinces were completed. Questions focused on the transition process, referral practices, information transfer and access to multidisciplinary resources. The need for expert guidance and transition-related quality indicators were identified. The interviews were audio-recorded, transcribed and coded in duplicate for qualitative thematic analysis. RESULTS: Transition practices included the following: transition clinic (n = 4) versus direct transfer (n = 5). Most transition patients were referred to academic centers. Transfer volume per center ranged from 12 to 100 per year. Transfer of information was optimized with shared electronic medical record and comprehensive referral package. The majority of the programs lacked consistent access to a multidisciplinary team. The strongest attributes related to health care provider interest in transition and complete information transfer. Areas for improvement included increased resource allocation: financial, logistical and personnel. All agreed that a consensus-based guidfmeline for adult phase of transition would be beneficial. Potential quality indicators included adherence to care, depression/anxiety scores and patient knowledge. CONCLUSIONS: This Canadian study of adult gastroenterologists revealed that while practice patterns vary, most agree that a transition clinic with access to multidisciplinary resources would be beneficial. A consensus-based guideline and quality indicators to assess performance may standardize the adult phase of transition and optimize outcomes.