Central Reading of Ulcerative Colitis Clinical Trial Videos Using Neural Networks.

BACKGROUND AND AIMS: Endoscopic disease activity scoring in ulcerative colitis (UC) is useful in clinical practice but done infrequently. It is required in clinical trials, where it is expensive and slow because human central readers are needed. A machine learning algorithm automating the process could elevate clinical care and facilitate clinical research. Prior work using single-institution databases and endoscopic still images has been promising. METHODS: Seven hundred and ninety-five full-length endoscopy videos were prospectively collected from a phase 2 trial of mirikizumab with 249 patients from 14 countries, totaling 19.5 million image frames. Expert central readers assigned each full-length endoscopy videos 1 endoscopic Mayo score (eMS) and 1 Ulcerative Colitis Endoscopic Index of Severity (UCEIS) score. Initially, video data were cleaned and abnormality features extracted using convolutional neural networks. Subsequently, a recurrent neural network was trained on the features to predict eMS and UCEIS from individual full-length endoscopy videos. RESULTS: The primary metric to assess the performance of the recurrent neural network model was quadratic weighted kappa (QWK) comparing the agreement of the machine-read endoscopy score with the human central reader score. QWK progressively penalizes disagreements that exceed 1 level. The model's agreement metric was excellent, with a QWK of 0.844 (95% confidence interval, 0.787-0.901) for eMS and 0.855 (95% confidence interval, 0.80-0.91) for UCEIS. CONCLUSIONS: We found that a deep learning algorithm can be trained to predict levels of UC severity from full-length endoscopy videos. Our data set was prospectively collected in a multinational clinical trial, videos rather than still images were used, UCEIS and eMS were reported, and machine learning algorithm performance metrics met or exceeded those previously published for UC severity scores.

Medication use among patients with Crohn's disease or ulcerative colitis before and after the initiation of advanced therapy.

BACKGROUND: Although various treatments help reduce abdominal pain, real-world pain medication utilization among patients with Crohn's disease (CD) or ulcerative colitis (UC) receiving advanced therapies is poorly understood. The aim is to understand the utilization of pain medication 12 months before and after the initiation of advanced therapies among patients with newly diagnosed CD or UC. METHODS: This retrospective, observational cohort study used administrative medical and pharmacy claims data of patients with CD or UC from HealthCore Integrated Research Database (HIRD®). The data from patients with use of pain medication over 12 months follow-up (after the initiation date of advanced therapies) were collected and analyzed. Differences in the use of pain medication 12 months before and after the initiation of advanced therapies were assessed using McNemar's and Wilcoxon signed-rank test. RESULTS: Prior to initiating advanced therapies, 23.1% of patients with CD (N = 540) received nonsteroidal anti-inflammatory drugs (NSAIDs), 78.1% glucocorticoids, 49.4% opioids, and 29.3% neuromodulators; similarly, 20.9% of patients with UC (N = 373) received NSAIDs, 91.4% glucocorticoids, 40.8% opioids, and 29.5% neuromodulators. After receiving advanced therapies for 12 months, patients reported a reduction in the use of steroids (78.1% vs. 58.9%, P < 0.001 in CD; 91.4% vs. 74.3%, P < 0.001 in UC), opioids (49.4% vs. 41.5%, P = 0.004 in CD; 40.8% vs. 36.5%, P = 0.194 in UC), and NSAIDs (23.1% vs. 15.0%, P < 0.001 in CD; 20.9% vs. 15.8%, P = 0.035 in UC), while the use of neuromodulators significantly increased (29.3% vs. 33.7%, P = 0.007 in CD; 29.5% vs. 35.7%; P = 0.006 in UC). CONCLUSIONS: The use of pain medications such as NSAIDs, glucocorticoids, opioids, and neuromodulators was common among patients with CD or UC. These results highlight that patients with CD or UC continued to receive pain medications even after initiating advanced therapies.

Endoscopy and central reading in inflammatory bowel disease clinical trials: achievements, challenges and future developments.

Central reading, that is, independent, off-site, blinded review or reading of imaging endpoints, has been identified as a crucial component in the conduct and analysis of inflammatory bowel disease clinical trials. Central reading is the final step in a workflow that has many parts, all of which can be improved. Furthermore, the best reading algorithm and the most intensive central reader training cannot make up for deficiencies in the acquisition stage (clinical trial endoscopy) or improve on the limitations of the underlying score (outcome instrument). In this review, academic and industry experts review scoring systems, and propose a theoretical framework for central reading that predicts when improvements in statistical power, affecting trial size and chances of success, can be expected: Multireader models can be conceptualised as statistical or non-statistical (social). Important organisational and operational factors, such as training and retraining of readers, optimal bowel preparation for colonoscopy, video quality, optimal or at least acceptable read duration times and other quality control matters, are addressed as well. The theory and practice of central reading and the conduct of endoscopy in clinical trials are interdisciplinary topics that should be of interest to many, regulators, clinical trial experts, gastroenterology societies and those in the academic community who endeavour to develop new scoring systems using traditional and machine learning approaches.

Comparison of the EMA and FDA Guidelines on Ulcerative Colitis Drug Development.

BACKGROUND & AIMS: In 2016, the European Medicines Agency (EMA) and the US Food and Drug Administration (FDA) released revised EMA and new FDA draft guidelines related to the development of drugs intended for the treatment of ulcerative colitis. We sought to compare and contrast the EMA draft guideline with the FDA draft guidance to facilitate further discussion and perhaps harmonization between the 2 guidelines when they are finalized. METHODS: A concordance document was created by arranging like or similar topics addressed by the guidelines side by side in a tabular format. This concordance table served as a source for writing the narrative. The first draft was subjected to repeated rounds of reviews and revisions by the authors and outside reviewers, all of them familiar with the subject matter from a regulatory science and/or academic perspective. RESULTS: The FDA guidance focuses on end points, whereas the EMA guideline additionally supplies much useful information for trial design. FDA guidance appears more aspirational, suggesting the development of entirely new patient-reported outcome instruments and the incorporation of a not-yet-validated histology instrument into the definition of mucosal healing. CONCLUSIONS: The guidelines by the FDA and the EMA complement each other and together are aimed to further practical drug development toward more clinically relevant end points in ulcerative colitis. Efforts are needed to harmonize the documents.

Electronic chromoendoscopy.

Electronic chromoendoscopy technologies provide image enhancement and may improve the diagnosis of mucosal lesions. Although strides have been made in standardization of image characterization, especially with NBI, further image-to-pathology correlation and validation are required. There is promise for the development of a resect and discard policy for diminutive adenomas by using electronic chromoendoscopy; however, before this can be adopted, further community-based studies are needed. Further validated training tools for NBI, FICE, and i-SCAN will also be required for the use of these techniques to become widespread.

Voting for image scoring and assessment (VISA)--theory and application of a 2 + 1 reader algorithm to improve accuracy of imaging endpoints in clinical trials.

Independent central reading or off-site reading of imaging endpoints is increasingly used in clinical trials. Clinician-reported outcomes, such as endoscopic disease activity scores, have been shown to be subject to bias and random error. Central reading attempts to limit bias and improve accuracy of the assessment, two factors that are critical to trial success. Whether one central reader is sufficient and how to best integrate the input of more than one central reader into one output measure, is currently not known.In this concept paper we develop the theoretical foundations of a reading algorithm that can achieve both objectives without jeopardizing operational efficiency We examine the role of expert versus competent reader, frame scoring of imaging as a classification task, and propose a voting algorithm (VISA: Voting for Image Scoring and Assessment) as the most appropriate solution which could also be used to operationally define imaging gold standards. We propose two image readers plus an optional third reader in cases of disagreement (2 + 1) for ordinary scoring tasks. We argue that it is critical in trials with endoscopically determined endpoints to include the score determined by the site reader, at least in endoscopy clinical trials. Juries with more than 3 readers could define a reference standard that would allow a transition from measuring reader agreement to measuring reader accuracy. We support VISA by applying concepts from engineering (triple-modular redundancy) and voting theory (Condorcet's jury theorem) and illustrate our points with examples from inflammatory bowel disease trials, specifically, the endoscopy component of the Mayo Clinic Score of ulcerative colitis disease activity. Detailed flow-diagrams (pseudo-code) are provided that can inform program design.The VISA "2 + 1" reading algorithm, based on voting, can translate individual reader scores into a final score in a fashion that is both mathematically sound (by avoiding averaging of ordinal data) and in a manner that is consistent with the scoring task at hand (based on decisions about the presence or absence of features, a subjective classification task). While the VISA 2 + 1 algorithm is currently being used in clinical trials, empirical data of its performance have not yet been reported.

Devices for the endoscopic treatment of hemorrhoids.

Multiple endoscopic methods are available to treat symptomatic internal hemorrhoids. Because of its low cost, ease of use, low rate of adverse events, and relative effectiveness, RBL is currently the most widely used technique.

Wireless capsule endoscopy.

Over the last decade, WCE has established itself as a valuable test for imaging the small intestine. It is a safe and relatively easy procedure to perform that can provide valuable information in the diagnosis of small-bowel conditions. Its applications still remain limited within the esophagus and colon. Future developments may include improving visualization within the esophagus and developing technologies that may allow manipulation of the capsule within the GI tract and biopsy capabilities.

Pancreatic and biliary stents.

Biliary and pancreatic stents are used in a variety of benign and malignant conditions including strictures and leaks and in the prevention of post-ERCP pancreatitis.Both plastic and metal stents are safe, effective, and easy to use. SEMSs have traditionally been used for inoperable malignant disease. Covered SEMSs are now being evaluated for use in benign disease. Increasing the duration of patency of both plastic and metal stents remains an important area for future research.

Identifying Measures of Suboptimal Healthcare Interaction (SOHI) to Develop a Claims-Based Model for Predicting Patients with Inflammatory Bowel Disease at Risk for SOHI.

BACKGROUND: Understanding the demographic and clinical characteristics of patients with Inflammatory Bowel Disease (IBD) who are likely to experience poor disease outcomes may allow early interventions that can improve health outcomes. OBJECTIVES: To describe demographic and clinical characteristics of patients with ulcerative colitis (UC) and Crohn's disease (CD) with the presence of at least one Suboptimal Healthcare Interaction (SOHI) event, which can inform the development of a model to predict SOHI in members with IBD based on insurance claims, with the goal of offering these patients some additional intervention. METHODS: We identified commercially insured individuals with IBD between 01 January 2019 and 31 December 2019 using Optum Labs' administrative claims database. The primary cohort was stratified on the presence or absence of ≥ 1 SOHI event (a SOHI-defining data point or characteristic at a specific time point) during the baseline observation period. SOHI was deployed as the basis for the development of a model to predict which individuals with IBD were most likely to continue to have SOHI within a 1-year timeframe (follow-up SOHI) using insurance claims data. All baseline characteristics were analyzed descriptively. Multivariable logistic regression was used to examine the association of follow-up SOHI with baseline characteristics. RESULTS: Of 19,824 individuals, 6872 (34.7%) were found to have follow-up SOHI. Individuals with follow-up SOHI were more likely to have had similar SOHI events in the baseline period than those with non-SOHI. A significantly greater proportion of individuals with SOHI had ≥ 1 claims-based C-reactive protein (CRP) test order and ≥ 1 CRP lab results compared with non-SOHI. Individuals with follow-up SOHI were more likely to incur higher healthcare expenditures and resource utilization as compared with non-SOHI individuals. A few of the most important variables used to predict follow-up SOHI included baseline mesalamine use, count of baseline opioid fills, count of baseline oral corticosteroid fills, baseline extraintestinal manifestations of disease, proxy for baseline SOHI, and index IBD provider specialty. CONCLUSION: Individuals with SOHI are likely to have higher expenditures, higher healthcare resource utilization, uncontrolled disease, and higher CRP lab results as compared with non-SOHI members. Distinguishing SOHI and non-SOHI patients in a dataset could efficiently identify potential cases of poor future IBD outcomes.

We have developed a model for identifying suboptimal healthcare interactions (SOHI) at follow-up and used it to predict the individuals with inflammatory bowel disease (IBD) who are likely to suffer poor healthcare outcomes. Our study showed that the SOHI and non-SOHI cohorts had notable differences in clinical baseline characteristics. Compared with non-SOHI members, individuals with SOHI experienced poor IBD outcomes and incurred higher healthcare resource utilization and costs. Understanding baseline characteristics of patients with SOHI to predict follow-up SOHI can improve health outcomes by early identification of patients with IBD who are likely to experience it. This can help in targeting efforts toward additional care, resulting in greater chances of a well-managed disease.

Can artificial intelligence replace endoscopists when assessing mucosal healing in ulcerative colitis? A systematic review and diagnostic test accuracy meta-analysis.

BACKGROUNDS AND AIMS: Mucosal healing (MH) in inflammatory bowel diseases (IBD) is an important landmark for clinical decision making. Artificial intelligence systems (AI) that automatically deliver the grade of endoscopic inflammation may solve moderate interobserver agreement and the need of central reading in clinical trials. METHODS: We performed a systematic review of EMBASE and MEDLINE databases up to 01/12/2022 following PRISMA and the Joanna Briggs Institute methodologies to answer the following question: "Can AI replace endoscopists when assessing MH in IBD?". The research was restricted to ulcerative colitis (UC), and a diagnostic odds ratio (DOR) meta-analysis was performed. Risk of bias was evaluated with QUADAS-2 tool. RESULTS: A total of 21 / 739 records were selected for full text evaluation, and 12 were included in the meta-analysis. Deep learning algorithms based on convolutional neural networks architecture achieved a satisfactory performance in evaluating MH on UC, with sensitivity, specificity, DOR and SROC of respectively 0.91(CI95 %:0.86-0.95);0.89(CI95 %:0.84-0.93);92.42(CI95 %:54.22-157.53) and 0.957 when evaluating fixed images (n = 8) and 0.86(CI95 %:0.75-0.93);0.91(CI95 %:0.87-0.94);70.86(CI95 %:24.63-203.86) and 0.941 when evaluating videos (n = 6). Moderate-high levels of heterogeneity were noted, limiting the quality of the evidence. CONCLUSIONS: AI systems showed high potential in detecting MH in UC with optimal diagnostic performance, although moderate-high heterogeneity of the data was noted. Standardised and shared AI training may reduce heterogeneity between systems.

Mirikizumab Regulates Genes Involved in Ulcerative Colitis Disease Activity and Anti-TNF Resistance: Results From a Phase 2 Study.

INTRODUCTION: Mirikizumab, a monoclonal antibody targeting the p19 subunit of interleukin (IL)-23, demonstrated efficacy and was well-tolerated in a phase 2 randomized clinical trial in patients with moderate-to-severe ulcerative colitis (UC) (NCT02589665). We explored gene expression changes in colonic tissue from study patients and their association with clinical outcomes. METHODS: Patients were randomized to receive intravenous placebo or 3 mirikizumab induction doses. Patient biopsies were collected at baseline and week 12, and differential gene expression was measured using a microarray platform and compared in all treatment groups to determine differential expression values between baseline and week 12. RESULTS: The greatest improvement in clinical outcomes and placebo-adjusted change from baseline in transcripts at week 12 was observed in the 200 mg mirikizumab group. Transcripts significantly modified by mirikizumab correlate with key UC disease activity indices (modified Mayo score, Geboes score, and Robarts Histopathology Index) and include MMP1, MMP3, S100A8, and IL1ß. Changes in transcripts associated with increased disease activity were decreased after 12 weeks of mirikizumab treatment. Mirikizumab treatment affected transcripts associated with resistance to current therapies, including IL-1ß, OSMR, FCGR3A and FCGR3B, and CXCL6, suggesting that anti-IL23p19 therapy modulates biological pathways involved in resistance to antitumor necrosis factor and Janus kinase inhibitors. DISCUSSION: This is the first large-scale gene expression study of inflamed mucosa from patients with UC treated with anti-IL23p19 therapy. These results provide molecular evidence for mucosal healing from an extensive survey of changes in transcripts that improve our understanding of the molecular effects of IL-23p19 inhibition in UC.

Pathologist-trained machine learning classifiers developed to quantitate celiac disease features differentiate endoscopic biopsies according to modified marsh score and dietary intervention response.

BACKGROUND: Histologic evaluation of the mucosal changes associated with celiac disease is important for establishing an accurate diagnosis and monitoring the impact of investigational therapies. While the Marsh-Oberhuber classification has been used to categorize the histologic findings into discrete stages (i.e., Type 0-3c), significant variability has been documented between observers using this ordinal scoring system. Therefore, we evaluated whether pathologist-trained machine learning classifiers can be developed to objectively quantitate the pathological changes of villus blunting, intraepithelial lymphocytosis, and crypt hyperplasia in small intestine endoscopic biopsies. METHODS: A convolutional neural network (CNN) was trained and combined with a secondary algorithm to quantitate intraepithelial lymphocytes (IEL) with 5 classes on CD3 immunohistochemistry whole slide images (WSI) and used to correlate feature outputs with ground truth modified Marsh scores in a total of 116 small intestine biopsies. RESULTS: Across all samples, median %CD3 counts (positive cells/enterocytes) from villous epithelium (VE) increased with higher Marsh scores (Type 0%CD3 VE = 13.4; Type 1-3%CD3 VE = 41.9, p < 0.0001). Indicators of villus blunting and crypt hyperplasia were also observed (Type 0-2 villous epithelium/lamina propria area ratio = 0.81; Type 3a-3c villous epithelium/lamina propria area ratio = 0.29, p < 0.0001), and Type 0-1 crypt/villous epithelial area ratio = 0.59; Type 2-3 crypt/villous epithelial area ratio = 1.64, p < 0.0001). Using these individual features, a combined feature machine learning score (MLS) was created to evaluate a set of 28 matched pre- and post-intervention biopsies captured before and after dietary gluten restriction. The disposition of the continuous MLS paired biopsy result aligned with the Marsh score in 96.4% (27/28) of the cohort. CONCLUSIONS: Machine learning classifiers can be developed to objectively quantify histologic features and capture additional data not achievable with manual scoring. Such approaches should be further investigated to improve biopsy evaluation, especially for clinical trials.

Baricitinib and primary biliary cholangitis.

BACKGROUND AND AIMS: There is an unmet need for alternative treatments for patients with primary biliary cholangitis (PBC) who do not respond to treatment with ursodeoxycholic acid (UDCA). A proof-of-concept study of baricitinib, an orally administered Janus kinase 1 and 2 inhibitor, was initiated to evaluate its use in PBC patients. APPROACH AND RESULTS: Patients with PBC showing inadequate response or intolerance to UDCA were eligible. This was a randomized, double-blinded placebo-controlled trial. Enrollees were assigned 1:1 to baricitinib (2 mg/day) or placebo. Endpoints included change in alkaline phosphatase (ALP), itch Numeric Rating Score (NRS), and fatigue NRS at 12 weeks post-baseline; exploratory markers included high sensitivity C-reactive protein (hs-CRP) and Enhanced Liver Fibrosis (ELF) score.Due to low enrollment, the study was terminated early. Two patients were enrolled and completed the trial; 1 was randomized to receive baricitinib and 1 to placebo. Over the treatment period, the baricitinib-treated patient demonstrated a 30% decrease in ALP and a 7-point improvement in the itch NRS, but a 2-point increase in the Fatigue NRS. Markers of inflammation and liver fibrosis (hs-CRP and ELF score) also improved over the study period. In contrast, the placebo-treated patient showed no improvement in primary or secondary endpoints. A single non-serious treatment-emergent adverse event of moderate sinusitis was reported by the baricitinib-treated patient at day 47. CONCLUSIONS: In a 12-week trial, a patient with PBC showing inadequate response to treatment with UDCA demonstrated a dramatic response to treatment with baricitinib.

Sequentially Determined Measures of Interobserver Agreement (Kappa) in Clinical Trials May Vary Independent of Changes in Observer Performance.

BACKGROUND: Cohen's kappa is a statistic that estimates interobserver agreement. It was originally introduced to help develop diagnostic tests. Interpretative readings of 2 observers, for example, of a mammogram or other imaging, were compared at a single point in time. It is known that kappa depends on the prevalence of disease and that, therefore, kappas across different settings are hard to compare. METHODS: Using simulation, we examine an analogous situation, not previously described, that occurs in clinical trials where sequential measurements are obtained to evaluate disease progression or clinical improvement over time. RESULTS: We show that weighted kappa, used for multilevel outcomes, changes during the trial even if we keep the performance of the observer constant. CONCLUSIONS: Kappa and closely related measures can therefore only be used with great difficulty, if at all, in quality assurance in clinical trials.

The mincemeat postmortem: forensic aspects of World War II's boldest counterintelligence operation.

On the 30th of April 1943 the waterlogged body of Major William Martin of the Royal Marines drifted toward the shores of the Spanish Atlantic-coast city of Huelva after having been floated from a British submarine. A train of events was set into motion, which helped to change the course of the war. Major Martin, although dead, played a key role in the allied deception operation code-named Mincemeat. Operation Mincemeat has been the subject of several books and a motion picture. The crucial postmortem examination of Major Martin conducted by the Spanish authorities has received surprisingly little attention in the general intelligence literature and details of the medical aspects have to our knowledge never been examined. This article is, in a manner of speaking, a postmortem itself. The events happened 65 years ago and although new material is presented, the interpretation of its significance in the context of the known facts may convince some readers but not others. Nevertheless, we hope that this literary postmortem will fascinate intelligence and medical professionals alike and contribute to the medical and intelligence history of World War II.

Sequentially Determined Measures of Interobserver Agreement (Kappa) in Clinical Trials May Vary Independent of Changes in Observer Performance.

BACKGROUND: Cohen's kappa is a statistic that estimates interobserver agreement. It was originally introduced to help develop diagnostic tests. Interpretative readings of 2 observers, for example, of a mammogram or other imaging, were compared at a single point in time. It is known that kappa depends on the prevalence of disease and that, therefore, kappas across different settings are hard to compare. METHODS: Using simulation, we examine an analogous situation, not previously described, that occurs in clinical trials where sequential measurements are obtained to evaluate disease progression or clinical improvement over time. RESULTS: We show that weighted kappa, used for multilevel outcomes, changes during the trial even if we keep the performance of the observer constant. CONCLUSIONS: Kappa and closely related measures can therefore only be used with great difficulty, if at all, in quality assurance in clinical trials.