Robust endoscopic image mosaicking via fusion of multimodal estimation.

We propose an endoscopic image mosaicking algorithm that is robust to light conditioning changes, specular reflections, and feature-less scenes. These conditions are especially common in minimally invasive surgery where the light source moves with the camera to dynamically illuminate close range scenes. This makes it difficult for a single image registration method to robustly track camera motion and then generate consistent mosaics of the expanded surgical scene across different and heterogeneous environments. Instead of relying on one specialised feature extractor or image registration method, we propose to fuse different image registration algorithms according to their uncertainties, formulating the problem as affine pose graph optimisation. This allows to combine landmarks, dense intensity registration, and learning-based approaches in a single framework. To demonstrate our application we consider deep learning-based optical flow, hand-crafted features, and intensity-based registration, however, the framework is general and could take as input other sources of motion estimation, including other sensor modalities. We validate the performance of our approach on three datasets with very different characteristics to highlighting its generalisability, demonstrating the advantages of our proposed fusion framework. While each individual registration algorithm eventually fails drastically on certain surgical scenes, the fusion approach flexibly determines which algorithms to use and in which proportion to more robustly obtain consistent mosaics.

Robotic Autonomy for Magnetic Endoscope Biopsy.

Magnetically actuated endoscopes are currently transitioning in to clinical use for procedures such as colonoscopy, presenting numerous benefits over their conventional counterparts. Intelligent and easy-to-use control strategies are an essential part of their clinical effectiveness due to the un-intuitive nature of magnetic field interaction. However, work on developing intelligent control for these devices has mainly been focused on general purpose endoscope navigation. In this work, we investigate the use of autonomous robotic control for magnetic colonoscope intervention via biopsy, another major component of clinical viability. We have developed control strategies with varying levels of robotic autonomy, including semi-autonomous routines for identifying and performing targeted biopsy, as well as random quadrant biopsy. We present and compare the performance of these approaches to magnetic endoscope biopsy against the use of a standard flexible endoscope on bench-top using a colonoscopy training simulator and silicone colon model. The semi-autonomous routines for targeted and random quadrant biopsy were shown to reduce user workload with comparable times to using a standard flexible endoscope.

Active Stabilization of Interventional Tasks Utilizing a Magnetically Manipulated Endoscope.

Magnetically actuated robots have become increasingly popular in medical endoscopy over the past decade. Despite the significant improvements in autonomy and control methods, progress within the field of medical magnetic endoscopes has mainly been in the domain of enhanced navigation. Interventional tasks such as biopsy, polyp removal, and clip placement are a major procedural component of endoscopy. Little advancement has been done in this area due to the problem of adequately controlling and stabilizing magnetically actuated endoscopes for interventional tasks. In the present paper we discuss a novel model-based Linear Parameter Varying (LPV) control approach to provide stability during interventional maneuvers. This method linearizes the non-linear dynamic interaction between the external actuation system and the endoscope in a set of equilibria, associated to different distances between the magnetic source and the endoscope, and computes different controllers for each equilibrium. This approach provides the global stability of the overall system and robustness against external disturbances. The performance of the LPV approach is compared to an intelligent teleoperation control method (based on a Proportional Integral Derivative (PID) controller), on the Magnetic Flexible Endoscope (MFE) platform. Four biopsies in different regions of the colon and at two different system equilibria are performed. Both controllers are asked to stabilize the endoscope in the presence of external disturbances (i.e. the introduction of the biopsy forceps through the working channel of the endoscope). The experiments, performed in a benchtop colon simulator, show a maximum reduction of the mean orientation error of the endoscope of 45.8% with the LPV control compared to the PID controller.

Rate of and Factors Associated with Palliative Care Referral among Patients Declined for Liver Transplantation.

Background: End-stage liver disease (ESLD) is associated with high morbidity and mortality, with liver transplantation as the only existing cure. Despite reduced quality of life and limited life expectancy, referral to palliative care (PC) rarely occurs. Moreover, there is scarcity of data on the appropriate timing and type of PC intervention needed. Aim: To evaluate PC utilization and documentation in ESLD patients declined or delisted for transplant at a tertiary care medical center with a large liver transplantation program. Methods: We performed a retrospective cohort study of all patients discussed in Liver Transplant Committee (LTC) at our academic medical center between August 2018 and May 2020 in the United States. Patients declined or delisted for liver transplantation were included. Baseline demographics, model for end-stage liver disease (MELD) score, decompensation events, and reason for transplant ineligibility were recorded. The primary outcome was PC referral. Secondary outcomes included survival from LTC decision, time from LTC decision to PC referral, and code status in relation to PC referral. Results: Of 769 patients discussed at LTC, 135 were declined for transplantation. Thirty-seven (27%) received referral to PC. When adjusting for body mass index and age, MELD score of 21-30 had odds ratio (OR) of 4.5 (95% confidence interval [CI]: 1.7-12.3) and MELD score >30 had OR of 12.8 (95% CI: 3.9-47.7) for PC referral when compared with MELD score <20. When adjusting for MELD score, presence of ascites had OR of 4.6 (95% CI: 1.1-19.1) and presence of multiple complications had OR of 2.2 (95% CI: 2.2-3.8). Conclusions: Only 37 (27%) patients delisted or declined for liver transplantation were referred to PC. MELD score and degree of decompensation were important factors associated with referral. Continued exploration of these data could help guide future studies and help determine timing and criteria for PC referral.

Guidelines for Robotic Flexible Endoscopy at the Time of COVID-19.

Flexible endoscopy involves the insertion of a long narrow flexible tube into the body for diagnostic and therapeutic procedures. In the gastrointestinal (GI) tract, flexible endoscopy plays a major role in cancer screening, surveillance, and treatment programs. As a result of gas insufflation during the procedure, both upper and lower GI endoscopy procedures have been classified as aerosol generating by the guidelines issued by the respective societies during the COVID-19 pandemic-although no quantifiable data on aerosol generation currently exists. Due to the risk of COVID-19 transmission to healthcare workers, most societies halted non-emergency and diagnostic procedures during the lockdown. The long-term implications of stoppage in cancer diagnoses and treatment is predicted to lead to a large increase in preventable deaths. Robotics may play a major role in this field by allowing healthcare operators to control the flexible endoscope from a safe distance and pave a path for protecting healthcare workers through minimizing the risk of virus transmission without reducing diagnostic and therapeutic capacities. This review focuses on the needs and challenges associated with the design of robotic flexible endoscopes for use during a pandemic. The authors propose that a few minor changes to existing platforms or considerations for platforms in development could lead to significant benefits for use during infection control scenarios.

Magnetic flexible endoscope for colonoscopy: an initial learning curve analysis.

Background and study aims Colonoscopy is a technically challenging procedure that requires extensive training to minimize discomfort and avoid trauma due to its drive mechanism. Our academic team developed a magnetic flexible endoscope (MFE) actuated by magnetic coupling under supervisory robotic control to enable a front-pull maneuvering mechanism, with a motion controller user interface, to minimize colon wall stress and potentially reduce the learning curve. We aimed to evaluate this learning curve and understand the user experience. Methods Five novices (no endoscopy experience), five experienced endoscopists, and five experienced MFE users each performed 40 trials on a model colon using 1:1 block randomization between a pediatric colonoscope (PCF) and the MFE. Cecal intubation (CI) success, time to cecum, and user experience (NASA task load index) were measured. Learning curves were determined by the number of trials needed to reach minimum and average proficiency-defined as the slowest average CI time by an experienced user and the average CI time by all experienced users, respectively. Results MFE minimum proficiency was achieved by all five novices (median 3.92 trials) and five experienced endoscopists (median 2.65 trials). MFE average proficiency was achieved by four novices (median 14.21 trials) and four experienced endoscopists (median 7.00 trials). PCF minimum and average proficiency levels were achieved by only one novice. Novices' perceived workload with the MFE significantly improved after obtaining minimum proficiency. Conclusions The MFE has a short learning curve for users with no prior experience-requiring relatively few attempts to reach proficiency and at a reduced perceived workload.

Enteral Stents for Malignant Gastric Outlet Obstruction: Low Reintervention Rates for Obstruction due to Pancreatic Adenocarcinoma Versus Other Etiologies.

BACKGROUND AND AIM: Enteral stents (ES) have emerged as first-line therapy for the treatment of malignant gastric outlet obstruction (GOO). Stent occlusion arising from tissue ingrowth may require endoscopic or surgical reintervention. The objective of this study was to compare rates of reintervention following palliative ES for patients with GOO due to pancreatic adenocarcinoma (PDAC) versus other malignant etiologies. METHODS: Patients who had undergone ES for palliation of malignant GOO between 2009 and 2018 were retrospectively identified and demographic, clinical, and procedural data were collected. Primary outcome was procedural reintervention for recurrent symptomatic GOO following ES placement. RESULTS: Forty-three patients were included in the study cohort. 62.8% (27/43) of patients had PDAC while 37.2% (16/43) of patients had other malignant etiologies. 11.6% (5/43) of patients were alive at follow-up. Thirty-day and 90-day mortality rates were 22.8% and 70.7% for PDAC and 25% and 56.3% for other malignant etiologies, respectively. Seven patients required reintervention for symptomatic GOO: 14.3% (1/7) had PDAC and 85.7% (6/7) had GOO due to other malignancy (P < .01). Ninety-six percent (26/27) of patients with PDAC required no further intervention for GOO prior to death or end of follow-up. On multivariate analysis, patients with PDAC were significantly less likely to require reintervention than patients with other malignant etiologies (OR 0.064, 95% CI 0.01-0.60). CONCLUSION: ES offer durable symptom palliation without requirement for reintervention for the overwhelming majority of patients with malignant GOO due to PDAC. Reintervention rates are higher following ES placement for GOO due to other malignant etiologies and future study may be needed to define the optimal palliative intervention for this group of patients.

The impact of distraction minimization on endoscopic mentoring and performance.

Background and study aims Endoscopic mentoring requires active attention by the preceptor. Unfortunately, sources of distraction are abundant during endoscopic precepting. The impact of distraction minimization on endoscopic mentoring and performance is unknown. Methods Fellow and attending preceptors were paired and randomized in a prospective crossover design to perform esophagogastroduodenoscopy (EGD) and/or colonoscopy in either a "distraction minimization" (DM) or a "standard" (S) room. Cell phones, pagers, music, and computers were not permitted in DM rooms. S rooms operated under typical conditions. Fellows and attendings then completed a survey. The primary outcome was fellow satisfaction with mentoring experience (visual analogue scale: 0 = min,100 = max). Additional fellow outcomes included satisfaction of attending attentiveness, identifying landmarks, communication, and distractedness; attending outcomes included satisfaction with mentoring, attentiveness, communication, and distractedness. Endoscopic performance measures included completion of EGD, cecal intubation rate, cecal intubation time, withdrawal time, total procedure time, attending assistance, and polyp detection rate. A paired t -test was used to compare mean differences (MD) between rooms; significance set at P  < 0.05. Results Eight fellows and seven attendings completed 164 procedures. Despite a trend toward less distraction between rooms (DM = 12.5 v. S = 18.3, MD =  4.1, P  = 0.17), there was no difference in fellow satisfaction with training/mentoring (DM = 93, S = 93, MD = -0.04, P  = 0.97), attentiveness (DM = 95, S = 92, MD = 0.86, P  = 0.77), identifying pathology/landmarks (DM = 94, S = 94, MD = -1.72, P  = 0.56), or communication (DM = 95, S = 95,MD = 1.0, P  = 0.37). Similarly, there was no difference between rooms for any attending outcome measures or performance metrics. Conclusions DM did not improve perceived quality of endoscopic mentoring or performance for fellows or attendings; however, reduced distraction may improve attending engagement/availability.

Enabling the future of colonoscopy with intelligent and autonomous magnetic manipulation.

Early diagnosis of colorectal cancer significantly improves survival. However, over half of cases are diagnosed late due to demand exceeding the capacity for colonoscopy - the "gold standard" for screening. Colonoscopy is limited by the outdated design of conventional endoscopes, associated with high complexity of use, cost and pain. Magnetic endoscopes represent a promising alternative, overcoming drawbacks of pain and cost, but struggle to reach the translational stage as magnetic manipulation is complex and unintuitive. In this work, we use machine vision to develop intelligent and autonomous control of a magnetic endoscope, for the first time enabling non-expert users to effectively perform magnetic colonoscopy in-vivo. We combine the use of robotics, computer vision and advanced control to offer an intuitive and effective endoscopic system. Moreover, we define the characteristics required to achieve autonomy in robotic endoscopy. The paradigm described here can be adopted in a variety of applications where navigation in unstructured environments is required, such as catheters, pancreatic endoscopy, bronchoscopy, and gastroscopy. This work brings alternative endoscopic technologies closer to the translational stage, increasing availability of early-stage cancer treatments.

The waterjet necrosectomy device for endoscopic management of pancreatic necrosis: design, development, and preclinical testing (with videos).

BACKGROUND AND AIMS: Endoscopic intervention has emerged as a first-line option for management of symptomatic pancreatic necrosis, yet endoscopic debridement is limited by the lack of dedicated endoscopic tools intended for this purpose. The objectives of this study were to design and build a prototype necrosectomy device compatible for use with a flexible endoscope and capable of selective tissue fragmentation, and to test the prototype in benchtop and porcine models. METHODS: A novel prototype, named the waterjet necrosectomy device (WAND), was designed and developed, consisting of a single-use disposable endoscopic waterjet instrument capable of waterjet selection and independent tip articulation while fitting through a 2.8-mm working channel of a standard adult upper GI endoscope. Benchtop, ex vivo, and in vivo (porcine) testing was performed in the initial stages of investigation. RESULTS: The WAND was capable of delivering a continuous waterjet force with a surface pressure of 0.72 bar at a flow rate of 0.37 L/minute. In phase 1 of testing, the WAND was able to achieve complete fragmentation of gelatin as a surrogate for pancreatic necrosis in benchtop testing. In phase 2 of testing, the WAND was able to achieve complete fragmentation of freshly explanted human pancreatic necrosis. In phase 3 of testing for safety in fresh necropsy swine, use of the WAND resulted in no significant tissue trauma, even when irrigation was applied at closer proximity and for more extended duration than would be anticipated in clinical use. CONCLUSION: The WAND prototype delivers irrigation capable of fragmenting necrotic debris ex vivo and avoiding trauma to healthy nontarget tissue. Planning is underway for first-in-human studies to assess the efficacy and safety of the WAND for endoscopic pancreatic necrosectomy.

Adaptive Dynamic Control for Magnetically Actuated Medical Robots.

In the present work we discuss a novel dynamic control approach for magnetically actuated robots, by proposing an adaptive control technique, robust towards parametric uncertainties and unknown bounded disturbances. The former generally arise due to partial knowledge of the robots' dynamic parameters, such as inertial factors, the latter are the outcome of unpredictable interaction with unstructured environments. In order to show the application of the proposed approach, we consider controlling the Magnetic Flexible Endoscope (MFE) which is composed of a soft-tethered Internal Permanent Magnet (IPM), actuated with a single External Permanent Magnet (EPM). We provide with experimental analysis to show the possibility of levitating the MFE - one of the most difficult tasks with this platform - in case of partial knowledge of the IPM's dynamics and no knowledge of the tether's behaviour. Experiments in an acrylic tube show a reduction of contact of the 32% compared to non-levitating techniques and 1.75 times faster task completion with respect to previously proposed levitating techniques. More realistic experiments, performed in a colon phantom, show that levitating the capsule achieves faster and smoother exploration and that the minimum time for completing the task is attained by the proposed approach.

Intelligent magnetic manipulation for gastrointestinal ultrasound.

Diagnostic endoscopy in the gastrointestinal tract has remained largely unchanged for decades and is limited to the visualization of the tissue surface, the collection of biopsy samples for diagnoses, and minor interventions such as clipping or tissue removal. In this work, we present the autonomous servoing of a magnetic capsule robot for in-situ, subsurface diagnostics of microanatomy. We investigated and showed the feasibility of closed-loop magnetic control using digitized microultrasound (µUS) feedback; this is crucial for obtaining robust imaging in an unknown and unconstrained environment. We demonstrated the functionality of an autonomous servoing algorithm that uses µUS feedback, both on benchtop trials as well as in-vivo in a porcine model. We have validated this magnetic-µUS servoing in instances of autonomous linear probe motion and were able to locate markers in an agar phantom with 1.0 ± 0.9 mm position accuracy using a fusion of robot localization and µUS image information. This work demonstrates the feasibility of closed-loop robotic µUS imaging in the bowel without the need for either a rigid physical link between the transducer and extracorporeal tools or complex manual manipulation.

Magnetic Levitation for Soft-Tethered Capsule Colonoscopy Actuated With a Single Permanent Magnet: A Dynamic Control Approach.

The present letter investigates a novel control approach for magnetically driven soft-tethered capsules for colonoscopy-a potentially painless approach for colon inspection. The focus of this work is on a class of devices composed of a magnetic capsule endoscope actuated by a single external permanent magnet. Actuation is achieved by manipulating the external magnet with a serial manipulator, which in turn produces forces and torques on the internal magnetic capsule. We propose a control strategy which, counteracting gravity, achieves levitation of the capsule. This technique, based on a nonlinear backstepping approach, is able to limit contact with the colon walls, reducing friction, avoiding contact with internal folds, and facilitating the inspection of nonplanar cavities. The approach is validated on an experimental setup, which embodies a general scenario faced in colonoscopy. The experiments show that we can attain 19.5% of contact with the colon wall, compared to the almost 100% of previously proposed approaches. Moreover, we show that the control can be used to navigate the capsule through a more realistic environment-a colon phantom-with reasonable completion time.

Autonomous Retroflexion of a Magnetic Flexible Endoscope.

Retroflexion during colonoscopy is typically only practiced in the wider proximal and distal ends of the large intestine owing to the stiff nature of the colonoscope. This inability to examine the proximal side of the majority of colon folds contributes to today's suboptimal colorectal cancer detection rates. We have developed an algorithm for autonomous retroflexion of a flexible endoscope that is actuated magnetically from the tip. The magnetic wrench applied on the tip of the endoscope is optimized in real-time with data from pose detection to compute motions of the actuating magnet. This is the first example of a completely autonomous maneuver by a magnetic endoscope for exploration of the gastrointestinal tract. The proposed approach was validated in plastic tubes of various diameters with a success rate of 98.8% for separation distances up to 50 mm. Additionally, a set of trials was conducted in an excised porcine colon observing a success rate of 100% with a mean time of 19.7 s. In terms of clinical safety, the maximum stress that is applied on the colon wall with our methodology is an order of magnitude below what would damage tissue.

Evaluation of a novel tablet application for improvement in colonoscopy training and mentoring (with video).

BACKGROUND AND AIMS: Endoscopic training can be challenging for the trainee and preceptor. Frustration can result from ineffective communication regarding areas of interest. Our team developed a novel tablet application for real-time mirroring of the colonoscopy examination that allows preceptors to make annotations directly on the viewing monitor. The potential for improvement in team proficiency and satisfaction is unknown. METHODS: The on-screen endoscopic image is mirrored to an Android tablet and permits real-time annotation directly on the in-room endoscopic image display. Preceptors can also "freeze-frame" an image and provide visual on-screen instruction (telestration). Trainees, precepted by a GI attending, were 1:1 randomized to perform colonoscopy on a training phantom using the application with traditional precepting or traditional precepting alone. Magnetized polyps (size < 5 mm) were placed in 1 of 5 preset location scenarios. Each trainee performed a total of 10 colonoscopies and completed each location scenario twice. During withdrawal, the trainee and the attending identified polyps. Outcome measures included number of polyps missed and participant satisfaction after each trial. RESULTS: Fifteen trainees (6 novice and 9 GI fellows) performed a total of 150 colonoscopies where 330 polyps in total were placed. Fellows missed fewer polyps using the tablet versus traditional precepting alone (4.2% vs 12.5%; P = .04). There was no significant difference in missed polyps for novices (12.5% vs 18.8%; P = .66). Overall, fellows missed fewer polyps when compared with novices regardless of the precepting method (P = .01). The attending and all trainees reported reduced stress with improved communication using the tablet. CONCLUSIONS: Fellows missed fewer polyps using the tablet when compared with traditional endoscopy precepting. All trainees reported reduced stress, quicker identification of polyps, and improved educational satisfaction using the tablet. Our application has the potential to improve trainee plus attending team lesion detection and to enhance the endoscopy training experience for both the trainee and attending preceptor.

Video on Diet Before Outpatient Colonoscopy Does Not Improve Quality of Bowel Preparation: A Prospective, Randomized, Controlled Trial.

OBJECTIVES: Successful outpatient colonoscopy (CLS) depends on many factors including the quality of a patient's bowel preparation. Although education on consumption of the pre-CLS purgative can improve bowel preparation quality, no study has evaluated dietary education alone. We have created an educational video on pre-CLS dietary instructions to determine whether dietary education would improve outpatient bowel preparation quality. METHODS: A prospective randomized, blinded, controlled study of patients undergoing outpatient CLS was performed. All patients received a 4 l polyethylene glycol-based split-dose bowel preparation and standard institutional pre-procedure instructions. Patients were then randomly assigned to an intervention arm or to a no intervention arm. A 4-min educational video detailing clear liquid diet restriction was made available to patients in the intervention arm, whereas those randomized to no intervention did not have access to the video. Patients randomized to the video were provided with the YouTube video link 48-72 h before CLS. An attending endoscopist blinded to randomization performed the CLS. Bowel preparation quality was scored using the Boston Bowel Preparation Scale (BBPS). Adequate preparation was defined as a BBPS total score of ≥6 with all segment scores ≥2. Wilcoxon rank-sum and Pearson's χ2-tests were performed to assess differences between groups. RESULTS: Ninety-two patients were randomized (video: n=42; control: n=50) with 47 total video views being tallied. There were no demographic differences between groups. There was no statistically significant difference in adequate preparation between groups (video=74%; control=68%; P=0.54). CONCLUSIONS: The availability of a supplementary patient educational video on clear liquid diet alone was insufficient to improve bowel preparation quality when compared with standard pre-procedure instruction at our institution.