Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method.

BACKGROUND : Artificial intelligence (AI) research in colonoscopy is progressing rapidly but widespread clinical implementation is not yet a reality. We aimed to identify the top implementation research priorities. METHODS : An established modified Delphi approach for research priority setting was used. Fifteen international experts, including endoscopists and translational computer scientists/engineers, from nine countries participated in an online survey over 9 months. Questions related to AI implementation in colonoscopy were generated as a long-list in the first round, and then scored in two subsequent rounds to identify the top 10 research questions. RESULTS : The top 10 ranked questions were categorized into five themes. Theme 1: clinical trial design/end points (4 questions), related to optimum trial designs for polyp detection and characterization, determining the optimal end points for evaluation of AI, and demonstrating impact on interval cancer rates. Theme 2: technological developments (3 questions), including improving detection of more challenging and advanced lesions, reduction of false-positive rates, and minimizing latency. Theme 3: clinical adoption/integration (1 question), concerning the effective combination of detection and characterization into one workflow. Theme 4: data access/annotation (1 question), concerning more efficient or automated data annotation methods to reduce the burden on human experts. Theme 5: regulatory approval (1 question), related to making regulatory approval processes more efficient. CONCLUSIONS : This is the first reported international research priority setting exercise for AI in colonoscopy. The study findings should be used as a framework to guide future research with key stakeholders to accelerate the clinical implementation of AI in endoscopy.

Closed-loop control of soft continuum manipulators under tip follower actuation.

Continuum manipulators, inspired by nature, have drawn significant interest within the robotics community. They can facilitate motion within complex environments where traditional rigid robots may be ineffective, while maintaining a reasonable degree of precision. Soft continuum manipulators have emerged as a growing subfield of continuum robotics, with promise for applications requiring high compliance, including certain medical procedures. This has driven demand for new control schemes designed to precisely control these highly flexible manipulators, whose kinematics may be sensitive to external loads, such as gravity. This article presents one such approach, utilizing a rapidly computed kinematic model based on Cosserat rod theory, coupled with sensor feedback to facilitate closed-loop control, for a soft continuum manipulator under tip follower actuation and external loading. This approach is suited to soft manipulators undergoing quasi-static deployment, where actuators apply a follower wrench (i.e., one that is in a constant body frame direction regardless of robot configuration) anywhere along the continuum structure, as can be done in water-jet propulsion. In this article we apply the framework specifically to a tip actuated soft continuum manipulator. The proposed control scheme employs both actuator feedback and pose feedback. The actuator feedback is utilized to both regulate the follower load and to compensate for non-linearities of the actuation system that can introduce kinematic model error. Pose feedback is required to maintain accurate path following. Experimental results demonstrate successful path following with the closed-loop control scheme, with significant performance improvements gained through the use of sensor feedback when compared with the open-loop case.

The waterjet necrosectomy device for endoscopic management of pancreatic necrosis: design, development, and preclinical testing (with videos).

BACKGROUND AND AIMS: Endoscopic intervention has emerged as a first-line option for management of symptomatic pancreatic necrosis, yet endoscopic debridement is limited by the lack of dedicated endoscopic tools intended for this purpose. The objectives of this study were to design and build a prototype necrosectomy device compatible for use with a flexible endoscope and capable of selective tissue fragmentation, and to test the prototype in benchtop and porcine models. METHODS: A novel prototype, named the waterjet necrosectomy device (WAND), was designed and developed, consisting of a single-use disposable endoscopic waterjet instrument capable of waterjet selection and independent tip articulation while fitting through a 2.8-mm working channel of a standard adult upper GI endoscope. Benchtop, ex vivo, and in vivo (porcine) testing was performed in the initial stages of investigation. RESULTS: The WAND was capable of delivering a continuous waterjet force with a surface pressure of 0.72 bar at a flow rate of 0.37 L/minute. In phase 1 of testing, the WAND was able to achieve complete fragmentation of gelatin as a surrogate for pancreatic necrosis in benchtop testing. In phase 2 of testing, the WAND was able to achieve complete fragmentation of freshly explanted human pancreatic necrosis. In phase 3 of testing for safety in fresh necropsy swine, use of the WAND resulted in no significant tissue trauma, even when irrigation was applied at closer proximity and for more extended duration than would be anticipated in clinical use. CONCLUSION: The WAND prototype delivers irrigation capable of fragmenting necrotic debris ex vivo and avoiding trauma to healthy nontarget tissue. Planning is underway for first-in-human studies to assess the efficacy and safety of the WAND for endoscopic pancreatic necrosectomy.

A Compression Valve for Sanitary Control of Fluid-Driven Actuators.

With significant research focused on integrating robotics into medical devices, sanitary control of pressurizing fluids in a precise, accurate and customizable way is highly desirable. Current sanitary flow control methods include pinch valves which clamp the pressure line locally to restrict fluid flow; resulting in damage and variable flow characteristics over time. This paper presents a sanitary compression valve based on an eccentric clamping mechanism. The proposed valve distributes clamping forces over a larger area, thereby reducing the plastic deformation and associated influence on flow characteristic. Using the proposed valve, significant reductions in plastic deformation (up to 96%) and flow-rate error (up to 98%) were found, when compared with a standard pinch valve. Additionally, an optimization strategy presents a method for improving linearity and resolution over the working range to suit specific control applications. The valve efficacy has been evaluated through controlled testing of a water jet propelled low-cost endoscopic device. In this case, use of the optimized valve shows a reduction in the average orientation error and its variation, resulting in smoother movement of the endoscopic tip when compared to alternative wet and dry valve solutions. The presented valve offers a customizable solution for sanitary control of fluid driven actuators.

Sensitivity Ellipsoids for Force Control of Magnetic Robots with Localization Uncertainty.

The navigation of magnetic medical robots typically relies on localizing an actuated, intracorporeal, ferromagnetic body and back-computing a necessary field and gradient that would result in a desired wrench on the device. Uncertainty in this localization degrades the precision of force transmission. Reducing applied force uncertainty may enhance tasks such as in-vivo navigation of miniature robots, actuation of magnetically guided catheters, tissue palpation, as well as simply ensuring a bound on forces applied on sensitive tissue. In this paper, we analyzed the effects of localization noise on force uncertainty by using sensitivity ellipsoids of the magnetic force Jacobian and introduced an algorithm for uncertainty reduction. We validated the algorithm in both a simulation study and in a physical experiment. In simulation, we observed reductions in estimated force uncertainty by factors of up to 2.8 and 3.1 when using one and two actuating magnets, respectively. On a physical platform, we demonstrated a force uncertainty reduction by a factor of up to 2.5 as measured using an external sensor. Being the first consideration of force uncertainty resulting from noisy localization, this work provides a strategy for investigators to minimize uncertainty in magnetic force transmission.

Enhanced Real-Time Pose Estimation for Closed-Loop Robotic Manipulation of Magnetically Actuated Capsule Endoscopes.

Pose estimation methods for robotically guided magnetic actuation of capsule endoscopes have recently enabled trajectory following and automation of repetitive endoscopic maneuvers. However, these methods face significant challenges in their path to clinical adoption including the presence of regions of magnetic field singularity, where the accuracy of the system degrades, and the need for accurate initialization of the capsule's pose. In particular, the singularity problem exists for any pose estimation method that utilizes a single source of magnetic field if the method does not rely on the motion of the magnet to obtain multiple measurements from different vantage points. We analyze the workspace of such pose estimation methods with the use of the point-dipole magnetic field model and show that singular regions exist in areas where the capsule is nominally located during magnetic actuation. Since the dipole model can approximate most magnetic field sources, the problem discussed herein pertains to a wider set of pose estimation techniques. We then propose a novel hybrid approach employing static and time-varying magnetic field sources and show that this system has no regions of singularity. The proposed system was experimentally validated for accuracy, workspace size, update rate and performance in regions of magnetic singularity. The system performed as well or better than prior pose estimation methods without requiring accurate initialization and was robust to magnetic singularity. Experimental demonstration of closed-loop control of a tethered magnetic device utilizing the developed pose estimation technique is provided to ascertain its suitability for robotically guided capsule endoscopy. Hence, advances in closed-loop control and intelligent automation of magnetically actuated capsule endoscopes can be further pursued toward clinical realization by employing this pose estimation system.

Evaluation of a novel tablet application for improvement in colonoscopy training and mentoring (with video).

BACKGROUND AND AIMS: Endoscopic training can be challenging for the trainee and preceptor. Frustration can result from ineffective communication regarding areas of interest. Our team developed a novel tablet application for real-time mirroring of the colonoscopy examination that allows preceptors to make annotations directly on the viewing monitor. The potential for improvement in team proficiency and satisfaction is unknown. METHODS: The on-screen endoscopic image is mirrored to an Android tablet and permits real-time annotation directly on the in-room endoscopic image display. Preceptors can also "freeze-frame" an image and provide visual on-screen instruction (telestration). Trainees, precepted by a GI attending, were 1:1 randomized to perform colonoscopy on a training phantom using the application with traditional precepting or traditional precepting alone. Magnetized polyps (size < 5 mm) were placed in 1 of 5 preset location scenarios. Each trainee performed a total of 10 colonoscopies and completed each location scenario twice. During withdrawal, the trainee and the attending identified polyps. Outcome measures included number of polyps missed and participant satisfaction after each trial. RESULTS: Fifteen trainees (6 novice and 9 GI fellows) performed a total of 150 colonoscopies where 330 polyps in total were placed. Fellows missed fewer polyps using the tablet versus traditional precepting alone (4.2% vs 12.5%; P = .04). There was no significant difference in missed polyps for novices (12.5% vs 18.8%; P = .66). Overall, fellows missed fewer polyps when compared with novices regardless of the precepting method (P = .01). The attending and all trainees reported reduced stress with improved communication using the tablet. CONCLUSIONS: Fellows missed fewer polyps using the tablet when compared with traditional endoscopy precepting. All trainees reported reduced stress, quicker identification of polyps, and improved educational satisfaction using the tablet. Our application has the potential to improve trainee plus attending team lesion detection and to enhance the endoscopy training experience for both the trainee and attending preceptor.

A magnetic levitation robotic camera for minimally invasive surgery: Useful for NOTES?

BACKGROUND: Minimally invasive surgery (MIS) is rising in popularity generating a revolution in operative medicine during the past few decades. Although laparoscopic techniques have not significantly changed in the last 10 years, several advances have been made in visualization devices and instrumentation. METHODS: Our team, composed of surgeons and biomedical engineers, developed a magnetic levitation camera (MLC) with a magnetic internal mechanism dedicated to MIS. Three animal trials were performed. Porcine acute model has been chosen after animal ethical committee approval, and laparoscopic cholecystectomy, nephrectomy and hernioplastic repair have been performed. RESULTS: MLC permits to complete efficiently several two-port laparoscopy surgeries reducing patients' invasiveness and at the same time saving surgeon's dexterity. CONCLUSIONS: We strongly believe that insertable and softly tethered devices like MLS camera will be an integral part of future surgical systems, thus improving procedures efficiency, minimizing invasiveness and enhancing surgeon dexterity and versatility of visions angles.

Jacobian-Based Iterative Method for Magnetic Localization in Robotic Capsule Endoscopy.

The purpose of this study is to validate a Jacobian-based iterative method for real-time localization of magnetically controlled endoscopic capsules. The proposed approach applies finite-element solutions to the magnetic field problem and least-squares interpolations to obtain closed-form and fast estimates of the magnetic field. By defining a closed-form expression for the Jacobian of the magnetic field relative to changes in the capsule pose, we are able to obtain an iterative localization at a faster computational time when compared with prior works, without suffering from the inaccuracies stemming from dipole assumptions. This new algorithm can be used in conjunction with an absolute localization technique that provides initialization values at a slower refresh rate. The proposed approach was assessed via simulation and experimental trials, adopting a wireless capsule equipped with a permanent magnet, six magnetic field sensors, and an inertial measurement unit. The overall refresh rate, including sensor data acquisition and wireless communication was 7 ms, thus enabling closed-loop control strategies for magnetic manipulation running faster than 100 Hz. The average localization error, expressed in cylindrical coordinates was below 7 mm in both the radial and axial components and 5° in the azimuthal component. The average error for the capsule orientation angles, obtained by fusing gyroscope and inclinometer measurements, was below 5°.

Emerging Issues and Future Developments in Capsule Endoscopy.

Capsule endoscopy (CE) has transformed from a research venture into a widely used clinical tool and the primary means for diagnosing small bowel pathology. These orally administered capsules traverse passively through the gastrointestinal tract via peristalsis and are used in the esophagus, stomach, small bowel, and colon. The primary focus of CE research in recent years has been enabling active CE manipulation and extension of the technology to therapeutic functionality; thus, widening the scope of the procedure. This review outlines clinical standards of the technology as well as recent advances in CE research. Clinical capsule applications are discussed with respect to each portion of the gastrointestinal tract. Promising research efforts are presented with an emphasis on enabling active capsule locomotion. The presented studies suggest, in particular, that the most viable solution for active capsule manipulation is actuation of a capsule via exterior permanent magnet held by a robot. Developing capsule procedures adhering to current healthcare standards, such as enabling a tool channel or irrigation in a therapeutic device, is a vital phase in the adaptation of CE in the clinical setting.

Controlled colonic insufflation by a remotely triggered capsule for improved mucosal visualization.

BACKGROUND AND STUDY AIMS: Capsule endoscopy is an attractive alternative to colorectal cancer screening by conventional colonoscopy, but is currently limited by compromised mucosal visibility because of the lack of safe, controlled colonic insufflation. We have therefore developed a novel system of untethered, wireless-controlled carbon dioxide (CO2) insufflation for use in colonic capsule endoscopy, which this study aims to assess in vivo. MATERIAL AND METHODS: This observational, nonsurvival, in vivo study used five Yorkshire-Landrace cross swine. A novel insufflation capsule was placed in the porcine colons, and we recorded volume of insufflation, time, force, visualization, and a pathologic assessment of the colon. RESULTS: The mean (standard deviation [SD]) diameter of insufflation was 32.1 (3.9) mm. The volume of CO2 produced successfully allowed complete endoscopic visualization of the mucosa and safe proximal passage of the endoscope. Pathologic examination demonstrated no evidence of trauma caused by the capsule. CONCLUSIONS: These results demonstrate the feasibility of a novel method of controlled colonic insufflation via an untethered capsule in vivo. This technological innovation addresses a critical need in colon capsule endoscopy.

Real-time surgical tool detection with multi-scale positional encoding and contrastive learning.

Real-time detection of surgical tools in laparoscopic data plays a vital role in understanding surgical procedures, evaluating the performance of trainees, facilitating learning, and ultimately supporting the autonomy of robotic systems. Existing detection methods for surgical data need to improve processing speed and high prediction accuracy. Most methods rely on anchors or region proposals, limiting their adaptability to variations in tool appearance and leading to sub-optimal detection results. Moreover, using non-anchor-based detectors to alleviate this problem has been partially explored without remarkable results. An anchor-free architecture based on a transformer that allows real-time tool detection is introduced. The proposal is to utilize multi-scale features within the feature extraction layer and at the transformer-based detection architecture through positional encoding that can refine and capture context-aware and structural information of different-sized tools. Furthermore, a supervised contrastive loss is introduced to optimize representations of object embeddings, resulting in improved feed-forward network performances for classifying localized bounding boxes. The strategy demonstrates superiority to state-of-the-art (SOTA) methods. Compared to the most accurate existing SOTA (DSSS) method, the approach has an improvement of nearly 4% on mAP and a reduction in the inference time by 113%. It also showed a 7% higher mAP than the baseline model.

Advanced technologies for gastrointestinal endoscopy.

The gastrointestinal tract is home to some of the most deadly human diseases. Exacerbating the problem is the difficulty of accessing it for diagnosis or intervention and the concomitant patient discomfort. Flexible endoscopy has established itself as the method of choice and its diagnostic accuracy is high, but there remain technical limitations in modern scopes, and the procedure is poorly tolerated by patients, leading to low rates of compliance with screening guidelines. Although advancement in clinical endoscope design has been slow in recent years, a critical mass of enabling technologies is now paving the way for the next generation of gastrointestinal endoscopes. This review describes current endoscopes and provides an overview of innovative flexible scopes and wireless capsules that can enable painless endoscopy and/or enhanced diagnostic and therapeutic capabilities. We provide a perspective on the potential of these new technologies to address the limitations of current endoscopes in mass cancer screening and other contexts and thus to save many lives.

Fine tilt tuning of a laparoscopic camera by local magnetic actuation: two-port nephrectomy experience on human cadavers.

BACKGROUND: The magnetic surgical camera is an emerging technology having the potential to improve visualization without taking up port site space. However, tilting the point of view downward/upward can be done only by constantly applying a pressure on the abdomen. This study aims to test the hypothesis that the novel concept of local magnetic actuation (LMA) is able to increase the tilt range available for a magnetic camera without the need for deforming the abdominal wall. The hypothesis that 2-port laparoscopic nephrectomy in fresh tissue human cadavers could be performed by using the LMA camera is also tested. METHODS: First, the 2 cameras were separately inserted, anchored, and moved inside the inflated abdomen. Tilting angles were quantified by image analysis while intra-abdominal pressure changes were monitored. Then, 5 two-port nephrectomies were performed by using the LMA camera while collecting quantitative outcomes. RESULTS: The magnetic camera required a constant pressure on the magnetic handle to achieve an average ±20° tilt from the horizontal position, with an average of 7 mm Hg loss of intra-abdominal pressure. The LMA camera allowed for 75° of tilt from the horizontal position with a resolution of ±1°, without any need to deform the abdomen. All the nephrectomies were completed successfully within an average time of 11 minutes. CONCLUSION: LMA is an effective strategy to provide magnetic cameras with wide-range and high-resolution vertical motion without the need to deform the abdominal wall.

Robust endoscopic image mosaicking via fusion of multimodal estimation.

We propose an endoscopic image mosaicking algorithm that is robust to light conditioning changes, specular reflections, and feature-less scenes. These conditions are especially common in minimally invasive surgery where the light source moves with the camera to dynamically illuminate close range scenes. This makes it difficult for a single image registration method to robustly track camera motion and then generate consistent mosaics of the expanded surgical scene across different and heterogeneous environments. Instead of relying on one specialised feature extractor or image registration method, we propose to fuse different image registration algorithms according to their uncertainties, formulating the problem as affine pose graph optimisation. This allows to combine landmarks, dense intensity registration, and learning-based approaches in a single framework. To demonstrate our application we consider deep learning-based optical flow, hand-crafted features, and intensity-based registration, however, the framework is general and could take as input other sources of motion estimation, including other sensor modalities. We validate the performance of our approach on three datasets with very different characteristics to highlighting its generalisability, demonstrating the advantages of our proposed fusion framework. While each individual registration algorithm eventually fails drastically on certain surgical scenes, the fusion approach flexibly determines which algorithms to use and in which proportion to more robustly obtain consistent mosaics.

Robotic, self-propelled, self-steerable, and disposable colonoscopes: Reality or pipe dream? A state of the art review.

Robotic colonoscopes could potentially provide a comfortable, less painful and safer alternative to standard colonoscopy. Recent exciting developments in this field are pushing the boundaries to what is possible in the future. This article provides a comprehensive review of the current work in robotic colonoscopes including self-propelled, steerable and disposable endoscopes that could be alternatives to standard colonoscopy. We discuss the advantages and disadvantages of these systems currently in development and highlight the technical readiness of each system to help the reader understand where and when such systems may be available for routine clinical use and get an idea of where and in which situation they can best be deployed.

Active Stabilization of Interventional Tasks Utilizing a Magnetically Manipulated Endoscope.

Magnetically actuated robots have become increasingly popular in medical endoscopy over the past decade. Despite the significant improvements in autonomy and control methods, progress within the field of medical magnetic endoscopes has mainly been in the domain of enhanced navigation. Interventional tasks such as biopsy, polyp removal, and clip placement are a major procedural component of endoscopy. Little advancement has been done in this area due to the problem of adequately controlling and stabilizing magnetically actuated endoscopes for interventional tasks. In the present paper we discuss a novel model-based Linear Parameter Varying (LPV) control approach to provide stability during interventional maneuvers. This method linearizes the non-linear dynamic interaction between the external actuation system and the endoscope in a set of equilibria, associated to different distances between the magnetic source and the endoscope, and computes different controllers for each equilibrium. This approach provides the global stability of the overall system and robustness against external disturbances. The performance of the LPV approach is compared to an intelligent teleoperation control method (based on a Proportional Integral Derivative (PID) controller), on the Magnetic Flexible Endoscope (MFE) platform. Four biopsies in different regions of the colon and at two different system equilibria are performed. Both controllers are asked to stabilize the endoscope in the presence of external disturbances (i.e. the introduction of the biopsy forceps through the working channel of the endoscope). The experiments, performed in a benchtop colon simulator, show a maximum reduction of the mean orientation error of the endoscope of 45.8% with the LPV control compared to the PID controller.