Fiberbots: Robotic fibers for high-precision minimally invasive surgery.

Precise manipulation of flexible surgical tools is crucial in minimally invasive surgical procedures, necessitating a miniature and flexible robotic probe that can precisely direct the surgical instruments. In this work, we developed a polymer-based robotic fiber with a thermal actuation mechanism by local heating along the sides of a single fiber. The fiber robot was fabricated by highly scalable fiber drawing technology using common low-cost materials. This low-profile (below 2 millimeters in diameter) robotic fiber exhibits remarkable motion precision (below 50 micrometers) and repeatability. We developed control algorithms coupling the robot with endoscopic instruments, demonstrating high-resolution in situ molecular and morphological tissue mapping. We assess its practicality and safety during in vivo laparoscopic surgery on a porcine model. High-precision motion of the fiber robot delivered endoscopically facilitates the effective use of cellular-level intraoperative tissue identification and ablation technologies, potentially enabling precise removal of cancer in challenging surgical sites.

Deep Graph-Based Multimodal Feature Embedding for Endomicroscopy Image Retrieval.

Representation learning is a critical task for medical image analysis in computer-aided diagnosis. However, it is challenging to learn discriminative features due to the limited size of the data set and the lack of labels. In this article, we propose a deep graph-based multimodal feature embedding (DGMFE) framework for medical image retrieval with application to breast tissue classification by learning discriminative features of probe-based confocal laser endomicroscopy (pCLE). We first build a multimodality graph model based on the visual similarity between pCLE data and reference histology images. The latent similar pCLE-histology pairs are extracted by walking with the cyclic path on the graph while the dissimilar pairs are extracted based on the geodesic distance. Given the similar and dissimilar pairs, the latent feature space is discovered by reconstructing the similarity between pCLE and histology images via deep Siamese neural networks. The proposed method is evaluated on a clinical database with 700 pCLE mosaics. The accuracy of image retrieval demonstrates that DGMFE can outperform previous works on feature learning. Especially, the top-1 accuracy in an eight-class retrieval task is 0.739, thus demonstrating a 10% improvement compared to the state-of-the-art method.

Transcutaneous fluorescence spectroscopy as a tool for non-invasive monitoring of gut function: first clinical experiences.

Gastro-intestinal function plays a vital role in conditions ranging from inflammatory bowel disease and HIV through to sepsis and malnutrition. However, the techniques that are currently used to assess gut function are either highly invasive or unreliable. Here we present an alternative, non-invasive sensing modality for assessment of gut function based on fluorescence spectroscopy. In this approach, patients receive an oral dose of a fluorescent contrast agent and a fibre-optic probe is used to make fluorescence measurements through the skin. This provides a readout of the degree to which fluorescent dyes have permeated from the gut into the blood stream. We present preliminary results from our first measurements in human volunteers demonstrating the potential of the technique for non-invasive monitoring of multiple aspects of gastro-intestinal health.

Hybrid Robot-assisted Frameworks for Endomicroscopy Scanning in Retinal Surgeries.

High-resolution real-time intraocular imaging of retina at the cellular level is very challenging due to the vulnerable and confined space within the eyeball as well as the limited availability of appropriate modalities. A probe-based confocal laser endomicroscopy (pCLE) system, can be a potential imaging modality for improved diagnosis. The ability to visualize the retina at the cellular level could provide information that may predict surgical outcomes. The adoption of intraocular pCLE scanning is currently limited due to the narrow field of view and the micron-scale range of focus. In the absence of motion compensation, physiological tremors of the surgeons' hand and patient movements also contribute to the deterioration of the image quality. Therefore, an image-based hybrid control strategy is proposed to mitigate the above challenges. The proposed hybrid control strategy enables a shared control of the pCLE probe between surgeons and robots to scan the retina precisely, with the absence of hand tremors and with the advantages of an image-based auto-focus algorithm that optimizes the quality of pCLE images. The hybrid control strategy is deployed on two frameworks - cooperative and teleoperated. Better image quality, smoother motion, and reduced workload are all achieved in a statistically significant manner with the hybrid control frameworks.

Line scanning, fiber bundle fluorescence HiLo endomicroscopy with confocal slit detection.

Fiber bundle fluorescence endomicroscopy is an effective method for in vivo imaging of biological tissue samples. Line-scanning confocal laser endomicroscopy realizes confocal imaging at a much higher frame rate compared to the point scanning system, but with reduced optical sectioning. To address this problem, we describe a fiber bundle endomicroscopy system that utilizes the HiLo technique to enhance the optical sectioning while still maintaining high image acquisition rates. Confocal HiLo endomicroscopy is achieved by synchronizing the scanning hybrid-illumination laser line with the rolling shutter of a CMOS camera. An evident improvement of axial sectioning is achieved as compared to the line-scanning confocal endomicroscopy without the HiLo technique. Comparisons are also made with epifluorescence endomicroscopy with and without HiLo. The optical sectioning enhancement is demonstrated on lens tissue as well as porcine kidney tissue.

En-face optical coherence tomography/fluorescence endomicroscopy for minimally invasive imaging using a robotic scanner.

We report a compact rigid instrument capable of delivering en-face optical coherence tomography (OCT) images alongside (epi)-fluorescence endomicroscopy (FEM) images by means of a robotic scanning device. Two working imaging channels are included: one for a one-dimensional scanning, forward-viewing OCT probe and another for a fiber bundle used for the FEM system. The robotic scanning system provides the second axis of scanning for the OCT channel while allowing the field of view (FoV) of the FEM channel to be increased by mosaicking. The OCT channel has resolutions of 25 / 60 µm (axial/lateral) and can provide en-face images with an FoV of 1.6 × 2.7 mm2. The FEM channel has a lateral resolution of better than 8 µm and can generate an FoV of 0.53 × 3.25 mm2 through mosaicking. The reproducibility of the scanning was determined using phantoms to be better than the lateral resolution of the OCT channel. Combined OCT and FEM imaging were validated with ex-vivo ovine and porcine tissues, with the instrument mounted on an arm to ensure constant contact of the probe with the tissue. The OCT imaging system alone was validated for in-vivo human dermal imaging with the handheld instrument. In both cases, the instrument was capable of resolving fine features such as the sweat glands in human dermal tissue and the alveoli in porcine lung tissue.

A Novel Semi-Autonomous Control Framework for Retina Confocal Endomicroscopy Scanning.

In this paper, a novel semi-autonomous control framework is presented for enabling probe-based confocal laser endomicroscopy (pCLE) scan of the retinal tissue. With pCLE, retinal layers such as nerve fiber layer (NFL) and retinal ganglion cell (RGC) can be scanned and characterized in real-time for an improved diagnosis and surgical outcome prediction. However, the limited field of view of the pCLE system and the micron-scale optimal focus distance of the probe, which are in the order of physiological hand tremor, act as barriers to successful manual scan of retinal tissue. Therefore, a novel sensorless framework is proposed for real-time semi-autonomous endomicroscopy scanning during retinal surgery. The framework consists of the Steady-Hand Eye Robot (SHER) integrated with a pCLE system, where the motion of the probe is controlled semi-autonomously. Through a hybrid motion control strategy, the system autonomously controls the confocal probe to optimize the sharpness and quality of the pCLE images, while providing the surgeon with the ability to scan the tissue in a tremor-free manner. Effectiveness of the proposed architecture is validated through experimental evaluations as well as a user study involving 9 participants. It is shown through statistical analyses that the proposed framework can reduce the work load experienced by the users in a statistically-significant manner, while also enhancing their performance in retaining pCLE images with optimized quality.

Transfer Recurrent Feature Learning for Endomicroscopy Image Recognition.

Probe-based confocal laser endomicroscopy (pCLE) is an emerging tool for epithelial cancer diagnosis, which enables in-vivo microscopic imaging during endoscopic procedures and facilitates the development of automatic recognition algorithms to identify the status of tissues. In this paper, we propose a transfer recurrent feature learning framework for classification tasks on pCLE videos. At the first stage, the discriminative feature of single pCLE frame is learned via generative adversarial networks based on both pCLE and histology modalities. At the second stage, we use recurrent neural networks to handle the varying length and irregular shape of pCLE mosaics taking the frame-based features as input. The experiments on real pCLE data sets demonstrate that our approach outperforms, with statistical significance, state-of-the-art approaches. A binary classification accuracy of 84.1% has been achieved.

Fiber bundle shifting endomicroscopy for high-resolution imaging.

Flexible endomicroscopes commonly use coherent fiber bundles with high core densities to facilitate high-resolution in vivo imaging during endoscopic and minimally-invasive procedures. However, under-sampling due to the inter-core spacing limits the spatial resolution, making it difficult to resolve smaller cellular features. Here, we report a compact and rapid piezoelectric transducer (PZT) based bundle-shifting endomicroscopy system in which a super-resolution (SR) image is restored from multiple pixelation-limited images by computational means. A miniaturized PZT tube actuates the fiber bundle behind a GRIN micro-lens and a Delaunay triangulation based algorithm reconstructs an enhanced SR image. To enable real-time cellular-level imaging, imaging is performed using a line-scan confocal laser endomicroscope system with a raw frame rate of 120 fps, delivering up to 2 times spatial resolution improvement for a field of view of 350 µm at a net frame rate of 30 fps. The resolution enhancement is confirmed using resolution phantoms and ex vivo fluorescence endomicroscopy imaging of human breast specimens is demonstrated.

Time-frequency analysis of human motion during rhythmic exercises.

Biomechanical signals due to human movements during exercise are represented in time-frequency domain using Wigner Distribution Function (WDF). Analysis based on WDF reveals instantaneous spectral and power changes during a rhythmic exercise. Investigations were carried out on 11 healthy subjects who performed 5 cycles of sun salutation, with a body-mounted Inertial Measurement Unit (IMU) as a motion sensor. Variance of Instantaneous Frequency (I.F) and Instantaneous Power (I.P) for performance analysis of the subject is estimated using one-way ANOVA model. Results reveal that joint Time-Frequency analysis of biomechanical signals during motion facilitates a better understanding of grace and consistency during rhythmic exercise.