Honeycomb Artifact Removal Using Convolutional Neural Network for Fiber Bundle Imaging.

We present a new deep learning framework for removing honeycomb artifacts yielded by optical path blocking of cladding layers in fiber bundle imaging. The proposed framework, HAR-CNN, provides an end-to-end mapping from a raw fiber bundle image to an artifact-free image via a convolution neural network (CNN). The synthesis of honeycomb patterns on ordinary images allows conveniently learning and validating the network without the enormous ground truth collection by extra hardware setups. As a result, HAR-CNN shows significant performance improvement in honeycomb pattern removal and also detailed preservation for the 1961 USAF chart sample, compared with other conventional methods. Finally, HAR-CNN is GPU-accelerated for real-time processing and enhanced image mosaicking performance.

NO Synthesis but Not Apoptosis, Mitosis or Inflammation Can Explain Correlations between Flow Directionality and Paracellular Permeability of Cultured Endothelium.

Haemodynamic wall shear stress varies from site to site within the arterial system and is thought to cause local variation in endothelial permeability to macromolecules. Our aim was to investigate mechanisms underlying the changes in paracellular permeability caused by different patterns of shear stress in long-term culture. We used the swirling well system and a substrate-binding tracer that permits visualisation of transport at the cellular level. Permeability increased in the centre of swirled wells, where flow is highly multidirectional, and decreased towards the edge, where flow is more uniaxial, compared to static controls. Overall, there was a reduction in permeability. There were also decreases in early- and late-stage apoptosis, proliferation and mitosis, and there were significant correlations between the first three and permeability when considering variation from the centre to the edge under flow. However, data from static controls did not fit the same relation, and a cell-by-cell analysis showed that <5% of uptake under shear was associated with each of these events. Nuclear translocation of NF-κB p65 increased and then decreased with the duration of applied shear, as did permeability, but the spatial correlation between them was not significant. Application of an NO synthase inhibitor abolished the overall decrease in permeability caused by chronic shear and the difference in permeability between the centre and the edge of the well. Hence, shear and paracellular permeability appear to be linked by NO synthesis and not by apoptosis, mitosis or inflammation. The effect was mediated by an increase in transport through tricellular junctions.

Ultra-high curvature sensors for multi-bend structures using fiber Bragg gratings.

Measuring high curvatures is essential in various applications such as structural engineering, medical treatment, and robotics. Herein, we present a novel ultra-high curvature sensor with an ultra-thin and highly flexible structure incorporating fiber Bragg gratings (FBGs). The sensor can measure curvature of bidirectional bending up to 200 m-1. In addition, the multi-bend curvature sensor in conjunction with the proposed calibration scheme enables the accurate reconstruction of a curve with varying curvature. The sensitivity and the accuracy of the curvature sensor are investigated for different sensor designs. Finally, we demonstrate the accurate shape sensing of various 2-D patterns using the multi-bend curvatures sensor.

Techniques for robot-aided intraocular surgery using monocular vision.

This paper presents techniques for robot-aided intraocular surgery using monocular vision in order to overcome erroneous stereo reconstruction in an intact eye. We propose a new retinal surface estimation method based on a structured-light approach. A handheld robot known as the Micron enables automatic scanning of a laser probe, creating projected beam patterns on the retinal surface. Geometric analysis of the patterns then allows planar reconstruction of the surface. To realize automated surgery in an intact eye, monocular hybrid visual servoing is accomplished through a scheme that incorporates surface reconstruction and partitioned visual servoing. We investigate the sensitivity of the estimation method according to relevant parameters and also evaluate its performance in both dry and wet conditions. The approach is validated through experiments for automated laser photocoagulation in a realistic eye phantom in vitro. Finally, we present the first demonstration of automated intraocular laser surgery in porcine eyes ex vivo.

Visualization of three pathways for macromolecule transport across cultured endothelium and their modification by flow.

Transport of macromolecules across vascular endothelium and its modification by fluid mechanical forces are important for normal tissue function and in the development of atherosclerosis. However, the routes by which macromolecules cross endothelium, the hemodynamic stresses that maintain endothelial physiology or trigger disease, and the dependence of transendothelial transport on hemodynamic stresses are controversial. We visualized pathways for macromolecule transport and determined the effect on these pathways of different types of flow. Endothelial monolayers were cultured under static conditions or on an orbital shaker producing different flow profiles in different parts of the wells. Fluorescent tracers that bound to the substrate after crossing the endothelium were used to identify transport pathways. Maps of tracer distribution were compared with numerical simulations of flow to determine effects of different shear stress metrics on permeability. Albumin-sized tracers dominantly crossed the cultured endothelium via junctions between neighboring cells, high-density lipoprotein-sized tracers crossed at tricellular junctions, and low-density lipoprotein-sized tracers crossed through cells. Cells aligned close to the angle that minimized shear stresses across their long axis. The rate of paracellular transport under flow correlated with the magnitude of these minimized transverse stresses, whereas transport across cells was uniformly reduced by all types of flow. These results contradict the long-standing two-pore theory of solute transport across microvessel walls and the consensus view that endothelial cells align with the mean shear vector. They suggest that endothelial cells minimize transverse shear, supporting its postulated proatherogenic role. Preliminary data show that similar tracer techniques are practicable in vivo.NEW & NOTEWORTHY Solutes of increasing size crossed cultured endothelium through intercellular junctions, through tricellular junctions, or transcellularly. Cells aligned to minimize the shear stress acting across their long axis. Paracellular transport correlated with the level of this minimized shear, but transcellular transport was reduced uniformly by flow regardless of the shear profile.

Comparative Evaluation of Handheld Robot-Aided Intraocular Laser Surgery.

This paper presents robot-aided intraocular laser surgery using a handheld robot known as Micron. The micromanipulator incorporated in Micron enables visual servoing of a laser probe, while maintaining a constant distance of the tool tip from the retinal surface. The comparative study was conducted with various control methods for evaluation of robot-aided intraocular laser surgery.

Handheld-automated microsurgical instrumentation for intraocular laser surgery.

BACKGROUND AND OBJECTIVE: Laser photocoagulation is a mainstay or adjuvant treatment for a variety of common retinal diseases. Automated laser photocoagulation during intraocular surgery has not yet been established. The authors introduce an automated laser photocoagulation system for intraocular surgery, based on a novel handheld instrument. The goals of the system are to enhance accuracy and efficiency and improve safety. MATERIALS AND METHODS: Triple-ring patterns are introduced as a typical arrangement for the treatment of proliferative retinopathy and registered to a preoperative fundus image. In total, 32 target locations are specified along the circumferences of three rings having diameters of 1, 2, and 3 mm, with a burn spacing of 600 µm. Given the initial system calibration, the retinal surface is reconstructed using stereo vision, and the targets specified on the preoperative image are registered with the control system. During automated operation, the laser probe attached to the manipulator of the active handheld instrument is deflected as needed via visual servoing in order to correct the error between the aiming beam and a specified target, regardless of any erroneous handle motion by the surgeon. A constant distance of the laser probe from the retinal surface is maintained in order to yield consistent size of burns and ensure safety during operation. Real-time tracking of anatomical features enables compensation for any movement of the eye. A graphical overlay system within operating microscope provides the surgeon with guidance cues for automated operation. Two retinal surgeons performed automated and manual trials in an artificial model of the eye, with each trial repeated three times. For the automated trials, various targeting thresholds (50-200 µm) were used to automatically trigger laser firing. In manual operation, fixed repetition rates were used, with frequencies of 1.0-2.5 Hz. The power of the 532 nm laser was set at 3.0 W with a duration of 20 ms. After completion of each trial, the speed of operation and placement error of burns were measured. The performance of the automated laser photocoagulation was compared with manual operation, using interpolated data for equivalent firing rates from 1.0 to 1.75 Hz. RESULTS: In automated trials, average error increased from 45 ± 27 to 60 ± 37 µm as the targeting threshold varied from 50 to 200 µm, while average firing rate significantly increased from 0.69 to 1.71 Hz. The average error in the manual trials increased from 102 ± 67 to 174 ± 98 µm as firing rate increased from 1.0 to 2.5 Hz. Compared to the manual trials, the average error in the automated trials was reduced by 53.0-56.4%, resulting in statistically significant differences (P ≤ 10(-20) ) for all equivalent frequencies (1.0-1.75 Hz). The depth of the laser tip in the automated trials was consistently maintained within 18 ± 2 µm root-mean-square (RMS) of its initial position, whereas it significantly varied in the manual trials, yielding an error of 296 ± 30 µm RMS. At high firing rates in manual trials, such as at 2.5 Hz, laser photocoagulation is marginally attained, yielding failed burns of 30% over the entire pattern, whereas no failed burns are found in automated trials. Relatively regular burn sizes are attained in the automated trials by the depth servoing of the laser tip, while burn sizes in the manual trials vary considerably. Automated avoidance of blood vessels was also successfully demonstrated, utilizing the retina-tracking feature to identify avoidance zones. CONCLUSION: Automated intraocular laser surgery can improve the accuracy of photocoagulation while ensuring safety during operation. This paper provides an initial demonstration of the technique under reasonably realistic laboratory conditions; development of a clinically applicable system requires further work.

Manipulator Design and Operation for a Six-Degree-of-Freedom Handheld Tremor-Canceling Microsurgical Instrument.

This paper presents the design and actuation of a six-degree-of-freedom (6-DOF) manipulator for a handheld instrument, known as "Micron," which performs active tremor compensation during microsurgery. The design incorporates a Gough-Stewart platform based on piezoelectric linear motor, with a specified minimum workspace of a cylinder 4 mm long and 4 mm in diameter at the end-effector. Given the stall force of the motors and the loading typically encountered in vitreoretinal microsurgery, the dimensions of the manipulator are optimized to tolerate a transverse load of 0.2 N on a remote center of motion near the midpoint of the tool shaft. The optimization yields a base diameter of 23 mm and a height of 37 mm. The fully handheld instrument includes a custom-built optical tracking system for control feedback, and an ergonomic housing to serve as a handle. The manipulation performance was investigated in both clamped and handheld conditions. In positioning experiments with varying side loads, the manipulator tolerates side load up to 0.25 N while tracking a sinusoidal target trajectory with less than 20 µm error. Physiological hand tremor is reduced by about 90% in a pointing task, and error less than 25 µm is achieved in handheld circle-tracing.

Design of endoscopic micro-robotic end effectors: safety and performance evaluation based on physical intestinal tissue damage characteristics.

During the last several years, legged locomotive mechanism has been considered as one of the main self-propelling mechanisms for future endoscopic microrobots due to its superior propulsion efficiency of an endoscopic microrobot inside the intestinal track. Nevertheless, its clinical application has been largely limited since the legged locomotive mechanism utilizes an end effector which has a sharp tip to generate sufficient traction by physically penetrating and interlocking with the intestinal tissue. This can cause excessive physical tissue damage or even complete perforation of the intestinal wall that can lead to abdominal inflammation. Hence, in this work two types of new end effectors, penetration-limited end effector (PLEE) and bi-material structured end effector (BMEE) were specially designed to acquire high medical safety as well as effective traction generation performance. The microscopic end effector specimens were fabricated with micro-wire electric discharge machining process. Traction generation performance of the end effectors was evaluated by direct measurement of resistance forces during contact-sliding tests using a custom-built contact-sliding tester. The safety of the end effector design was evaluated by examination of microscopic intestinal tissue damage using a scanning electron microscope (SEM). Physical damage characteristics of the intestinal tissue and related contact physics of the end effectors were discussed. From the results, the end effectors were evaluated with respect to their prospects in future medical applications as safe end effectors as well as micro-surgical tools.

Tentonin 3 is a pore-forming subunit of a slow inactivation mechanosensitive channel.

Mechanically activating (MA) channels transduce numerous physiological functions. Tentonin 3/TMEM150C (TTN3) confers MA currents with slow inactivation kinetics in somato- and barosensory neurons. However, questions were raised about its role as a Piezo1 regulator and its potential as a channel pore. Here, we demonstrate that purified TTN3 proteins incorporated into the lipid bilayer displayed spontaneous and pressure-sensitive channel currents. These MA currents were conserved across vertebrates and differ from Piezo1 in activation threshold and pharmacological response. Deep neural network structure prediction programs coupled with mutagenetic analysis predicted a rectangular-shaped, tetrameric structure with six transmembrane helices and a pore at the inter-subunit center. The putative pore aligned with two helices of each subunit and had constriction sites whose mutations changed the MA currents. These findings suggest that TTN3 is a pore-forming subunit of a distinct slow inactivation MA channel, potentially possessing a tetrameric structure.

Real-Time Endomicroscopic Image Mosaicking with an EKF-based Sensor Fusion Approach.

This study presents a real-time sensor fusion framework based on the extended Kalman filter (EKF) for accurate and robust endomicroscopic image mosaicking. The sensor fusion framework incorporates an optical tracking system that can track 6-DOF pose of the imaging probe with high accuracy in real time in conjunction with 2D local image registration from image feature matching between two consecutive frames. We evaluated the performance of the real-time image mosaicking based on the sensor fusion compared with the image or tracker only approach. As a result, it could retain the microscopic level of image detail from the image-based approach and also achieve a robust image mosaic without any drift by using the accurate optical tracking system.

In Vivo Cellular-Level 3D Imaging of Peripheral Nerves Using a Dual-Focusing Technique for Intra-Neural Interface Implantation.

In vivo volumetric imaging of the microstructural changes of peripheral nerves with an inserted electrode could be key for solving the chronic implantation failure of an intra-neural interface necessary to provide amputated patients with natural motion and sensation. Thus far, no imaging devices can provide a cellular-level three-dimensional (3D) structural images of a peripheral nerve in vivo. In this study, an optical coherence tomography-based peripheral nerve imaging platform that employs a newly proposed depth of focus extension technique is reported. A point spread function with the finest transverse resolution of 1.27 µm enables the cellular-level volumetric visualization of the metal wire and microstructural changes in a rat sciatic nerve with the metal wire inserted in vivo. Further, the feasibility of applying the imaging platform to large animals for a preclinical study is confirmed through in vivo rabbit sciatic nerve imaging. It is expected that new possibilities for the successful chronic implantation of an intra-neural interface will open up by providing the 3D microstructural changes of nerves around the inserted electrode.

Evaluation of Applied Force During Nasopharyngeal Swab Sampling Using Handheld Sensorized Instrument.

Nasopharyngeal swab is the most widely used diagnostic test for COVID-19 detection. However, enormous tests have posed a high risk of infection to medical professionals due to close contact with patients and substantial health burden. While automation of the nasopharyngeal swab is regarded as a potential solution to address these problems, the quantitative study of force for safe and effective control has not been widely performed yet. Hence, this study presents applied force during the standard nasopharyngeal swab sampling procedure using a handheld sensorized instrument. The sensorized instrument can simultaneously measure multi-axis forces and 6-DOF hand motion while allowing natural hand motion as is used in the standard swab sampling. To accurately measure force from the handheld instrument, the compensation of gravity bias is accomplished online while estimating the orientation of the hand with an embedded IMU sensor. As a result, the instrument can measure all three-axes forces by an error below 5 mN. A simulated test on a phantom model using the sensorized instrument shows that how the forces vary during the sampling sequences.

Active locomotion of a paddling-based capsule endoscope in an in vitro and in vivo experiment (with videos).

BACKGROUND: Capsule endoscopy that could actively move and approach a specific site might be more valuable for the diagnosis or treatment of GI diseases. OBJECTIVE: We tested the performance of active locomotion of a novel wired capsule endoscope with a paddling-based locomotion mechanism, using 3 models: a silicone tube, an extracted porcine colon, and a living pig. DESIGN: In vitro, ex vivo, and in vivo experiments in a pig model. SETTING: Study in an animal laboratory. INTERVENTIONS: For the in vitro test, the locomotive capsule was controlled to actively move from one side of a silicone tube to the other by a controller-operated automatic traveling program. The velocity was calculated by following a video recording. We performed ex vivo tests by using an extracted porcine colon in the same manner we performed the in vitro test. In in vivo experiments, the capsule was inserted into the rectum of a living pig under anesthesia, and was controlled to move automatically forward. After 8 consecutive trials, the velocity was calculated. MAIN OUTCOME MEASUREMENTS: Elapsed time, velocity, and mucosal damage. RESULTS: The locomotive capsule showed stable and active movement inside the lumen both in vitro and ex vivo. The velocity was 60 cm/min in the silicone tube, and 36.8 and 37.5 cm/min in the extracted porcine colon. In the in vivo experiments, the capsule stably moved forward inside the colon of a living pig without any serious complications. The mean velocity was 17 cm/min over 40 cm length. We noted pinpoint erythematous mucosal injuries in the colon. LIMITATION: Porcine model experiments, wired capsule endoscope. CONCLUSIONS: The novel paddling-based locomotive capsule endoscope performed fast and stable movement in a living pig colon with consistent velocity. Further investigation is necessary for practical use in humans.

An improved measurement of dsDNA elasticity using AFM.

The mechanical properties of a small fragment (30 bp) of an individual double-stranded deoxyribonucleic acid (dsDNA) in water have been investigated by atomic force microscopy (AFM). We have stretched three systems including ssDNA, double-fixed dsDNA (one strand of the dsDNA molecules was biotinylated at the 3'-end and thiolated at the 5'-end, this was reversed for the other complementary strand) and single-fixed dsDNA (one strand of the dsDNA molecules was biotinylated at the 3'-end and thiolated at the 5'-end, whereas the other complementary strand was biotinylated at only the 5'-end). The achieved thiolation and biotinylation were to bind ds- or ssDNA to the gold surface and streptavidin-coated AFM tip, respectively. Analysis of the force versus displacement (F-D) curves from tip-DNA-substrate systems shows that the pull-off length (L(o)) and stretch length (delta) from the double-fixed system were shorter than those observed in the ssDNA and the single-fixed system. The obtained stretch force (F(st)) from the single-fixed dsDNA was much greater than that from the ssDNA even though it was about 10 pN greater than the one obtained in the double-fixed system. As a result, the Young's modulus of the double-fixed dsDNA was greater than that of the single-fixed dsDNA and the ssDNA. A more reliable stiffness of the dsDNA was observed via the double-fixed system, since there is no effect of the unpaired molecules during stretching, which always occurred in the single-fixed system. The unpaired molecules were also observed by comparing the stiffness of ssDNA and single-fixed dsDNA in which the end of one strand was left free.

One-step Implantation of a 3D Neural Microelectrode Array.

This paper presents a neurosurgical device called NEIT 2 (Nerve Electrode Insertion Tool) to implant a 3D microelectrode array into a peripheral nervous system. Using an elastomer-made nerve holder, the device is able to stable target a flexible nerve, and then safely inserts an electrode array into the fixed nerve. Finally, a nerve containment assembly is made at once. We conducted animal experiments to evaluate the proposed scenario using a 3D printed prototype and commercial microelectrodes. The results show that microelectrodes are successfully implanted into sciatic nerves of rats and neural signals are recorded through the chronically implanted electrodes.

Quantitative evaluation of cardiomyocyte contractility in a 3D microenvironment.

Three-dimensional cultures in a microfabricated environment provide in vivo-like conditions for cells, and have been used in a variety of applications in basic and clinical studies. In this study, the contractility of cardiomyocytes in a 3D environment using complex 3D hybrid biopolymer microcantilevers was quantified and compared with that observed in a 2D environment. By measuring the deflections of the microcantilevers with different surfaces and carrying out finite element modeling (FEM) of the focal pressures of the microcantilevers, it was found that the contractile force of high-density cardiomyocytes on 3D grooved surfaces was 65-85% higher than that of cardiomyocytes on flat surfaces. These results were supported by immunostaining, which showed alignment of the cytoskeleton and elongation of the nuclei, as well as by quantitative RT-PCR, which revealed that cells on the grooved surface had experienced sustained stimuli and tighter cell-to-cell interactions.

Towards Finger Motion Capture System Using FBG Sensors.

This paper introduces a novel finger motion capture system using FBG (fiber Bragg grating) optical sensors. We develop two types of sensors to seamlessly reconstruct finger motion from strains induced in the FBGs. First, the shape sensor incorporates three optical fibers with multiple FBGs to reconstruct the position and orientation of a finger joint in 3D. In addition, the angle sensor is designed to measure the high curvature of bending on the finger joints. By deploying the two types of sensors on the fingers, we can reconstruct various finger motion in real time without drift over time. The accuracies of the fabricated FBG sensors are evaluated, resulting in an average error of 1.49 mm for the shape sensor at the distal tip (1.9% for the full length of the sensor) and 0.21° error for the angle sensor. We finally demonstrate finger motion tracking with the FBG sensors in real time, while measuring the multi-DOF motion at the carpometacarpal joint of the thumb and also the high curvatures of bending motion at the metacarpophalangeal and interphalangeal joints of the thumb and the index finger.

EyeSLAM: Real-time simultaneous localization and mapping of retinal vessels during intraocular microsurgery.

BACKGROUND: Fast and accurate mapping and localization of the retinal vasculature is critical to increasing the effectiveness and clinical utility of robot-assisted intraocular microsurgery such as laser photocoagulation and retinal vessel cannulation. METHODS: The proposed EyeSLAM algorithm delivers 30 Hz real-time simultaneous localization and mapping of the human retina and vasculature during intraocular surgery, combining fast vessel detection with 2D scan-matching techniques to build and localize a probabilistic map of the vasculature. RESULTS: In the harsh imaging environment of retinal surgery with high magnification, quick shaky motions, textureless retina background, variable lighting and tool occlusion, EyeSLAM can map 75% of the vessels within two seconds of initialization and localize the retina in real time with a root mean squared (RMS) error of under 5.0 pixels (translation) and 1° (rotation). CONCLUSIONS: EyeSLAM robustly provides retinal maps and registration that enable intelligent surgical micromanipulators to aid surgeons in simulated retinal vessel tracing and photocoagulation tasks.

Establishment of a fabrication method for a long-term actuated hybrid cell robot.

We developed a novel method to fabricate a crab-like microrobot that can actuate for a long period in a physiological condition. The microrobot backbone was built with a biocompatible and elastic material-polydimethylsiloxane (PDMS)-by using a specially designed 3D molding aligner, and consisted of three strips of PDMS "legs" connected across a "body." Cardiomyocytes were then plated on the grooved top surface of the backbone, resulting in a high concentration of pulsating cells. These key techniques enabled the microrobot to walk continuously for over ten days. The performance of our crab-like microrobot was measured at an average velocity of 100 microm s(-1), and the estimated total distance it travelled was 50 m over a one-week period. Thus, we have demonstrated for the first time a walking robot that exhibited reliable and long-term actuation performances.