A Respiratory Motion Prediction Method Based on LSTM-AE with Attention Mechanism for Spine Surgery.

Respiratory motion-induced vertebral movements can adversely impact intraoperative spine surgery, resulting in inaccurate positional information of the target region and unexpected damage during the operation. In this paper, we propose a novel deep learning architecture for respiratory motion prediction, which can adapt to different patients. The proposed method utilizes an LSTM-AE with attention mechanism network that can be trained using few-shot datasets during operation. To ensure real-time performance, a dimension reduction method based on the respiration-induced physical movement of spine vertebral bodies is introduced. The experiment collected data from prone-positioned patients under general anaesthesia to validate the prediction accuracy and time efficiency of the LSTM-AE-based motion prediction method. The experimental results demonstrate that the presented method (RMSE: 4.39%) outperforms other methods in terms of accuracy within a learning time of 2 min. The maximum predictive errors under the latency of 333 ms with respect to the x, y, and z axes of the optical camera system were 0.13, 0.07, and 0.10 mm, respectively, within a motion range of 2 mm.

Robotic-assisted navigation system for preoperative lung nodule localization: a pilot study.

Background: Preoperative percutaneous computed tomography (CT)-guided localization of pulmonary nodules plays a pivotal role in the diagnosis and treatment of early-stage lung cancer. However, conventional manual localization techniques have inherent limitations in achieving a high degree of accuracy. Consequently, a novel robotic-assisted navigation system was developed to attain precise localization of small lung nodules. This study aims to investigate the accuracy and safety of this system in clinical applications. Methods: Patients with peripheral solitary pulmonary nodules measuring less than 20 mm were enrolled. The robotic-assisted navigation system generated a three-dimensional (3D) model based on the patient's CT images, determining the optimal puncture path. The robotic arm then accurately located the nodule and, following percutaneous puncture, indocyanine green (ICG) was injected. The primary outcome measure was the accuracy of pulmonary nodule localization, while secondary outcomes included the complication rate, procedural duration, and total radiation exposure. Results: A total of 33 nodules were successfully localized using the robotic-assisted navigation system and resected through video-assisted thoracoscopic surgery (VATS). The first-pass success rate was 100%, with a median deviation of 6.1 mm [interquartile range (IQR), 2.5-7.2 mm] between the localizer and the nodule. The median localization time was 25.0 minutes, and the single and cumulative exam dose-length products (DLP) were 534.0 and 1491.0 mGy·cm, respectively. Notably, no observable complications were reported during the procedures. Conclusions: The innovative robotic-assisted navigation system demonstrated satisfactory accuracy and holds promise for improving the percutaneous localization of lung nodules. This method represents a safe and viable alternative to traditional CT-guided manual localization techniques.

Robot-assisted navigation for percutaneous localization of peripheral pulmonary nodule: an in vivo swine study.

Background: Robot-assisted surgery (RAS) systems have been developed but rarely applied to lung nodule localization. This study aimed to assess the feasibility and safety of using a robot-assisted navigation system in percutaneous lung nodule localization. Methods: A computed tomography (CT)-guided robot-assisted navigation system was used to localize the simulated peripheral nodule in the swine lung through fluorescent agent injection. After the localization, fluorescent thoracoscopic wedge resection was performed. The deviation between the target point and the needle tip was measured using a professional 3-dimensional (3D) distance measurement software. The primary outcome was the localization accuracy (deviation) of the localization. The secondary outcomes were the localization-related complication rate, the localization duration, and the success rate. Results: A total of 4 pigs were enrolled, and 20 peripheral lung nodules were created and localized successfully. All nodules underwent subsequent wedge resection for verification. The mean deviation by measuring the 3D distance was 3.81 mm [standard deviation (SD): 1.29 mm, 95% confidence interval (CI): 2.936-4.536 mm]. The technical success rate for localization was 100%, and the mean localization time was 14.69 minutes (SD: 4.67 minutes). The complication rate was 5% (1/20), with 1 pneumothorax after localization, and no mortality occurred. Conclusions: This pilot animal study demonstrated the promising potential of the robot-assisted navigation technique in peripheral lung nodule localization, with high accuracy and feasibility. Further clinical trials are needed to validate its safety compared to traditional manual localization.

Fixed-Time Recurrent NN Learning Control of Uncertain Robotic Manipulators with Time-Varying Constraints: Experimental Verification.

This paper proposes a learning control framework for the robotic manipulator's dynamic tracking task demanding fixed-time convergence and constrained output. In contrast with model-dependent methods, the proposed solution deals with unknown manipulator dynamics and external disturbances by virtue of a recurrent neural network (RNN)-based online approximator. First, a time-varying tangent-type barrier Lyapunov function (BLF) is introduced to construct a fixed-time virtual controller. Then, the RNN approximator is embedded in the closed-loop system to compensate for the lumped unknown term in the feedforward loop. Finally, we devise a novel fixed-time, output-constrained neural learning controller by integrating the BLF and RNN approximator into the main framework of the dynamic surface control (DSC). The proposed scheme not only guarantees the tracking errors converge to the small neighborhoods about the origin in a fixed time, but also preserves the actual trajectories always within the prescribed ranges and thus improves the tracking accuracy. Experiment results illustrate the excellent tracking performance and verify the effectiveness of the online RNN estimate for unknown dynamics and external disturbances.

A Novel Robotic Bronchoscope System for Navigation and Biopsy of Pulmonary Lesions.

Transbronchial biopsy sampling, as a minimally invasive method with relatively low risk, has been proved to be a promising treatment in the field of respiratory surgery. Although several robotic bronchoscopes have been developed, it remains a great challenge to balance size and flexibility, while integrating multisensors to realize navigation during complex airway networks. This paper proposes a novel robotic bronchoscope system composed by end effector with relatively small size, relevant actuation unit, and navigation system with path planning and surgical guidance capability. The main part of the end effector is machined by bidirectional groove on a nickel-titanium tube, which can realize bending, rotation, and translation 3 degrees of freedom. A prototype of the proposed robotic bronchoscope system is designed and fabricated, and its performance is tested through several experiments to verify the stiffness, flexibility, and navigation performance. The results show that the proposed system is with good environment adaptiveness, and it can become a promising biopsy method through natural cavity of the human body.

A Robust Control via a Fuzzy System with PID for the ROV.

Uncertainty and nonlinearity in the depth control of remotely operated vehicles (ROVs) have been widely studied, especially in complex underwater environments. To improve the motion performance of ROVs and enhance their robustness, the model of ROV depth control in complex water environments was developed. The developed control scheme of interval type-2 fuzzy proportional-integral-derivative control (IT2FPID) is based on proportional-integral-derivative control (PID) and interval type-2 fuzzy logic control (IT2FLC). The performance indicators were used to evaluate the immunity of the controller type to external disturbances. The overshoot of 0.3% and settling time of 7.5 s of IT2FPID seem to be more robust compared to those of type-1 fuzzy proportional-integral-derivative (T1FPID) and PID.

Unsupervised learning of depth estimation from imperfect rectified stereo laparoscopic images.

BACKGROUND: Learning-based methods have achieved remarkable performances on depth estimation. However, the premise of most self-learning and unsupervised learning methods is built on rigorous, geometrically-aligned stereo rectification. The performances of these methods degrade when the rectification is not accurate. Therefore, we explore an approach for unsupervised depth estimation from stereo images that can handle imperfect camera parameters. METHODS: We propose an unsupervised deep convolutional network that takes rectified stereo image pairs as input and outputs corresponding dense disparity maps. First, a new vertical correction module is designed for predicting a correction map to compensate for the imperfect geometry alignment. Second, the left and right images, which are reconstructed based on the input image pair and corresponding disparities as well as the vertical correction maps, are regarded as the outputs of the generative term of the generative adversarial network (GAN). Then, the discriminator term of the GAN is used to distinguish the reconstructed images from the original inputs to force the generator to output increasingly realistic images. In addition, a residual mask is introduced to exclude pixels that conflict with the appearance of the original image in the loss calculation. RESULTS: The proposed model is validated on the publicly available Stereo Correspondence and Reconstruction of Endoscopic Data (SCARED) dataset and the average MAE is 3.054 mm. CONCLUSION: Our model can effectively handle imperfect rectified stereo images for depth estimation.

Segmentation and recognition of breast ultrasound images based on an expanded U-Net.

This paper establishes a fully automatic real-time image segmentation and recognition system for breast ultrasound intervention robots. It adopts the basic architecture of a U-shaped convolutional network (U-Net), analyses the actual application scenarios of semantic segmentation of breast ultrasound images, and adds dropout layers to the U-Net architecture to reduce the redundancy in texture details and prevent overfitting. The main innovation of this paper is proposing an expanded training approach to obtain an expanded of U-Net. The output map of the expanded U-Net can retain texture details and edge features of breast tumours. Using the grey-level probability labels to train the U-Net is faster than using ordinary labels. The average Dice coefficient (standard deviation) and the average IOU coefficient (standard deviation) are 90.5% (±0.02) and 82.7% (±0.02), respectively, when using the expanded training approach. The Dice coefficient of the expanded U-Net is 7.6 larger than that of a general U-Net, and the IOU coefficient of the expanded U-Net is 11 larger than that of the general U-Net. The context of breast ultrasound images can be extracted, and texture details and edge features of tumours can be retained by the expanded U-Net. Using an expanded U-Net can quickly and automatically achieve precise segmentation and multi-class recognition of breast ultrasound images.

A Flexible Transoral Robot Towards COVID-19 Swab Sampling.

There are high risks of infection for surgeons during the face-to-face COVID-19 swab sampling due to the novel coronavirus's infectivity. To address this issue, we propose a flexible transoral robot with a teleoperated configuration for swab sampling. The robot comprises a flexible manipulator, an endoscope with a monitor, and a master device. A 3-prismatic-universal (3-PU) flexible parallel mechanism with 3 degrees of freedom (DOF) is used to realize the manipulator's movements. The flexibility of the manipulator improves the safety of testees. Besides, the master device is similar to the manipulator in structure. It is easy to use for operators. Under the guidance of the vision from the endoscope, the surgeon can operate the master device to control the swab's motion attached to the manipulator for sampling. In this paper, the robotic system, the workspace, and the operation procedure are described in detail. The tongue depressor, which is used to prevent the tongue's interference during the sampling, is also tested. The accuracy of the manipulator under visual guidance is validated intuitively. Finally, the experiment on a human phantom is conducted to demonstrate the feasibility of the robot preliminarily.

Modelling and Experiment Based on a Navigation System for a Cranio-Maxillofacial Surgical Robot.

In view of the characteristics of high risk and high accuracy in cranio-maxillofacial surgery, we present a novel surgical robot system that can be used in a variety of surgeries. The surgical robot system can assist surgeons in completing biopsy of skull base lesions, radiofrequency thermocoagulation of the trigeminal ganglion, and radioactive particle implantation of skull base malignant tumors. This paper focuses on modelling and experimental analyses of the robot system based on navigation technology. Firstly, the transformation relationship between the subsystems is realized based on the quaternion and the iterative closest point registration algorithm. The hand-eye coordination model based on optical navigation is established to control the end effector of the robot moving to the target position along the planning path. The closed-loop control method, "kinematics + optics" hybrid motion control method, is presented to improve the positioning accuracy of the system. Secondly, the accuracy of the system model was tested by model experiments. And the feasibility of the closed-loop control method was verified by comparing the positioning accuracy before and after the application of the method. Finally, the skull model experiments were performed to evaluate the function of the surgical robot system. The results validate its feasibility and are consistent with the preoperative surgical planning.

Towards human-controlled, real-time shape sensing based flexible needle steering for MRI-guided percutaneous therapies.

BACKGROUND: Accurate needle placement into soft tissue is essential to percutaneous prostate cancer diagnosis and treatment procedures. METHODS: This paper discusses the steering of a 20 gauge (G) FBG-integrated needle with three sets of Fiber Bragg Grating (FBG) sensors. A fourth-order polynomial shape reconstruction method is introduced and compared with previous approaches. To control the needle, a bicycle model based navigation method is developed to provide visual guidance lines for clinicians. A real-time model updating method is proposed for needle steering inside inhomogeneous tissue. A series of experiments were performed to evaluate the proposed needle shape reconstruction, visual guidance and real-time model updating methods. RESULTS: Targeting experiments were performed in soft plastic phantoms and in vitro tissues with insertion depths ranging between 90 and 120 mm. Average targeting errors calculated based upon the acquired camera images were 0.40 ± 0.35 mm in homogeneous plastic phantoms, 0.61 ± 0.45 mm in multilayer plastic phantoms and 0.69 ± 0.25 mm in ex vivo tissue. CONCLUSIONS: Results endorse the feasibility and accuracy of the needle shape reconstruction and visual guidance methods developed in this work. The approach implemented for the multilayer phantom study could facilitate accurate needle placement efforts in real inhomogeneous tissues. Copyright © 2016 John Wiley & Sons, Ltd.

A medical manipulator system with lasers in photodynamic therapy of port wine stains.

Port wine stains (PWS) are a congenital malformation and dilation of the superficial dermal capillary. Photodynamic therapy (PDT) with lasers is an effective treatment of PWS with good results. However, because the laser density is uneven and nonuniform, the treatment is carried out manually by a doctor thus providing little accuracy. Additionally, since the treatment of a single lesion can take between 30 and 60 minutes, the doctor can become fatigued after only a few applications. To assist the medical staff with this treatment method, a medical manipulator system (MMS) was built to operate the lasers. The manipulator holds the laser fiber and, using a combination of active and passive joints, the fiber can be operated automatically. In addition to the control input from the doctor over a human-computer interface, information from a binocular vision system is used to guide and supervise the operation. Clinical results are compared in nonparametric values between treatments with and without the use of the MMS. The MMS, which can significantly reduce the workload of doctors and improve the uniformity of laser irradiation, was safely and helpfully applied in PDT treatment of PWS with good therapeutic results.

Development and clinical evaluation of medical robot assisted photodynamic therapy of port wine stains.

BACKGROUND: Port wine stains (PWS) are a kind of skin disease for which photodynamic therapy (PDT) has already achieved good results. With manual operation of clinical PDT, the laser density is uneven and laser irradiation of the lesion is arbitrary and non-uniform. In addition, lengthy manual operation tires doctors; thus a robot system has been developed to assist them. METHODS: First, a novel medical manipulator consisting of five passive joints (robot arm) and two active joints (robot wrist) was developed to automatically improve the uniformity of laser irradiation. Second, image processing of the lesion was introduced. Third, kinematics and path planning of the robot were analysed, and safety precautions were introduced. Then, accuracy tests of the robot wrist and robot system were conducted separately before clinical application. Finally, a total of 50 PWS cases were treated using the robot system. The clinical outcomes and comparison of non-parametric values were employed to evaluate the robot system. RESULTS: The accuracies of the robot wrist and robot system were shown to meet the requirements of clinical PDT treatment. The robot system performed successfully in 50 PWS cases. Doctors can devote more energy to clinical judgments during treatment with the assistance of the robot system. All the PWS have shown different degrees of improvement. The results show that the robot system is useful in assisting doctors for the PDT treatment of PWS. CONCLUSIONS: The experiments show the feasibility and usefulness of the robot system in assisting doctors giving PDT treatment for PWS. The robot system can lighten the load on doctors and improve the therapeutic effect.