A Double-Hurdle Quantification Model for Freezing of Gait of Parkinson's Patients.

Freezing of gait (FOG) leads to an increased risk of falls and limited mobility in individuals with Parkinson's disease (PD). However, existing research ignores the fine-grained quantitative assessment of FOG severity. This paper provides a double-hurdle model that uses typical spatiotemporal gait features to quantify the FOG severity in patients with PD. Moreover, a novel multi-output random forest algorithm is used as one hurdle of the double-hurdle model, further enhancing the model's performance. We conduct six experiments on a public PD gait database. Results demonstrate that the designed random forest algorithm in the double-hurdle model-hyperparameter independence framework achieves outstanding performances with the highest correlation coefficient (CC) of 0.972 and the lowest root mean square error (RMSE) of 2.488. Furthermore, we study the effect of drug state on the gait patterns of PD patients with or without FOG. Results show that "OFF" state amplifies the visibility of FOG symptoms in PD patients. Therefore, this study holds significant implications for the management and treatment of PD.

Design, Optimization, and Modeling of a Hydraulic Soft Robot for Chronic Total Occlusions.

Chronic total occlusion (CTO) is one of the most severe and sophisticated vascular stenosis because of complete blockage, greater operation difficulty, and lower procedural success rate. This study proposes a hydraulic-driven soft robot imitating the earthworm's locomotion to assist doctors or operators in actively opening thrombi in coronary or peripheral artery vessels. Firstly, a three-actuator bionic soft robot is developed based on earthworms' physiological structure. The soft robot's locomotion gait inspired by the earthworm's mechanism is designed. Secondly, the influence of structure parameters on actuator deformation, stress, and strain is explored, which can help us determine the soft actuators' optimal structure parameters. Thirdly, the relationship between hydraulic pressure and actuator deformation is investigated by performing finite element analysis using the bidirectional fluid-structure interaction (FSI) method. The kinematic models of the soft actuators are established to provide a valuable reference for the soft actuators' motion control.

Learning Skill Characteristics From Manipulations.

Percutaneous coronary intervention (PCI) has increasingly become the main treatment for coronary artery disease. The procedure requires high experienced skills and dexterous manipulations. However, there are few techniques to model PCI skill so far. In this study, a learning framework with local and ensemble learning is proposed to learn skill characteristics of different skill-level subjects from their PCI manipulations. Ten interventional cardiologists (four experts and six novices) were recruited to deliver a medical guidewire to two target arteries on a porcine model for in vivo studies. Simultaneously, translation and twist manipulations of thumb, forefinger, and wrist are acquired with electromagnetic (EM) and fiber-optic bend (FOB) sensors, respectively. These behavior data are then processed with wavelet packet decomposition (WPD) under 1-10 levels for feature extraction. The feature vectors are further fed into three candidate individual classifiers in the local learning layer. Furthermore, the local learning results from different manipulation behaviors are fused in the ensemble learning layer with three rule-based ensemble learning algorithms. In subject-dependent skill characteristics learning, the ensemble learning can achieve 100% accuracy, significantly outperforming the best local result (90%). Furthermore, ensemble learning can also maintain 73% accuracy in subject-independent schemes. These promising results demonstrate the great potential of the proposed method to facilitate skill learning in surgical robotics and skill assessment in clinical practice.

A Novel Spatial Position Prediction Navigation System Makes Surgery More Accurate.

During intravascular interventional surgery, the 3D surgical navigation system can provide doctors with 3D spatial information of the vascular lumen, reducing the impact of missing dimension caused by digital subtraction angiography (DSA) guidance and further improving the success rate of surgeries. Nevertheless, this task often comes with the challenge of complex registration problems due to vessel deformation caused by respiratory motion and high requirements for the surgical environment because of the dependence on external electromagnetic sensors. This article proposes a novel 3D spatial predictive positioning navigation (SPPN) technique to predict the real-time tip position of surgical instruments. In the first stage, we propose a trajectory prediction algorithm integrated with instrumental morphological constraints to generate the initial trajectory. Then, a novel hybrid physical model is designed to estimate the trajectory's energy and mechanics. In the second stage, a point cloud clustering algorithm applies multi-information fusion to generate the maximum probability endpoint cloud. Then, an energy-weighted probability density function is introduced using statistical analysis to achieve the prediction of the 3D spatial location of instrument endpoints. Extensive experiments are conducted on 3D-printed human artery and vein models based on a high-precision electromagnetic tracking system. Experimental results demonstrate the outstanding performance of our method, reaching 98.2% of the achievement ratio and less than 3 mm of the average positioning accuracy. This work is the first 3D surgical navigation algorithm that entirely relies on vascular interventional robot sensors, effectively improving the accuracy of interventional surgery and making it more accessible for primary surgeons.

SurgiNet: Pyramid Attention Aggregation and Class-wise Self-Distillation for Surgical Instrument Segmentation.

Surgical instrument segmentation plays a promising role in robot-assisted surgery. However, illumination issues often appear in surgical scenes, altering the color and texture of surgical instruments. Changes in visual features make surgical instrument segmentation difficult. To address illumination issues, the SurgiNet is proposed to learn pyramid attention features. The double attention module is designed to capture the semantic dependencies between locations and channels. Based on semantic dependencies, the semantic features in the disturbed area can be inferred for addressing illumination issues. Pyramid attention is aggregated to capture multi-scale features and make predictions more accurate. To perform model compression, class-wise self-distillation is proposed to enhance the representation learning of the network, which performs feature distillation within the class to eliminate interference from other classes. Top-down and multi-stage knowledge distillation is designed to distill class probability maps. By inter-layer supervision, high-level probability maps are applied to calibrate the probability distribution of low-level probability maps. Since class-wise distillation enhances the self-learning of the network, the network can get excellent performance with a lightweight backbone. The proposed network achieves the state-of-the-art performance of 89.14% mIoU on CataIS with only 1.66 GFlops and 2.05 M parameters. It also takes first place on EndoVis 2017 with 66.30% mIoU.

Hepatocellular carcinoma-derived exosomal miRNA-761 regulates the tumor microenvironment by targeting the SOCS2/JAK2/STAT3 pathway.

BACKGROUND: Exosomes and exosomal microRNAs have been implicated in tumor occurrence and metastasis. Our previous study showed that microRNA-761 (miR-761) is overexpressed in hepatocellular carcinoma (HCC) tissues and that its inhibition affects mitochondrial function and inhibits HCC metastasis. The mechanism by which exosomal miR-761 modulates the tumor microenvironment has not been elucidated. METHODS: Exosomal miR-761 was detected in six cell lines. Cell counting kit-8 (CCK-8) and transwell migration assays were performed to determine the function of exosomal miR-761 in HCC cells. The luciferase reporter assay was used to analyze miR-761 target genes in normal fibroblasts (NFs). The inhibitors AZD1480 and C188-9 were employed to determine the role of the Janus kinase 2/signal transducer and activator of transcription 3 (JAK2/STAT3) signaling pathway in the transformation of cancer-associated fibroblasts (CAFs). RESULTS: In this study, we characterized the mechanism by which miR-761 reprogrammed the tumor microenvironment. We found that HCC-derived exosomal miR-761 was taken up by NFs. Moreover, HCC exosomes affected the tumor microenvironment by activating NFs via suppressor of cytokine signaling 2 (SOCS2) and the JAK2/STAT3 signaling pathway. CONCLUSIONS: These results demonstrated that exosomal miR-761 modulated the tumor microenvironment via SOCS2/JAK2/STAT3 pathway-dependent activation of CAFs. Our findings may inspire new strategies for HCC prevention and therapy.

Space Squeeze Reasoning and Low-Rank Bilinear Feature Fusion for Surgical Image Segmentation.

Surgical image segmentation is critical for surgical robot control and computer-assisted surgery. In the surgical scene, the local features of objects are highly similar, and the illumination interference is strong, which makes surgical image segmentation challenging. To address the above issues, a bilinear squeeze reasoning network is proposed for surgical image segmentation. In it, the space squeeze reasoning module is proposed, which adopts height pooling and width pooling to squeeze global contexts in the vertical and horizontal directions, respectively. The similarity between each horizontal position and each vertical position is calculated to encode long-range semantic dependencies and establish the affinity matrix. The feature maps are also squeezed from both the vertical and horizontal directions to model channel relations. Guided by channel relations, the affinity matrix is expanded to the same size as the input features. It captures long-range semantic dependencies from different directions, helping address the local similarity issue. Besides, a low-rank bilinear fusion module is proposed to enhance the model's ability to recognize similar features. This module is based on the low-rank bilinear model to capture the inter-layer feature relations. It integrates the location details from low-level features and semantic information from high-level features. Various semantics can be represented more accurately, which effectively improves feature representation. The proposed network achieves state-of-the-art performance on cataract image segmentation dataset CataSeg and robotic image segmentation dataset EndoVis 2018.

A Multilayer and Multimodal-Fusion Architecture for Simultaneous Recognition of Endovascular Manipulations and Assessment of Technical Skills.

The clinical success of the percutaneous coronary intervention (PCI) is highly dependent on endovascular manipulation skills and dexterous manipulation strategies of interventionalists. However, the analysis of endovascular manipulations and related discussion for technical skill assessment are limited. In this study, a multilayer and multimodal-fusion architecture is proposed to recognize six typical endovascular manipulations. The synchronously acquired multimodal motion signals from ten subjects are used as the inputs of the architecture independently. Six classification-based and two rule-based fusion algorithms are evaluated for performance comparisons. The recognition metrics under the determined architecture are further used to assess technical skills. The experimental results indicate that the proposed architecture can achieve the overall accuracy of 96.41%, much higher than that of a single-layer recognition architecture (92.85%). In addition, the multimodal fusion brings significant performance improvement in comparison with single-modal schemes. Furthermore, the K -means-based skill assessment can obtain an accuracy of 95% to cluster the attempts made by different skill-level groups. These hopeful results indicate the great possibility of the architecture to facilitate clinical skill assessment and skill learning.

A Unified Framework for Multi-Guidewire Endpoint Localization in Fluoroscopy Images.

OBJECTIVE: In this paper, Keypoint Localization Region-based CNN (KL R-CNN) is proposed, which can simultaneously accomplish the guidewire detection and endpoint localization in a unified model. METHODS: KL R-CNN modifies Mask R-CNN by replacing the mask branch with a novel keypoint localization branch. Besides, some settings of Mask R-CNN are also modified to generate the keypoint localization results at a higher detail level. At the same time, based on the existing metrics of Average Precision (AP) and Percentage of Correct Keypoints (PCK), a new metric named APPCK is proposed to evaluate the overall performance on the multi-guidewire endpoint localization task. Compared with existing metrics, APPCK is easy to use and its results are more intuitive. RESULTS: Compared with existing methods, KL R-CNN has better performance when the threshold is loose, reaching a mean APPCK of 90.65% when the threshold is 9 pixels. CONCLUSION: KL R-CNN achieves the state-of-the-art performance on the multi-guidewire endpoint localization task and has application potentials. SIGNIFICANCE: KL R-CNN can achieve the localization of guidewire endpoints in fluoroscopy images, which is a prerequisite for computer-assisted percutaneous coronary intervention. KL R-CNN can also be extended to other multi-instrument localization tasks.

TR-GAN: Multi-Session Future MRI Prediction With Temporal Recurrent Generative Adversarial Network.

Magnetic Resonance Imaging (MRI) has been proven to be an efficient way to diagnose Alzheimer's disease (AD). Recent dramatic progress on deep learning greatly promotes the MRI analysis based on data-driven CNN methods using a large-scale longitudinal MRI dataset. However, most of the existing MRI datasets are fragmented due to unexpected quits of volunteers. To tackle this problem, we propose a novel Temporal Recurrent Generative Adversarial Network (TR-GAN) to complete missing sessions of MRI datasets. Unlike existing GAN-based methods, which either fail to generate future sessions or only generate fixed-length sessions, TR-GAN takes all past sessions to recurrently and smoothly generate future ones with variant length. Specifically, TR-GAN adopts recurrent connection to deal with variant input sequence length and flexibly generate future variant sessions. Besides, we also design a multiple scale & location (MSL) module and a SWAP module to encourage the model to better focus on detailed information, which helps to generate high-quality MRI data. Compared with other popular GAN architectures, TR-GAN achieved the best performance in all evaluation metrics of two datasets. After expanding the Whole MRI dataset, the balanced accuracy of AD vs. cognitively normal (CN) vs. mild cognitive impairment (MCI) and stable MCI vs. progressive MCI classification can be increased by 3.61% and 4.00%, respectively.

An Improved Networked Predictive Controller for Vascular Robot Using 5G Networks.

Percutaneous coronary intervention (PCI) has gradually become the most common treatment of coronary artery disease (CAD) in clinical practice due to its advantages of small trauma and quick recovery. However, the availability of hospitals with cardiac catheterization facilities and trained interventionalists is extremely limited in remote and underdeveloped areas. Remote vascular robotic system can assist interventionalists to complete operations precisely, and reduce occupational health hazards occurrence. In this paper, a bionic remote vascular robot is introduced in detail from three parts: mechanism, communication architecture, and controller model. Firstly, human finger-like mechanisms in vascular robot enable the interventionalists to advance, retract and rotate the guidewires or balloons. Secondly, a 5G-based communication system is built to satisfy the end-to-end requirements of strong data transmission and packet priority setting in remote robot control. Thirdly, a generalized predictive controller (GPC) is developed to suppress the effect of time-varying network delay and parameter identification error, while adding a designed polynomial compensation module to reduce tracking error and improve system responsiveness. Then, the simulation experiment verifies the system performance in comparison with different algorithms, network delay, and packet loss rate. Finally, the improved control system conducted PCI on an experimental pig, which reduced the delivery integral absolute error (IAE) by at least 20% compared with traditional methods.

Design and Performance Evaluation of a Novel Vascular Robotic System for Complex Percutaneous Coronary Interventions.

The robotic-assisted percutaneous coronary intervention is an emerging technology with great potential to solve the shortcomings of existing treatments. However, the current robotic systems can not manipulate two guidewires or ballons/stents simultaneously for coronary bifurcation lesions. This paper presents VasCure, a novel bio-inspired vascular robotic system, to deliver two guidewires and stents into the main branch and side branch of bifurcation lesions in sequence. The system is designed in master-slave architecture to reduce occupational hazards of radiation exposure and orthopedic injury to interventional surgeons. The slave delivery device has one active roller and two passive rollers to manipulate two interventional devices. The performance of the VasCure was verified by in vitro and in vivo animal experiments. In vitro results showed the robotic system has good accuracy to deliver guidewires and the maximum error is 0.38mm. In an animal experiment, the interventional surgeon delivered two guidewires and balloons to the left circumflex branch and the left anterior descending branch of the pig, which confirmed the feasibility of the vascular robotic system.

Real-Time Multi-Guidewire Endpoint Localization in Fluoroscopy Images.

The real-time localization of the guidewire endpoints is a stepping stone to computer-assisted percutaneous coronary intervention (PCI). However, methods for multi-guidewire endpoint localization in fluoroscopy images are still scarce. In this paper, we introduce a framework for real-time multi-guidewire endpoint localization in fluoroscopy images. The framework consists of two stages, first detecting all guidewire instances in the fluoroscopy image, and then locating the endpoints of each single guidewire instance. In the first stage, a YOLOv3 detector is used for guidewire detection, and a post-processing algorithm is proposed to refine the guidewire detection results. In the second stage, a Segmentation Attention-hourglass (SA-hourglass) network is proposed to predict the endpoint locations of each single guidewire instance. The SA-hourglass network can be generalized to the keypoint localization of other surgical instruments. In our experiments, the SA-hourglass network is applied not only on a guidewire dataset but also on a retinal microsurgery dataset, reaching the mean pixel error (MPE) of 2.20 pixels on the guidewire dataset and the MPE of 5.30 pixels on the retinal microsurgery dataset, both achieving the state-of-the-art localization results. Besides, the inference rate of our framework is at least 20FPS, which meets the real-time requirement of fluoroscopy images (6-12FPS).

Qualitative and Quantitative Assessment of Technical Skills in Percutaneous Coronary Intervention: In Vivo Porcine Studies.

OBJECTIVE: Technical skill assessment plays an important role in the professional development of an interventionalist in percutaneous coronary intervention (PCI). However, most of the traditional assessment methods are time consuming and subjective. This paper aims to develop objective assessment techniques. METHODS: In this study, a natural-behavior-based assessment framework is proposed to qualitatively and quantitatively assess technical skills in PCI. In vivo porcine studies were conducted to deliver a medical guidewire to two target coronaries of left circumflex arteries by six novice and four expert interventionalists. Simultaneously, four types of natural behaviors (i.e., hand motion, proximal force, muscle activity, and finger motion) were acquired from the subjects' dominant hand and arm. The features extracted from the behaviors of different skill-level groups were compared using the Mann-Whitney U-test for effective behavior selection. The effective ones were further applied in the Gaussian-mixture-model-based qualitative assessment and Mahalanobis-distance-based quantitative assessment. RESULTS: The qualitative assessment achieves an accuracy of 92% to distinguish the novice and expert attempts, which is significantly higher than that of using single guidewire motions. Furthermore, the quantitative assessment can assign objective and effective scores for all attempts, indicating high correlation ( R = 0.9225) to those obtained by traditional methods. CONCLUSION: The objective, effective, and comprehensive assessment of technical skills can be provided by qualitatively and quantitatively analyzing interventionalists' natural behaviors in PCI. SIGNIFICANCE: This paper suggests a novel approach for the technical skill assessment and the promising results demonstrate the great importance and effectiveness of the proposed method for promoting the development of objective assessment techniques.

Pyramid attention recurrent networks for real-time guidewire segmentation and tracking in intraoperative X-ray fluoroscopy.

In endovascular and cardiovascular surgery, real-time and accurate segmentation and tracking of interventional instruments can aid in reducing radiation exposure, contrast agent and processing time. Nevertheless, this task often comes with the challenges of the elongated deformable structures with low contrast in noisy X-ray fluoroscopy. To address these issues, a novel efficient network architecture, termed pyramid attention recurrent networks (PAR-Net), is proposed for real-time guidewire segmentation and tracking. The proposed PAR-Net contains three major modules, namely pyramid attention module, recurrent residual module and pre-trained MobileNetV2 encoder. Specifically, a hybrid loss function of both reinforced focal loss and dice loss is proposed to better address the issues of class imbalance and misclassified examples. Quantitative and qualitative evaluations on clinical intraoperative images demonstrate that the proposed approach significantly outperforms simpler baselines as well as the best previously published result for this task, achieving the state-of-the-art performance.

[Review of digital ground object spectral library].

A higher spectral resolution is the main direction of developing remote sensing technology, and it is quite important to set up the digital ground object reflectance spectral database library, one of fundamental research fields in remote sensing application. Remote sensing application has been increasingly relying on ground object spectral characteristics, and quantitative analysis has been developed to a new stage. The present article summarized and systematically introduced the research status quo and development trend of digital ground object reflectance spectral libraries at home and in the world in recent years. Introducing the spectral libraries has been established, including desertification spectral database library, plants spectral database library, geological spectral database library, soil spectral database library, minerals spectral database library, cloud spectral database library, snow spectral database library, the atmosphere spectral database library, rocks spectral database library, water spectral database library, meteorites spectral database library, moon rock spectral database library, and man-made materials spectral database library, mixture spectral database library, volatile compounds spectral database library, and liquids spectral database library. In the process of establishing spectral database libraries, there have been some problems, such as the lack of uniform national spectral database standard and uniform standards for the ground object features as well as the comparability between different databases. In addition, data sharing mechanism can not be carried out, etc. This article also put forward some suggestions on those problems.

Analysis of Interventionalists' Natural Behaviors for Recognizing Motion Patterns of Endovascular Tools During Percutaneous Coronary Interventions.

Many robotic platforms can indeed reduce radiation exposure to clinicians during percutaneous coronary intervention (PCI), however, interventionalists' natural manipulations are rarely involved in robot-assisted PCI. This requires more attention to analyze interventionalists' natural behaviors during conventional PCI. In this study, four types of natural behavior (i.e., muscle activity, hand motion, proximal force, and finger motion) were synchronously acquired from ten subjects while performing six typical types of guidewire manipulation. These behaviors are evaluated by a hidden Markov model (HMM) based analysis framework for relevant behavior selection. Relevant behaviors are further used as the input of two HMM-based classification frameworks to recognize guidewire motion patterns. Experimental results show that under the basic classification framework (BCF), 91.01% and 93.32% recognition accuracies can be achieved by using all behaviors and relevant behaviors, respectively. Furthermore, the hierarchical classification framework can significantly enhance the recognition ability of relevant behaviors with an accuracy of 96.39%. These promising results demonstrate great potential of proposed methods for promoting the future design of human-robot interfaces in robot-assisted PCI.

RASNet: Segmentation for Tracking Surgical Instruments in Surgical Videos Using Refined Attention Segmentation Network.

Segmentation for tracking surgical instruments plays an important role in robot-assisted surgery. Segmentation of surgical instruments contributes to capturing accurate spatial information for tracking. In this paper, a novel network, Refined Attention Segmentation Network, is proposed to simultaneously segment surgical instruments and identify their categories. The U-shape network which is popular in segmentation is used. Different from previous work, an attention module is adopted to help the network focus on key regions, which can improve the segmentation accuracy. To solve the class imbalance problem, the weighted sum of the cross entropy loss and the logarithm of the Jaccard index is used as loss function. Furthermore, transfer learning is adopted in our network. The encoder is pre-trained on ImageNet. The dataset from the MICCAI EndoVis Challenge 2017 is used to evaluate our network. Based on this dataset, our network achieves state-of-the-art performance 94.65% mean Dice and 90.33% mean IOU.

Recognition of Endovascular Manipulations using Recurrent Neural Networks.

The ability to accurately recognize elementary surgical gestures is a stepping stone to automated surgical assessment and surgical training. In this paper, a long short-term memory (LSTM) recurrent neural network is applied to the task of recognizing six typical manipulations in percutaneous coronary intervention (PCI). The manipulation mentioned above is referring to the atomic surgical operation, also called surgeme in many research. Instead of using the video data or kinematic data of surgical instruments, we propose to use the kinematic data of the operator's hand acquired by our wearable data glove to recognize the manipulations. To establish a baseline for comparison, a method based on Hidden Markov Model (HMM) is applied because HMM is frequently used in the tasks of surgical sequence learning. Two cross-validation schemes are used in our experiments, they both illustrate that our LSTM-based method far outperforms the HMM-based method. To our knowledge, this is the first paper to apply the LSTM recurrent neural network in the field of PCI.

Romidepsin induces G2/M phase arrest via Erk/cdc25C/cdc2/cyclinB pathway and apoptosis induction through JNK/c-Jun/caspase3 pathway in hepatocellular carcinoma cells.

The aim of the study is to demonstrate the effect of Romidepsin in hepatocellular carcinoma (HCC) by inducing G2/M phase arrest via Erk/cdc25C/cdc2/cyclinB pathway and apoptosis through JNK/c-Jun/caspase3 pathway in vitro and in vivo. Human HCC cell lines were cultured with Romidepsin and DMSO (negative control) and 5-fluorouracil (positive control). Then the cells' viability and apoptosis were determined by cell proliferation assay and flow cytometry. Protein concentrations and expression changes were measured by Western blot. Subsequently, Huh7 cells were subcutaneously inoculated into the nude mice, which were employed to further probe the tumor-suppressive effect of Romidepsin in vivo. Romidepsin treatment led to a time- and dose-dependent induction of cell cycle arrest in the G2/M phase and apoptosis. G2/M phase arrest inhibited the proliferation of HCC cells by alterations in p21/cdc25C/cdc2/cyclinB proteins. Increased concentrations of Erk and JNK phosphorylations were observed in a dose-dependent manner in the Romidepsin group, but p38 phosphorylation was not affected. G2/M phase arrest and the apoptosis of HCC cells induced by Romidepsin were mediated by the activation of Erk/MAPK pathways and JNK/MAPK pathways. The tumor size was significantly larger in the negative control group compared to Romidepsin group and no significant loss in body weight was observed in the Romidepsin group. Our findings offer proof-of-concept for use of Romidepsin as a novel class of chemotherapy in the treatment of HCC.