Complementary multi-modality molecular self-supervised learning via non-overlapping masking for property prediction.

Self-supervised learning plays an important role in molecular representation learning because labeled molecular data are usually limited in many tasks, such as chemical property prediction and virtual screening. However, most existing molecular pre-training methods focus on one modality of molecular data, and the complementary information of two important modalities, SMILES and graph, is not fully explored. In this study, we propose an effective multi-modality self-supervised learning framework for molecular SMILES and graph. Specifically, SMILES data and graph data are first tokenized so that they can be processed by a unified Transformer-based backbone network, which is trained by a masked reconstruction strategy. In addition, we introduce a specialized non-overlapping masking strategy to encourage fine-grained interaction between these two modalities. Experimental results show that our framework achieves state-of-the-art performance in a series of molecular property prediction tasks, and a detailed ablation study demonstrates efficacy of the multi-modality framework and the masking strategy.

ProteinMAE: masked autoencoder for protein surface self-supervised learning.

SUMMARY: The biological functions of proteins are determined by the chemical and geometric properties of their surfaces. Recently, with the booming progress of deep learning, a series of learning-based surface descriptors have been proposed and achieved inspirational performance in many tasks such as protein design, protein-protein interaction prediction, etc. However, they are still limited by the problem of label scarcity, since the labels are typically obtained through wet experiments. Inspired by the great success of self-supervised learning in natural language processing and computer vision, we introduce ProteinMAE, a self-supervised framework specifically designed for protein surface representation to mitigate label scarcity. Specifically, we propose an efficient network and utilize a large number of accessible unlabeled protein data to pretrain it by self-supervised learning. Then we use the pretrained weights as initialization and fine-tune the network on downstream tasks. To demonstrate the effectiveness of our method, we conduct experiments on three different downstream tasks including binding site identification in protein surface, ligand-binding protein pocket classification, and protein-protein interaction prediction. The extensive experiments show that our method not only successfully improves the network's performance on all downstream tasks, but also achieves competitive performance with state-of-the-art methods. Moreover, our proposed network also exhibits significant advantages in terms of computational cost, which only requires less than a tenth of memory cost of previous methods. AVAILABILITY AND IMPLEMENTATION: https://github.com/phdymz/ProteinMAE.

A Transformer-Based microvascular invasion classifier enhances prognostic stratification in HCC following radiofrequency ablation.

BACKGROUND & AIMS: We aimed to develop a Transformer-based deep learning (DL) network for prognostic stratification in hepatocellular carcinoma (HCC) patients undergoing RFA. METHODS: A Swin Transformer DL network was trained to establish associations between magnetic resonance imaging (MRI) datasets and the ground truth of microvascular invasion (MVI) based on 696 surgical resection (SR) patients with solitary HCC ≤3 cm, and was validated in an external cohort (n = 180). The multiphase MRI-based DL risk outputs using an optimal threshold of .5 was employed as a MVI classifier for prognosis stratification in the RFA cohort (n = 180). RESULTS: Over 90% of all enrolled patients exhibited hepatitis B virus infection. Liver cirrhosis was significantly more prevalent in the RFA cohort compared to the SR cohort (72.2% vs. 44.1%, p < .001). The MVI risk outputs exhibited good performance (area under the curve values = .938 and .883) for predicting MVI in the training and validation cohort, respectively. The RFA patients at high risk of MVI classified by the MVI classifier demonstrated significantly lower recurrence-free survival (RFS) and overall survival rates at 1, 3 and 5 years compared to those classified as low risk (p < .001). Multivariate cox regression modelling of a-fetoprotein > 20 ng/mL [hazard ratio (HR) = 1.53; 95% confidence interval (95% CI): 1.02-2.33, p = .047], high risk of MVI (HR = 3.76; 95% CI: 2.40-5.88, p < .001) and unfavourable tumour location (HR = 2.15; 95% CI: 1.40-3.29, p = .001) yielded a c-index of .731 (bootstrapped 95% CI: .667-.778) for evaluating RFS after RFA. Among the three risk factors, MVI was the most powerful predictor for intrahepatic distance recurrence. CONCLUSIONS: The proposed MVI classifier can serve as a valuable imaging biomarker for prognostic stratification in early-stage HCC patients undergoing RFA.

Deep Learning Prediction for Distal Aortic Remodeling After Thoracic Endovascular Aortic Repair in Stanford Type B Aortic Dissection.

PURPOSE: This study aimed to develop a deep learning model for predicting distal aortic remodeling after proximal thoracic endovascular aortic repair (TEVAR) in patients with Stanford type B aortic dissection (TBAD) using computed tomography angiography (CTA). METHODS: A total of 147 patients with acute or subacute TBAD who underwent proximal TEVAR at a single center were retrospectively reviewed. The boundary of aorta was manually segmented, and the point clouds of each aorta were obtained. Prediction of negative aortic remodeling or reintervention was accomplished by a convolutional neural network (CNN) and a point cloud neural network (PC-NN), respectively. The discriminatory value of the established models was mainly evaluated by the area under the receiver operating characteristic curve (AUC) in the test set. RESULTS: The mean follow-up time was 34.0 months (range: 12-108 months). During follow-up, a total of 25 (17.0%) patients were identified as having negative aortic remodeling, and 16 (10.9%) patients received reintervention. The AUC (0.876) by PC-NN for predicting negative aortic remodeling was superior to that obtained by CNN (0.612, p=0.034) and similar to the AUC by PC-NN combined with clinical features (0.884, p=0.92). As to reintervention, the AUC by PC-NN was significantly higher than that by CNN (0.805 vs 0.579; p=0.042), and AUCs by PC-NN combined with clinical features and PC-NN alone were comparable (0.836 vs 0.805; p=0.81). CONCLUSION: The CTA-based deep learning algorithms may assist clinicians in automated prediction of distal aortic remodeling after TEVAR for acute or subacute TBAD. CLINICAL IMPACT: Negative aortic remodeling is the leading cause of late reintervention after proximal thoracic endovascular aortic repair (TEVAR) for Stanford type B aortic dissection (TBAD), and possesses great challenge to endovascular repair. Early recognizing high-risk patients is of supreme importance for optimizing the follow-up interval and therapy strategy. Currently, clinicians predict the prognosis of these patients based on several imaging signs, which is subjective. The computed tomography angiography-based deep learning algorithms may incorporate abundant morphological information of aorta, provide with a definite and objective output value, and finally assist clinicians in automated prediction of distal aortic remodeling after TEVAR for acute or subacute TBAD.

Multi-task learning-based histologic subtype classification of non-small cell lung cancer.

PURPOSE: In clinical applications, accurate histologic subtype classification of lung cancer is important for determining appropriate treatment plans. The purpose of this paper is to evaluate the role of multi-task learning in the classification of adenocarcinoma and squamous cell carcinoma. MATERIAL AND METHODS: In this paper, we propose a novel multi-task learning model for histologic subtype classification of non-small cell lung cancer based on computed tomography (CT) images. The model consists of a histologic subtype classification branch and a staging branch, which share a part of the feature extraction layers and are simultaneously trained. By optimizing on the two tasks simultaneously, our model could achieve high accuracy in histologic subtype classification of non-small cell lung cancer without relying on physician's precise labeling of tumor areas. In this study, 402 cases from The Cancer Imaging Archive (TCIA) were used in total, and they were split into training set (n = 258), internal test set (n = 66) and external test set (n = 78). RESULTS: Compared with the radiomics method and single-task networks, our multi-task model could reach an AUC of 0.843 and 0.732 on internal and external test set, respectively. In addition, multi-task network can achieve higher accuracy and specificity than single-task network. CONCLUSION: Compared with the radiomics methods and single-task networks, our multi-task learning model could improve the accuracy of histologic subtype classification of non-small cell lung cancer by sharing network layers, which no longer relies on the physician's precise labeling of lesion regions and could further reduce the manual workload of physicians.

Label correlation transformer for automated chest X-ray diagnosis with reliable interpretability.

Computer-aided diagnosis of chest X-ray (CXR) images can help reduce the huge workload of radiologists and avoid the inter-observer variability in large-scale early disease screening. Recently, most state-of-the-art studies employ deep learning techniques to address this problem through multi-label classification. However, existing methods still suffer from low classification accuracy and poor interpretability for each diagnostic task. This study aims to propose a novel transformer-based deep learning model for automated CXR diagnosis with high performance and reliable interpretability. We introduce a novel transformer architecture into this problem and utilize the unique query structure of transformer to capture the global and local information of the images and the correlation between labels. In addition, we propose a new loss function to help the model find correlations between the labels in CXR images. To achieve accurate and reliable interpretability, we generate heatmaps using the proposed transformer model and compare with the true pathogenic regions labeled by the physicians. The proposed model achieves a mean AUC of 0.831 on chest X-ray 14 and 0.875 on PadChest dataset, which outperforms existing state-of-the-art methods. The attention heatmaps show that our model could focus on the exact corresponding areas of related truly labeled pathogenic regions. The proposed model effectively improves the performance of CXR multi-label classification and the interpretability of label correlations, thus providing new evidence and methods for automated clinical diagnosis.

A Multiparametric Fusion Deep Learning Model Based on DCE-MRI for Preoperative Prediction of Microvascular Invasion in Intrahepatic Cholangiocarcinoma.

BACKGROUND: Assessment of microvascular invasion (MVI) in intrahepatic cholangiocarcinoma (ICC) by using a noninvasive method is an unresolved issue. Deep learning (DL) methods based on multiparametric fusion of MR images have the potential of preoperative assessment of MVI. PURPOSE: To investigate whether a multiparametric fusion DL model based on MR images can be used for preoperative assessment of MVI in ICC. STUDY TYPE: Retrospective. POPULATION: A total of 519 patients (200 females and 319 males) with a single ICC were categorized as a training (n = 361), validation (n = 90), and an external test cohort (n = 68). FIELD STRENGTH/SEQUENCE: A 1.5 T and 3.0 T; axial T2-weighted turbo spin-echo sequence, diffusion-weighted imaging with a single-shot spin-echo planar sequence, and dynamic contrast-enhanced (DCE) imaging with T1-weighted three-dimensional quick spoiled gradient echo sequence. ASSESSMENT: DL models of multiparametric fusion convolutional neural network (CNN) and late fusion CNN were both constructed for evaluating MVI in ICC. Gradient-weighted class activation mapping was used for visual interpretation of MVI status in ICC. STATISTICAL TESTS: The DL model performance was assessed through the receiver operating characteristic curve (ROC) analysis, and the area under the ROC curve (AUC) with the accuracy, sensitivity, and specificity were measured. P value < 0.05 was considered as statistical significance. RESULTS: In the external test cohort, the proposed multiparametric fusion DL model achieved an AUC of 0.888 with an accuracy of 86.8%, sensitivity of 85.7%, and specificity of 87.0% for evaluating MVI in ICC, and the positive predictive value and negative predictive value were 63.2% and 95.9%, respectively. The late fusion DL model achieved a lower AUC of 0.866, with an accuracy of 83.8%, sensitivity of 78.6%, specificity of 85.2% for evaluating MVI in ICC. DATA CONCLUSION: Our DL model based on multiparametric fusion of MRI achieved a good diagnostic performance in the evaluation of MVI in ICC. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY: Stage 2.

A predictive model integrating deep and radiomics features based on gadobenate dimeglumine-enhanced MRI for postoperative early recurrence of hepatocellular carcinoma.

PURPOSE: Hepatocellular carcinoma (HCC) is the most common liver cancer worldwide, and early recurrence of HCC after curative hepatic resection is indicative of poor prognoses. We aim to develop a predictive model for postoperative early recurrence of HCC based on deep and radiomics features from multi-phasic magnetic resonance imaging (MRI). MATERIALS AND METHODS: A total of 472 HCC patients were included and divided into the training (n = 378) and validation (n = 94) cohorts in the retrospective study. We separately extracted radiomics features and deep features from eight phases of gadoxetic acid-enhanced MRI and utilized the least absolute shrinkage and selection operator logistic regression algorithm for feature selection and model construction. We integrated the selected two types of features into a combined model and established a radiomics model as well as a deep learning (DL) model for comparison. RESULTS: In the training and validation cohorts, the combined model demonstrated better performance for stratifying patients at high risk of early recurrence (AUC of 0.911 and 0.840, accuracy of 0.779 and 0.777, sensitivity of 0.927 and 0.769, specificity 0.720 and 0.779) than the radiomics model (AUC of 0.740 and 0.780) and the DL model (AUC of 0.887 and 0.813). CONCLUSION: The combined model integrating deep and radiomics features from multi-phasic MRI is efficient for noninvasively stratifying patients at high risk of early HCC recurrence after resection.

Efficient Similarity Point Set Registration by Transformation Decomposition.

Point set registration is one of the basic problems in computer vision. When the overlap ratio between point sets is small or the relative transformation is large, local methods cannot guarantee the accuracy. However, the time complexity of the branch and bound (BnB) optimization used in most existing global methods is exponential in the dimensionality of parameter space. Therefore, seven-Degrees of Freedom (7-DoF) similarity transformation is a big challenge for BnB. In this paper, a novel rotation and scale invariant feature is introduced to decouple the optimization of translation, rotation, and scale in similarity point set registration, so that BnB optimization can be done in two lower dimensional spaces. With the transformation decomposition, the translation is first estimated and then the rotation is optimized by maximizing a robust objective function defined on consensus set. Finally, the scale is estimated according to the potential correspondences in the obtained consensus set. Experiments on synthetic data and clinical data show that our method is approximately two orders of magnitude faster than the state-of-the-art global method and more accurate than a typical local method. When the outlier ratio with respect to the inliers is up to 1.0, our method still achieves accurate registration.

A Surface-Based Spatial Registration Method Based on Sense Three-Dimensional Scanner.

OBJECTIVE: The purpose of this study was to investigate the feasibility of a surface-based registration method based on a low-cost, hand-held Sense three-dimensional (3D) scanner in image-guided neurosurgery system. METHODS: The scanner was calibrated prior and fixed on a tripod before registration. During registration, a part of the head surface was scanned at first and the spatial position of the adapter was recorded. Then the scanner was taken off from the tripod and the entire head surface was scanned by moving the scanner around the patient's head. All the scan points were aligned to the recorded spatial position to form a unique point cloud of the head by the automatic mosaic function of the scanner. The coordinates of the scan points were transformed from the device space to the adapter space by a calibration matrix, and then to the patient space. A 2-step patient-to-image registration method was then performed to register the patient space to the image space. RESULTS: The experimental results showed that the mean target registration error of 15 targets on the surface of the phantom was 1.61±0.09 mm. In a clinical experiment, the mean target registration error of 7 targets on the patient's head surface was 2.50±0.31 mm, which was sufficient to meet clinical requirements. CONCLUSIONS: It is feasible to use the Sense 3D scanner for patient-to-image registration, and the low-cost Sense 3D scanner can take the place of the current used scanner in the image-guided neurosurgery system.

Easy-to-use augmented reality neuronavigation using a wireless tablet PC.

BACKGROUND/AIMS: Augmented reality (AR) technology solves the problem of view switching in traditional image-guided neurosurgery systems by integrating computer-generated objects into the actual scene. However, the state-of-the-art AR solution using head-mounted displays has not been widely accepted in clinical applications because it causes some inconvenience for the surgeon during surgery. METHODS: In this paper, we present a Tablet-AR system that transmits navigation information to a movable tablet PC via a wireless local area network and overlays this information on the tablet screen, which simultaneously displays the actual scene captured by its back-facing camera. With this system, the surgeon can directly observe the intracranial anatomical structure of the patient with the overlaid virtual projection images to guide the surgery. RESULTS: The alignment errors in the skull specimen study and clinical experiment were 4.6 pixels (approx. 1.6 mm) and 6 pixels (approx. 2.1 mm), respectively. The system was also used for navigation in 2 actual clinical cases of neurosurgery, which demonstrated its feasibility in a clinical application. CONCLUSIONS: The easy-to-use Tablet-AR system presented in this study is accurate and feasible in clinical applications and has the potential to become a routine device in AR neuronavigation.

Rigid point cloud registration based on correspondence cloud for image-to-patient registration in image-guided surgery.

BACKGROUND: Image-to-patient registration aligns preoperative images to intra-operative anatomical structures and it is a critical step in image-guided surgery (IGS). The accuracy and speed of this step significantly influence the performance of IGS systems. Rigid registration based on paired points has been widely used in IGS, but studies have shown its limitations in terms of cost, accuracy, and registration time. Therefore, rigid registration of point clouds representing the human anatomical surfaces has become an alternative way for image-to-patient registration in the IGS systems. PURPOSE: We propose a novel correspondence-based rigid point cloud registration method that can achieve global registration without the need for pose initialization. The proposed method is less sensitive to outliers compared to the widely used RANSAC-based registration methods and it achieves high accuracy at a high speed, which is particularly suitable for the image-to-patient registration in IGS. METHODS: We use the rotation axis and angle to represent the rigid spatial transformation between two coordinate systems. Given a set of correspondences between two point clouds in two coordinate systems, we first construct a 3D correspondence cloud (CC) from the inlier correspondences and prove that the CC distributes on a plane, whose normal is the rotation axis between the two point clouds. Thus, the rotation axis can be estimated by fitting the CP. Then, we further show that when projecting the normals of a pair of corresponding points onto the CP, the angle between the projected normal pairs is equal to the rotation angle. Therefore, the rotation angle can be estimated from the angle histogram. Besides, this two-stage estimation also produces a high-quality correspondence subset with high inlier rate. With the estimated rotation axis, rotation angle, and the correspondence subset, the spatial transformation can be computed directly, or be estimated using RANSAC in a fast and robust way within only 100 iterations. RESULTS: To validate the performance of the proposed registration method, we conducted experiments on the CT-Skull dataset. We first conducted a simulation experiment by controlling the initial inlier rate of the correspondence set, and the results showed that the proposed method can effectively obtain a correspondence subset with much higher inlier rate. We then compared our method with traditional approaches such as ICP, Go-ICP, and RANSAC, as well as recently proposed methods like TEASER, SC2-PCR, and MAC. Our method outperformed all traditional methods in terms of registration accuracy and speed. While achieving a registration accuracy comparable to the recently proposed methods, our method demonstrated superior speed, being almost three times faster than TEASER. CONCLUSIONS: Experiments on the CT-Skull dataset demonstrate that the proposed method can effectively obtain a high-quality correspondence subset with high inlier rate, and a tiny RANSAC with 100 iterations is sufficient to estimate the optimal transformation for point cloud registration. Our method achieves higher registration accuracy and faster speed than existing widely used methods, demonstrating great potential for the image-to-patient registration, where a rigid spatial transformation is needed to align preoperative images to intra-operative patient anatomy.

Negative Instance Guided Self-Distillation Framework for Whole Slide Image Analysis.

Histopathology image classification is an important clinical task, and current deep learning-based whole-slide image (WSI) classification methods typically cut WSIs into small patches and cast the problem as multi-instance learning. The mainstream approach is to train a bag-level classifier, but their performance on both slide classification and positive patch localization is limited because the instance-level information is not fully explored. In this article, we propose a negative instance-guided, self-distillation framework to directly train an instance-level classifier end-to-end. Instead of depending only on the self-supervised training of the teacher and the student classifiers in a typical self-distillation framework, we input the true negative instances into the student classifier to guide the classifier to better distinguish positive and negative instances. In addition, we propose a prediction bank to constrain the distribution of pseudo instance labels generated by the teacher classifier to prevent the self-distillation from falling into the degeneration of classifying all instances as negative. We conduct extensive experiments and analysis on three publicly available pathological datasets: CAMELYON16, PANDA, and TCGA, as well as an in-house pathological dataset for cervical cancer lymph node metastasis prediction. The results show that our method outperforms existing methods by a large margin. Code will be publicly available.

A comprehensive survey on deep active learning in medical image analysis.

Deep learning has achieved widespread success in medical image analysis, leading to an increasing demand for large-scale expert-annotated medical image datasets. Yet, the high cost of annotating medical images severely hampers the development of deep learning in this field. To reduce annotation costs, active learning aims to select the most informative samples for annotation and train high-performance models with as few labeled samples as possible. In this survey, we review the core methods of active learning, including the evaluation of informativeness and sampling strategy. For the first time, we provide a detailed summary of the integration of active learning with other label-efficient techniques, such as semi-supervised, self-supervised learning, and so on. We also summarize active learning works that are specifically tailored to medical image analysis. Additionally, we conduct a thorough comparative analysis of the performance of different AL methods in medical image analysis with experiments. In the end, we offer our perspectives on the future trends and challenges of active learning and its applications in medical image analysis. An accompanying paper list and code for the comparative analysis is available in https://github.com/LightersWang/Awesome-Active-Learning-for-Medical-Image-Analysis.

SlZIP11 mediates zinc accumulation and sugar storage in tomato fruits.

Background: Zinc (Zn) is a vital micronutrient essential for plant growth and development. Transporter proteins of the ZRT/IRT-like protein (ZIP) family play crucial roles in maintaining Zn homeostasis. Although the acquisition, translocation, and intracellular transport of Zn are well understood in plant roots and leaves, the genes that regulate these pathways in fruits remain largely unexplored. In this study, we aimed to investigate the function of SlZIP11 in regulating tomato fruit development. Methods: We used Solanum lycopersicum L. 'Micro-Tom' SlZIP11 (Solanum lycopersicum) is highly expressed in tomato fruit, particularly in mature green (MG) stages. For obtaining results, we employed reverse transcription-quantitative polymerase chain reaction (RT-qPCR), yeast two-hybrid assay, bimolecular fluorescent complementation, subcellular localization assay, virus-induced gene silencing (VIGS), SlZIP11 overexpression, determination of Zn content, sugar extraction and content determination, and statistical analysis. Results: RT-qPCR analysis showed elevated SlZIP11 expression in MG tomato fruits. SlZIP11 expression was inhibited and induced by Zn deficiency and toxicity treatments, respectively. Silencing SlZIP11 via the VIGS technology resulted in a significant increase in the Zn content of tomato fruits. In contrast, overexpression of SlZIP11 led to reduced Zn content in MG fruits. Moreover, both silencing and overexpression of SlZIP11 caused alterations in the fructose and glucose contents of tomato fruits. Additionally, SlSWEEET7a interacted with SlZIP11. The heterodimerization between SlSWEET7a and SlZIP11 affected subcellular targeting, thereby increasing the amount of intracellularly localized oligomeric complexes. Overall, this study elucidates the role of SlZIP11 in mediating Zn accumulation and sugar transport during tomato fruit ripening. These findings underscore the significance of SlZIP11 in regulating Zn levels and sugar content, providing insights into its potential implications for plant physiology and agricultural practices.

Donor-derived Anti-CD19 CAR T cells GC007g for relapsed or refractory B-cell acute lymphoblastic leukemia after allogeneic HSCT: a phase 1 trial.

Background: Although chimeric antigen receptor-modified T cells (CAR T) cell therapy has been widely reported in improving the outcomes of B-cell acute lymphoblastic leukemia (B-ALL), less research about the feasibility and safety of donor-derived CAR T after allogeneic hematopoietic stem cell transplantation (allo-HSCT) was reported. Methods: This phase 1 clinical trial aims to evaluate safety and efficacy of donor-derived anti-CD19 CAR T cells (GC007g) in B-ALL patients who relapsed after allo-HSCT. This trial is registered with ClinicalTrials.gov, NCT04516551. Findings: Between 15 March 2021 and 19 May 2022, fifteen patients were screened, three patients were excluded due to withdraw of consent, donor's reason, and death, respectively. Patients received donor-derived CAR T cells infusions at 6 × 105/kg (n = 3) or 2 × 106/kg (n = 6) dose level. The median time from HSCT to relapse was 185 days (range, 81-2063). The median age of patients was 31 years (range 21-48). Seven patients (77.8%) had BCR-ABL fusion gene. CAR T cells expanded in vivo and the median time to reach Cmax was 9 days (range, 7-11). One patient had hyperbilirubinemia after GC007g infusion which was defined as a dose-limiting toxicity. All patients experienced CRS and hematological adverse events. Three patients had acute graft-versus-host-disease (grade I, n = 1; grade II, n = 1; grade IV, n = 1) and all resolved after treatment. They received CAR T cells from matched sister, haploidentical matched father and sisiter, respectively. At 28 days after infusion, all patients achieved complete remission with/without incomplete hematologic recovery (CRi/CR) with undetectable MRD. At a median follow-up of 475 days (range 322-732), seven patients remained in CR/CRi while two had CD19-negative relapse. The overall response rates (ORR) were 100% (9/9), 88.9% (8/9), and 75% (6/8) at 3 month, 6 month, and 12 month, respectively. The 1-year progression-free and overall survival were 77.8% and 85.7%, respectively. Interpretation: GC007g expanded and induced durable remission in patients with B-ALL relapsed after allo-HSCT, with manageable safety profiles. Funding: Gracell Biotechnologies Inc.

Robust Point Cloud Registration Framework Based on Deep Graph Matching.

3D point cloud registration is a fundamental problem in computer vision and robotics. Recently, learning-based point cloud registration methods have made great progress. However, these methods are sensitive to outliers, which lead to more incorrect correspondences. In this paper, we propose a novel deep graph matching-based framework for point cloud registration. Specifically, we first transform point clouds into graphs and extract deep features for each point. Then, we develop a module based on deep graph matching to calculate a soft correspondence matrix. By using graph matching, not only the local geometry of each point but also its structure and topology in a larger range are considered in establishing correspondences, so that more correct correspondences are found. We train the network with a loss directly defined on the correspondences, and in the test stage the soft correspondences are transformed into hard one-to-one correspondences so that registration can be performed by a correspondence-based solver. Furthermore, we introduce a transformer-based method to generate edges for graph construction, which further improves the quality of the correspondences. Extensive experiments on object-level and scene-level benchmark datasets show that the proposed method achieves state-of-the-art performance.

A Structure-Aware Framework of Unsupervised Cross-Modality Domain Adaptation via Frequency and Spatial Knowledge Distillation.

Unsupervised domain adaptation (UDA) aims to train a model on a labeled source domain and adapt it to an unlabeled target domain. In medical image segmentation field, most existing UDA methods rely on adversarial learning to address the domain gap between different image modalities. However, this process is complicated and inefficient. In this paper, we propose a simple yet effective UDA method based on both frequency and spatial domain transfer under a multi-teacher distillation framework. In the frequency domain, we introduce non-subsampled contourlet transform for identifying domain-invariant and domain-variant frequency components (DIFs and DVFs) and replace the DVFs of the source domain images with those of the target domain images while keeping the DIFs unchanged to narrow the domain gap. In the spatial domain, we propose a batch momentum update-based histogram matching strategy to minimize the domain-variant image style bias. Additionally, we further propose a dual contrastive learning module at both image and pixel levels to learn structure-related information. Our proposed method outperforms state-of-the-art methods on two cross-modality medical image segmentation datasets (cardiac and abdominal). Codes are avaliable at https://github.com/slliuEric/FSUDA.

Improving brain tumor segmentation with anatomical prior-informed pre-training.

Introduction: Precise delineation of glioblastoma in multi-parameter magnetic resonance images is pivotal for neurosurgery and subsequent treatment monitoring. Transformer models have shown promise in brain tumor segmentation, but their efficacy heavily depends on a substantial amount of annotated data. To address the scarcity of annotated data and improve model robustness, self-supervised learning methods using masked autoencoders have been devised. Nevertheless, these methods have not incorporated the anatomical priors of brain structures. Methods: This study proposed an anatomical prior-informed masking strategy to enhance the pre-training of masked autoencoders, which combines data-driven reconstruction with anatomical knowledge. We investigate the likelihood of tumor presence in various brain structures, and this information is then utilized to guide the masking procedure. Results: Compared with random masking, our method enables the pre-training to concentrate on regions that are more pertinent to downstream segmentation. Experiments conducted on the BraTS21 dataset demonstrate that our proposed method surpasses the performance of state-of-the-art self-supervised learning techniques. It enhances brain tumor segmentation in terms of both accuracy and data efficiency. Discussion: Tailored mechanisms designed to extract valuable information from extensive data could enhance computational efficiency and performance, resulting in increased precision. It's still promising to integrate anatomical priors and vision approaches.

Prediction of cerebral aneurysm rupture using a point cloud neural network.

OBJECTIVE: Accurate prediction of cerebral aneurysm (CA) rupture is of great significance. We intended to evaluate the accuracy of the point cloud neural network (PC-NN) in predicting CA rupture using MR angiography (MRA) and CT angiography (CTA) data. METHODS: 418 CAs in 411 consecutive patients confirmed by CTA (n=180) or MRA (n=238) in a single hospital were retrospectively analyzed. A PC-NN aneurysm model with/without parent artery involvement was used for CA rupture prediction and compared with ridge regression, support vector machine (SVM) and neural network (NN) models based on radiomics features. Furthermore, the performance of the trained PC-NN and radiomics-based models was prospectively evaluated in 258 CAs of 254 patients from five external centers. RESULTS: In the internal test data, the area under the curve (AUC) of the PC-NN model trained with parent artery (AUC=0.913) was significantly higher than that of the PC-NN model trained without parent artery (AUC=0.851; p=0.041) and of the ridge regression (AUC=0.803; p=0.019), SVM (AUC=0.788; p=0.013) and NN (AUC=0.805; p=0.023) radiomics-based models. Additionally, the PC-NN model trained with MRA source data achieved a higher prediction accuracy (AUC=0.936) than that trained with CTA source data (AUC=0.824; p=0.043). In external data of prospective cohort patients, the AUC of PC-NN was 0.835, significantly higher than ridge regression (0.692; p<0.001), SVM (0.701; p<0.001) and NN (0.681; p<0.001) models. CONCLUSION: PC-NNs can achieve more accurate CA rupture prediction than traditional radiomics-based models. Furthermore, the performance of the PC-NN model trained with MRA data was superior to that trained with CTA data.