Constructing fine-grained spatiotemporal neonatal functional atlases with spectral functional network learning.

The early stages of human development are increasingly acknowledged as pivotal in laying the groundwork for subsequent behavioral and cognitive development. Spatiotemporal (4D) brain functional atlases are important in elucidating the development of human brain functions. However, the scarcity of such atlases for early life stages stems from two primary challenges: (1) the significant noise in functional magnetic resonance imaging (fMRI) that complicates the generation of high-quality atlases for each age group, and (2) the rapid and complex changes in the early human brain that hinder the maintenance of temporal consistency in 4D atlases. This study tackles these challenges by integrating low-rank tensor learning with spectral embedding, thereby proposing a novel, data-driven 4D functional atlas generation framework based on spectral functional network learning (SFNL). This method utilizes low-rank tensor learning to capture common functional connectivity (FC) patterns across different ages, thus optimizing FCs for each age group to improve the temporal consistency of functional networks. Incorporating spectral embedding aids in mitigating potential noise in FC networks derived from fMRI data by reconstructing networks in the spectral space. Utilizing SFNL-generated functional networks enables the creation of consistent and highly qualified spatiotemporal functional atlases. The framework was applied to the developing Human Connectome Project (dHCP) dataset, generating the first neonatal 4D functional atlases with fine-grained temporal and spatial resolutions. Experimental evaluations focusing on functional homogeneity, reliability, and temporal consistency demonstrated the superiority of our framework compared to existing methods for constructing 4D atlases. Additionally, network analysis experiments, including individual identification, functional systems development, and local efficiency assessments, further corroborate the efficacy and robustness of the generated atlases. The 4D atlases and related codes will be made publicly accessible (https://github.com/zhaoyunxi/neonate-atlases).

D-MHGCN: An End-to-End Individual Behavioral Prediction Model Using Dual Multi-Hop Graph Convolutional Network.

Predicting individual behavior is a crucial area of research in neuroscience. Graph Neural Networks (GNNs), as powerful tools for extracting graph-structured features, are increasingly being utilized in various functional connectivity (FC) based behavioral prediction tasks. However, current predictive models primarily focus on enhancing GNNs' ability to extract features from FC networks while neglecting the importance of upstream individual network construction quality. This oversight results in constructed functional networks that fail to adequately represent individual behavioral capacity, thereby affecting the subsequent prediction accuracy. To address this issue, we proposed a new GNN-based behavioral prediction framework, named Dual Multi-Hop Graph Convolutional Network (D-MHGCN). Through the joint training of two GCNs, this framework integrates individual functional network construction and behavioral prediction into a unified optimization model. It allows the model to dynamically adjust the individual functional cortical parcellation according to the downstream tasks, thus creating task-aware, individual-specific FCNs that largely enhance its ability to predict behavior scores. Additionally, we employed multi-hop graph convolution layers instead of traditional single-hop methods in GCN to capture complex hierarchical connectivity patterns in brain networks. Our experimental evaluations, conducted on the large, public Human Connectome Project dataset, demonstrate that our proposed method outperforms existing methods in various behavioral prediction tasks. Moreover, it produces more functionally homogeneous cortical parcellation, showcasing its practical utility and effectiveness. Our work not only enhances the accuracy of individual behavioral prediction but also provides deeper insights into the neural mechanisms underlying individual differences in behavior.

A Dual-Adversarial Model for Cross-Time and Cross-Subject Cognitive Workload Decoding.

Electroencephalogram (EEG) signals are widely utilized in the field of cognitive workload decoding (CWD). However, when the recognition scenario is shifted from subject-dependent to subject-independent or spans a long period, the accuracy of CWD deteriorates significantly. Current solutions are either dependent on extensive training datasets or fail to maintain clear distinctions between categories, additionally lacking a robust feature extraction mechanism. In this paper, we tackle these issues by proposing a Bi-Classifier Joint Domain Adaptation (BCJDA) model for EEG-based cross-time and cross-subject CWD. Specifically, the model consists of a feature extractor, a domain discriminator, and a Bi-Classifier, containing two sets of adversarial processes for domain-wise alignment and class-wise alignment. In the adversarial domain adaptation, the feature extractor is forced to learn the common domain features deliberately. The Bi-Classifier also fosters the feature extractor to retain the category discrepancies of the unlabeled domain, so that its classification boundary is consistent with the labeled domain. Furthermore, different adversarial distance functions of the Bi-Classifier are adopted and evaluated in this model. We conduct classification experiments on a publicly available BCI competition dataset for recognizing low, medium, and high cognitive workload levels. The experimental results demonstrate that our proposed BCJDA model based on cross-gradient difference maximization achieves the best performance.

Cross-cohort replicable resting-state functional connectivity in predicting symptoms and cognition of schizophrenia.

Schizophrenia (SZ) is a debilitating mental illness characterized by adolescence or early adulthood onset of psychosis, positive and negative symptoms, as well as cognitive impairments. Despite a plethora of studies leveraging functional connectivity (FC) from functional magnetic resonance imaging (fMRI) to predict symptoms and cognitive impairments of SZ, the findings have exhibited great heterogeneity. We aimed to identify congruous and replicable connectivity patterns capable of predicting positive and negative symptoms as well as cognitive impairments in SZ. Predictable functional connections (FCs) were identified by employing an individualized prediction model, whose replicability was further evaluated across three independent cohorts (BSNIP, SZ = 174; COBRE, SZ = 100; FBIRN, SZ = 161). Across cohorts, we observed that altered FCs in frontal-temporal-cingulate-thalamic network were replicable in prediction of positive symptoms, while sensorimotor network was predictive of negative symptoms. Temporal-parahippocampal network was consistently identified to be associated with reduced cognitive function. These replicable 23 FCs effectively distinguished SZ from healthy controls (HC) across three cohorts (82.7%, 90.2%, and 86.1%). Furthermore, models built using these replicable FCs showed comparable accuracies to those built using the whole-brain features in predicting symptoms/cognition of SZ across the three cohorts (r = .17-.33, p < .05). Overall, our findings provide new insights into the neural underpinnings of SZ symptoms/cognition and offer potential targets for further research and possible clinical interventions.

Do as Sonographers Think: Contrast-enhanced Ultrasound for Thyroid Nodules Diagnosis via Microvascular Infiltrative Awareness.

Dynamic contrast-enhanced ultrasound (CEUS) imaging can reflect the microvascular distribution and blood flow perfusion, thereby holding clinical significance in distinguishing between malignant and benign thyroid nodules. Notably, CEUS offers a meticulous visualization of the microvascular distribution surrounding the nodule, leading to an apparent increase in tumor size compared to gray-scale ultrasound (US). In the dual-image obtained, the lesion size enlarged from gray-scale US to CEUS, as the microvascular appeared to be continuously infiltrating the surrounding tissue. Although the infiltrative dilatation of microvasculature remains ambiguous, sonographers believe it may promote the diagnosis of thyroid nodules. We propose a deep learning model designed to emulate the diagnostic reasoning process employed by sonographers. This model integrates the observation of microvascular infiltration on dynamic CEUS, leveraging the additional insights provided by gray-scale US for enhanced diagnostic support. Specifically, temporal projection attention is implemented on time dimension of dynamic CEUS to represent the microvascular perfusion. Additionally, we employ a group of confidence maps with flexible Sigmoid Alpha Functions to aware and describe the infiltrative dilatation process. Moreover, a self-adaptive integration mechanism is introduced to dynamically integrate the assisted gray-scale US and the confidence maps of CEUS for individual patients, ensuring a trustworthy diagnosis of thyroid nodules. In this retrospective study, we collected a thyroid nodule dataset of 282 CEUS videos. The method achieves a superior diagnostic accuracy and sensitivity of 89.52% and 93.75%, respectively. These results suggest that imitating the diagnostic thinking of sonographers, encompassing dynamic microvascular perfusion and infiltrative expansion, proves beneficial for CEUS-based thyroid nodule diagnosis.

Automated individual cortical parcellation via consensus graph representation learning.

Cortical parcellation plays a pivotal role in elucidating the brain organization. Despite the growing efforts to develop parcellation algorithms using functional magnetic resonance imaging, achieving a balance between intra-individual specificity and inter-individual consistency proves challenging, making the generation of high-quality, subject-consistent cortical parcellations particularly elusive. To solve this problem, our paper proposes a fully automated individual cortical parcellation method based on consensus graph representation learning. The method integrates spectral embedding with low-rank tensor learning into a unified optimization model, which uses group-common connectivity patterns captured by low-rank tensor learning to optimize subjects' functional networks. This not only ensures consistency in brain representations across different subjects but also enhances the quality of each subject's representation matrix by eliminating spurious connections. More importantly, it achieves an adaptive balance between intra-individual specificity and inter-individual consistency during this process. Experiments conducted on a test-retest dataset from the Human Connectome Project (HCP) demonstrate that our method outperforms existing methods in terms of reproducibility, functional homogeneity, and alignment with task activation. Extensive network-based comparisons on the HCP S900 dataset reveal that the functional network derived from our cortical parcellation method exhibits greater capabilities in gender identification and behavior prediction than other approaches.

T-S2Inet: Transformer-based sequence-to-image network for accurate nanopore sequence recognition.

MOTIVATION: Nanopore sequencing is a new macromolecular recognition and perception technology that enables high-throughput sequencing of DNA, RNA, even protein molecules. The sequences generated by nanopore sequencing span a large time frame, and the labor and time costs incurred by traditional analysis methods are substantial. Recently, research on nanopore data analysis using machine learning algorithms has gained unceasing momentum, but there is often a significant gap between traditional and deep learning methods in terms of classification results. To analyze nanopore data using deep learning technologies, measures such as sequence completion and sequence transformation can be employed. However, these technologies do not preserve the local features of the sequences. To address this issue, we propose a sequence-to-image (S2I) module that transforms sequences of unequal length into images. Additionally, we propose the Transformer-based T-S2Inet model to capture the important information and improve the classification accuracy. RESULTS: Quantitative and qualitative analysis shows that the experimental results have an improvement of around 2% in accuracy compared to previous methods. The proposed method is adaptable to other nanopore platforms, such as the Oxford nanopore. It is worth noting that the proposed method not only aims to achieve the most advanced performance, but also provides a general idea for the analysis of nanopore sequences of unequal length. AVAILABILITY AND IMPLEMENTATION: The main program is available at https://github.com/guanxiaoyu11/S2Inet.

Spatio-Temporal Graph Hubness Propagation Model for Dynamic Brain Network Classification.

Dynamic brain network has the advantage over static brain network in characterizing the variation pattern of functional brain connectivity, and it has attracted increasing attention in brain disease diagnosis. However, most of the existing dynamic brain networks analysis methods rely on extracting features from independent brain networks divided by sliding windows, making them hard to reveal the high-order dynamic evolution laws of functional brain networks. Additionally, they cannot effectively extract the spatio-temporal topology features in dynamic brain networks. In this paper, we propose to use optimal transport (OT) theory to capture the topology evolution of the dynamic brain networks, and develop a multi-channel spatio-temporal graph convolutional network that collaboratively extracts the temporal and spatial features from the evolution networks. Specifically, we first adaptively evaluate the graph hubness of brain regions in the brain network of each time window, which comprehensively models information transmission among multiple brain regions. Second, the hubness propagation information across adjacent time windows is captured by optimal transport, describing high-order topology evolution of dynamic brain networks. Moreover, we develop a spatio-temporal graph convolutional network with attention mechanism to collaboratively extract the intrinsic temporal and spatial topology information from the above networks. Finally, the multi-layer perceptron is adopted for classifying the dynamic brain network. The extensive experiment on the collected epilepsy dataset and the public ADNI dataset show that our proposed method not only outperforms several state-of-the-art methods in brain disease diagnosis, but also reveals the key dynamic alterations of brain connectivities between patients and healthy controls.

Multi-Instance Multi-Task Learning for Joint Clinical Outcome and Genomic Profile Predictions From the Histopathological Images.

With the remarkable success of digital histopathology and the deep learning technology, many whole-slide pathological images (WSIs) based deep learning models are designed to help pathologists diagnose human cancers. Recently, rather than predicting categorical variables as in cancer diagnosis, several deep learning studies are also proposed to estimate the continuous variables such as the patients' survival or their transcriptional profile. However, most of the existing studies focus on conducting these predicting tasks separately, which overlooks the useful intrinsic correlation among them that can boost the prediction performance of each individual task. In addition, it is sill challenge to design the WSI-based deep learning models, since a WSI is with huge size but annotated with coarse label. In this study, we propose a general multi-instance multi-task learning framework (HistMIMT) for multi-purpose prediction from WSIs. Specifically, we firstly propose a novel multi-instance learning module (TMICS) considering both common and specific task information across different tasks to generate bag representation for each individual task. Then, a soft-mask based fusion module with channel attention (SFCA) is developed to leverage useful information from the related tasks to help improve the prediction performance on target task. We evaluate our method on three cancer cohorts derived from the Cancer Genome Atlas (TCGA). For each cohort, our multi-purpose prediction tasks range from cancer diagnosis, survival prediction and estimating the transcriptional profile of gene TP53. The experimental results demonstrated that HistMIMT can yield better outcome on all clinical prediction tasks than its competitors.

MSEF-Net: Multi-scale edge fusion network for lumbosacral plexus segmentation with MR image.

Nerve damage of spine areas is a common cause of disability and paralysis. The lumbosacral plexus segmentation from magnetic resonance imaging (MRI) scans plays an important role in many computer-aided diagnoses and surgery of spinal nerve lesions. Due to the complex structure and low contrast of the lumbosacral plexus, it is difficult to delineate the regions of edges accurately. To address this issue, we propose a Multi-Scale Edge Fusion Network (MSEF-Net) to fully enhance the edge feature in the encoder and adaptively fuse multi-scale features in the decoder. Specifically, to highlight the edge structure feature, we propose an edge feature fusion module (EFFM) by combining the Sobel operator edge detection and the edge-guided attention module (EAM), respectively. To adaptively fuse the multi-scale feature map in the decoder, we introduce an adaptive multi-scale fusion module (AMSF). Our proposed MSEF-Net method was evaluated on the collected spinal MRI dataset with 89 patients (a total of 2848 MR images). Experimental results demonstrate that our MSEF-Net is effective for lumbosacral plexus segmentation with MR images, when compared with several state-of-the-art segmentation methods.

The role of lysosomes in airborne particulate matter-induced pulmonary toxicity.

An investigation of the potential role of lysosomes in airborne particulate matter (APM) induced health risks is essential to fully comprehend the pathogenic mechanisms of respiratory diseases. It is commonly accepted that APM-induced lung injury is caused by oxidative stress, inflammatory responses, and DNA damage. In addition, there exists abundant evidence that changes in lysosomal function are essential for cellular adaptation to a variety of particulate stimuli. This review emphasizes that disruption of the lysosomal structure/function is a key step in the cellular metabolic imbalance induced by APMs. After being ingested by cells, most particles are localized within lysosomes. Thus, lysosomes become the primary locus where APMs accumulate, and here they undergo degradation and release toxic components. Recent studies have provided incontrovertible evidence that a wide variety of APMs interfere with the normal function of lysosomes. After being stimulated by APMs, lysosome rupture leads to a loss of lysosomal acidic conditions and the inactivation of proteolytic enzymes, promoting an inflammatory response by activating the nucleotide-binding oligomerization domain-like receptor family pyrin domain-containing 3 (NLRP3) inflammasome. Moreover, APMs interfere with autophagosome production or block autophagic flux, resulting in autophagy dysfunction. Additionally, APMs disrupt the normal function of lysosomes in iron metabolism, leading to disruption on iron homeostasis. Therefore, understanding the impacts of APM exposure from the perspective of lysosomes will provide new insights into the detrimental consequences of air pollution.

Multi-Scale FC-Based Multi-Order GCN: A Novel Model for Predicting Individual Behavior From fMRI.

Predicting individual behavior from brain imaging data using machine learning is a rapidly growing field in neuroscience. Functional connectivity (FC), which captures interactions between different brain regions, contains valuable information about the organization of the brain and is considered a crucial feature for modeling human behavior. Graph convolutional networks (GCN) have proven to be a powerful tool for extracting graph structure features and have shown promising results in various FC-based classification tasks, such as disease classification and prognosis prediction. Despite this success, few behavior prediction models currently exist based on GCN, and their performance is not satisfactory. To address this gap, a new model called the Multi-Scale FC-based Multi-Order GCN (MSFC-MO-GCN) was proposed in this paper. The model considers the hierarchical structure of the brain system and utilizes FCs inferred from multiple spatial scales as input to comprehensively characterize individual brain organization. To enhance the feature learning ability of GCN, a multi-order graph convolutional layer is incorporated, which uses multi-order neighbors to guide message passing and learns high-order graph information of nodal connections. Additionally, an inter-subject contrast constraint is designed to control the potential information redundancy of FCs among different spatial scales during the feature learning process. Experimental evaluation were conducted on the publicly available dataset from human connectome project. A total of 805 healthy subjects were included and 5 representative behavior metrics were used. The experimental results show that our proposed method outperforms the existing behavior prediction models in all behavior prediction tasks.

Ordinal Pattern Tree: A New Representation Method for Brain Network Analysis.

Brain networks, describing the functional or structural interactions of brain with graph theory, have been widely used for brain imaging analysis. Currently, several network representation methods have been developed for describing and analyzing brain networks. However, most of these methods ignored the valuable weighted information of the edges in brain networks. In this paper, we propose a new representation method (i.e., ordinal pattern tree) for brain network analysis. Compared with the existing network representation methods, the proposed ordinal pattern tree (OPT) can not only leverage the weighted information of the edges but also express the hierarchical relationships of nodes in brain networks. On OPT, nodes are connected by ordinal edges which are constructed by using the ordinal pattern relationships of weighted edges. We represent brain networks as OPTs and further develop a new graph kernel called optimal transport (OT) based ordinal pattern tree (OT-OPT) kernel to measure the similarity between paired brain networks. In OT-OPT kernel, the OT distances are used to calculate the transport costs between the nodes on the OPTs. Based on these OT distances, we use exponential function to calculate OT-OPT kernel which is proved to be positive definite. To evaluate the effectiveness of the proposed method, we perform classification and regression experiments on ADHD-200, ABIDE and ADNI datasets. The experimental results demonstrate that our proposed method outperforms the state-of-the-art graph methods in the classification and regression tasks.

A Generalized Full-to-Partial Registration Framework of 3D Point Sets for Computer-Aided Orthopedic Surgery.

OBJECTIVE: The precise alignment of full and partial 3D point sets is a crucial technique in computer-aided orthopedic surgery, but remains a significant challenge. This registration process is complicated by the partial overlap between the full and partial 3D point sets, as well as the susceptibility of 3D point sets to noise interference and poor initialization conditions. METHODS: To address these issues, we propose a novel full-to-partial registration framework for computer-aided orthopedic surgery that utilizes reinforcement learning. Our proposed framework is both generalized and robust, effectively handling the challenges of noise, poor initialization, and partial overlap. Moreover, this framework demonstrates exceptional generalization capabilities for various bones, including the pelvis, femurs, and tibias. RESULTS: Extensive experimentation on several bone datasets has demonstrated that the proposed method achieves a superior C.D. error of 8.211 e-05 and our method consistently outperforms state-of-the-art registration techniques. CONCLUSION AND SIGNIFICANCE: Hence, our proposed method is capable of achieving precise bone alignments for computer-aided orthopedic surgery.

Hypergraph convolutional network for longitudinal data analysis in Alzheimer's disease.

Alzheimer's disease (AD) is an irreversible and progressive neurodegenerative disease. Longitudinal structural magnetic resonance imaging (sMRI) data have been widely used for tracking AD pathogenesis and diagnosis. However, existing methods tend to treat each time point equally without considering the temporal characteristics of longitudinal data. In this paper, we propose a weighted hypergraph convolution network (WHGCN) to use the internal correlations among different time points and leverage high-order relationships between subjects for AD detection. Specifically, we construct hypergraphs for sMRI data at each time point using the K-nearest neighbor (KNN) method to represent relationships between subjects, and then fuse the hypergraphs according to the importance of the data at each time point to obtain the final hypergraph. Subsequently, we use hypergraph convolution to learn high-order information between subjects while performing feature dimensionality reduction. Finally, we conduct experiments on 518 subjects selected from the Alzheimer's disease neuroimaging initiative (ADNI) database, and the results show that the WHGCN can get higher AD detection performance and has the potential to improve our understanding of the pathogenesis of AD.

Identifying psychosis subtypes use individualized covariance structural differential networks and multi-site clustering.

BACKGROUND: Similarities among schizophrenia (SZ), schizoaffective disorder (SAD) and bipolar disorder (BP) including clinical phenotypes, brain alterations and risk genes, make it challenging to perform reliable separation among them. However, previous subtype identification that transcend traditional diagnostic boundaries were based on group-level neuroimaging features, ignoring individual-level inferences. METHODS: 455 psychoses (178 SZs, 134 SADs and 143 BPs), their first-degree relatives (N = 453) and healthy controls (HCs, N = 220) were collected from Bipolar-Schizophrenia Network on Intermediate Phenotypes (B-SNIP I) consortium. Individualized covariance structural differential networks (ICSDNs) were constructed for each patient and multi-site clustering was used to identify psychosis subtypes. Group differences between subtypes in clinical phenotypes and voxel-wise fractional amplitude of low frequency fluctuation (fALFF) were calculated, as well as between the corresponding relatives. RESULTS: Two psychosis subtypes were identified with increased whole brain structural covariance, with decreased connectivity between amygdala-hippocampus and temporal-occipital cortex in subtype I (S-I) compared to subtype II (S-II), which was replicated under different clustering methods, number of edges and across datasets (B-SNIP II) and different brain atlases. S-I had higher emotional-related symptoms than S-II and showed significant fALFF decrease in temporal and occipital cortex, while S-II was more similar to HC. This pattern was consistently validated on relatives of S-I and S-II in both fALFF and clinical symptoms. CONCLUSIONS: These findings reconcile categorical and dimensional perspectives of psychosis neurobiological heterogeneity, indicating that relatives of S-I might have greater predisposition in developing psychosis, while relatives of S-II are more likely to be healthy. This study contributes to the development of neuroimaging informed diagnostic classifications within psychosis spectrum.

Uncertainty-inspired open set learning for retinal anomaly identification.

Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies.

T-MGCL: Molecule Graph Contrastive Learning Based on Transformer for Molecular Property Prediction.

In recent years, machine learning has gained increasing traction in the study of molecules, enabling researchers to tackle challenging tasks including molecular property prediction and drug design.Consequently, there remains an open challenge to develop a neural network architecture that can make use of extensive amounts of unlabeled data for training while still providing competitive results in various molecular property prediction tasks. To address this challenge, we propose a Molecule Graph Contrastive Learning approach based on the Transformer framework (T-MGCL). Our approach involves expanding numerous unsupervised molecular graphs and using a contrast estimator to ensure consistency among various graph expansions of the same molecule. Transformer framework is employed to consider the distance between atoms and molecular graph attributes, thereby accounting for structural information that may be overlooked by traditional graph neural networks. Our experimental results demonstrate that the T-MGCL model outperforms other models in several molecular property prediction tasks. Additionally, we observe that the attention weight learned by T-MGCL can be easily explained from a chemical perspective.

Deep Semi-Supervised Ultrasound Image Segmentation by Using a Shadow Aware Network With Boundary Refinement.

Accurate ultrasound (US) image segmentation is crucial for the screening and diagnosis of diseases. However, it faces two significant challenges: 1) pixel-level annotation is a time-consuming and laborious process; 2) the presence of shadow artifacts leads to missing anatomy and ambiguous boundaries, which negatively impact reliable segmentation results. To address these challenges, we propose a novel semi-supervised shadow aware network with boundary refinement (SABR-Net). Specifically, we add shadow imitation regions to the original US, and design shadow-masked transformer blocks to perceive missing anatomy of shadow regions. Shadow-masked transformer block contains an adaptive shadow attention mechanism that introduces an adaptive mask, which is updated automatically to promote the network training. Additionally, we utilize unlabeled US images to train a missing structure inpainting path with shadow-masked transformer, which further facilitates semi-supervised segmentation. Experiments on two public US datasets demonstrate the superior performance of the SABR-Net over other state-of-the-art semi-supervised segmentation methods. In addition, experiments on a private breast US dataset prove that our method has a good generalization to clinical small-scale US datasets.

Anatomy preserving GAN for realistic simulation of intraoperative liver ultrasound images.

In ultrasound-guided liver surgery, the lack of large-scale intraoperative ultrasound images with important anatomical structures remains an obstacle hindering the successful application of AI to ultrasound guidance. In this case, intraoperative ultrasound (iUS) simulation should be conducted from preoperative magnetic resonance (pMR), which not only helps doctors understand the characteristics of iUS in advance, but also expands the iUS dataset from various imaging positions, thereby promoting the automatic iUS analysis in ultrasound guidance. Herein, a novel anatomy preserving generative adversarial network (ApGAN) framework was proposed to generate simulated intraoperative ultrasound (Sim-iUS) of liver with precise structure information from pMR. Specifically, the low-rank factors based bimodal fusion was first established focusing on the effective information of hepatic parenchyma. Then, a deformation field based correction module was introduced to learn and correct the slight structural distortion from surgical operations. Meanwhile, the multiple loss functions were designed to constrain the simulation of the content, structures, and style. Empirical results of clinical data showed that the proposed ApGAN obtained higher Structural Similarity (SSIM) of 0.74 and Fr´echet Inception Distance (FID) of 35.54 compared to existing methods. Furthermore, the average Hausdorff Distance (HD) error of the liver capsule structure was less than 0.25 mm, and the average relative (Euclidean Distance) ED error for polyps was 0.12 mm, indicating the high-level precision of this ApGAN in simulating the anatomical structures and focal areas.