Aligning the domains in cross domain model inversion attack.

Model Inversion Attack reconstructs confidential training dataset from a target deep learning model. Most of the existing methods assume the adversary has an auxiliary dataset that has similar distribution with the private dataset. However, this assumption does not always hold in real-world scenarios. Since the private dataset is unknown, the domain divergence between the auxiliary dataset and the private dataset is inevitable. In this paper, we use Cross Domain Model Inversion Attack to represent the distribution divergence scenario in MIA. With the distribution divergence between the private images and auxiliary images, the distribution between the feature vectors of the private images and those of the auxiliary images is also different. Moreover, the outputted prediction vectors of the auxiliary images are also misclassified. The inversion attack is thus hard to be performed. We perform both the feature vector inversion task and prediction vector inversion task in this cross domain setting. For feature vector inversion, Domain Alignment MIA (DA-MIA) is proposed. While performing the reconstruction task, DA-MIA aligns the feature vectors of auxiliary images with the feature vectors of private images in an adversarial manner to mitigate the domain divergence between them. Thus, semantically meaningful images can be reconstructed. For prediction vector inversion, we further introduce an auxiliary classifier and propose Domain Alignment MIA with Auxiliary Classifier (DA-MIA-AC). The auxiliary classifier is pretrained by the auxiliary dataset and fine-tuned during the adversarial training stage. Thus, the misclassification problem caused by domain divergence can be solved, and the images can be reconstructed correctly. Various experiments are performed to show the advancement of our methods, the results show that DA-MIA can improve the SSIM score of the reconstructed images for up to 191%, DA-MIA-AC can increase the classification accuracy score of the reconstructed images from 9.18% to 81.32% in Cross Domain Model Inversion Attack.

HCA-DAN: hierarchical class-aware domain adaptive network for gastric tumor segmentation in 3D CT images.

BACKGROUND: Accurate segmentation of gastric tumors from CT scans provides useful image information for guiding the diagnosis and treatment of gastric cancer. However, automated gastric tumor segmentation from 3D CT images faces several challenges. The large variation of anisotropic spatial resolution limits the ability of 3D convolutional neural networks (CNNs) to learn features from different views. The background texture of gastric tumor is complex, and its size, shape and intensity distribution are highly variable, which makes it more difficult for deep learning methods to capture the boundary. In particular, while multi-center datasets increase sample size and representation ability, they suffer from inter-center heterogeneity. METHODS: In this study, we propose a new cross-center 3D tumor segmentation method named Hierarchical Class-Aware Domain Adaptive Network (HCA-DAN), which includes a new 3D neural network that efficiently bridges an Anisotropic neural network and a Transformer (AsTr) for extracting multi-scale context features from the CT images with anisotropic resolution, and a hierarchical class-aware domain alignment (HCADA) module for adaptively aligning multi-scale context features across two domains by integrating a class attention map with class-specific information. We evaluate the proposed method on an in-house CT image dataset collected from four medical centers and validate its segmentation performance in both in-center and cross-center test scenarios. RESULTS: Our baseline segmentation network (i.e., AsTr) achieves best results compared to other 3D segmentation models, with a mean dice similarity coefficient (DSC) of 59.26%, 55.97%, 48.83% and 67.28% in four in-center test tasks, and with a DSC of 56.42%, 55.94%, 46.54% and 60.62% in four cross-center test tasks. In addition, the proposed cross-center segmentation network (i.e., HCA-DAN) obtains excellent results compared to other unsupervised domain adaptation methods, with a DSC of 58.36%, 56.72%, 49.25%, and 62.20% in four cross-center test tasks. CONCLUSIONS: Comprehensive experimental results demonstrate that the proposed method outperforms compared methods on this multi-center database and is promising for routine clinical workflows.

Enhanced Nuclei Segmentation and Classification via Category Descriptors in the SAM Model.

Segmenting and classifying nuclei in H&E histopathology images is often limited by the long-tailed distribution of nuclei types. However, the strong generalization ability of image segmentation foundation models like the Segment Anything Model (SAM) can help improve the detection quality of rare types of nuclei. In this work, we introduce category descriptors to perform nuclei segmentation and classification by prompting the SAM model. We close the domain gap between histopathology and natural scene images by aligning features in low-level space while preserving the high-level representations of SAM. We performed extensive experiments on the Lizard dataset, validating the ability of our model to perform automatic nuclei segmentation and classification, especially for rare nuclei types, where achieved a significant detection improvement in the F1 score of up to 12%. Our model also maintains compatibility with manual point prompts for interactive refinement during inference without requiring any additional training.

Graph embedding-based heterogeneous domain adaptation with domain-invariant feature learning and distributional order preserving.

Heterogeneous domain adaptation (HDA) methods leverage prior knowledge from the source domain to train models for the target domain and address the differences in their feature spaces. However, incorrect alignment of categories and distribution structure disruption may be caused by unlabeled target samples during the domain alignment process for most existing methods, resulting in negative transfer. Additionally, the previous works rarely focus on the robustness and interpretability of the model. To address these issues, we propose a novel Graph embedding-based Heterogeneous domain-Invariant feature learning and Distributional order preserving framework (GHID). Specifically, a bidirectional robust cross-domain alignment graph embedding structure is proposed to globally align two domains, which learns the domain-invariant and discriminative features simultaneously. In addition, the interpretability of the proposed graph structures is demonstrated through two theoretical analyses, which can elucidate the correlation between important samples from a global perspective in heterogeneous domain alignment scenarios. Then, a heterogeneous discriminative distributional order preserving graph embedding structure is designed to preserve the original distribution relationship of each domain to prevent negative transfer. Moreover, the dynamic centroid strategy is incorporated into the graph structures to improve the robustness of the model. Comprehensive experimental results on four benchmarks demonstrate that the proposed method outperforms other state-of-the-art approaches in effectiveness.

Partial adversarial domain adaptation by dual-domain alignment for fault diagnosis of rotating machines.

Domain adaptation (DA) techniques have succeeded in solving domain shift problem for fault diagnosis (FD), where the research assumption is that the target domain (TD) and source domain (SD) share identical label spaces. However, when the SD label spaces subsume the TD, heterogeneity occurs, which is a partial domain adaptation (PDA) problem. In this paper, we propose a dual-domain alignment approach for partial adversarial DA (DDA-PADA) for FD, including (1) traditional domain-adversarial neural network (DANN) modules (feature extractors, feature classifiers and a domain discriminator); (2) a SD alignment (SDA) module designed based on the feature alignment of SD extracted in two stages; and (3) a cross-domain alignment (CDA) module designed based on the feature alignment of SD and TD extracted in the second stage. Specifically, SDA and CDA are implemented by a unilateral feature alignment approach, which maintains the feature consistency of the SD and attempts to mitigate cross-domain variation by correcting the feature distribution of TD, achieving feature alignment from a dual-domain perspective. Thus, DDA-PADA can effectively align the SD and TD without affecting the feature distribution of SD. Experimental results obtained on two rotating mechanical datasets show that DDA-PADA exhibits satisfactory performance in handling PDA problems. The various analysis results validate the advantages of DDA-PADA.

Atomistic Insight into the Epitaxial Growth Mechanism of Single-Crystal Two-Dimensional Transition-Metal Dichalcogenides on Au(111) Substrate.

A mechanistic understanding of interactions between atomically thin two-dimensional (2D) transition-metal dichalcogenides (TMDs) and their growth substrates is important for achieving the unidirectional alignment of nuclei and seamless stitching of 2D TMD domains and thus 2D wafers. In this work, we conduct a cross-sectional scanning transmission electron microscopy (STEM) study to investigate the atomic-scale nucleation and early stage growth behaviors of chemical vapor deposited monolayer (ML-) MoS2 and molecular beam epitaxy ML-MoSe2 on a Au(111) substrate. Statistical analysis reveals the majority of as-grown domains, i.e., ∼88% for MoS2 and 90% for MoSe2, nucleate on surface terraces, with the rest (i.e., ∼12% for MoS2 and 10% for MoSe2) on surface steps. Moreover, within the latter case, step-associated nucleation, ∼64% of them are terminated with a Mo-zigzag edge in connection with the Au surface steps, with the rest (∼36%) being S-zigzag edges. In conjunction with ab initio density functional theory calculations, the results confirm that van der Waals epitaxy, rather than the surface step guided epitaxy, plays deterministic roles for the realization of unidirectional ML-MoS2 (MoSe2) domains on a Au(111) substrate. In contrast, surface steps, particularly their step height, are mainly responsible for the integrity and thickness of MoS2/MoSe2 films. In detail, it is found that the lateral growth of monolayer thick MoS2/MoSe2 domains only proceeds across mono-Au-atom high surface steps (∼2.4 Å), but fail for higher ones (bi-Au atom step and higher) during the growth. Our cross-sectional STEM study also confirms the existence of considerable compressive residual strain that reaches ∼3.0% for ML-MoS2/MoSe2 domains on Au(111). The present study aims to understand the growth mechanism of 2D TMD wafers.

Brain graph synthesis by dual adversarial domain alignment and target graph prediction from a source graph.

Developing predictive intelligence in neuroscience for learning how to generate multimodal medical data from a single modality can improve neurological disorder diagnosis with minimal data acquisition resources. Existing deep learning frameworks are mainly tailored for images, which might fail in handling geometric data (e.g., brain graphs). Specifically, predicting a target brain graph from a single source brain graph remains largely unexplored. Solving such problem is generally challenged with domain fracturecaused by the difference in distribution between source and target domains. Besides, solving the prediction and domain fracture independently might not be optimal for both tasks. To address these challenges, we unprecedentedly propose a Learning-guided Graph Dual Adversarial Domain Alignment (LG-DADA) framework for predicting a target brain graph from a source brain graph. The proposed LG-DADA is grounded in three fundamental contributions: (1) a source data pre-clustering step using manifold learning to firstly handle source data heterogeneity and secondly circumvent mode collapse in generative adversarial learning, (2) a domain alignment of source domain to the target domain by adversarially learning their latent representations, and (3) a dual adversarial regularization that jointly learns a source embedding of training and testing brain graphs using two discriminators and predict the training target graphs. Results on morphological brain graphs synthesis showed that our method produces better prediction accuracy and visual quality as compared to other graph synthesis methods.