Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences.

What specific features should visual neurons encode, given the infinity of real-world images and the limited number of neurons available to represent them? We investigated neuronal selectivity in monkey inferotemporal cortex via the vast hypothesis space of a generative deep neural network, avoiding assumptions about features or semantic categories. A genetic algorithm searched this space for stimuli that maximized neuronal firing. This led to the evolution of rich synthetic images of objects with complex combinations of shapes, colors, and textures, sometimes resembling animals or familiar people, other times revealing novel patterns that did not map to any clear semantic category. These results expand our conception of the dictionary of features encoded in the cortex, and the approach can potentially reveal the internal representations of any system whose input can be captured by a generative model.

ctGAN: combined transformation of gene expression and survival data with generative adversarial network.

Recent studies have extensively used deep learning algorithms to analyze gene expression to predict disease diagnosis, treatment effectiveness, and survival outcomes. Survival analysis studies on diseases with high mortality rates, such as cancer, are indispensable. However, deep learning models are plagued by overfitting owing to the limited sample size relative to the large number of genes. Consequently, the latest style-transfer deep generative models have been implemented to generate gene expression data. However, these models are limited in their applicability for clinical purposes because they generate only transcriptomic data. Therefore, this study proposes ctGAN, which enables the combined transformation of gene expression and survival data using a generative adversarial network (GAN). ctGAN improves survival analysis by augmenting data through style transformations between breast cancer and 11 other cancer types. We evaluated the concordance index (C-index) enhancements compared with previous models to demonstrate its superiority. Performance improvements were observed in nine of the 11 cancer types. Moreover, ctGAN outperformed previous models in seven out of the 11 cancer types, with colon adenocarcinoma (COAD) exhibiting the most significant improvement (median C-index increase of ~15.70%). Furthermore, integrating the generated COAD enhanced the log-rank p-value (0.041) compared with using only the real COAD (p-value = 0.797). Based on the data distribution, we demonstrated that the model generated highly plausible data. In clustering evaluation, ctGAN exhibited the highest performance in most cases (89.62%). These findings suggest that ctGAN can be meaningfully utilized to predict disease progression and select personalized treatments in the medical field.

DaDL-SChlo: protein subchloroplast localization prediction based on generative adversarial networks and pre-trained protein language model.

Chloroplast is a crucial site for photosynthesis in plants. Determining the location and distribution of proteins in subchloroplasts is significant for studying the energy conversion of chloroplasts and regulating the utilization of light energy in crop production. However, the prediction accuracy of the currently developed protein subcellular site predictors is still limited due to the complex protein sequence features and the scarcity of labeled samples. We propose DaDL-SChlo, a multi-location protein subchloroplast localization predictor, which addresses the above problems by fusing pre-trained protein language model deep learning features with traditional handcrafted features and using generative adversarial networks for data augmentation. The experimental results of cross-validation and independent testing show that DaDL-SChlo has greatly improved the prediction performance of protein subchloroplast compared with the state-of-the-art predictors. Specifically, the overall actual accuracy outperforms the state-of-the-art predictors by 10.7% on 10-fold cross-validation and 12.6% on independent testing. DaDL-SChlo is a promising and efficient predictor for protein subchloroplast localization. The datasets and codes of DaDL-SChlo are available at https://github.com/xwanggroup/DaDL-SChlo.

scGAL: unmask tumor clonal substructure by jointly analyzing independent single-cell copy number and scRNA-seq data.

BACKGROUND: Accurately deciphering clonal copy number substructure can provide insights into the evolutionary mechanism of cancer, and clustering single-cell copy number profiles has become an effective means to unmask intra-tumor heterogeneity (ITH). However, copy numbers inferred from single-cell DNA sequencing (scDNA-seq) data are error-prone due to technically confounding factors such as amplification bias and allele-dropout, and this makes it difficult to precisely identify the ITH. RESULTS: We introduce a hybrid model called scGAL to infer clonal copy number substructure. It combines an autoencoder with a generative adversarial network to jointly analyze independent single-cell copy number profiles and gene expression data from same cell line. Under an adversarial learning framework, scGAL exploits complementary information from gene expression data to relieve the effects of noise in copy number data, and learns latent representations of scDNA-seq cells for accurate inference of the ITH. Evaluation results on three real cancer datasets suggest scGAL is able to accurately infer clonal architecture and surpasses other similar methods. In addition, assessment of scGAL on various simulated datasets demonstrates its high robustness against the changes of data size and distribution. scGAL can be accessed at: https://github.com/zhyu-lab/scgal . CONCLUSIONS: Joint analysis of independent single-cell copy number and gene expression data from a same cell line can effectively exploit complementary information from individual omics, and thus gives more refined indication of clonal copy number substructure.

GAN-MAT: Generative adversarial network-based microstructural profile covariance analysis toolbox.

Multimodal magnetic resonance imaging (MRI) provides complementary information for investigating brain structure and function; for example, an in vivo microstructure-sensitive proxy can be estimated using the ratio between T1- and T2-weighted structural MRI. However, acquiring multiple imaging modalities is challenging in patients with inattentive disorders. In this study, we proposed a comprehensive framework to provide multiple imaging features related to the brain microstructure using only T1-weighted MRI. Our toolbox consists of (i) synthesizing T2-weighted MRI from T1-weighted MRI using a conditional generative adversarial network; (ii) estimating microstructural features, including intracortical covariance and moment features of cortical layer-wise microstructural profiles; and (iii) generating a microstructural gradient, which is a low-dimensional representation of the intracortical microstructure profile. We trained and tested our toolbox using T1- and T2-weighted MRI scans of 1,104 healthy young adults obtained from the Human Connectome Project database. We found that the synthesized T2-weighted MRI was very similar to the actual image and that the synthesized data successfully reproduced the microstructural features. The toolbox was validated using an independent dataset containing healthy controls and patients with episodic migraine as well as the atypical developmental condition of autism spectrum disorder. Our toolbox may provide a new paradigm for analyzing multimodal structural MRI in the neuroscience community and is openly accessible at https://github.com/CAMIN-neuro/GAN-MAT.

Harmonic field extension for QSM with reduced spatial coverage using physics-informed generative adversarial network.

Quantitative susceptibility mapping (QSM) is frequently employed in investigating brain iron related to brain development and diseases within deep gray matter (DGM). Nonetheless, the acquisition of whole-brain QSM data is time-intensive. An alternative approach, focusing the QSM specifically on areas of interest such as the DGM by reducing the field-of-view (FOV), can significantly decrease scan times. However, severe susceptibility value underestimations have been reported during QSM reconstruction with a limited FOV, largely attributable to artifacts from incorrect background field removal in the boundary region. This presents a considerable barrier to the clinical use of QSM with small spatial coverages using conventional methods alone. To mitigate the propagation of these errors, we proposed a harmonic field extension method based on a physics-informed generative adversarial network. Both quantitative and qualitative results demonstrate that our method outperforms conventional methods and delivers results comparable to those obtained with full FOV. Furthermore, we demonstrate the versatility of our method by applying it to data acquired prospectively with limited FOV and to data from patients with Parkinson's disease. The method has shown significant improvements in local field results, with QSM outcomes. In a clear illustration of its feasibility and effectiveness in real clinical environments, our proposed method addresses the prevalent issue of susceptibility underestimation in QSM with small spatial coverage.

Detection of individual brain tau deposition in Alzheimer's disease based on latent feature-enhanced generative adversarial network.

OBJECTIVE: The conventional methods for interpreting tau PET imaging in Alzheimer's disease (AD), including visual assessment and semi-quantitative analysis of fixed hallmark regions, are insensitive to detect individual small lesions because of the spatiotemporal neuropathology's heterogeneity. In this study, we proposed a latent feature-enhanced generative adversarial network model for the automatic extraction of individual brain tau deposition regions. METHODS: The latent feature-enhanced generative adversarial network we propose can learn the distribution characteristics of tau PET images of cognitively normal individuals and output the abnormal distribution regions of patients. This model was trained and validated using 1131 tau PET images from multiple centres (with distinct races, i.e., Caucasian and Mongoloid) with different tau PET ligands. The overall quality of synthetic imaging was evaluated using structural similarity (SSIM), peak signal to noise ratio (PSNR), and mean square error (MSE). The model was compared to the fixed templates method for diagnosing and predicting AD. RESULTS: The reconstructed images archived good quality, with SSIM = 0.967 ± 0.008, PSNR = 31.377 ± 3.633, and MSE = 0.0011 ± 0.0007 in the independent test set. The model showed higher classification accuracy (AUC = 0.843, 95 % CI = 0.796-0.890) and stronger correlation with clinical scales (r = 0.508, P < 0.0001). The model also achieved superior predictive performance in the survival analysis of cognitive decline, with a higher hazard ratio: 3.662, P < 0.001. INTERPRETATION: The LFGAN4Tau model presents a promising new approach for more accurate detection of individualized tau deposition. Its robustness across tracers and races makes it a potentially reliable diagnostic tool for AD in practice.

Improving lesion detection in mammograms by leveraging a Cycle-GAN-based lesion remover.

BACKGROUND: The wide heterogeneity in the appearance of breast lesions and normal breast structures can confuse computerized detection algorithms. Our purpose was therefore to develop a Lesion Highlighter (LH) that can improve the performance of computer-aided detection algorithms for detecting breast cancer on screening mammograms. METHODS: We hypothesized that a Cycle-GAN based Lesion Remover (LR) could act as an LH, which can improve the performance of lesion detection algorithms. We used 10,310 screening mammograms from 4,832 women that included 4,942 recalled lesions (BI-RADS 0) and 5,368 normal results (BI-RADS 1). We divided the dataset into Train:Validate:Test folds with the ratios of 0.64:0.16:0.2. We segmented image patches (400 × 400 pixels) from either lesions marked by MQSA radiologists or normal tissue in mammograms. We trained a Cycle-GAN to develop two GANs, where each GAN transferred the style of one image to another. We refer to the GAN transferring the style of a lesion to normal breast tissue as the LR. We then highlighted the lesion by color-fusing the mammogram after applying the LR to its original. Using ResNet18, DenseNet201, EfficientNetV2, and Vision Transformer as backbone architectures, we trained three deep networks for each architecture, one trained on lesion highlighted mammograms (Highlighted), another trained on the original mammograms (Baseline), and Highlighted and Baseline combined (Combined). We conducted ROC analysis for the three versions of each deep network on the test set. RESULTS: The Combined version of all networks achieved AUCs ranging from 0.963 to 0.974 for identifying the image with a recalled lesion from a normal breast tissue image, which was statistically improved (p-value < 0.001) over their Baseline versions with AUCs that ranged from 0.914 to 0.967. CONCLUSIONS: Our results showed that a Cycle-GAN based LR is effective for enhancing lesion conspicuity and this can improve the performance of a detection algorithm.

Frequency-specific dual-attention based adversarial network for blood oxygen level-dependent time series prediction.

Functional magnetic resonance imaging (fMRI) is currently one of the most popular technologies for measuring brain activity in both research and clinical contexts. However, clinical constraints often result in short fMRI scan durations, limiting the diagnostic performance for brain disorders. To address this limitation, we developed an end-to-end frequency-specific dual-attention-based adversarial network (FDAA-Net) to extend the time series of existing blood oxygen level-dependent (BOLD) data, enhancing their diagnostic utility. Our approach leverages the frequency-dependent nature of fMRI signals using variational mode decomposition (VMD), which adaptively tracks brain activity across different frequency bands. We integrated the generative adversarial network (GAN) with a spatial-temporal attention mechanism to fully capture relationships among spatially distributed brain regions and temporally continuous time windows. We also introduced a novel loss function to estimate the upward and downward trends of each frequency component. We validated FDAA-Net on the Human Connectome Project (HCP) database by comparing the original and predicted time series of brain regions in the default mode network (DMN), a key network activated during rest. FDAA-Net effectively overcame linear frequency-specific challenges and outperformed other popular prediction models. Test-retest reliability experiments demonstrated high consistency between the functional connectivity of predicted outcomes and targets. Furthermore, we examined the clinical applicability of FDAA-Net using short-term fMRI data from individuals with autism spectrum disorder (ASD) and major depressive disorder (MDD). The model achieved a maximum predicted sequence length of 40% of the original scan durations. The prolonged time series improved diagnostic performance by 8.0% for ASD and 11.3% for MDD compared with the original sequences. These findings highlight the potential of fMRI time series prediction to enhance diagnostic power of brain disorders in short fMRI scans.

In Silico Design of Heterogeneous Microvascular Trees Using Generative Adversarial Networks and Constrained Constructive Optimization.

OBJECTIVE: Designing physiologically adequate microvascular trees is of crucial relevance for bioengineering functional tissues and organs. Yet, currently available methods are poorly suited to replicate the morphological and topological heterogeneity of real microvascular trees because the parameters used to control tree generation are too simplistic to mimic results of the complex angiogenetic and structural adaptation processes in vivo. METHODS: We propose a method to overcome this limitation by integrating a conditional deep convolutional generative adversarial network (cDCGAN) with a local fractal dimension-oriented constrained constructive optimization (LFDO-CCO) strategy. The cDCGAN learns the patterns of real microvascular bifurcations allowing for their artificial replication. The LFDO-CCO strategy connects the generated bifurcations hierarchically to form microvascular trees with a vessel density corresponding to that observed in healthy tissues. RESULTS: The generated artificial microvascular trees are consistent with real microvascular trees regarding characteristics such as fractal dimension, vascular density, and coefficient of variation of diameter, length, and tortuosity. CONCLUSIONS: These results support the adoption of the proposed strategy for the generation of artificial microvascular trees in tissue engineering as well as for computational modeling and simulations of microcirculatory physiology.