Learning Numerosity Representations with Transformers: Number Generation Tasks and Out-of-Distribution Generalization.

One of the most rapidly advancing areas of deep learning research aims at creating models that learn to disentangle the latent factors of variation from a data distribution. However, modeling joint probability mass functions is usually prohibitive, which motivates the use of conditional models assuming that some information is given as input. In the domain of numerical cognition, deep learning architectures have successfully demonstrated that approximate numerosity representations can emerge in multi-layer networks that build latent representations of a set of images with a varying number of items. However, existing models have focused on tasks requiring to conditionally estimate numerosity information from a given image. Here, we focus on a set of much more challenging tasks, which require to conditionally generate synthetic images containing a given number of items. We show that attention-based architectures operating at the pixel level can learn to produce well-formed images approximately containing a specific number of items, even when the target numerosity was not present in the training distribution.

Enabling uncertainty estimation in neural networks through weight perturbation for improved Alzheimer's disease classification.

Background: The willingness to trust predictions formulated by automatic algorithms is key in a wide range of domains. However, a vast number of deep architectures are only able to formulate predictions without associated uncertainty. Purpose: In this study, we propose a method to convert a standard neural network into a Bayesian neural network and estimate the variability of predictions by sampling different networks similar to the original one at each forward pass. Methods: We combine our method with a tunable rejection-based approach that employs only the fraction of the data, i.e., the share that the model can classify with an uncertainty below a user-set threshold. We test our model in a large cohort of brain images from patients with Alzheimer's disease and healthy controls, discriminating the former and latter classes based on morphometric images exclusively. Results: We demonstrate how combining estimated uncertainty with a rejection-based approach increases classification accuracy from 0.86 to 0.95 while retaining 75% of the test set. In addition, the model can select the cases to be recommended for, e.g., expert human evaluation due to excessive uncertainty. Importantly, our framework circumvents additional workload during the training phase by using our network "turned into Bayesian" to implicitly investigate the loss landscape in the neighborhood of each test sample in order to determine the reliability of the predictions. Conclusion: We believe that being able to estimate the uncertainty of a prediction, along with tools that can modulate the behavior of the network to a degree of confidence that the user is informed about (and comfortable with), can represent a crucial step in the direction of user compliance and easier integration of deep learning tools into everyday tasks currently performed by human operators.

Retrieving and reconstructing conceptually similar images from fMRI with latent diffusion models and a neuro-inspired brain decoding model.

Objective.Brain decoding is a field of computational neuroscience that aims to infer mental states or internal representations of perceptual inputs from measurable brain activity. This study proposes a novel approach to brain decoding that relies on semantic and contextual similarity.Approach.We use several functional magnetic resonance imaging (fMRI) datasets of natural images as stimuli and create a deep learning decoding pipeline inspired by the bottom-up and top-down processes in human vision. Our pipeline includes a linear brain-to-feature model that maps fMRI activity to semantic visual stimuli features. We assume that the brain projects visual information onto a space that is homeomorphic to the latent space of last layer of a pretrained neural network, which summarizes and highlights similarities and differences between concepts. These features are categorized in the latent space using a nearest-neighbor strategy, and the results are used to retrieve images or condition a generative latent diffusion model to create novel images.Main results.We demonstrate semantic classification and image retrieval on three different fMRI datasets: Generic Object Decoding (vision perception and imagination), BOLD5000, and NSD. In all cases, a simple mapping between fMRI and a deep semantic representation of the visual stimulus resulted in meaningful classification and retrieved or generated images. We assessed quality using quantitative metrics and a human evaluation experiment that reproduces the multiplicity of conscious and unconscious criteria that humans use to evaluate image similarity. Our method achieved correct evaluation in over 80% of the test set.Significance.Our study proposes a novel approach to brain decoding that relies on semantic and contextual similarity. The results demonstrate that measurable neural correlates can be linearly mapped onto the latent space of a neural network to synthesize images that match the original content. These findings have implications for both cognitive neuroscience and artificial intelligence.

Decoding visual brain representations from electroencephalography through knowledge distillation and latent diffusion models.

Decoding visual representations from human brain activity has emerged as a thriving research domain, particularly in the context of brain-computer interfaces. Our study presents an innovative method that employs knowledge distillation to train an EEG classifier and reconstruct images from the ImageNet and THINGS-EEG 2 datasets using only electroencephalography (EEG) data from participants who have viewed the images themselves (i.e. "brain decoding"). We analyzed EEG recordings from 6 participants for the ImageNet dataset and 10 for the THINGS-EEG 2 dataset, exposed to images spanning unique semantic categories. These EEG readings were converted into spectrograms, which were then used to train a convolutional neural network (CNN), integrated with a knowledge distillation procedure based on a pre-trained Contrastive Language-Image Pre-Training (CLIP)-based image classification teacher network. This strategy allowed our model to attain a top-5 accuracy of 87%, significantly outperforming a standard CNN and various RNN-based benchmarks. Additionally, we incorporated an image reconstruction mechanism based on pre-trained latent diffusion models, which allowed us to generate an estimate of the images that had elicited EEG activity. Therefore, our architecture not only decodes images from neural activity but also offers a credible image reconstruction from EEG only, paving the way for, e.g., swift, individualized feedback experiments.

Beyond multilayer perceptrons: Investigating complex topologies in neural networks.

This study delves into the crucial aspect of network topology in artificial neural networks (NNs) and its impact on model performance. Addressing the need to comprehend how network structures influence learning capabilities, the research contrasts traditional multilayer perceptrons (MLPs) with models built on various complex topologies using novel network generation techniques. Drawing insights from synthetic datasets, the study reveals the remarkable accuracy of complex NNs, particularly in high-difficulty scenarios, outperforming MLPs. Our exploration extends to real-world datasets, highlighting the task-specific nature of optimal network topologies and unveiling trade-offs, including increased computational demands and reduced robustness to graph damage in complex NNs compared to MLPs. This research underscores the pivotal role of complex topologies in addressing challenging learning tasks. However, it also signals the necessity for deeper insights into the complex interplay among topological attributes influencing NN performance. By shedding light on the advantages and limitations of complex topologies, this study provides valuable guidance for practitioners and paves the way for future endeavors to design more efficient and adaptable neural architectures across various applications.

Dynomics: A Novel and Promising Approach for Improved Breast Cancer Prognosis Prediction.

Traditional imaging techniques for breast cancer (BC) diagnosis and prediction, such as X-rays and magnetic resonance imaging (MRI), demonstrate varying sensitivity and specificity due to clinical and technological factors. Consequently, positron emission tomography (PET), capable of detecting abnormal metabolic activity, has emerged as a more effective tool, providing critical quantitative and qualitative tumor-related metabolic information. This study leverages a public clinical dataset of dynamic 18F-Fluorothymidine (FLT) PET scans from BC patients, extending conventional static radiomics methods to the time domain-termed as 'Dynomics'. Radiomic features were extracted from both static and dynamic PET images on lesion and reference tissue masks. The extracted features were used to train an XGBoost model for classifying tumor versus reference tissue and complete versus partial responders to neoadjuvant chemotherapy. The results underscored the superiority of dynamic and static radiomics over standard PET imaging, achieving accuracy of 94% in tumor tissue classification. Notably, in predicting BC prognosis, dynomics delivered the highest performance, achieving accuracy of 86%, thereby outperforming both static radiomics and standard PET data. This study illustrates the enhanced clinical utility of dynomics in yielding more precise and reliable information for BC diagnosis and prognosis, paving the way for improved treatment strategies.

The age-specific comorbidity burden of mild cognitive impairment: a US claims database study.

BACKGROUND: Identifying individuals with mild cognitive impairment (MCI) who are likely to progress to Alzheimer's disease and related dementia disorders (ADRD) would facilitate the development of individualized prevention plans. We investigated the association between MCI and comorbidities of ADRD. We examined the predictive potential of these comorbidities for MCI risk determination using a machine learning algorithm. METHODS: Using a retrospective matched case-control design, 5185 MCI and 15,555 non-MCI individuals aged ≥50 years were identified from MarketScan databases. Predictive models included ADRD comorbidities, age, and sex. RESULTS: Associations between 25 ADRD comorbidities and MCI were significant but weakened with increasing age groups. The odds ratios (MCI vs non-MCI) in 50-64, 65-79, and ≥ 80 years, respectively, for depression (4.4, 3.1, 2.9) and stroke/transient ischemic attack (6.4, 3.0, 2.1). The predictive potential decreased with older age groups, with ROC-AUCs 0.75, 0.70, and 0.66 respectively. Certain comorbidities were age-specific predictors. CONCLUSIONS: The comorbidity burden of MCI relative to non-MCI is age-dependent. A model based on comorbidities alone predicted an MCI diagnosis with reasonable accuracy.

Spatiotemporal learning of dynamic positron emission tomography data improves diagnostic accuracy in breast cancer.

Positron emission tomography (PET) can reveal metabolic activity in a voxelwise manner. PET analysis is commonly performed in a static manner by analyzing the standardized uptake value (SUV) obtained from the plateau region of PET acquisitions. A dynamic PET acquisition can provide a map of the spatiotemporal concentration of the tracer in vivo, hence conveying information about radiotracer delivery to tissue, its interaction with the target and washout. Therefore, tissue-specific biochemical properties are embedded in the shape of time activity curves (TACs), which are generally used for kinetic analysis. Conventionally, TACs are employed along with information about blood plasma activity concentration, i.e., the arterial input function (AIF), and specific compartmental models to obtain a full quantitative analysis of PET data. The main drawback of this approach is the need for invasive procedures requiring arterial blood sample collection during the whole PET scan. In this paper, we address the challenge of improving PET diagnostic accuracy through an alternative approach based on the analysis of time signal intensity patterns. Specifically, we demonstrate the diagnostic potential of tissue TACs provided by dynamic PET acquisition using various deep learning models. Our framework is shown to outperform the discriminative potential of classical SUV analysis, hence paving the way for more accurate PET-based lesion discrimination without additional acquisition time or invasive procedures. Clinical Relevance- The diagnostic accuracy of dynamic PET data exploited by deep-learning based time signal intensity pattern analysis is superior to that of static SUV imaging.