Graph neural network interatomic potential ensembles with calibrated aleatoric and epistemic uncertainty on energy and forces.

Inexpensive machine learning (ML) potentials are increasingly being used to speed up structural optimization and molecular dynamics simulations of materials by iteratively predicting and applying interatomic forces. In these settings, it is crucial to detect when predictions are unreliable to avoid wrong or misleading results. Here, we present a complete framework for training and recalibrating graph neural network ensemble models to produce accurate predictions of energy and forces with calibrated uncertainty estimates. The proposed method considers both epistemic and aleatoric uncertainty and the total uncertainties are recalibrated post hoc using a nonlinear scaling function to achieve good calibration on previously unseen data, without loss of predictive accuracy. The method is demonstrated and evaluated on two challenging, publicly available datasets, ANI-1x (Smith et al. J. Chem. Phys., 2018, 148, 241733.) and Transition1x (Schreiner et al. Sci. Data, 2022, 9, 779.), both containing diverse conformations far from equilibrium. A detailed analysis of the predictive performance and uncertainty calibration is provided. In all experiments, the proposed method achieved low prediction error and good uncertainty calibration, with predicted uncertainty correlating with expected error, on energy and forces. To the best of our knowledge, the method presented in this paper is the first to consider a complete framework for obtaining calibrated epistemic and aleatoric uncertainty predictions on both energy and forces in ML potentials.

Nitroaromatic explosives' detection and quantification using an attention-based transformer on surface-enhanced Raman spectroscopy maps.

Rapidly and accurately detecting and quantifying the concentrations of nitroaromatic explosives is critical for public health and security. Among existing approaches, explosives' detection with Surface-Enhanced Raman Spectroscopy (SERS) has received considerable attention due to its high sensitivity. Typically, a preprocessed single spectrum that is the average of the entire or a selected subset of a SERS map is used to train various machine learning models for detection and quantification. Designing an appropriate averaging and preprocessing procedure for SERS maps across different concentrations is time-consuming and computationally costly, and the averaging of spectra may lead to the loss of crucial spectral information. We propose an attention-based vision transformer neural network for nitroaromatic explosives' detection and quantification that takes raw SERS maps as the input without any preprocessing. We produce two novel SERS datasets, 2,4-dinitrophenols (DNP) and picric acid (PA), and one benchmark SERS dataset, 4-nitrobenzenethiol (4-NBT), which have repeated measurements down to concentrations of 1 nM to illustrate the detection limit. We experimentally show that our approach outperforms or is on par with the existing methods in terms of detection and concentration prediction accuracy. With the produced attention maps, we can further identify the regions with a higher signal-to-noise ratio in the SERS maps. Based on our findings, the molecule of interest detection and concentration prediction using raw SERS maps is a promising alternative to existing approaches.

Raman spectrum matching with contrastive representation learning.

Raman spectroscopy is an important, low-cost, non-intrusive technique often used for chemical identification. Typical approaches identify a spectrum by comparing it with a reference database using supervised machine learning, which usually requires careful preprocessing and multiple spectra available per analyte. We propose a new machine learning technique for spectrum identification using contrastive representation learning. Our approach requires no preprocessing and works with as little as a single reference spectrum per analyte. We have significantly improved or are on par with the existing state-of-the-art analyte identification accuracy on two Raman spectral datasets and one SERS dataset that include a single component. We demonstrate that the identification accuracy can be further increased by slightly increasing the candidate set size using conformal prediction on the SERS dataset. Based on our findings, we believe contrastive representation learning is a promising alternative to the existing methods for Raman spectrum matching.

Uncovering Cortical Units of Processing From Multi-Layered Connectomes.

Modern diffusion and functional magnetic resonance imaging (dMRI/fMRI) provide non-invasive high-resolution images from which multi-layered networks of whole-brain structural and functional connectivity can be derived. Unfortunately, the lack of observed correspondence between the connectivity profiles of the two modalities challenges the understanding of the relationship between the functional and structural connectome. Rather than focusing on correspondence at the level of connections we presently investigate correspondence in terms of modular organization according to shared canonical processing units. We use a stochastic block-model (SBM) as a data-driven approach for clustering high-resolution multi-layer whole-brain connectivity networks and use prediction to quantify the extent to which a given clustering accounts for the connectome within a modality. The employed SBM assumes a single underlying parcellation exists across modalities whilst permitting each modality to possess an independent connectivity structure between parcels thereby imposing concurrent functional and structural units but different structural and functional connectivity profiles. We contrast the joint processing units to their modality specific counterparts and find that even though data-driven structural and functional parcellations exhibit substantial differences, attributed to modality specific biases, the joint model is able to achieve a consensus representation that well accounts for both the functional and structural connectome providing improved representations of functional connectivity compared to using functional data alone. This implies that a representation persists in the consensus model that is shared by the individual modalities. We find additional support for this viewpoint when the anatomical correspondence between modalities is removed from the joint modeling. The resultant drop in predictive performance is in general substantial, confirming that the anatomical correspondence of processing units is indeed present between the two modalities. Our findings illustrate how multi-modal integration admits consensus representations well-characterizing each individual modality despite their biases and points to the importance of multi-layered connectomes as providing supplementary information regarding the brain's canonical processing units.

Using connectomics for predictive assessment of brain parcellations.

The organization of the human brain remains elusive, yet is of great importance to the mechanisms of integrative brain function. At the macroscale, its structural and functional interpretation is conventionally assessed at the level of cortical units. However, the definition and validation of such cortical parcellations are problematic due to the absence of a true gold standard. We propose a framework for quantitative evaluation of brain parcellations via statistical prediction of connectomics data. Specifically, we evaluate the extent in which the network representation at the level of cortical units (defined as parcels) accounts for high-resolution brain connectivity. Herein, we assess the pertinence and comparative ranking of ten existing parcellation atlases to account for functional (FC) and structural connectivity (SC) data based on data from the Human Connectome Project (HCP), and compare them to data-driven as well as spatially-homogeneous geometric parcellations including geodesic parcellations with similar size distributions as the atlases. We find substantial discrepancy in parcellation structures that well characterize FC and SC and differences in what well represents an individual's functional connectome when compared against the FC structure that is preserved across individuals. Surprisingly, simple spatial homogenous parcellations generally provide good representations of both FC and SC, but are inferior when their within-parcellation distribution of individual parcel sizes is matched to that of a valid atlas. This suggests that the choice of fine grained and coarse representations used by existing atlases are important. However, we find that resolution is more critical than the exact border location of parcels.

Validation of structural brain connectivity networks: The impact of scanning parameters.

Evaluation of the structural connectivity (SC) of the brain based on tractography has mainly focused on the choice of diffusion model, tractography algorithm, and their respective parameter settings. Here, we systematically validate SC derived from a post mortem monkey brain, while varying key acquisition parameters such as the b-value, gradient angular resolution and image resolution. As gold standard we use the connectivity matrix obtained invasively with histological tracers by Markov et al. (2014). As performance metric, we use cross entropy as a measure that enables comparison of the relative tracer labeled neuron counts to the streamline counts from tractography. We find that high angular resolution and high signal-to-noise ratio are important to estimate SC, and that SC derived from low image resolution (1.03 mm3) are in better agreement with the tracer network, than those derived from high image resolution (0.53 mm3) or at an even lower image resolution (2.03 mm3). In contradiction, sensitivity and specificity analyses suggest that if the angular resolution is sufficient, the balanced compromise in which sensitivity and specificity are identical remains 60-64% regardless of the other scanning parameters. Interestingly, the tracer graph is assumed to be the gold standard but by thresholding, the balanced compromise increases to 70-75%. Hence, by using performance metrics based on binarized tracer graphs, one risks losing important information, changing the performance of SC graphs derived by tractography and their dependence of different scanning parameters.

Deep Learning Spectroscopy: Neural Networks for Molecular Excitation Spectra.

Deep learning methods for the prediction of molecular excitation spectra are presented. For the example of the electronic density of states of 132k organic molecules, three different neural network architectures: multilayer perceptron (MLP), convolutional neural network (CNN), and deep tensor neural network (DTNN) are trained and assessed. The inputs for the neural networks are the coordinates and charges of the constituent atoms of each molecule. Already, the MLP is able to learn spectra, but the root mean square error (RMSE) is still as high as 0.3 eV. The learning quality improves significantly for the CNN (RMSE = 0.23 eV) and reaches its best performance for the DTNN (RMSE = 0.19 eV). Both CNN and DTNN capture even small nuances in the spectral shape. In a showcase application of this method, the structures of 10k previously unseen organic molecules are scanned and instant spectra predictions are obtained to identify molecules for potential applications.

Machine learning-based screening of complex molecules for polymer solar cells.

Polymer solar cells admit numerous potential advantages including low energy payback time and scalable high-speed manufacturing, but the power conversion efficiency is currently lower than for their inorganic counterparts. In a Phenyl-C_61-Butyric-Acid-Methyl-Ester (PCBM)-based blended polymer solar cell, the optical gap of the polymer and the energetic alignment of the lowest unoccupied molecular orbital (LUMO) of the polymer and the PCBM are crucial for the device efficiency. Searching for new and better materials for polymer solar cells is a computationally costly affair using density functional theory (DFT) calculations. In this work, we propose a screening procedure using a simple string representation for a promising class of donor-acceptor polymers in conjunction with a grammar variational autoencoder. The model is trained on a dataset of 3989 monomers obtained from DFT calculations and is able to predict LUMO and the lowest optical transition energy for unseen molecules with mean absolute errors of 43 and 74 meV, respectively, without knowledge of the atomic positions. We demonstrate the merit of the model for generating new molecules with the desired LUMO and optical gap energies which increases the chance of finding suitable polymers by more than a factor of five in comparison to the randomised search used in gathering the training set.

Deep Generative Models for Molecular Science.

Generative deep machine learning models now rival traditional quantum-mechanical computations in predicting properties of new structures, and they come with a significantly lower computational cost, opening new avenues in computational molecular science. In the last few years, a variety of deep generative models have been proposed for modeling molecules, which differ in both their model structure and choice of input features. We review these recent advances within deep generative models for predicting molecular properties, with particular focus on models based on the probabilistic autoencoder (or variational autoencoder, VAE) approach in which the molecular structure is embedded in a latent vector space from which its properties can be predicted and its structure can be restored.

Predictive assessment of models for dynamic functional connectivity.

In neuroimaging, it has become evident that models of dynamic functional connectivity (dFC), which characterize how intrinsic brain organization changes over time, can provide a more detailed representation of brain function than traditional static analyses. Many dFC models in the literature represent functional brain networks as a meta-stable process with a discrete number of states; however, there is a lack of consensus on how to perform model selection and learn the number of states, as well as a lack of understanding of how different modeling assumptions influence the estimated state dynamics. To address these issues, we consider a predictive likelihood approach to model assessment, where models are evaluated based on their predictive performance on held-out test data. Examining several prominent models of dFC (in their probabilistic formulations) we demonstrate our framework on synthetic data, and apply it on two real-world examples: a face recognition EEG experiment and resting-state fMRI. Our results evidence that both EEG and fMRI are better characterized using dynamic modeling approaches than by their static counterparts, but we also demonstrate that one must be cautious when interpreting dFC because parameter settings and modeling assumptions, such as window lengths and emission models, can have a large impact on the estimated states and consequently on the interpretation of the brain dynamics.

Infinite von Mises-Fisher Mixture Modeling of Whole Brain fMRI Data.

Cluster analysis of functional magnetic resonance imaging (fMRI) data is often performed using gaussian mixture models, but when the time series are standardized such that the data reside on a hypersphere, this modeling assumption is questionable. The consequences of ignoring the underlying spherical manifold are rarely analyzed, in part due to the computational challenges imposed by directional statistics. In this letter, we discuss a Bayesian von Mises-Fisher (vMF) mixture model for data on the unit hypersphere and present an efficient inference procedure based on collapsed Markov chain Monte Carlo sampling. Comparing the vMF and gaussian mixture models on synthetic data, we demonstrate that the vMF model has a slight advantage inferring the true underlying clustering when compared to gaussian-based models on data generated from both a mixture of vMFs and a mixture of gaussians subsequently normalized. Thus, when performing model selection, the two models are not in agreement. Analyzing multisubject whole brain resting-state fMRI data from healthy adult subjects, we find that the vMF mixture model is considerably more reliable than the gaussian mixture model when comparing solutions across models trained on different groups of subjects, and again we find that the two models disagree on the optimal number of components. The analysis indicates that the fMRI data support more than a thousand clusters, and we confirm this is not a result of overfitting by demonstrating better prediction on data from held-out subjects. Our results highlight the utility of using directional statistics to model standardized fMRI data and demonstrate that whole brain segmentation of fMRI data requires a very large number of functional units in order to adequately account for the discernible statistical patterns in the data.

Infinite-degree-corrected stochastic block model.

In stochastic block models, which are among the most prominent statistical models for cluster analysis of complex networks, clusters are defined as groups of nodes with statistically similar link probabilities within and between groups. A recent extension by Karrer and Newman [Karrer and Newman, Phys. Rev. E 83, 016107 (2011)] incorporates a node degree correction to model degree heterogeneity within each group. Although this demonstrably leads to better performance on several networks, it is not obvious whether modeling node degree is always appropriate or necessary. We formulate the degree corrected stochastic block model as a nonparametric Bayesian model, incorporating a parameter to control the amount of degree correction that can then be inferred from data. Additionally, our formulation yields principled ways of inferring the number of groups as well as predicting missing links in the network that can be used to quantify the model's predictive performance. On synthetic data we demonstrate that including the degree correction yields better performance on both recovering the true group structure and predicting missing links when degree heterogeneity is present, whereas performance is on par for data with no degree heterogeneity within clusters. On seven real networks (with no ground truth group structure available) we show that predictive performance is about equal whether or not degree correction is included; however, for some networks significantly fewer clusters are discovered when correcting for degree, indicating that the data can be more compactly explained by clusters of heterogenous degree nodes.

Non-parametric Bayesian graph models reveal community structure in resting state fMRI.

Modeling of resting state functional magnetic resonance imaging (rs-fMRI) data using network models is of increasing interest. It is often desirable to group nodes into clusters to interpret the communication patterns between nodes. In this study we consider three different nonparametric Bayesian models for node clustering in complex networks. In particular, we test their ability to predict unseen data and their ability to reproduce clustering across datasets. The three generative models considered are the Infinite Relational Model (IRM), Bayesian Community Detection (BCD), and the Infinite Diagonal Model (IDM). The models define probabilities of generating links within and between clusters and the difference between the models lies in the restrictions they impose upon the between-cluster link probabilities. IRM is the most flexible model with no restrictions on the probabilities of links between clusters. BCD restricts the between-cluster link probabilities to be strictly lower than within-cluster link probabilities to conform to the community structure typically seen in social networks. IDM only models a single between-cluster link probability, which can be interpreted as a background noise probability. These probabilistic models are compared against three other approaches for node clustering, namely Infomap, Louvain modularity, and hierarchical clustering. Using 3 different datasets comprising healthy volunteers' rs-fMRI we found that the BCD model was in general the most predictive and reproducible model. This suggests that rs-fMRI data exhibits community structure and furthermore points to the significance of modeling heterogeneous between-cluster link probabilities.

Nonlinear impairment compensation using expectation maximization for dispersion managed and unmanaged PDM 16-QAM transmission.

In this paper, we show numerically and experimentally that expectation maximization (EM) algorithm is a powerful tool in combating system impairments such as fibre nonlinearities, inphase and quadrature (I/Q) modulator imperfections and laser linewidth. The EM algorithm is an iterative algorithm that can be used to compensate for the impairments which have an imprint on a signal constellation, i.e. rotation and distortion of the constellation points. The EM is especially effective for combating non-linear phase noise (NLPN). It is because NLPN severely distorts the signal constellation and this can be tracked by the EM. The gain in the nonlinear system tolerance for the system under consideration is shown to be dependent on the transmission scenario. We show experimentally that for a dispersion managed polarization multiplexed 16-QAM system at 14 Gbaud a gain in the nonlinear system tolerance of up to 3 dB can be obtained. For, a dispersion unmanaged system this gain reduces to 0.5 dB.

Bayesian community detection.

Many networks of scientific interest naturally decompose into clusters or communities with comparatively fewer external than internal links; however, current Bayesian models of network communities do not exert this intuitive notion of communities. We formulate a nonparametric Bayesian model for community detection consistent with an intuitive definition of communities and present a Markov chain Monte Carlo procedure for inferring the community structure. A Matlab toolbox with the proposed inference procedure is available for download. On synthetic and real networks, our model detects communities consistent with ground truth, and on real networks, it outperforms existing approaches in predicting missing links. This suggests that community structure is an important structural property of networks that should be explicitly modeled.

Nonnegative matrix factorization with Gaussian process priors.

We present a general method for including prior knowledge in a nonnegative matrix factorization (NMF), based on Gaussian process priors. We assume that the nonnegative factors in the NMF are linked by a strictly increasing function to an underlying Gaussian process specified by its covariance function. This allows us to find NMF decompositions that agree with our prior knowledge of the distribution of the factors, such as sparseness, smoothness, and symmetries. The method is demonstrated with an example from chemical shift brain imaging.