Exploring the Performance of Linear and Nonlinear Models of Time-of-Flight Secondary Ion Mass Spectrometry Spectra.

Multivariate statistical tools and machine learning (ML) techniques can deconvolute hyperspectral data and control the disparity between the number of samples and features in materials science. Nevertheless, the importance of generating sufficient high-quality sample replicates in training data cannot be overlooked, as it fundamentally affects the performance of ML models. Here, we present a quantitative analysis of time-of-flight secondary ion mass spectrometry (ToF-SIMS) spectra of a simple microarray system of two food dyes using partial least-squares (PLS, linear) and random forest (RF, nonlinear) algorithms. This microarray was generated by a high-throughput sample preparation and analysis workflow for fast and efficient acquisition of quality and reproducible spectra via ToF-SIMS. We drew insights from the bias-variance trade-off, investigated the performances of PLS and RF regression models as a function of training data size, and inferred the amount of data needed to construct accurate and reliable regression models. In addition, we found that the spectral concatenation of positive and negative ToF-SIMS spectra improved the model performances. This study provides an empirical basis for future design of high-throughput microarrays and multicomponent systems, for the purpose of analysis with ToF-SIMS and ML.

A Machine Learning-Driven Comparison of Ion Images Obtained by MALDI and MALDI-2 Mass Spectrometry Imaging.

Matrix-assisted laser desorption ionization mass spectrometry imaging (MALDI-MSI) enables label-free imaging of biomolecules in biological tissues. However, many species remain undetected due to their poor ionization efficiencies. MALDI-2 (laser-induced post-ionization) is the most widely used post-ionization method for improving analyte ionization efficiencies. Mass spectra acquired using MALDI-2 constitute a combination of ions generated by both MALDI and MALDI-2 processes. Until now, no studies have focused on a detailed comparison between the ion images (as opposed to the generated m/z values) produced by MALDI and MALDI-2 for mass spectrometry imaging (MSI) experiments. Herein, we investigated the ion images produced by both MALDI and MALDI-2 on the same tissue section using correlation analysis (to explore similarities in ion images for ions common to both MALDI and MALDI-2) and a deep learning approach. For the latter, we used an analytical workflow based on the Xception convolutional neural network, which was originally trained for human-like natural image classification but which we adapted to elucidate similarities and differences in ion images obtained using the two MSI techniques. Correlation analysis demonstrated that common ions yielded similar spatial distributions with low-correlation species explained by either poor signal intensity in MALDI or the generation of additional unresolved signals using MALDI-2. Using the Xception-based method, we identified many regions in the t-SNE space of spatially similar ion images containing MALDI and MALDI-2-related signals. More notably, the method revealed distinct regions containing only MALDI-2 ion images with unique spatial distributions that were not observed using MALDI. These data explicitly demonstrate the ability of MALDI-2 to reveal molecular features and patterns as well as histological regions of interest that are not visible when using conventional MALDI.

Markedly Enhanced Analysis of Mass Spectrometry Images Using Weakly Supervised Machine Learning.

Supervised and unsupervised machine learning algorithms are routinely applied to time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging data and, more broadly, to mass spectrometry imaging (MSI). These algorithms have accelerated large-scale, single-pixel analysis, classification, and regression. However, there is relatively little research on methods suited for so-called weakly supervised problems, where ground-truth class labels exist at the image level, but not at the individual pixel level. Unsupervised learning methods are usually applied to these problems. However, these methods cannot make use of available labels. Here a novel method specifically designed for weakly supervised MSI data is presented. A dual-stream multiple instance learning (MIL) approach is adapted from computational pathology that reveals the spatial-spectral characteristics distinguishing different classes of MSI images. The method uses an information entropy-regularized attention mechanism to identify characteristic class pixels that are then used to extract characteristic mass spectra. This work provides a proof-of-concept exemplification using printed ink samples imaged by ToF-SIMS. A second application-oriented study is also presented, focusing on the analysis of a mixed powder sample type. Results demonstrate the potential of the MIL method for broader application in MSI, with implications for understanding subtle spatial-spectral characteristics in various applications and contexts.

Comparison of Tiling Artifact Removal Methods in Secondary Ion Mass Spectrometry Images.

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging is used across many fields for the atomic and molecular characterization of surfaces, with both high sensitivity and high spatial resolution. When large analysis areas are required, standard ToF-SIMS instruments allow for the acquisition of adjoining tiles, which are acquired by rastering the primary ion beam. For such large area scans, tiling artifacts are a ubiquitous challenge, manifesting as intensity gradients across each tile and/or sudden changes in intensity between tiles. Such artifacts are thought to be related to a combination of sample charging, local detector sensitivity issues, and misalignment of the primary ion gun, among other instrumental factors. In this work, we investigated six different computational tiling artifact removal methods: tensor decomposition, multiplicative linear correction, linear discriminant analysis, seamless stitching, simple averaging, and simple interpolating. To ensure robustness in the study, we applied these methods to three hyperspectral ToF-SIMS data sets and one OrbiTrapSIMS data set. Our study includes a carefully designed statistical analysis and a quantitative survey that subjectively assessed the quality of the various methods employed. Our results demonstrate that while certain methods are useful and preferred more often, no one particular approach can be considered universally acceptable and that the effectiveness of the artifact removal method is strongly dependent on the particulars of the data set analyzed. As examples, the multiplicative linear correction and seamless stitching methods tended to score more highly on the subjective survey; however, for some data sets, this led to the introduction of new artifacts. In contrast, simple averaging and interpolation methods scored subjectively poorly on the biological data set, but more highly on the microarray data sets. We discuss and explore these findings in depth and present general recommendations given our findings to conclude the work.

Multi-Dimensional Machine Learning Analysis of Polyaniline Films Using Stitched Hyperspectral ToF-SIMS Data.

The self-organizing map with relational perspective mapping (SOM-RPM) is an unsupervised machine learning method that can be used to visualize and interpret high-dimensional hyperspectral data. We have previously used SOM-RPM for the analysis of time-of-flight secondary ion mass spectrometry (ToF-SIMS) hyperspectral images and three-dimensional (3D) depth profiles. This provides insightful visualization of features and trends of 3D depth profile data, using a slice-by-slice view, which can be useful for highlighting structural flaws including molecular characteristics and transport of contaminants to a buried interface and characterization of spectra. Here, we apply SOM-RPM to stitched ToF-SIMS data sets, whereby the stitched data are used to train the same model to provide a direct comparison in both 2D and 3D. We conduct an analysis of spin-coated polyaniline (PANI) films on indium tin oxide-coated glass slides that were subjected to heat treatment under atmospheric conditions to model PANI as a conformal aerospace industry coating. Replicates were shown to be precisely equivalent, both spatially and by composition, indicating a clear threshold for annealing of the film. Quantitative assessment was performed on the chemical breakdown trends accompanying annealing based on peak ratios, while spectral analysis alone shows only very subtle differences which are difficult to evaluate quantitatively. The SOM-RPM method considers data sets in their totality and highlights subtle differences between samples often simply differences in peak intensity ratios.

Two-Dimensional and Three-Dimensional Time-of-Flight Secondary Ion Mass Spectrometry Image Feature Extraction Using a Spatially Aware Convolutional Autoencoder.

Feature extraction algorithms are an important class of unsupervised methods used to reduce data dimensionality. They have been applied extensively for time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging─commonly, matrix factorization (MF) techniques such as principal component analysis have been used. A limitation of MF is the assumption of linearity, which is generally not accurate for ToF-SIMS data. Recently, nonlinear autoencoders have been shown to outperform MF techniques for ToF-SIMS image feature extraction. However, another limitation of most feature extraction methods (including autoencoders) that is particularly important for hyperspectral data is that they do not consider spatial information. To address this limitation, we describe the application of the convolutional autoencoder (CNNAE) to hyperspectral ToF-SIMS imaging data. The CNNAE is an artificial neural network developed specifically for hyperspectral data that uses convolutional layers for image encoding, thereby explicitly incorporating pixel neighborhood information. We compared the performance of the CNNAE with other common feature extraction algorithms for two biological ToF-SIMS imaging data sets. We investigated the extracted features and used the dimensionality-reduced data to train additional ML algorithms. By converting two-dimensional convolutional layers to three-dimensional (3D), we also showed how the CNNAE can be extended to 3D ToF-SIMS images. In general, the CNNAE produced features with significantly higher contrast and autocorrelation than other techniques. Furthermore, histologically recognizable features in the data were more accurately represented. The extension of the CNNAE to 3D data also provided an important proof of principle for the analysis of more complex 3D data sets.

Applications of multivariate analysis and unsupervised machine learning to ToF-SIMS images of organic, bioorganic, and biological systems.

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging offers a powerful, label-free method for exploring organic, bioorganic, and biological systems. The technique is capable of very high spatial resolution, while also producing an enormous amount of information about the chemical and molecular composition of a surface. However, this information is inherently complex, making interpretation and analysis of the vast amount of data produced by a single ToF-SIMS experiment a considerable challenge. Much research over the past few decades has focused on the application and development of multivariate analysis (MVA) and machine learning (ML) techniques that find meaningful patterns and relationships in these datasets. Here, we review the unsupervised algorithms-that is, algorithms that do not require ground truth labels-that have been applied to ToF-SIMS images, as well as other algorithms and approaches that have been used in the broader family of mass spectrometry imaging (MSI) techniques. We first give a nontechnical overview of several commonly used classes of unsupervised algorithms, such as matrix factorization, clustering, and nonlinear dimensionality reduction. We then review the application of unsupervised algorithms to various organic, bioorganic, and biological systems including cells and tissues, organic films, residues and coatings, and spatially structured systems such as polymer microarrays. We then cover several novel algorithms employed for other MSI techniques that have received little attention from ToF-SIMS imaging researchers. We conclude with a brief outline of potential future directions for the application of MVA and ML algorithms to ToF-SIMS images.

Understanding mass spectrometry images: complexity to clarity with machine learning.

The application of artificial intelligence and machine learning to hyperspectral mass spectrometry imaging (MSI) data has received considerable attention over recent years. Various methodologies have shown great promise in their ability to handle the complexity and size of MSI data sets. Advances in this area have been particularly appealing for MSI of biological samples, which typically produce highly complicated data with often subtle relationships between features. There are many different machine learning approaches that have been applied to MSI data over the past two decades. In this review, we focus on a subset of non-linear machine learning techniques that have mostly only been applied in the past 5 years. Specifically, we review the use of the self-organizing map (SOM), SOM with relational perspective mapping (SOM-RPM), t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP). While not their only functionality, we have grouped these techniques based on their ability to produce what we refer to as similarity maps. Similarity maps are color representations of hyperspectral data, in which spectral similarity between pixels-that is, their distance in high-dimensional space-is represented by relative color similarity. In discussing these techniques, we describe, briefly, their associated algorithms and functionalities, and also outline applications in MSI research with a strong focus on biological sample types. The aim of this review is therefore to introduce this relatively recent paradigm for visualizing and exploring hyperspectral MSI, while also providing a comparison between each technique discussed.

Analyzing 3D hyperspectral TOF-SIMS depth profile data using self-organizing map-relational perspective mapping.

The advantages of applying multivariate analysis to mass spectrometry imaging (MSI) data have been thoroughly demonstrated in recent decades. The identification and visualization of complex relationships between pixels in a hyperspectral data set can provide unique insights into the underlying surface chemistry. It is now recognized that most MSI data contain nonlinear relationships, which has led to increased application of machine learning approaches. Previously, we exemplified the use of the self-organizing map (SOM), a type of artificial neural network, for analyzing time-of-flight secondary ion mass spectrometry (TOF-SIMS) hyperspectral images. Recently, we developed a novel methodology, SOM-relational perspective mapping (RPM), which incorporates the algorithm RPM to improve visualization of the SOM for 2D TOF-SIMS images. Here, we use SOM-RPM to characterize and interpret 3D TOF-SIMS depth profile data, voxel-by-voxel. An organic Irganox™ multilayer standard sample was depth profiled using TOF-SIMS, and SOM-RPM was used to create 3D similarity maps of the depth-profiled sample, in which the mass spectral similarity of individual voxels is modeled with color similarity. We used this similarity map to segment the data into spatial features, demonstrating that the unsupervised method meaningfully differentiated between Irganox-3114 and Irganox-1010 nanometer-thin multilayer films. The method also identified unique clusters at the surface associated with environmental exposure and sample degradation. Key fragment ions characteristic of each cluster were identified, tying clusters to their underlying chemistries. SOM-RPM has the demonstrable ability to reduce vast data sets to simple 3D visualizations that can be used for clustering data and visualizing the complex relationships within.

Self-Organizing Map and Relational Perspective Mapping for the Accurate Visualization of High-Dimensional Hyperspectral Data.

We present an optimization of the toroidal self-organizing map (SOM) algorithm for the accurate visualization of hyperspectral data. This represents a significant advancement on our previous work, in which we demonstrated the use of toroidal SOMs for the visualization of time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging data. We have previously shown that the toroidal SOM can be used, unsupervised, to produce a multicolor similarity map of the analysis area, in which pixels with similar mass spectra are assigned a similar color. Here, we use an additional algorithm, relational perspective mapping (RPM), to produce more accurate visualizations of hyperspectral data. The SOM output is used as an input for the RPM algorithm, which is a nonlinear dimensionality reduction technique designed to produce a two-dimensional map of high-dimensional data. Using the topological information provided by the SOM, RPM provides complementary distance information. The result is a color scheme that more accurately reflects the local spectral distances between pixels in the data. We exemplify SOM-RPM using ToF-SIMS imaging data from a mouse tumor tissue section. The similarity maps produced are compared with those produced by two leading hyperspectral visualization techniques in the field of mass spectrometry imaging: t-distributed stochastic neighborhood embedding (t-SNE) and uniform manifold approximation and projection (UMAP). We evaluate the performance of each technique both qualitatively and quantitatively, investigating the correlations between distances in the models and distances in the data. SOM-RPM is demonstrably highly competitive with t-SNE and UMAP, according to our evaluations. Furthermore, the use of a neural network offers distinct advantages in data characterization, which we discuss. We also show how spectra extracted from regions of interest identified by SOM-RPM can be further analyzed using linear discriminant analysis for the validation and characterization of the surface chemistry.

ToF-SIMS and Machine Learning for Single-Pixel Molecular Discrimination of an Acrylate Polymer Microarray.

Combinatorial approaches to materials discovery offer promising potential for the rapid development of novel polymer systems. Polymer microarrays enable the high-throughput comparison of material physical and chemical properties-such as surface chemistry and properties like cell attachment or protein adsorption-in order to identify correlations that can progress materials development. A challenge for this approach is to accurately discriminate between highly similar polymer chemistries or identify heterogeneities within individual polymer spots. Time-of-flight secondary ion mass spectrometry (ToF-SIMS) offers unique potential in this regard, capable of describing the chemistry associated with the outermost layer of a sample with high spatial resolution and chemical sensitivity. However, this comes at the cost of generating large scale, complex hyperspectral imaging data sets. We have demonstrated previously that machine learning is a powerful tool for interpreting ToF-SIMS images, describing a method for color-tagging the output of a self-organizing map (SOM). This reduces the entire hyperspectral data set to a single reconstructed color similarity map, in which the spectral similarity between pixels is represented by color similarity in the map. Here, we apply the same methodology to a ToF-SIMS image of a printed polymer microarray for the first time. We report complete, single-pixel molecular discrimination of the 70 unique homopolymer spots on the array while also identifying intraspot heterogeneities thought to be related to intermixing of the polymer and the pHEMA coating. In this way, we show that the SOM can identify layers of similarity and clusters in the data, both with respect to polymer backbone structures and their individual side groups. Finally, we relate the output of the SOM analysis with fluorescence data from polymer-protein adsorption studies, highlighting how polymer performance can be visualized within the context of the global topology of the data set.

Visualizing ToF-SIMS Hyperspectral Imaging Data Using Color-Tagged Toroidal Self-Organizing Maps.

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) is a powerful surface characterization technique capable of producing high spatial resolution hyperspectral images, in which each pixel comprises an entire mass spectrum. Such images can provide insight into the chemical composition across a surface. However, issues arise due to the size and complexity of the data produced. Data are particularly complicated for biological samples, primarily due to overlapping spectra produced by similar components. The traditional approach of selecting individual ion peaks as representative of particular components is insufficient for such complex data sets. Multivariate analysis (MVA) can help to overcome this significant hurdle. We demonstrate that Kohonen self-organizing maps (SOMs) with a toroidal topology can be used to analyze a ToF-SIMS hyperspectral imaging data set and identify spectral similarities between pixels. We present a method for color-tagging the toroidal SOM output, which reduces the entire data set to a single RGB image in which similar pixels-based on their associated mass spectra-are assigned a similar color. This method was exemplified using a ToF-SIMS image of dried large multilamellar vesicles (LMVs), loaded with the antibiotic cefditoren pivoxil (CP). We successfully identified CP-loaded and empty LMVs without the need for any prior knowledge of the sample, despite their highly similar spectra. We also identified which specific ion peaks were most important in differentiating the two LMV populations. This approach is entirely unsupervised and requires minimal experimenter input. It was developed with the aim of providing a user-friendly yet sophisticated workflow for understanding complex biological samples using ToF-SIMS images.