Multimodal MALDI imaging mass spectrometry for improved diagnosis of melanoma.

Imaging mass spectrometry (IMS) provides promising avenues to augment histopathological investigation with rich spatio-molecular information. We have previously developed a classification model to differentiate melanoma from nevi lesions based on IMS protein data, a task that is challenging solely by histopathologic evaluation. Most IMS-focused studies collect microscopy in tandem with IMS data, but this microscopy data is generally omitted in downstream data analysis. Microscopy, nevertheless, forms the basis for traditional histopathology and thus contains invaluable morphological information. In this work, we developed a multimodal classification pipeline that uses deep learning, in the form of a pre-trained artificial neural network, to extract the meaningful morphological features from histopathological images, and combine it with the IMS data. To test whether this deep learning-based classification strategy can improve on our previous results in classification of melanocytic neoplasia, we utilized MALDI IMS data with collected serial H&E stained sections for 331 patients, and compared this multimodal classification pipeline to classifiers using either exclusively microscopy or IMS data. The multimodal pipeline achieved the best performance, with ROC-AUCs of 0.968 vs. 0.938 vs. 0.931 for the multimodal, unimodal microscopy and unimodal IMS pipelines respectively. Due to the use of a pre-trained network to perform the morphological feature extraction, this pipeline does not require any training on large amounts of microscopy data. As such, this framework can be readily applied to improve classification performance in other experimental settings where microscopy data is acquired in tandem with IMS experiments.

Topological-Preserving Membrane Skeleton Segmentation in Multiplex Immunofluorescence Imaging.

Multiplex immunofluorescence (MxIF) is an emerging imaging technology whose downstream molecular analytics highly rely upon the effectiveness of cell segmentation. In practice, multiple membrane markers (e.g., NaKATPase, PanCK and ß-catenin) are employed to stain membranes for different cell types, so as to achieve a more comprehensive cell segmentation since no single marker fits all cell types. However, prevalent watershed-based image processing might yield inferior capability for modeling complicated relationships between markers. For example, some markers can be misleading due to questionable stain quality. In this paper, we propose a deep learning based membrane segmentation method to aggregate complementary information that is uniquely provided by large scale MxIF markers. We aim to segment tubular membrane structure in MxIF data using global (membrane markers z-stack projection image) and local (separate individual markers) information to maximize topology preservation with deep learning. Specifically, we investigate the feasibility of four SOTA 2D deep networks and four volumetric-based loss functions. We conducted a comprehensive ablation study to assess the sensitivity of the proposed method with various combinations of input channels. Beyond using adjusted rand index (ARI) as the evaluation metric, which was inspired by the clDice, we propose a novel volumetric metric that is specific for skeletal structure, denoted as clDiceSKEL. In total, 80 membrane MxIF images were manually traced for 5-fold cross-validation. Our model outperforms the baseline with a 20.2% and 41.3% increase in clDiceSKEL and ARI performance, which is significant (p<0.05) using the Wilcoxon signed rank test. Our work explores a promising direction for advancing MxIF imaging cell segmentation with deep learning membrane segmentation. Tools are available at https://github.com/MASILab/MxIF_Membrane_Segmentation.

Addressing persistent challenges in digital image analysis of cancerous tissues.

The National Cancer Institute (NCI) supports many research programs and consortia, many of which use imaging as a major modality for characterizing cancerous tissue. A trans-consortia Image Analysis Working Group (IAWG) was established in 2019 with a mission to disseminate imaging-related work and foster collaborations. In 2022, the IAWG held a virtual hackathon focused on addressing challenges of analyzing high dimensional datasets from fixed cancerous tissues. Standard image processing techniques have automated feature extraction, but the next generation of imaging data requires more advanced methods to fully utilize the available information. In this perspective, we discuss current limitations of the automated analysis of multiplexed tissue images, the first steps toward deeper understanding of these limitations, what possible solutions have been developed, any new or refined approaches that were developed during the Image Analysis Hackathon 2022, and where further effort is required. The outstanding problems addressed in the hackathon fell into three main themes: 1) challenges to cell type classification and assessment, 2) translation and visual representation of spatial aspects of high dimensional data, and 3) scaling digital image analyses to large (multi-TB) datasets. We describe the rationale for each specific challenge and the progress made toward addressing it during the hackathon. We also suggest areas that would benefit from more focus and offer insight into broader challenges that the community will need to address as new technologies are developed and integrated into the broad range of image-based modalities and analytical resources already in use within the cancer research community.

Segmentation of human functional tissue units in support of a Human Reference Atlas.

The Human BioMolecular Atlas Program (HuBMAP) aims to compile a Human Reference Atlas (HRA) for the healthy adult body at the cellular level. Functional tissue units (FTUs), relevant for HRA construction, are of pathobiological significance. Manual segmentation of FTUs does not scale; highly accurate and performant, open-source machine-learning algorithms are needed. We designed and hosted a Kaggle competition that focused on development of such algorithms and 1200 teams from 60 countries participated. We present the competition outcomes and an expanded analysis of the winning algorithms on additional kidney and colon tissue data, and conduct a pilot study to understand spatial location and density of FTUs across the kidney. The top algorithm from the competition, Tom, outperforms other algorithms in the expanded study, while using fewer computational resources. Tom was added to the HuBMAP infrastructure to run kidney FTU segmentation at scale-showcasing the value of Kaggle competitions for advancing research.

High-Resolution Imaging Mass Spectrometry of Human Donor Eye: Photoreceptors Cells and Basal Laminar Deposit of Age-Related Macular Degeneration.

Pathologies of the retina are clinically visualized in vivo with OCT and ex vivo with immunohistochemistry. Although both techniques provide valuable information on prognosis and disease state, a comprehensive method for fully elucidating molecular constituents present in locations of interest is desirable. The purpose of this work was to use multimodal imaging technologies to localize the vast number of molecular species observed with matrix-assisted laser desorption ionization imaging mass spectrometry (MALDI IMS) in aged and diseased retinal tissues. Herein, MALDI IMS was utilized to observe molecular species that reside in photoreceptor cells and also a basal laminar deposit from two human donor eyes. The molecular species observed to accumulate in these discrete regions can be further identified and studied to attempt to gain a greater understanding of biological processes occurring in debilitating eye diseases such as age-related macular degeneration (AMD).

Microscopy-Directed Imaging Mass Spectrometry for Rapid High Spatial Resolution Molecular Imaging of Glomeruli.

The glomerulus is a multicellular functional tissue unit (FTU) of the nephron that is responsible for blood filtration. Each glomerulus contains multiple substructures and cell types that are crucial for their function. To understand normal aging and disease in kidneys, methods for high spatial resolution molecular imaging within these FTUs across whole slide images is required. Here we demonstrate a workflow using microscopy-driven selected sampling to enable 5 µm pixel size matrix-assisted laser desorption/ionization imaging mass spectrometry (MALDI IMS) of all glomeruli within whole slide human kidney tissues. Such high spatial resolution imaging entails large numbers of pixels, increasing the data acquisition times. Automating FTU-specific tissue sampling enables high-resolution analysis of critical tissue structures, while concurrently maintaining throughput. Glomeruli were automatically segmented using coregistered autofluorescence microscopy data, and these segmentations were translated into MALDI IMS measurement regions. This allowed high-throughput acquisition of 268 glomeruli from a single whole slide human kidney tissue section. Unsupervised machine learning methods were used to discover molecular profiles of glomerular subregions and differentiate between healthy and diseased glomeruli. Average spectra for each glomerulus were analyzed using Uniform Manifold Approximation and Projection (UMAP) and k-means clustering, yielding 7 distinct groups of differentiated healthy and diseased glomeruli. Pixel-wise k-means clustering was applied to all glomeruli, showing unique molecular profiles localized to subregions within each glomerulus. Automated microscopy-driven, FTU-targeted acquisition for high spatial resolution molecular imaging maintains high-throughput and enables rapid assessment of whole slide images at cellular resolution and identification of tissue features associated with normal aging and disease.

Prospective on Imaging Mass Spectrometry in Clinical Diagnostics.

Imaging mass spectrometry (IMS) is a molecular technology utilized for spatially driven research, providing molecular maps from tissue sections. This article reviews matrix-assisted laser desorption ionization (MALDI) IMS and its progress as a primary tool in the clinical laboratory. MALDI mass spectrometry has been used to classify bacteria and perform other bulk analyses for plate-based assays for many years. However, the clinical application of spatial data within a tissue biopsy for diagnoses and prognoses is still an emerging opportunity in molecular diagnostics. This work considers spatially driven mass spectrometry approaches for clinical diagnostics and addresses aspects of new imaging-based assays that include analyte selection, quality control/assurance metrics, data reproducibility, data classification, and data scoring. It is necessary to implement these tasks for the rigorous translation of IMS to the clinical laboratory; however, this requires detailed standardized protocols for introducing IMS into the clinical laboratory to deliver reliable and reproducible results that inform and guide patient care.

MALDI IMS-Derived Molecular Contour Maps: Augmenting Histology Whole-Slide Images.

Imaging mass spectrometry (IMS) provides untargeted, highly multiplexed maps of molecular distributions in tissue. Ion images are routinely presented as heatmaps and can be overlaid onto complementary microscopy images that provide greater context. However, heatmaps use transparency blending to visualize both images, obscuring subtle quantitative differences and distribution gradients. Here, we developed a contour mapping approach that combines information from IMS ion intensity distributions with that of stained microscopy. As a case study, we applied this approach to imaging data from Staphylococcus aureus-infected murine kidney. In a univariate, or single molecular species, use-case of the contour map representation of IMS data, certain lipids colocalizing with regions of infection were selected using Pearson's correlation coefficient. Contour maps of these lipids overlaid with stained microscopy showed enhanced visualization of lipid distributions and spatial gradients in and around the bacterial abscess as compared to traditional heatmaps. The full IMS data set comprising hundreds of individual ion images was then grouped into a smaller subset of representative patterns using non-negative matrix factorization (NMF). Contour maps of these multivariate NMF images revealed distinct molecular profiles of the major abscesses and surrounding immune response. This contour mapping workflow also enabled a molecular visualization of the transition zone at the host-pathogen interface, providing potential clues about the spatial molecular dynamics beyond what histological staining alone provides. In summary, we developed a new IMS-based contour mapping approach to augment classical stained microscopy images, providing an enhanced and more interpretable visualization of IMS-microscopy multimodal molecular imaging data sets.

Rapid Multivariate Analysis Approach to Explore Differential Spatial Protein Profiles in Tissue.

Spatially targeted proteomics analyzes the proteome of specific cell types and functional regions within tissue. While spatial context is often essential to understanding biological processes, interpreting sub-region-specific protein profiles can pose a challenge due to the high-dimensional nature of the data. Here, we develop a multivariate approach for rapid exploration of differential protein profiles acquired from distinct tissue regions and apply it to analyze a published spatially targeted proteomics data set collected from Staphylococcus aureus-infected murine kidney, 4 and 10 days postinfection. The data analysis process rapidly filters high-dimensional proteomic data to reveal relevant differentiating species among hundreds to thousands of measured molecules. We employ principal component analysis (PCA) for dimensionality reduction of protein profiles measured by microliquid extraction surface analysis mass spectrometry. Subsequently, k-means clustering of the PCA-processed data groups samples by chemical similarity. Cluster center interpretation revealed a subset of proteins that differentiate between spatial regions of infection over two time points. These proteins appear involved in tricarboxylic acid metabolomic pathways, calcium-dependent processes, and cytoskeletal organization. Gene ontology analysis further uncovered relationships to tissue damage/repair and calcium-related defense mechanisms. Applying our analysis in infectious disease highlighted differential proteomic changes across abscess regions over time, reflecting the dynamic nature of host-pathogen interactions.

Lysolipids are prominent in subretinal drusenoid deposits, a high-risk phenotype in age-related macular degeneration.

Introduction: Age related macular degeneration (AMD) causes legal blindness worldwide, with few therapeutic targets in early disease and no treatments for 80% of cases. Extracellular deposits, including drusen and subretinal drusenoid deposits (SDD; also called reticular pseudodrusen), disrupt cone and rod photoreceptor functions and strongly confer risk for advanced disease. Due to the differential cholesterol composition of drusen and SDD, lipid transfer and cycling between photoreceptors and support cells are candidate dysregulated pathways leading to deposit formation. The current study explores this hypothesis through a comprehensive lipid compositional analysis of SDD. Methods: Histology and transmission electron microscopy were used to characterize the morphology of SDD. Highly sensitive tools of imaging mass spectrometry (IMS) and nano liquid chromatography tandem mass spectrometry (nLC-MS/MS) in positive and negative ion modes were used to spatially map and identify SDD lipids, respectively. An interpretable supervised machine learning approach was utilized to compare the lipid composition of SDD to regions of uninvolved retina across 1873 IMS features and to automatically discern candidate markers for SDD. Immunohistochemistry (IHC) was used to localize secretory phospholipase A2 group 5 (PLA2G5). Results: Among the 1873 detected features in IMS data, three lipid classes, including lysophosphatidylcholine (LysoPC), lysophosphatidylethanolamine (LysoPE) and lysophosphatidic acid (LysoPA) were observed nearly exclusively in SDD while presumed precursors, including phosphatidylcholine (PC), phosphatidylethanolamine (PE) and phosphatidic acid (PA) lipids were detected in SDD and adjacent photoreceptor outer segments. Molecular signals specific to SDD were found in central retina and elsewhere. IHC results indicated abundant PLA2G5 in photoreceptors and retinal pigment epithelium (RPE). Discussion: The abundance of lysolipids in SDD implicates lipid remodeling or degradation in deposit formation, consistent with ultrastructural evidence of electron dense lipid-containing structures distinct from photoreceptor outer segment disks and immunolocalization of secretory PLA2G5 in photoreceptors and RPE. Further studies are required to understand the role of lipid signals observed in and around SDD.

Highly multiplexed immunofluorescence of the human kidney using co-detection by indexing.

The human kidney is composed of many cell types that vary in their abundance and distribution from normal to diseased organ. As these cell types perform unique and essential functions, it is important to confidently label each within a single tissue to accurately assess tissue architecture and microenvironments. Towards this goal, we demonstrate the use of co-detection by indexing (CODEX) multiplexed immunofluorescence for visualizing 23 antigens within the human kidney. Using CODEX, many of the major cell types and substructures, such as collecting ducts, glomeruli, and thick ascending limb, were visualized within a single tissue section. Of these antibodies, 19 were conjugated in-house, demonstrating the flexibility and utility of this approach for studying the human kidney using custom and commercially available antibodies. We performed a pilot study that compared both fresh frozen and formalin-fixed paraffin-embedded healthy non-neoplastic and diabetic nephropathy kidney tissues. The largest cellular differences between the two groups was observed in cells labeled with aquaporin 1, cytokeratin 7, and α-smooth muscle actin. Thus, our data show the power of CODEX multiplexed immunofluorescence for surveying the cellular diversity of the human kidney and the potential for applications within pathology, histology, and building anatomical atlases.

Tissue fixation effects on human retinal lipid analysis by MALDI imaging and LC-MS/MS technologies.

Imaging mass spectrometry (IMS) allows the location and abundance of lipids to be mapped across tissue sections of human retina. For reproducible and accurate information, sample preparation methods need to be optimized. Paraformaldehyde fixation of a delicate multilayer structure like human retina facilitates the preservation of tissue morphology by forming methylene bridge crosslinks between formaldehyde and amine/thiols in biomolecules; however, retina sections analyzed by IMS are typically fresh-frozen. To determine if clinically significant inferences could be reliably based on fixed tissue, we evaluated the effect of fixation on analyte detection, spatial localization, and introduction of artifactual signals. Hence, we assessed the molecular identity of lipids generated by matrix-assisted laser desorption ionization (MALDI-IMS) and liquid chromatography coupled tandem mass spectrometry (LC-MS/MS) for fixed and fresh-frozen retina tissues in positive and negative ion modes. Based on MALDI-IMS analysis, more lipid signals were observed in fixed compared with fresh-frozen retina. More potassium adducts were observed in fresh-frozen tissues than fixed as the fixation process caused displacement of potassium adducts to protonated and sodiated species in ion positive ion mode. LC-MS/MS analysis revealed an overall decrease in lipid signals due to fixation that reduced glycerophospholipids and glycerolipids and conserved most sphingolipids and cholesteryl esters. The high quality and reproducible information from untargeted lipidomics analysis of fixed retina informs on all major lipid classes, similar to fresh-frozen retina, and serves as a steppingstone towards understanding of lipid alterations in retinal diseases.

Automated biomarker candidate discovery in imaging mass spectrometry data through spatially localized Shapley additive explanations.

The search for molecular species that are differentially expressed between biological states is an important step towards discovering promising biomarker candidates. In imaging mass spectrometry (IMS), performing this search manually is often impractical due to the large size and high-dimensionality of IMS datasets. Instead, we propose an interpretable machine learning workflow that automatically identifies biomarker candidates by their mass-to-charge ratios, and that quantitatively estimates their relevance to recognizing a given biological class using Shapley additive explanations (SHAP). The task of biomarker candidate discovery is translated into a feature ranking problem: given a classification model that assigns pixels to different biological classes on the basis of their mass spectra, the molecular species that the model uses as features are ranked in descending order of relative predictive importance such that the top-ranking features have a higher likelihood of being useful biomarkers. Besides providing the user with an experiment-wide measure of a molecular species' biomarker potential, our workflow delivers spatially localized explanations of the classification model's decision-making process in the form of a novel representation called SHAP maps. SHAP maps deliver insight into the spatial specificity of biomarker candidates by highlighting in which regions of the tissue sample each feature provides discriminative information and in which regions it does not. SHAP maps also enable one to determine whether the relationship between a biomarker candidate and a biological state of interest is correlative or anticorrelative. Our automated approach to estimating a molecular species' potential for characterizing a user-provided biological class, combined with the untargeted and multiplexed nature of IMS, allows for the rapid screening of thousands of molecular species and the obtention of a broader biomarker candidate shortlist than would be possible through targeted manual assessment. Our biomarker candidate discovery workflow is demonstrated on mouse-pup and rat kidney case studies.

Protocol for multimodal analysis of human kidney tissue by imaging mass spectrometry and CODEX multiplexed immunofluorescence.

Here, we describe the preservation and preparation of human kidney tissue for interrogation by histopathology, imaging mass spectrometry, and multiplexed immunofluorescence. Custom image registration and integration techniques are used to create cellular and molecular atlases of this organ system. Through careful optimization, we ensure high-quality and reproducible datasets suitable for cross-patient comparisons that are essential to understanding human health and disease. Moreover, each of these steps can be adapted to other organ systems or diseases, enabling additional atlas efforts.

α-Cyano-4-hydroxycinnamic Acid and Tri-Potassium Citrate Salt Pre-Coated Silicon Nanopost Array Provides Enhanced Lipid Detection for High Spatial Resolution MALDI Imaging Mass Spectrometry.

We have developed a pre-coated substrate for matrix-assisted laser desorption/ionization (MALDI) imaging mass spectrometry (IMS) that enables high spatial resolution mapping of both phospholipids and neutral lipid classes in positive ion mode as metal cation adducts. The MALDI substrates are constructed by depositing a layer of α-cyano-4-hydroxycinnamic acid (CHCA) and potassium salts onto silicon nanopost arrays (NAPA) prior to tissue mounting. The matrix/salt pre-coated NAPA substrate significantly enhances all detected lipid signals allowing lipids to be detected at lower laser energies than bare NAPA. The improved sensitivity at lower laser energy enabled ion images to be generated at 10 µm spatial resolution from rat retinal tissue. Optimization of matrix pre-coated NAPA consisted of testing lithium, sodium, and potassium salts along with various matrices to investigate the increased sensitivity toward lipids for MALDI IMS experiments. It was determined that pre-coating NAPA with CHCA and potassium salts before thaw-mounting of tissue resulted in a signal intensity increase of at least 5.8 ± 0.1-fold for phospholipids and 2.0 ± 0.1-fold for neutral lipids compared to bare NAPA. Pre-coating NAPA with matrix and salt also reduced the necessary laser power to achieve desorption/ionization by ∼35%. This reduced the effective diameter of the ablation area from 13 ± 2 µm down to 8 ± 1 µm, enabling high spatial resolution MALDI IMS. Using pre-coated NAPA with CHCA and potassium salts offers a MALDI IMS substrate with broad molecular coverage of lipids in a single polarity that eliminates the need for extensive sample preparation after sectioning.

Diagnosis of melanoma by imaging mass spectrometry: Development and validation of a melanoma prediction model.

BACKGROUND: The definitive diagnosis of melanocytic neoplasia using solely histopathologic evaluation can be challenging. Novel techniques that objectively confirm diagnoses are needed. This study details the development and validation of a melanoma prediction model from spatially resolved multivariate protein expression profiles generated by imaging mass spectrometry (IMS). METHODS: Three board-certified dermatopathologists blindly evaluated 333 samples. Samples with triply concordant diagnoses were included in this study, divided into a training set (n = 241) and a test set (n = 92). Both the training and test sets included various representative subclasses of unambiguous nevi and melanomas. A prediction model was developed from the training set using a linear support vector machine classification model. RESULTS: We validated the prediction model on the independent test set of 92 specimens (75 classified correctly, 2 misclassified, and 15 indeterminate). IMS detects melanoma with a sensitivity of 97.6% and a specificity of 96.4% when evaluating each unique spot. IMS predicts melanoma at the sample level with a sensitivity of 97.3% and a specificity of 97.5%. Indeterminate results were excluded from sensitivity and specificity calculations. CONCLUSION: This study provides evidence that IMS-based proteomics results are highly concordant to diagnostic results obtained by careful histopathologic evaluation from a panel of expert dermatopathologists.

An orthogonal methods assessment of topical drug concentrations in skin and the impact for risk assessment in the viable epidermis.

Systemic toxicity assessments for oral or parenteral drugs often utilize the concentration of drug in plasma to enable safety margin calculations for human risk assessment. For topical drugs, there is no standard method for measuring drug concentrations in the stratum basale of the viable epidermis. This is particularly important since the superficial part of the epidermis, the stratum corneum (SC), is nonviable and where most of a topically applied drug remains, never penetrating deeper into the skin. We investigated the relative concentrations of a prototype kinase inhibitor using punch biopsy, laser capture microdissection, and imaging mass spectrometry methods in the SC, stratum basale, and dermis of minipig skin following topical application as a cream formulation. The results highlight the value of laser capture microdissection and mass spectrometry imaging in quantifying the large difference in drug concentration across the skin and even within the epidermis, and supports use of these methods for threshold-based toxicity risk assessments in specific anatomic locations of the skin, like of the stratum basale.

Imaging Mass Spectrometry Is an Accurate Tool in Differentiating Clear Cell Renal Cell Carcinoma and Chromophobe Renal Cell Carcinoma: A Proof-of-concept Study.

Clear cell renal cell carcinoma (ccRCC) and chromophobe renal cell carcinoma (chRCC) are relatively common tumors that can have significant risk for mortality. Treatment and prognostication in renal cell carcinoma (RCC) are dependent upon correct histologic typing. ccRCC and chRCC are generally straightforward to diagnose based on histomorphology alone. However, high-grade ccRCC and chRCC can sometimes resemble each other morphologically, particularly in small biopsies. Multiple immunostains and/or colloidal iron stain are sometimes required to differentiate the two. Imaging mass spectrometry (IMS) allows simultaneous spatial mapping of thousands of biomarkers, using formalin-fixed paraffin-embedded tissue sections. In this study, we evaluate the ability of IMS to differentiate between World Health Organization/International Society for Urological Pathology grade 3 ccRCC and chRCC. IMS spectra from a training set of 14 ccRCC and 13 chRCC were evaluated via support vector machine algorithm with a linear kernel for machine learning, building a classification model. The classification model was applied to a separate validation set of 6 ccRCC and 6 chRCC, with 19 to 20, 150-µm diameter tumor foci in each case sampled by IMS. Most evaluated tumor foci were classified correctly as ccRCC versus chRCC (99% accuracy, kappa=0.98), demonstrating that IMS is an accurate tool in differentiating high-grade ccRCC and chRCC.