ComPRePS: An Automated Cloud-based Image Analysis tool to democratize AI in Digital Pathology.

Artificial intelligence (AI) has extensive applications in a wide range of disciplines including healthcare and clinical practice. Advances in high-resolution whole-slide brightfield microscopy allow for the digitization of histologically stained tissue sections, producing gigapixel-scale whole-slide images (WSI). The significant improvement in computing and revolution of deep neural network (DNN)-based AI technologies over the last decade allow us to integrate massively parallelized computational power, cutting-edge AI algorithms, and big data storage, management, and processing. Applied to WSIs, AI has created opportunities for improved disease diagnostics and prognostics with the ultimate goal of enhancing precision medicine and resulting patient care. The National Institutes of Health (NIH) has recognized the importance of developing standardized principles for data management and discovery for the advancement of science and proposed the Findable, Accessible, Interoperable, Reusable, (FAIR) Data Principles1 with the goal of building a modernized biomedical data resource ecosystem to establish collaborative research communities. In line with this mission and to democratize AI-based image analysis in digital pathology, we propose ComPRePS: an end-to-end automated Computational Renal Pathology Suite which combines massive scalability, on-demand cloud computing, and an easy-to-use web-based user interface for data upload, storage, management, slide-level visualization, and domain expert interaction. Moreover, our platform is equipped with both in-house and collaborator developed sophisticated AI algorithms in the back-end server for image analysis to identify clinically relevant micro-anatomic functional tissue units (FTU) and to extract image features.

ComPRePS: An Automated Cloud-based Image Analysis tool to democratize AI in Digital Pathology.

Artificial intelligence (AI) has extensive applications in a wide range of disciplines including healthcare and clinical practice. Advances in high-resolution whole-slide brightfield microscopy allow for the digitization of histologically stained tissue sections, producing gigapixel-scale whole-slide images (WSI). The significant improvement in computing and revolution of deep neural network (DNN)-based AI technologies over the last decade allow us to integrate massively parallelized computational power, cutting-edge AI algorithms, and big data storage, management, and processing. Applied to WSIs, AI has created opportunities for improved disease diagnostics and prognostics with the ultimate goal of enhancing precision medicine and resulting patient care. The National Institutes of Health (NIH) has recognized the importance of developing standardized principles for data management and discovery for the advancement of science and proposed the Findable, Accessible, Interoperable, Reusable, (FAIR) Data Principles1 with the goal of building a modernized biomedical data resource ecosystem to establish collaborative research communities. In line with this mission and to democratize AI-based image analysis in digital pathology, we propose ComPRePS: an end-to-end automated Computational Renal Pathology Suite which combines massive scalability, on-demand cloud computing, and an easy-to-use web-based user interface for data upload, storage, management, slide-level visualization, and domain expert interaction. Moreover, our platform is equipped with both in-house and collaborator developed sophisticated AI algorithms in the back-end server for image analysis to identify clinically relevant micro-anatomic functional tissue units (FTU) and to extract image features.

Cosmic kidney disease: an integrated pan-omic, physiological and morphological study into spaceflight-induced renal dysfunction.

Missions into Deep Space are planned this decade. Yet the health consequences of exposure to microgravity and galactic cosmic radiation (GCR) over years-long missions on indispensable visceral organs such as the kidney are largely unexplored. We performed biomolecular (epigenomic, transcriptomic, proteomic, epiproteomic, metabolomic, metagenomic), clinical chemistry (electrolytes, endocrinology, biochemistry) and morphometry (histology, 3D imaging, miRNA-ISH, tissue weights) analyses using samples and datasets available from 11 spaceflight-exposed mouse and 5 human, 1 simulated microgravity rat and 4 simulated GCR-exposed mouse missions. We found that spaceflight induces: 1) renal transporter dephosphorylation which may indicate astronauts' increased risk of nephrolithiasis is in part a primary renal phenomenon rather than solely a secondary consequence of bone loss; 2) remodelling of the nephron that results in expansion of distal convoluted tubule size but loss of overall tubule density; 3) renal damage and dysfunction when exposed to a Mars roundtrip dose-equivalent of simulated GCR.

Generative Modeling of Histology Tissue Reduces Human Annotation Effort for Segmentation Model Development.

Segmentation of histology tissue whole side images is an important step for tissue analysis. Given enough annotated training data, modern neural networks are capable of accurate reproducible segmentation; however, the annotation of training datasets is time consuming. Techniques such as human-in-the-loop annotation attempt to reduce this annotation burden, but still require vast initial annotation. Semi-supervised learning-a technique which leverages both labeled and unlabeled data to learn features-has shown promise for easing the burden of annotation. Towards this goal, we employ a recently published semi-supervised method, datasetGAN, for the segmentation of glomeruli from renal biopsy images. We compare the performance of models trained using datasetGAN and traditional annotation and show that datasetGAN significantly reduces the amount of annotation required to develop a highly performing segmentation model. We also explore the usefulness of datasetGAN for transfer learning and find that this method greatly enhances the performance when a limited number of whole slide images are used for training.

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images.

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that suggest opportunities for future spatially aware WSI research using limited pathology datasets.

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images.

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that sugest opportunities for future spatially aware WSI research using limited pathology datasets.

Automated Reference Kidney Histomorphometry using a Panoptic Segmentation Neural Network Correlates to Patient Demographics and Creatinine.

Reference histomorphometric data of healthy human kidneys are lacking due to laborious quantitation requirements. We leveraged deep learning to investigate the relationship of histomorphometry with patient age, sex, and serum creatinine in a multinational set of reference kidney tissue sections. A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in digitized images of 79 periodic acid-Schiff (PAS)-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were measured from the segmented classes. Regression analysis was used to determine the relationship of histomorphometric parameters with age, sex, and serum creatinine. The model achieved high segmentation performance for all test compartments. We found that the size and density of nephrons, arteries/arterioles, and the baseline level of interstitium vary significantly among healthy humans, with potentially large differences between subjects from different geographic locations. Nephron size in any region of the kidney was significantly dependent on patient creatinine. Slight differences in renal vasculature and interstitium were observed between sexes. Finally, glomerulosclerosis percentage increased and cortical density of arteries/arterioles decreased as a function of age. We show that precise measurements of kidney histomorphometric parameters can be automated. Even in reference kidney tissue sections with minimal pathologic changes, several histomorphometric parameters demonstrated significant correlation to patient demographics and serum creatinine. These robust tools support the feasibility of deep learning to increase efficiency and rigor in histomorphometric analysis and pave the way for future large-scale studies.

Discovery of Novel Digital Biomarkers for Type 2 Diabetic Nephropathy Classification via Integration of Urinary Proteomics and Pathology.

Background: The heterogeneous phenotype of diabetic nephropathy (DN) from type 2 diabetes complicates appropriate treatment approaches and outcome prediction. Kidney histology helps diagnose DN and predict its outcomes, and an artificial intelligence (AI)-based approach will maximize clinical utility of histopathological evaluation. Herein, we addressed whether AI-based integration of urine proteomics and image features improves DN classification and its outcome prediction, altogether augmenting and advancing pathology practice. Methods: We studied whole slide images (WSIs) of periodic acid-Schiff-stained kidney biopsies from 56 DN patients with associated urinary proteomics data. We identified urinary proteins differentially expressed in patients who developed end-stage kidney disease (ESKD) within two years of biopsy. Extending our previously published human-AI-loop pipeline, six renal sub-compartments were computationally segmented from each WSI. Hand-engineered image features for glomeruli and tubules, and urinary protein measurements, were used as inputs to deep-learning frameworks to predict ESKD outcome. Differential expression was correlated with digital image features using the Spearman rank sum coefficient. Results: A total of 45 urinary proteins were differentially detected in progressors, which was most predictive of ESKD (AUC=0.95), while tubular and glomerular features were less predictive (AUC=0.71 and AUC=0.63, respectively). Accordingly, a correlation map between canonical cell-type proteins, such as epidermal growth factor and secreted phosphoprotein 1, and AI-based image features was obtained, which supports previous pathobiological results. Conclusions: Computational method-based integration of urinary and image biomarkers may improve the pathophysiological understanding of DN progression as well as carry clinical implications in histopathological evaluation.

Correlating Deep Learning-Based Automated Reference Kidney Histomorphometry with Patient Demographics and Creatinine.

Background: Reference histomorphometric data of healthy human kidneys are largely lacking due to laborious quantitation requirements. Correlating histomorphometric features with clinical parameters through machine learning approaches can provide valuable information about natural population variance. To this end, we leveraged deep learning, computational image analysis, and feature analysis to investigate the relationship of histomorphometry with patient age, sex, and serum creatinine (SCr) in a multinational set of reference kidney tissue sections. Methods: A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in the digitized images of 79 periodic acid-Schiff-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were quantified from the segmented classes. Regression analysis aided in determining the relationship of histomorphometric parameters with age, sex, and SCr. Results: Our deep-learning model achieved high segmentation performance for all test compartments. The size and density of nephrons and arteries/arterioles varied significantly among healthy humans, with potentially large differences between geographically diverse patients. Nephron size was significantly dependent on SCr. Slight, albeit significant, differences in renal vasculature were observed between sexes. Glomerulosclerosis percentage increased, and cortical density of arteries/arterioles decreased, as a function of age. Conclusions: Using deep learning, we automated precise measurements of kidney histomorphometric features. In the reference kidney tissue, several histomorphometric features demonstrated significant correlation to patient demographics and SCr. Deep learning tools can increase the efficiency and rigor of histomorphometric analysis.

Automated Tubular Morphometric Visualization for Whole Kidney Biopsy.

One of the strongest prognostic predictors of chronic kidney disease is interstitial fibrosis and tubular atrophy (IFTA). The ultimate goal of IFTA calculation is an estimation of the functional nephritic area. However, the clinical gold standard of estimation by pathologist is imprecise, primarily due to the overwhelming number of tubules sampled in a standard kidney biopsy. Artificial intelligence algorithms could provide significant benefit in this aspect as their high-throughput could identify and quantitatively measure thousands of tubules in mere minutes. Towards this goal, we use a custom panoptic convolutional network similar to Panoptic-DeepLab to detect tubules from 87 WSIs of biopsies from native diabetic kidneys and transplant kidneys. We measure 206 features on each tubule, including commonly understood features like tubular basement membrane thickness and tubular diameter. Finally, we have developed a tool which allows a user to select a range of tubule morphometric features to be highlighted in corresponding WSIs. The tool can also highlight tubules in WSI leveraging multiple morphometric features through selection of regions-of-interest in a uniform manifold approximation and projection plot.

Computational Integration of Renal Histology and Urinary Proteomics using Neural Networks.

Histological image data and molecular profiles provide context into renal condition. Often, a biopsy is drawn to diagnose or monitor a suspected kidney problem. However, molecular profiles can go beyond a pathologist's ability to see and diagnose. Using AI, we computationally incorporated urinary proteomic profiles with microstructural morphology from renal biopsy to investigate new and existing molecular links to image phenotypes. We studied whole slide images of periodic acid-Schiff stained renal biopsies from 56 DN patients matched with 2,038 proteins measured from each patient's urine. Using Seurat, we identified differentially expressed proteins in patients that developed end-stage renal disease within 2 years of biopsy. Glomeruli, globally sclerotic glomeruli, and tubules were segmented from WSI using our previously published HAIL pipeline. For each glomerulus, 315 handcrafted digital image features were measured, and for tubules, 207 features. We trained fully connected networks to predict urinary protein measurements that were differentially expressed between patients who did/ did not progress to ESRD within 2 years of biopsy. The input to this network was either glomerular or tubular histomorphological features in biopsy. Trained network weights were used as a proxy to rank which morphological features correlated most highly with specific urinary proteins. We identified significant image feature-protein pairs by ranking network weights by magnitude. We also looked at which features on average were most significant in predicting proteins. For both glomeruli and tubules, RGB color values and variance in PAS+ areas (specifically basement membrane for tubules) were, on average, more predictive of molecular profiles than other features. There is a strong connection between molecular profile and image phenotype, which can be elucidated through computational methods. These discovered links can provide insight to disease pathways, and discover new factors contributing to incidence and progression.

A Distributed System Improves Inter-Observer and AI Concordance in Annotating Interstitial Fibrosis and Tubular Atrophy.

Histologic examination of interstitial fibrosis and tubular atrophy (IFTA) is critical to determine the extent of irreversible kidney injury in renal disease. The current clinical standard involves pathologist's visual assessment of IFTA, which is prone to inter-observer variability. To address this diagnostic variability, we designed two case studies (CSs), including seven pathologists, using HistomicsTK- a distributed system developed by Kitware Inc. (Clifton Park, NY). Twenty-five whole slide images (WSIs) were classified into a training set of 21 and a validation set of four. The training set was composed of seven unique subsets, each provided to an individual pathologist along with four common WSIs from the validation set. In CS 1, all pathologists individually annotated IFTA in their respective slides. These annotations were then used to train a deep learning algorithm to computationally segment IFTA. In CS 2, manual and computational annotations from CS 1 were first reviewed by the annotators to improve concordance of IFTA annotation. Both the manual and computational annotation processes were then repeated as in CS1. The inter-observer concordance in the validation set was measured by Krippendorff's alpha (KA). The KA for the seven pathologists in CS1 was 0.62 with CI [0.57, 0.67], and after reviewing each other's annotations in CS2, 0.66 with CI [0.60, 0.72]. The respective CS1 and CS2 KA were 0.58 with CI [0.52, 0.64] and 0.63 with CI [0.56, 0.69] when including the deep learner as an eighth annotator. These results suggest that our designed annotation framework refines agreement of spatial annotation of IFTA and demonstrates a human-AI approach to significantly improve the development of computational models.