Sex, racial, and APOE-ε4 allele differences in longitudinal white matter microstructure in multiple cohorts of aging and Alzheimer's disease.

INTRODUCTION: The effects of sex, race, and Apolipoprotein E (APOE) - Alzheimer's disease (AD) risk factors - on white matter integrity are not well characterized. METHODS: Diffusion MRI data from nine well-established longitudinal cohorts of aging were free-water (FW)-corrected and harmonized. This dataset included 4,702 participants (age=73.06 ± 9.75) with 9,671 imaging sessions over time. FW and FW-corrected fractional anisotropy (FAFWcorr) were used to assess differences in white matter microstructure by sex, race, and APOE-ε4 carrier status. RESULTS: Sex differences in FAFWcorr in association and projection tracts, racial differences in FAFWcorr in projection tracts, and APOE-ε4 differences in FW limbic and occipital transcallosal tracts were most pronounced. DISCUSSION: There are prominent differences in white matter microstructure by sex, race, and APOE-ε4 carrier status. This work adds to our understanding of disparities in AD. Additional work to understand the etiology of these differences is warranted.

MidRISH: Unbiased harmonization of rotationally invariant harmonics of the diffusion signal.

Data harmonization is necessary for removing confounding effects in multi-site diffusion image analysis. One such harmonization method, LinearRISH, scales rotationally invariant spherical harmonic (RISH) features from one site ("target") to the second ("reference") to reduce confounding scanner effects. However, reference and target site designations are not arbitrary and resultant diffusion metrics (fractional anisotropy, mean diffusivity) are biased by this choice. In this work we propose MidRISH: rather than scaling reference RISH features to target RISH features, we project both sites to a mid-space. We validate MidRISH with the following experiments: harmonizing scanner differences from 37 matched patients free of cognitive impairment, and harmonizing acquisition and study differences on 117 matched patients free of cognitive impairment. We find that MidRISH reduces bias of reference selection while preserving harmonization efficacy of LinearRISH. Users should be cautious when performing LinearRISH harmonization. To select a reference site is to choose diffusion metric effect-size. Our proposed method eliminates the bias-inducing site selection step.

DeepN4: Learning N4ITK Bias Field Correction for T1-weighted Images.

T1-weighted (T1w) MRI has low frequency intensity artifacts due to magnetic field inhomogeneities. Removal of these biases in T1w MRI images is a critical preprocessing step to ensure spatially consistent image interpretation. N4ITK bias field correction, the current state-of-the-art, is implemented in such a way that makes it difficult to port between different pipelines and workflows, thus making it hard to reimplement and reproduce results across local, cloud, and edge platforms. Moreover, N4ITK is opaque to optimization before and after its application, meaning that methodological development must work around the inhomogeneity correction step. Given the importance of bias fields correction in structural preprocessing and flexible implementation, we pursue a deep learning approximation / reinterpretation of the N4ITK bias fields correction to create a method which is portable, flexible, and fully differentiable. In this paper, we trained a deep learning network "DeepN4" on eight independent cohorts from 72 different scanners and age ranges with N4ITK-corrected T1w MRI and bias field for supervision in log space. We found that we can closely approximate N4ITK bias fields correction with naïve networks. We evaluate the peak signal to noise ratio (PSNR) in test dataset against the N4ITK corrected images. The median PSNR of corrected images between N4ITK and DeepN4 was 47.96 dB. In addition, we assess the DeepN4 model on eight additional external datasets and show the generalizability of the approach. This study establishes that incompatible N4ITK preprocessing steps can be closely approximated by naïve deep neural networks, facilitating more flexibility. All code and models are released at https://github.com/MASILab/DeepN4 .

Effect of Oral Prednisone on the Prevention and Management of Proliferative Vitreoretinopathy After Open-Globe Injury.

Purpose: To determine the impact of oral prednisone on the final visual acuity (VA) and prevention of proliferative vitreoretinopathy (PVR) in patients having pars plana vitrectomy (PPV) for globe injuries. Methods: A retrospective chart review was performed of all globe injuries with an initial repair and subsequent PPV between 2009 and 2018. Data included the initial VA, zones of injury, initial closure date, time to secondary intervention (PPV), oral prednisone (1 mg/kg/day) use, the final VA, and enucleation rate. Multivariable regression models were used to assess the impact of oral prednisone use on anatomic and functional outcomes. Results: The mean (±SD) patient age was 46.25 ±18.56 years (range, 13-92); 131 (83.9%) were men. Oral prednisone intake was recorded in 81 patients (52.3%). The prednisone group had significantly more zone 3 involvement (P = .001), worse initial VA (2.28 vs 1.92 logMAR; P = .003), and a greater mean number of surgeries (P = .020) than the no-steroids (control) group but an equivalent final logMAR VA (1.57 vs 1.52; P = .881). The prednisone group had significant VA improvement (P = .025); however, oral prednisone use did not predict the development of PVR (29.23% vs 12.90%; odds ratio [OR], 2.81; 95% CI, 0.89-8.85) or retinal detachment (27.27% vs 29.58%; OR, 0.59; 95% CI, 0.23-1.56). Conclusions: Despite a worse initial clinical presentation, patients who received oral prednisone had significant visual improvement compared with the control group. However, oral prednisone (1 mg/kg/day) use at the time of injury did not decrease the PVR rate.

Fractal Changes of the Retinal Microvasculature in Syphilitic Uveitis.

PURPOSE: To quantify chorioretinal microvascular damage and recovery post-treatment in patients with acute syphilitic posterior placoid chorioretinitis (ASPPC) using fractal dimension (FD). METHODS: Retrospective cohort study of patients with serologically confirmed syphilitic uveitis. We obtained optical coherence tomography angiography (OCTA) scans at baseline and follow-up after intravenous penicillin treatment and computed FD of the superficial capillary plexus (SCP), deep capillary plexus (DCP), and choriocapillaris (CC) using ImageJ. RESULTS: We enrolled seven patients with ASPPC (11 eyes), and 17 control subjects (34 eyes). Pre-treatment averages of FD-SCP, FD-DCP, and FD-CC were: 1.672 (±0.115), 1.638 (±0.097), and 1.72 (±0.137); post-treatment: 1.760 (±0.071), 1.764 (±0.043), and 1.898 (±0.047). After treatment FD-CC increased in all 11 eyes with an average of 0.163 (p = 0.003); FD-DCP increased in 10 (91%) eyes with an average of 0.126 (p = 0.003); and FD-SCP increased in seven (64%) eyes with an average of 0.089 (p = 0.059). Compared to the post-treatment FD values in the syphilitic group, controls had similar FD-SCP (p = 0.266), FD-DCP (p = 0.078), and FD-CC (p = 0.449). CONCLUSIONS: CC and DCP are mostly affected in ASPPC with minimal changes in the SCP. All vascular layers FD recovered after completing antibiotic treatment.

Robust fiber orientation distribution function estimation using deep constrained spherical deconvolution for diffusion-weighted magnetic resonance imaging.

Purpose: Diffusion-weighted magnetic resonance imaging (DW-MRI) is a critical imaging method for capturing and modeling tissue microarchitecture at a millimeter scale. A common practice to model the measured DW-MRI signal is via fiber orientation distribution function (fODF). This function is the essential first step for the downstream tractography and connectivity analyses. With recent advantages in data sharing, large-scale multisite DW-MRI datasets are being made available for multisite studies. However, measurement variabilities (e.g., inter- and intrasite variability, hardware performance, and sequence design) are inevitable during the acquisition of DW-MRI. Most existing model-based methods [e.g., constrained spherical deconvolution (CSD)] and learning-based methods (e.g., deep learning) do not explicitly consider such variabilities in fODF modeling, which consequently leads to inferior performance on multisite and/or longitudinal diffusion studies. Approach: In this paper, we propose a data-driven deep CSD method to explicitly constrain the scan-rescan variabilities for a more reproducible and robust estimation of brain microstructure from repeated DW-MRI scans. Specifically, the proposed method introduces a three-dimensional volumetric scanner-invariant regularization scheme during the fODF estimation. We study the Human Connectome Project (HCP) young adults test-retest group as well as the MASiVar dataset (with inter- and intrasite scan/rescan data). The Baltimore Longitudinal Study of Aging dataset is employed for external validation. Results: From the experimental results, the proposed data-driven framework outperforms the existing benchmarks in repeated fODF estimation. By introducing the contrastive loss with scan/rescan data, the proposed method achieved a higher consistency while maintaining higher angular correlation coefficients with the CSD modeling. The proposed method is assessing the downstream connectivity analysis and shows increased performance in distinguishing subjects with different biomarkers. Conclusion: We propose a deep CSD method to explicitly reduce the scan-rescan variabilities, so as to model a more reproducible and robust brain microstructure from repeated DW-MRI scans. The plug-and-play design of the proposed approach is potentially applicable to a wider range of data harmonization problems in neuroimaging.

Characterizing patterns of DTI variance in aging brains.

Background: As large analyses merge data across sites, a deeper understanding of variance in statistical assessment across the sources of data becomes critical for valid analyses. Diffusion tensor imaging (DTI) exhibits spatially varying and correlated noise, so care must be taken with distributional assumptions. Purpose: We characterize the role of physiology, subject compliance, and the interaction of subject with the scanner in the understanding of DTI variability, as modeled in spatial variance of derived metrics in homogeneous regions. Methods: We analyze DTI data from 1035 subjects in the Baltimore Longitudinal Study of Aging (BLSA), with ages ranging from 22.4 to 103 years old. For each subject, up to 12 longitudinal sessions were conducted. We assess variance of DTI scalars within regions of interest (ROIs) defined by four segmentation methods and investigate the relationships between the variance and covariates, including baseline age, time from the baseline (referred to as "interval"), motion, sex, and whether it is the first scan or the second scan in the session. Results: Covariate effects are heterogeneous and bilaterally symmetric across ROIs. Inter-session interval is positively related (p ≪ 0.001) to FA variance in the cuneus and occipital gyrus, but negatively (p ≪ 0.001) in the caudate nucleus. Males show significantly (p ≪ 0.001) higher FA variance in the right putamen, thalamus, body of the corpus callosum, and cingulate gyrus. In 62 out of 176 ROIs defined by the Eve type-1 atlas, an increase in motion is associated (p < 0.05) with a decrease in FA variance. Head motion increases during the rescan of DTI (Δµ = 0.045 millimeters per volume). Conclusions: The effects of each covariate on DTI variance, and their relationships across ROIs are complex. Ultimately, we encourage researchers to include estimates of variance when sharing data and consider models of heteroscedasticity in analysis. This work provides a foundation for study planning to account for regional variations in metric variance.

DeepN4: Learning N4ITK Bias Field Correction for T1-weighted Images.

T1-weighted (T1w) MRI has low frequency intensity artifacts due to magnetic field inhomogeneities. Removal of these biases in T1w MRI images is a critical preprocessing step to ensure spatially consistent image interpretation. N4ITK bias field correction, the current state-of-the-art, is implemented in such a way that makes it difficult to port between different pipelines and workflows, thus making it hard to reimplement and reproduce results across local, cloud, and edge platforms. Moreover, N4ITK is opaque to optimization before and after its application, meaning that methodological development must work around the inhomogeneity correction step. Given the importance of bias fields correction in structural preprocessing and flexible implementation, we pursue a deep learning approximation / reinterpretation of the N4ITK bias fields correction to create a method which is portable, flexible, and fully differentiable. In this paper, we trained a deep learning network "DeepN4" on eight independent cohorts from 72 different scanners and age ranges with N4ITK-corrected T1w MRI and bias field for supervision in log space. We found that we can closely approximate N4ITK bias fields correction with naïve networks. We evaluate the peak signal to noise ratio (PSNR) in test dataset against the N4ITK corrected images. The median PSNR of corrected images between N4ITK and DeepN4 was 47.96 dB. In addition, we assess the DeepN4 model on eight additional external datasets and show the generalizability of the approach. This study establishes that incompatible N4ITK preprocessing steps can be closely approximated by naïve deep neural networks, facilitating more flexibility. All code and models are released at https://github.com/MASILab/DeepN4.

MidRISH: Unbiased harmonization of rotationally invariant harmonics of the diffusion signal.

Objective: Data harmonization is necessary for removing confounding effects in multi-site diffusion image analysis. One such harmonization method, LinearRISH, scales rotationally invariant spherical harmonic (RISH) features from one site ("target") to the second ("reference") to reduce confounding scanner effects. However, reference and target site designations are not arbitrary and resultant diffusion metrics (fractional anisotropy, mean diffusivity) are biased by this choice. In this work we propose MidRISH: rather than scaling reference RISH features to target RISH features, we project both sites to a mid-space. Methods: We validate MidRISH with the following experiments: harmonizing scanner differences from 37 matched patients free of cognitive impairment, and harmonizing acquisition and study differences on 117 matched patients free of cognitive impairment. Conclusion: MidRISH reduces bias of reference selection while preserving harmonization efficacy of LinearRISH. Significance: Users should be cautious when performing LinearRISH harmonization. To select a reference site is to choose diffusion metric effect-size. Our proposed method eliminates the bias-inducing site selection step.

Comparing voxel- and feature-wise harmonization of complex graph measures from multiple sites for structural brain network investigation of aging.

Complex graph theory measures of brain structural connectomes derived from diffusion weighted images (DWI) provide insight into the network structure of the brain. Further, as the number of available DWI datasets grows, so does the ability to investigate associations in these measures with major biological factors, like age. However, one key hurdle that remains is the presence of scanner effects that can arise from different DWI datasets and confound multisite analyses. Two common approaches to correct these effects are voxel-wise and feature-wise harmonization. However, it is still unclear how to best leverage them for graph-theory analysis of an aging population. Thus, there is a need to better characterize the impact of each harmonization method and their ability to preserve age related features. We investigate this by characterizing four complex graph theory measures (modularity, characteristic path length, global efficiency, and betweenness centrality) in 48 participants aged 55 to 86 from Baltimore Longitudinal Study of Aging (BLSA) and Vanderbilt Memory and Aging Project (VMAP) before and after voxel- and feature-wise harmonization with the Null Space Deep Network (NSDN) and ComBat, respectively. First, we characterize across dataset coefficients of variation (CoV) and find the combination of NSDN and ComBat causes the greatest reduction in CoV followed by ComBat alone then NSDN alone. Second, we reproduce published associations of modularity with age after correcting for other covariates with linear models. We find that harmonization with ComBat or ComBat and NSDN together improves the significance of existing age effects, reduces model residuals, and qualitatively reduces separation between datasets. These results reinforce the efficiency of statistical harmonization on the feature-level with ComBat and suggest that harmonization on the voxel-level is synergistic but may have reduced effect after running through the multiple layers of the connectomics pipeline. Thus, we conclude that feature-wise harmonization improves statistical results, but the addition of biologically informed voxel-based harmonization offers further improvement.

Deep Constrained Spherical Deconvolution for Robust Harmonization.

Diffusion weighted magnetic resonance imaging (DW-MRI) captures tissue microarchitecture at millimeter scale. With recent advantages in data sharing, large-scale multi-site DW-MRI datasets are being made available for multi-site studies. However, DW-MRI suffers from measurement variability (e.g., inter- and intra-site variability, hardware performance, and sequence design), which consequently yields inferior performance on multi-site and/or longitudinal diffusion studies. In this study, we propose a novel, deep learning-based method to harmonize DW-MRI signals for a more reproducible and robust estimation of microstructure. Our method introduces a data-driven scanner-invariant regularization scheme to model a more robust fiber orientation distribution function (FODF) estimation. We study the Human Connectome Project (HCP) young adults test-retest group as well as the MASiVar dataset (with inter- and intra-site scan/rescan data). The 8th order spherical harmonics coefficients are employed as data representation. The results show that the proposed harmonization approach maintains higher angular correlation coefficients (ACC) with the ground truth signals (0.954 versus 0.942), while achieves higher consistency of FODF signals for intra-scanner data (0.891 versus 0.826), as compared with the baseline supervised deep learning scheme. Furthermore, the proposed data-driven framework is flexible and potentially applicable to a wider range of data harmonization problems in neuroimaging.

Omni-Seg: A Scale-Aware Dynamic Network for Renal Pathological Image Segmentation.

Comprehensive semantic segmentation on renal pathological images is challenging due to the heterogeneous scales of the objects. For example, on a whole slide image (WSI), the cross-sectional areas of glomeruli can be 64 times larger than that of the peritubular capillaries, making it impractical to segment both objects on the same patch, at the same scale. To handle this scaling issue, prior studies have typically trained multiple segmentation networks in order to match the optimal pixel resolution of heterogeneous tissue types. This multi-network solution is resource-intensive and fails to model the spatial relationship between tissue types. In this article, we propose the Omni-Seg network, a scale-aware dynamic neural network that achieves multi-object (six tissue types) and multi-scale (5× to 40× scale) pathological image segmentation via a single neural network. The contribution of this article is three-fold: (1) a novel scale-aware controller is proposed to generalize the dynamic neural network from single-scale to multi-scale; (2) semi-supervised consistency regularization of pseudo-labels is introduced to model the inter-scale correlation of unannotated tissue types into a single end-to-end learning paradigm; and (3) superior scale-aware generalization is evidenced by directly applying a model trained on human kidney images to mouse kidney images, without retraining. By learning from 150,000 human pathological image patches from six tissue types at three different resolutions, our approach achieved superior segmentation performance according to human visual assessment and evaluation of image-omics (i.e., spatial transcriptomics).

Enhancing thoracic disease detection using chest X-rays from PubMed Central Open Access.

Large chest X-rays (CXR) datasets have been collected to train deep learning models to detect thorax pathology on CXR. However, most CXR datasets are from single-center studies and the collected pathologies are often imbalanced. The aim of this study was to automatically construct a public, weakly-labeled CXR database from articles in PubMed Central Open Access (PMC-OA) and to assess model performance on CXR pathology classification by using this database as additional training data. Our framework includes text extraction, CXR pathology verification, subfigure separation, and image modality classification. We have extensively validated the utility of the automatically generated image database on thoracic disease detection tasks, including Hernia, Lung Lesion, Pneumonia, and pneumothorax. We pick these diseases due to their historically poor performance in existing datasets: the NIH-CXR dataset (112,120 CXR) and the MIMIC-CXR dataset (243,324 CXR). We find that classifiers fine-tuned with additional PMC-CXR extracted by the proposed framework consistently and significantly achieved better performance than those without (e.g., Hernia: 0.9335 vs 0.9154; Lung Lesion: 0.7394 vs. 0.7207; Pneumonia: 0.7074 vs. 0.6709; Pneumothorax 0.8185 vs. 0.7517, all in AUC with p< 0.0001) for CXR pathology detection. In contrast to previous approaches that manually submit the medical images to the repository, our framework can automatically collect figures and their accompanied figure legends. Compared to previous studies, the proposed framework improved subfigure segmentation and incorporates our advanced self-developed NLP technique for CXR pathology verification. We hope it complements existing resources and improves our ability to make biomedical image data findable, accessible, interoperable, and reusable.

Efficient approximate signal reconstruction for correction of gradient nonlinearities in diffusion-weighted imaging.

In diffusion weighted MRI (DW-MRI), hardware nonlinearities lead to spatial variations in the orientation and magnitude of diffusion weighting. While the correction of these spatial distortions has been well established for analyses of DW-MRI, the existing voxel-wise empirical correction for gradient nonlinearities requires reimplementation of existing models, as the resultant gradients vary by voxel. Herein, we propose a two-step signal approximation after voxel-wise correction of gradient nonlinearity effects in DW-MRI. The proposed technique (1) scales the diffusion signal and (2) resamples the gradient orientations. This results in uniform gradients across the corrected image and provides the key advantage of seamless integration into current diffusion workflows. We investigated the validity of our technique by fitting a multi-compartment neurite orientation dispersion and density imaging (NODDI) model to the empirical correction and proposed approximation in five subjects from the MASiVar pediatric dataset. We evaluated intra-cellular volume fraction (iVF), CSF volume fraction (cVF), and orientation dispersion index (ODI) from NODDI. The Cohen's d of iVF, cVF and ODI between the techniques was <0.2 indicating the proposed technique does not exhibit significant differences from the voxel-wise correction technique. Our two-step signal approximation is an efficient representation of the voxel-wise gradient table correction. Using this approximation, correction of gradient nonlinearities can be easily incorporated into existing diffusion preprocessing pipelines and is implemented in "PreQual: An automated pipeline for integrated preprocessing and quality assurance of diffusion weighted MRI images".

Closed Collaborative Surgical Intensive Care Unit Modeling and Its Association With Trauma Patient Outcomes.

INTRODUCTION: The optimization of intensive care unit (ICU) care impacts clinical outcomes and resource utilization. In 2017, our surgical ICU (SICU) adopted a "closed-collaborative" model. The aim of this study is to compare patient outcomes in the closed-collaborative model versus the previous open model in a cohort of trauma surgical patients admitted to our adult level 1 trauma center. METHODS: A retrospective review of trauma patients in the SICU from August 1, 2015 to July 31, 2019 was performed. Patients were divided into those admitted prior to August 1, 2017 (the "open" cohort) and those admitted after August 1, 2017 (the "closed-collaborative" cohort). Demographic variables and clinical outcomes were analyzed. Trauma severity was assessed using injury severity score (ISS). RESULTS: We identified 1669 patients (O: 895; C: 774). While no differences in demographics were observed, the closed-collaborative cohort had a higher overall ISS (O: 21.5 ± 12.14; C: 25.10 ± 2.72; P < 0.0001). There were no significant differences between the two cohorts in the incidence of strokes (O: 1.90%; C: 2.58%, P = 0.3435), pulmonary embolism (O: 0.78%; C: 0.65%; P = 0.7427), sepsis (O: 5.25%; C: 7.49%; P = 0.0599), median ICU charges (O: $7784.50; C: $8986.53; P = 0.5286), mortality (O: 11.40%; C: 13.18%; P = 0.2678), or ICU length of stay (LOS) (O: 4.85 ± 6.23; C: 4.37 ± 4.94; P = 0.0795). CONCLUSIONS: Patients in the closed-collaborative cohort had similar clinical outcomes despite having a sicker cohort of patients. We hypothesize that the closed-collaborative ICU model was able to maintain equivalent outcomes due to the dedicated multidisciplinary critical care team caring for these patients. Further research is warranted to determine the optimal model of ICU care for trauma patients.

Glo-In-One: holistic glomerular detection, segmentation, and lesion characterization with large-scale web image mining.

Purpose: The quantitative detection, segmentation, and characterization of glomeruli from high-resolution whole slide imaging (WSI) play essential roles in the computer-assisted diagnosis and scientific research in digital renal pathology. Historically, such comprehensive quantification requires extensive programming skills to be able to handle heterogeneous and customized computational tools. To bridge the gap of performing glomerular quantification for non-technical users, we develop the Glo-In-One toolkit to achieve holistic glomerular detection, segmentation, and characterization via a single line of command. Additionally, we release a large-scale collection of 30,000 unlabeled glomerular images to further facilitate the algorithmic development of self-supervised deep learning. Approach: The inputs of the Glo-In-One toolkit are WSIs, while the outputs are (1) WSI-level multi-class circle glomerular detection results (which can be directly manipulated with ImageScope), (2) glomerular image patches with segmentation masks, and (3) different lesion types. In the current version, the fine-grained global glomerulosclerosis (GGS) characterization is provided, including assessed-solidified-GSS (associated with hypertension-related injury), disappearing-GSS (a further end result of the SGGS becoming contiguous with fibrotic interstitium), and obsolescent-GSS (nonspecific GGS increasing with aging) glomeruli. To leverage the performance of the Glo-In-One toolkit, we introduce self-supervised deep learning to glomerular quantification via large-scale web image mining. Results: The GGS fine-grained classification model achieved a decent performance compared with baseline supervised methods while only using 10% of the annotated data. The glomerular detection achieved an average precision of 0.627 with circle representations, while the glomerular segmentation achieved a 0.955 patch-wise Dice dimilarity coefficient. Conclusion: We develop and release an open-source Glo-In-One toolkit, a software with holistic glomerular detection, segmentation, and lesion characterization. This toolkit is user-friendly to non-technical users via a single line of command. The toolbox and the 30,000 web mined glomerular images have been made publicly available at https://github.com/hrlblab/Glo-In-One.

Holistic fine-grained global glomerulosclerosis characterization: from detection to unbalanced classification.

Purpose: Recent studies have demonstrated the diagnostic and prognostic values of global glomerulosclerosis (GGS) in IgA nephropathy, aging, and end-stage renal disease. However, the fine-grained quantitative analysis of multiple GGS subtypes (e.g., obsolescent, solidified, and disappearing glomerulosclerosis) is typically a resource extensive manual process. Very few automatic methods, if any, have been developed to bridge this gap for such analytics. We present a holistic pipeline to quantify GGS (with both detection and classification) from a whole slide image in a fully automatic manner. In addition, we conduct the fine-grained classification for the subtypes of GGS. Our study releases the open-source quantitative analytical tool for fine-grained GGS characterization while tackling the technical challenges in unbalanced classification and integrating detection and classification. Approach: We present a deep learning-based framework to perform fine-grained detection and classification of GGS, with a hierarchical two-stage design. Moreover, we incorporate the state-of-the-art transfer learning techniques to achieve a more generalizable deep learning model for tackling the imbalanced distribution of our dataset. This way, we build a highly efficient WSI-to-results GGS characterization pipeline. Meanwhile, we investigated the largest fine-grained GGS cohort as of yet with 11,462 glomeruli and 10,619 nonglomeruli, which include 7841 globally sclerotic glomeruli of three distinct categories. With these data, we apply deep learning techniques to achieve (1) fine-grained GGS characterization, (2) GGS versus non-GGS classification, and (3) improved glomeruli detection results. Results: For fine-grained GGS characterization, when pretrained on the larger dataset, our model can achieve a 0.778-macro- F 1 score, compared to a 0.746-macro- F 1 score when using the regular ImageNet-pretrained weights. On the external dataset, our best model achieves an area under the curve (AUC) score of 0.994 when tasked with differentiating GGS from normal glomeruli. Using our dataset, we are able to build algorithms that allow for fine-grained classification of glomeruli lesions and are robust to distribution shifts. Conclusion: Our study showed that the proposed methods consistently improve the detection and fine-grained classification performance through both cross validation and external validation. Our code and pretrained models have been released for public use at https://github.com/luyuzhe111/glomeruli.

Compound Figure Separation of Biomedical Images: Mining Large Datasets for Self-supervised Learning.

With the rapid development of self-supervised learning (e.g., contrastive learning), the importance of having large-scale images (even without annotations) for training a more generalizable AI model has been widely recognized in medical image analysis. However, collecting large-scale task-specific unannotated data at scale can be challenging for individual labs. Existing online resources, such as digital books, publications, and search engines, provide a new resource for obtaining large-scale images. However, published images in healthcare (e.g., radiology and pathology) consist of a considerable amount of compound figures with subplots. In order to extract and separate compound figures into usable individual images for downstream learning, we propose a simple compound figure separation (SimCFS) framework without using the traditionally required detection bounding box annotations, with a new loss function and a hard case simulation. Our technical contribution is four-fold: (1) we introduce a simulation-based training framework that minimizes the need for resource extensive bounding box annotations; (2) we propose a new side loss that is optimized for compound figure separation; (3) we propose an intra-class image augmentation method to simulate hard cases; and (4) to the best of our knowledge, this is the first study that evaluates the efficacy of leveraging self-supervised learning with compound image separation. From the results, the proposed SimCFS achieved state-of-the-art performance on the ImageCLEF 2016 Compound Figure Separation Database. The pretrained self-supervised learning model using large-scale mined figures improved the accuracy of downstream image classification tasks with a contrastive learning algorithm. The source code of SimCFS is made publicly available at https://github.com/hrlblab/ImageSeperation.