Brain-wide Electrical Spatiotemporal Dynamics Encode Depression Vulnerability.

Brain-wide fluctuations in local field potential oscillations reflect emergent network-level signals that mediate behavior. Cracking the code whereby these oscillations coordinate in time and space (spatiotemporal dynamics) to represent complex behaviors would provide fundamental insights into how the brain signals emotional pathology. Using machine learning, we discover a spatiotemporal dynamic network that predicts the emergence of major depressive disorder (MDD)-related behavioral dysfunction in mice subjected to chronic social defeat stress. Activity patterns in this network originate in prefrontal cortex and ventral striatum, relay through amygdala and ventral tegmental area, and converge in ventral hippocampus. This network is increased by acute threat, and it is also enhanced in three independent models of MDD vulnerability. Finally, we demonstrate that this vulnerability network is biologically distinct from the networks that encode dysfunction after stress. Thus, these findings reveal a convergent mechanism through which MDD vulnerability is mediated in the brain.

Deep-Learning-Based Screening and Ancillary Testing for Thyroid Cytopathology.

Thyroid cancer is the most common malignant endocrine tumor. The key test to assess preoperative risk of malignancy is cytologic evaluation of fine-needle aspiration biopsies (FNABs). The evaluation findings can often be indeterminate, leading to unnecessary surgery for benign post-surgical diagnoses. We have developed a deep-learning algorithm to analyze thyroid FNAB whole-slide images (WSIs). We show, on the largest reported data set of thyroid FNAB WSIs, clinical-grade performance in the screening of determinate cases and indications for its use as an ancillary test to disambiguate indeterminate cases. The algorithm screened and definitively classified 45.1% (130/288) of the WSIs as either benign or malignant with risk of malignancy rates of 2.7% and 94.7%, respectively. It reduced the number of indeterminate cases (N = 108) by reclassifying 21.3% (N = 23) as benign with a resultant risk of malignancy rate of 1.8%. Similar results were reproduced using a data set of consecutive FNABs collected during an entire calendar year, achieving clinically acceptable margins of error for thyroid FNAB classification.

Thyroid Cytopathology Cancer Diagnosis from Smartphone Images Using Machine Learning.

We examined the performance of deep learning models on the classification of thyroid fine-needle aspiration biopsies using microscope images captured in 2 ways: with a high-resolution scanner and with a mobile phone camera. Our training set consisted of images from 964 whole-slide images captured with a high-resolution scanner. Our test set consisted of 100 slides; 20 manually selected regions of interest (ROIs) from each slide were captured in 2 ways as mentioned above. Applying a baseline machine learning algorithm trained on scanner ROIs resulted in performance deterioration when applied to the smartphone ROIs (97.8% area under the receiver operating characteristic curve [AUC], CI = [95.4%, 100.0%] for scanner images vs 89.5% AUC, CI = [82.3%, 96.6%] for mobile images, P = .019). Preliminary analysis via histogram matching showed that the baseline model was overly sensitive to slight color variations in the images (specifically, to color differences between mobile and scanner images). Adding color augmentation during training reduces this sensitivity and narrows the performance gap between mobile and scanner images (97.6% AUC, CI = [95.0%, 100.0%] for scanner images vs 96.0% AUC, CI = [91.8%, 100.0%] for mobile images, P = .309), with both modalities on par with human pathologist performance (95.6% AUC, CI = [91.6%, 99.5%]) for malignancy prediction (P = .398 for pathologist vs scanner and P = .875 for pathologist vs mobile). For indeterminate cases (pathologist-assigned Bethesda category of 3, 4, or 5), color augmentations confer some improvement (88.3% AUC, CI = [73.7%, 100.0%] for the baseline model vs 96.2% AUC, CI = [90.9%, 100.0%] with color augmentations, P = .158). In addition, we found that our model's performance levels off after 15 ROIs, a promising indication that ROI data collection would not be time-consuming for our diagnostic system. Finally, we showed that the model has sensible Bethesda category (TBS) predictions (increasing risk malignancy rate with predicted TBS category, with 0% malignancy for predicted TBS 2 and 100% malignancy for TBS 6).

SAGES consensus recommendations on an annotation framework for surgical video.

BACKGROUND: The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration. METHODS: Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups. RESULTS: After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established. CONCLUSIONS: While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.

Efficient patch-based approach for compressive depth imaging.

We present efficient camera hardware and algorithms to capture images with extended depth of field. The camera moves its focal plane via a liquid lens and modulates the scene at different focal planes by shifting a fixed binary mask, with synchronization achieved by using the same triangular wave to control the focal plane and the pizeoelectronic translator that shifts the mask. Efficient algorithms are developed to reconstruct the all-in-focus image and the depth map from a single coded exposure, and various sparsity priors are investigated to enhance the reconstruction, including group sparsity, tree structure, and dictionary learning. The algorithms naturally admit a parallel computational structure due to the independent patch-level operations. Experimental results on both simulation and real datasets demonstrate the efficacy of the new hardware and the inversion algorithms.

Bayesian joint analysis of heterogeneous genomics data.

SUMMARY: A non-parametric Bayesian factor model is proposed for joint analysis of multi-platform genomics data. The approach is based on factorizing the latent space (feature space) into a shared component and a data-specific component with the dimensionality of these components (spaces) inferred via a beta-Bernoulli process. The proposed approach is demonstrated by jointly analyzing gene expression/copy number variations and gene expression/methylation data for ovarian cancer patients, showing that the proposed model can potentially uncover key drivers related to cancer. AVAILABILITY AND IMPLEMENTATION: The source code for this model is written in MATLAB and has been made publicly available at https://sites.google.com/site/jointgenomics/. CONTACT: catherine.ll.zheng@gmail.com SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Spectral-temporal compressive imaging.

This Letter presents a compressive camera that integrates mechanical translation and spectral dispersion to compress a multi-spectral, high-speed scene onto a monochrome, video-rate detector. Experimental reconstructions of 17 spectral channels and 11 temporal channels from a single measurement are reported for a megapixel-scale monochrome camera.

Estimation of the CSA-ODF using Bayesian compressed sensing of multi-shell HARDI.

PURPOSE: Diffusion MRI provides important information about the brain white matter structures and has opened new avenues for neuroscience and translational research. However, acquisition time needed for advanced applications can still be a challenge in clinical settings. There is consequently a need to accelerate diffusion MRI acquisitions. METHODS: A multi-task Bayesian compressive sensing (MT-BCS) framework is proposed to directly estimate the constant solid angle orientation distribution function (CSA-ODF) from under-sampled (i.e., accelerated image acquisition) multi-shell high angular resolution diffusion imaging (HARDI) datasets, and accurately recover HARDI data at higher resolution in q-space. The proposed MT-BCS approach exploits the spatial redundancy of the data by modeling the statistical relationships within groups (clusters) of diffusion signal. This framework also provides uncertainty estimates of the computed CSA-ODF and diffusion signal, directly computed from the compressive measurements. Experiments validating the proposed framework are performed using realistic multi-shell synthetic images and in vivo multi-shell high angular resolution HARDI datasets. RESULTS: Results indicate a practical reduction in the number of required diffusion volumes (q-space samples) by at least a factor of four to estimate the CSA-ODF from multi-shell data. CONCLUSION: This work presents, for the first time, a multi-task Bayesian compressive sensing approach to simultaneously estimate the full posterior of the CSA-ODF and diffusion-weighted volumes from multi-shell HARDI acquisitions. It demonstrates improvement of the quality of acquired datasets by means of CS de-noising, and accurate estimation of the CSA-ODF, as well as enables a reduction in the acquisition time by a factor of two to four, especially when "staggered" q-space sampling schemes are used. The proposed MT-BCS framework can naturally be combined with parallel MR imaging to further accelerate HARDI acquisitions.

Compressed sampling strategies for tomography.

We investigate new sampling strategies for projection tomography, enabling one to employ fewer measurements than expected from classical sampling theory without significant loss of information. Inspired by compressed sensing, our approach is based on the understanding that many real objects are compressible in some known representation, implying that the number of degrees of freedom defining an object is often much smaller than the number of pixels/voxels. We propose a new approach based on quasi-random detector subsampling, whereas previous approaches only addressed subsampling with respect to source location (view angle). The performance of different sampling strategies is considered using object-independent figures of merit, and also based on reconstructions for specific objects, with synthetic and real data. The proposed approach can be implemented using a structured illumination of the interrogated object or the detector array by placing a coded aperture/mask at the source or detector side, respectively. Advantages of the proposed approach include (i) for structured illumination of the detector array, it leads to fewer detector pixels and allows one to integrate detectors for scattered radiation in the unused space; (ii) for structured illumination of the object, it leads to a reduced radiation dose for patients in medical scans; (iii) in the latter case, the blocking of rays reduces scattered radiation while keeping the same energy in the transmitted rays, resulting in a higher signal-to-noise ratio than that achieved by lowering exposure times or the energy of the source; (iv) compared to view-angle subsampling, it allows one to use fewer measurements for the same image quality, or leads to better image quality for the same number of measurements. The proposed approach can also be combined with view-angle subsampling.

Temporal dynamics of host molecular responses differentiate symptomatic and asymptomatic influenza a infection.

Exposure to influenza viruses is necessary, but not sufficient, for healthy human hosts to develop symptomatic illness. The host response is an important determinant of disease progression. In order to delineate host molecular responses that differentiate symptomatic and asymptomatic Influenza A infection, we inoculated 17 healthy adults with live influenza (H3N2/Wisconsin) and examined changes in host peripheral blood gene expression at 16 timepoints over 132 hours. Here we present distinct transcriptional dynamics of host responses unique to asymptomatic and symptomatic infections. We show that symptomatic hosts invoke, simultaneously, multiple pattern recognition receptors-mediated antiviral and inflammatory responses that may relate to virus-induced oxidative stress. In contrast, asymptomatic subjects tightly regulate these responses and exhibit elevated expression of genes that function in antioxidant responses and cell-mediated responses. We reveal an ab initio molecular signature that strongly correlates to symptomatic clinical disease and biomarkers whose expression patterns best discriminate early from late phases of infection. Our results establish a temporal pattern of host molecular responses that differentiates symptomatic from asymptomatic infections and reveals an asymptomatic host-unique non-passive response signature, suggesting novel putative molecular targets for both prognostic assessment and ameliorative therapeutic intervention in seasonal and pandemic influenza.

A machine learning algorithm improves the diagnostic accuracy of the histologic component of antibody mediated rejection (AMR-H) in cardiac transplant endomyocardial biopsies.

BACKGROUND: Pathologic antibody mediated rejection (pAMR) remains a major driver of graft failure in cardiac transplant patients. The endomyocardial biopsy remains the primary diagnostic tool but presents with challenges, particularly in distinguishing the histologic component (pAMR-H) defined by 1) intravascular macrophage accumulation in capillaries and 2) activated endothelial cells that expand the cytoplasm to narrow or occlude the vascular lumen. Frequently, pAMR-H is difficult to distinguish from acute cellular rejection (ACR) and healing injury. With the advent of digital slide scanning and advances in machine deep learning, artificial intelligence technology is widely under investigation in the areas of oncologic pathology, but in its infancy in transplant pathology. For the first time, we determined if a machine learning algorithm could distinguish pAMR-H from normal myocardium, healing injury and ACR. MATERIALS AND METHODS: A total of 4,212 annotations (1,053 regions of normal, 1,053 pAMR-H, 1,053 healing injury and 1,053 ACR) were completed from 300 hematoxylin and eosin slides scanned using a Leica Aperio GT450 digital whole slide scanner at 40X magnification. All regions of pAMR-H were annotated from patients confirmed with a previous diagnosis of pAMR2 (>50% positive C4d immunofluorescence and/or >10% CD68 positive intravascular macrophages). Annotations were imported into a Python 3.7 development environment using the OpenSlide™ package and a convolutional neural network approach utilizing transfer learning was performed. RESULTS: The machine learning algorithm showed 98% overall validation accuracy and pAMR-H was correctly distinguished from specific categories with the following accuracies: normal myocardium (99.2%), healing injury (99.5%) and ACR (99.5%). CONCLUSION: Our novel deep learning algorithm can reach acceptable, and possibly surpass, performance of current diagnostic standards of identifying pAMR-H. Such a tool may serve as an adjunct diagnostic aid for improving the pathologist's accuracy and reproducibility, especially in difficult cases with high inter-observer variability. This is one of the first studies that provides evidence that an artificial intelligence machine learning algorithm can be trained and validated to diagnose pAMR-H in cardiac transplant patients. Ongoing studies include multi-institutional verification testing to ensure generalizability.

Coded aperture compressive temporal imaging.

We use mechanical translation of a coded aperture for code division multiple access compression of video. We discuss the compressed video's temporal resolution and present experimental results for reconstructions of > 10 frames of temporal data per coded snapshot.

Differentiable Hierarchical Optimal Transport for Robust Multi-View Learning.

Traditional multi-view learning methods often rely on two assumptions: ( i) the samples in different views are well-aligned, and ( ii) their representations obey the same distribution in a latent space. Unfortunately, these two assumptions may be questionable in practice, which limits the application of multi-view learning. In this work, we propose a differentiable hierarchical optimal transport (DHOT) method to mitigate the dependency of multi-view learning on these two assumptions. Given arbitrary two views of unaligned multi-view data, the DHOT method calculates the sliced Wasserstein distance between their latent distributions. Based on these sliced Wasserstein distances, the DHOT method further calculates the entropic optimal transport across different views and explicitly indicates the clustering structure of the views. Accordingly, the entropic optimal transport, together with the underlying sliced Wasserstein distances, leads to a hierarchical optimal transport distance defined for unaligned multi-view data, which works as the objective function of multi-view learning and leads to a bi-level optimization task. Moreover, our DHOT method treats the entropic optimal transport as a differentiable operator of model parameters. It considers the gradient of the entropic optimal transport in the backpropagation step and thus helps improve the descent direction for the model in the training phase. We demonstrate the superiority of our bi-level optimization strategy by comparing it to the traditional alternating optimization strategy. The DHOT method is applicable for both unsupervised and semi-supervised learning. Experimental results show that our DHOT method is at least comparable to state-of-the-art multi-view learning methods on both synthetic and real-world tasks, especially for challenging scenarios with unaligned multi-view data.

Representing Graphs via Gromov-Wasserstein Factorization.

Graph representation is a challenging and significant problem for many real-world applications. In this work, we propose a novel paradigm called "Gromov-Wasserstein Factorization" (GWF) to learn graph representations in a flexible and interpretable way. Given a set of graphs, whose correspondence between nodes is unknown and whose sizes can be different, our GWF model reconstructs each graph by a weighted combination of some "graph factors" under a pseudo-metric called Gromov-Wasserstein (GW) discrepancy. This model leads to a new nonlinear factorization mechanism of the graphs. The graph factors are shared by all the graphs, which represent the typical patterns of the graphs' structures. The weights associated with each graph indicate the graph factors' contributions to the graph's reconstruction, which lead to a permutation-invariant graph representation. We learn the graph factors of the GWF model and the weights of the graphs jointly by minimizing the overall reconstruction error. When learning the model, we reparametrize the graph factors and the weights to unconstrained model parameters and simplify the backpropagation of gradient with the help of the envelope theorem. For the GW discrepancy (the critical training step), we consider two algorithms to compute it, which correspond to the proximal point algorithm (PPA) and Bregman alternating direction method of multipliers (BADMM), respectively. Furthermore, we propose some extensions of the GWF model, including (i) combining with a graph neural network and learning graph representations in an auto-encoding manner, (ii) representing the graphs with node attributes, and (iii) working as a regularizer for semi-supervised graph classification. Experiments on various datasets demonstrate that our GWF model is comparable to the state-of-the-art methods. The graph representations derived by it perform well in graph clustering and classification tasks.

Learning Hierarchical Document Graphs From Multilevel Sentence Relations.

Organizing the implicit topology of a document as a graph, and further performing feature extraction via the graph convolutional network (GCN), has proven effective in document analysis. However, existing document graphs are often restricted to expressing single-level relations, which are predefined and independent of downstream learning. A set of learnable hierarchical graphs are built to explore multilevel sentence relations, assisted by a hierarchical probabilistic topic model. Based on these graphs, multiple parallel GCNs are used to extract multilevel semantic features, which are aggregated by an attention mechanism for different document-comprehension tasks. Equipped with variational inference, the graph construction and GCN are learned jointly, allowing the graphs to evolve dynamically to better match the downstream task. The effectiveness and efficiency of the proposed multilevel sentence relation graph convolutional network (MuserGCN) is demonstrated via experiments on document classification, abstractive summarization, and matching.

Calibration and Uncertainty in Neural Time-to-Event Modeling.

Models for predicting the time of a future event are crucial for risk assessment, across a diverse range of applications. Existing time-to-event (survival) models have focused primarily on preserving pairwise ordering of estimated event times (i.e., relative risk). We propose neural time-to-event models that account for calibration and uncertainty while predicting accurate absolute event times. Specifically, an adversarial nonparametric model is introduced for estimating matched time-to-event distributions for probabilistically concentrated and accurate predictions. We also consider replacing the discriminator of the adversarial nonparametric model with a survival-function matching estimator that accounts for model calibration. The proposed estimator can be used as a means of estimating and comparing conditional survival distributions while accounting for the predictive uncertainty of probabilistic models. Extensive experiments show that the distribution matching methods outperform existing approaches in terms of both calibration and concentration of time-to-event distributions.

Deep Learning-Assisted Detection of Glaucoma Progression in Spectral-Domain OCT.

PURPOSE: To develop and validate a deep learning (DL) model for detection of glaucoma progression using spectral-domain (SD)-OCT measurements of retinal nerve fiber layer (RNFL) thickness. DESIGN: Retrospective cohort study. PARTICIPANTS: A total of 14 034 SD-OCT scans from 816 eyes from 462 individuals. METHODS: A DL convolutional neural network was trained to assess SD-OCT RNFL thickness measurements of 2 visits (a baseline and a follow-up visit) along with time between visits to predict the probability of glaucoma progression. The ground truth was defined by consensus from subjective grading by glaucoma specialists. Diagnostic performance was summarized by the area under the receiver operator characteristic curve (AUC), sensitivity, and specificity, and was compared with conventional trend-based analyses of change. Interval likelihood ratios were calculated to determine the impact of DL model results in changing the post-test probability of progression. MAIN OUTCOME MEASURES: The AUC, sensitivity, and specificity of the DL model. RESULTS: The DL model had an AUC of 0.938 (95% confidence interval [CI], 0.921-0.955), with sensitivity of 87.3% (95% CI, 83.6%-91.6%) and specificity of 86.4% (95% CI, 79.9%-89.6%). When matched for the same specificity, the DL model significantly outperformed trend-based analyses. Likelihood ratios for the DL model were associated with large changes in the probability of progression in the vast majority of SD-OCT tests. CONCLUSIONS: A DL model was able to assess the probability of glaucomatous structural progression from SD-OCT RNFL thickness measurements. The model agreed well with expert judgments and outperformed conventional trend-based analyses of change, while also providing indication of the likely locations of change. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found after the references.

Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors.

Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists' annotation of region of interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) - based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.

A Deep-Learning Algorithm to Predict Short-Term Progression to Geographic Atrophy on Spectral-Domain Optical Coherence Tomography.

Importance: The identification of patients at risk of progressing from intermediate age-related macular degeneration (iAMD) to geographic atrophy (GA) is essential for clinical trials aimed at preventing disease progression. DeepGAze is a fully automated and accurate convolutional neural network-based deep learning algorithm for predicting progression from iAMD to GA within 1 year from spectral-domain optical coherence tomography (SD-OCT) scans. Objective: To develop a deep-learning algorithm based on volumetric SD-OCT scans to predict the progression from iAMD to GA during the year following the scan. Design, Setting, and Participants: This retrospective cohort study included participants with iAMD at baseline and who either progressed or did not progress to GA within the subsequent 13 months. Participants were included from centers in 4 US states. Data set 1 included patients from the Age-Related Eye Disease Study 2 AREDS2 (Ancillary Spectral-Domain Optical Coherence Tomography) A2A study (July 2008 to August 2015). Data sets 2 and 3 included patients with imaging taken in routine clinical care at a tertiary referral center and associated satellites between January 2013 and January 2023. The stored imaging data were retrieved for the purpose of this study from July 1, 2022, to February 1, 2023. Data were analyzed from May 2021 to July 2023. Exposure: A position-aware convolutional neural network with proactive pseudointervention was trained and cross-validated on Bioptigen SD-OCT volumes (data set 1) and validated on 2 external data sets comprising Heidelberg Spectralis SD-OCT scans (data sets 2 and 3). Main Outcomes and Measures: Prediction of progression to GA within 13 months was evaluated with area under the receiver-operator characteristic curves (AUROC) as well as area under the precision-recall curve (AUPRC), sensitivity, specificity, positive predictive value, negative predictive value, and accuracy. Results: The study included a total of 417 patients: 316 in data set 1 (mean [SD] age, 74 [8]; 185 [59%] female), 53 in data set 2, (mean [SD] age, 83 [8]; 32 [60%] female), and 48 in data set 3 (mean [SD] age, 81 [8]; 32 [67%] female). The AUROC for prediction of progression from iAMD to GA within 1 year was 0.94 (95% CI, 0.92-0.95; AUPRC, 0.90 [95% CI, 0.85-0.95]; sensitivity, 0.88 [95% CI, 0.84-0.92]; specificity, 0.90 [95% CI, 0.87-0.92]) for data set 1. The addition of expert-annotated SD-OCT features to the model resulted in no improvement compared to the fully autonomous model (AUROC, 0.95; 95% CI, 0.92-0.95; P = .19). On an independent validation data set (data set 2), the model predicted progression to GA with an AUROC of 0.94 (95% CI, 0.91-0.96; AUPRC, 0.92 [0.89-0.94]; sensitivity, 0.91 [95% CI, 0.74-0.98]; specificity, 0.80 [95% CI, 0.63-0.91]). At a high-specificity operating point, simulated clinical trial recruitment was enriched for patients progressing to GA within 1 year by 8.3- to 20.7-fold (data sets 2 and 3). Conclusions and Relevance: The fully automated, position-aware deep-learning algorithm assessed in this study successfully predicted progression from iAMD to GA over a clinically meaningful time frame. The ability to predict imminent GA progression could facilitate clinical trials aimed at preventing the condition and could guide clinical decision-making regarding screening frequency or treatment initiation.