A comprehensive AI model development framework for consistent Gleason grading.

BACKGROUND: Artificial Intelligence(AI)-based solutions for Gleason grading hold promise for pathologists, while image quality inconsistency, continuous data integration needs, and limited generalizability hinder their adoption and scalability. METHODS: We present a comprehensive digital pathology workflow for AI-assisted Gleason grading. It incorporates A!MagQC (image quality control), A!HistoClouds (cloud-based annotation), Pathologist-AI Interaction (PAI) for continuous model improvement, Trained on Akoya-scanned images only, the model utilizes color augmentation and image appearance migration to address scanner variations. We evaluate it on Whole Slide Images (WSI) from another five scanners and conduct validations with pathologists to assess AI efficacy and PAI. RESULTS: Our model achieves an average F1 score of 0.80 on annotations and 0.71 Quadratic Weighted Kappa on WSIs for Akoya-scanned images. Applying our generalization solution increases the average F1 score for Gleason pattern detection from 0.73 to 0.88 on images from other scanners. The model accelerates Gleason scoring time by 43% while maintaining accuracy. Additionally, PAI improve annotation efficiency by 2.5 times and led to further improvements in model performance. CONCLUSIONS: This pipeline represents a notable advancement in AI-assisted Gleason grading for improved consistency, accuracy, and efficiency. Unlike previous methods limited by scanner specificity, our model achieves outstanding performance across diverse scanners. This improvement paves the way for its seamless integration into clinical workflows.

Gleason grading is a well-accepted diagnostic standard to assess the severity of prostate cancer in patients' tissue samples, based on how abnormal the cells in their prostate tumor look under a microscope. This process can be complex and time-consuming. We explore how artificial intelligence (AI) can help pathologists perform Gleason grading more efficiently and consistently. We build an AI-based system which automatically checks image quality, standardizes the appearance of images from different equipment, learns from pathologists' feedback, and constantly improves model performance. Testing shows that our approach achieves consistent results across different equipment and improves efficiency of the grading process. With further testing and implementation in the clinic, our approach could potentially improve prostate cancer diagnosis and management.

BANKSY unifies cell typing and tissue domain segmentation for scalable spatial omics data analysis.

Spatial omics data are clustered to define both cell types and tissue domains. We present Building Aggregates with a Neighborhood Kernel and Spatial Yardstick (BANKSY), an algorithm that unifies these two spatial clustering problems by embedding cells in a product space of their own and the local neighborhood transcriptome, representing cell state and microenvironment, respectively. BANKSY's spatial feature augmentation strategy improved performance on both tasks when tested on diverse RNA (imaging, sequencing) and protein (imaging) datasets. BANKSY revealed unexpected niche-dependent cell states in the mouse brain and outperformed competing methods on domain segmentation and cell typing benchmarks. BANKSY can also be used for quality control of spatial transcriptomics data and for spatially aware batch effect correction. Importantly, it is substantially faster and more scalable than existing methods, enabling the processing of millions of cell datasets. In summary, BANKSY provides an accurate, biologically motivated, scalable and versatile framework for analyzing spatially resolved omics data.

Distribution based MIL pooling filters: Experiments on a lymph node metastases dataset.

Histopathology is a crucial diagnostic tool in cancer and involves the analysis of gigapixel slides. Multiple instance learning (MIL) promises success in digital histopathology thanks to its ability to handle gigapixel slides and work with weak labels. MIL is a machine learning paradigm that learns the mapping between bags of instances and bag labels. It represents a slide as a bag of patches and uses the slide's weak label as the bag's label. This paper introduces distribution-based pooling filters that obtain a bag-level representation by estimating marginal distributions of instance features. We formally prove that the distribution-based pooling filters are more expressive than the classical point estimate-based counterparts, like 'max' and 'mean' pooling, in terms of the amount of information captured while obtaining bag-level representations. Moreover, we empirically show that models with distribution-based pooling filters perform equal to or better than those with point estimate-based pooling filters on distinct real-world MIL tasks defined on the CAMELYON16 lymph node metastases dataset. Our model with a distribution pooling filter achieves an area under the receiver operating characteristics curve value of 0.9325 (95% confidence interval: 0.8798 - 0.9743) in the tumor vs. normal slide classification task.

Improving transparency and representational generalizability through parallel continual learning.

This paper takes a parallel learning approach in continual learning scenarios. We define parallel continual learning as learning a sequence of tasks where the data for the previous tasks, whose distribution may have shifted over time, are also available while learning new tasks. We propose a parallel continual learning method by assigning subnetworks to each task, and simultaneously training only the assigned subnetworks on their corresponding tasks. In doing so, some parts of the network will be shared across multiple tasks. This is unlike the existing literature in continual learning which aims at learning incoming tasks sequentially, with the assumption that the data for the previous tasks have a fixed distribution. Our proposed method offers promises in: (1) Transparency in the network and in the relationship across tasks by enabling examination of the learned representations by independent and shared subnetworks, (2) Representation generalizability through sharing and training subnetworks on multiple tasks simultaneously. Our analysis shows that compared to many competing approaches such as continual learning, neural architecture search, and multi-task learning, parallel continual learning is capable of learning more generalizable representations. Also, (3)Parallel continual learning overcomes the common issue of catastrophic forgetting in continual learning algorithms. This is the first effort to train a neural network on multiple tasks and input domains simultaneously in a continual learning scenario. Our code is available at https://github.com/yours-anonym/PaRT.

An AI-assisted tool for efficient prostate cancer diagnosis in low-grade and low-volume cases.

Pathologists diagnose prostate cancer by core needle biopsy. In low-grade and low-volume cases, they look for a few malignant glands out of hundreds within a core. They may miss a few malignant glands, resulting in repeat biopsies or missed therapeutic opportunities. This study developed a multi-resolution deep-learning pipeline to assist pathologists in detecting malignant glands in core needle biopsies of low-grade and low-volume cases. Analyzing a gland at multiple resolutions, our model exploited morphology and neighborhood information, which were crucial in prostate gland classification. We developed and tested our pipeline on the slides of a local cohort of 99 patients in Singapore. Besides, we made the images publicly available, becoming the first digital histopathology dataset of patients of Asian ancestry with prostatic carcinoma. Our multi-resolution classification model achieved an area under the receiver operating characteristic curve (AUROC) value of 0.992 (95% confidence interval [CI]: 0.985-0.997) in the external validation study, showing the generalizability of our multi-resolution approach.

Obtaining spatially resolved tumor purity maps using deep multiple instance learning in a pan-cancer study.

Tumor purity is the percentage of cancer cells within a tissue section. Pathologists estimate tumor purity to select samples for genomic analysis by manually reading hematoxylin-eosin (H&E)-stained slides, which is tedious, time consuming, and prone to inter-observer variability. Besides, pathologists' estimates do not correlate well with genomic tumor purity values, which are inferred from genomic data and accepted as accurate for downstream analysis. We developed a deep multiple instance learning model predicting tumor purity from H&E-stained digital histopathology slides. Our model successfully predicted tumor purity in eight The Cancer Genome Atlas (TCGA) cohorts and a local Singapore cohort. The predictions were highly consistent with genomic tumor purity values. Thus, our model can be utilized to select samples for genomic analysis, which will help reduce pathologists' workload and decrease inter-observer variability. Furthermore, our model provided tumor purity maps showing the spatial variation within sections. They can help better understand the tumor microenvironment.

Oner, Sung, and Lee: Researchers in digital pathology for the future of modern medicine.

Oner, an early-career researcher, and Lee and Sung, group leaders, have developed a deep learning model for accurate prediction of the proportion of cancer cells within tumor tissue. This is a necessary step for precision oncology and target therapy in cancer. They talk about their view of data science and the evolution of pathology in the coming years.

Automated interpretation of systolic and diastolic function on the echocardiogram: a multicohort study.

BACKGROUND: Echocardiography is the diagnostic modality for assessing cardiac systolic and diastolic function to diagnose and manage heart failure. However, manual interpretation of echocardiograms can be time consuming and subject to human error. Therefore, we developed a fully automated deep learning workflow to classify, segment, and annotate two-dimensional (2D) videos and Doppler modalities in echocardiograms. METHODS: We developed the workflow using a training dataset of 1145 echocardiograms and an internal test set of 406 echocardiograms from the prospective heart failure research platform (Asian Network for Translational Research and Cardiovascular Trials; ATTRaCT) in Asia, with previous manual tracings by expert sonographers. We validated the workflow against manual measurements in a curated dataset from Canada (Alberta Heart Failure Etiology and Analysis Research Team; HEART; n=1029 echocardiograms), a real-world dataset from Taiwan (n=31 241), the US-based EchoNet-Dynamic dataset (n=10 030), and in an independent prospective assessment of the Asian (ATTRaCT) and Canadian (Alberta HEART) datasets (n=142) with repeated independent measurements by two expert sonographers. FINDINGS: In the ATTRaCT test set, the automated workflow classified 2D videos and Doppler modalities with accuracies (number of correct predictions divided by the total number of predictions) ranging from 0·91 to 0·99. Segmentations of the left ventricle and left atrium were accurate, with a mean Dice similarity coefficient greater than 93% for all. In the external datasets (n=1029 to 10 030 echocardiograms used as input), automated measurements showed good agreement with locally measured values, with a mean absolute error range of 9-25 mL for left ventricular volumes, 6-10% for left ventricular ejection fraction (LVEF), and 1·8-2·2 for the ratio of the mitral inflow E wave to the tissue Doppler e' wave (E/e' ratio); and reliably classified systolic dysfunction (LVEF <40%, area under the receiver operating characteristic curve [AUC] range 0·90-0·92) and diastolic dysfunction (E/e' ratio ≥13, AUC range 0·91-0·91), with narrow 95% CIs for AUC values. Independent prospective evaluation confirmed less variance of automated compared with human expert measurements, with all individual equivalence coefficients being less than 0 for all measurements. INTERPRETATION: Deep learning algorithms can automatically annotate 2D videos and Doppler modalities with similar accuracy to manual measurements by expert sonographers. Use of an automated workflow might accelerate access, improve quality, and reduce costs in diagnosing and managing heart failure globally. FUNDING: A*STAR Biomedical Research Council and A*STAR Exploit Technologies.

Model learning analysis of 3D optoacoustic mesoscopy images for the classification of atopic dermatitis.

Atopic dermatitis (AD) is a skin inflammatory disease affecting 10% of the population worldwide. Raster-scanning optoacoustic mesoscopy (RSOM) has recently shown promise in dermatological imaging. We conducted a comprehensive analysis using three machine-learning models, random forest (RF), support vector machine (SVM), and convolutional neural network (CNN) for classifying healthy versus AD conditions, and sub-classifying different AD severities using RSOM images and clinical information. CNN model successfully differentiates healthy from AD patients with 97% accuracy. With limited data, RF achieved 65% accuracy in sub-classifying AD patients into mild versus moderate-severe cases. Identification of disease severities is vital in managing AD treatment.

Solving the inverse problem of time independent Fokker-Planck equation with a self supervised neural network method.

The Fokker-Planck equation (FPE) has been used in many important applications to study stochastic processes with the evolution of the probability density function (pdf). Previous studies on FPE mainly focus on solving the forward problem which is to predict the time-evolution of the pdf from the underlying FPE terms. However, in many applications the FPE terms are usually unknown and roughly estimated, and solving the forward problem becomes more challenging. In this work, we take a different approach of starting with the observed pdfs to recover the FPE terms using a self-supervised machine learning method. This approach, known as the inverse problem, has the advantage of requiring minimal assumptions on the FPE terms and allows data-driven scientific discovery of unknown FPE mechanisms. Specifically, we propose an FPE-based neural network (FPE-NN) which directly incorporates the FPE terms as neural network weights. By training the network on observed pdfs, we recover the FPE terms. Additionally, to account for noise in real-world observations, FPE-NN is able to denoise the observed pdfs by training the pdfs alongside the network weights. Our experimental results on various forms of FPE show that FPE-NN can accurately recover FPE terms and denoising the pdf plays an essential role.

Inverse renormalization group based on image super-resolution using deep convolutional networks.

The inverse renormalization group is studied based on the image super-resolution using the deep convolutional neural networks. We consider the improved correlation configuration instead of spin configuration for the spin models, such as the two-dimensional Ising and three-state Potts models. We propose a block-cluster transformation as an alternative to the block-spin transformation in dealing with the improved estimators. In the framework of the dual Monte Carlo algorithm, the block-cluster transformation is regarded as a transformation in the graph degrees of freedom, whereas the block-spin transformation is that in the spin degrees of freedom. We demonstrate that the renormalized improved correlation configuration successfully reproduces the original configuration at all the temperatures by the super-resolution scheme. Using the rule of enlargement, we repeatedly make inverse renormalization procedure to generate larger correlation configurations. To connect thermodynamics, an approximate temperature rescaling is discussed. The enlarged systems generated using the super-resolution satisfy the finite-size scaling.

Machine learning improves early prediction of small-for-gestational-age births and reveals nuchal fold thickness as unexpected predictor.

OBJECTIVE: To investigate the performance of the machine learning (ML) model in predicting small-for-gestational-age (SGA) at birth, using second-trimester data. METHODS: Retrospective data of 347 patients, consisting of maternal demographics and ultrasound parameters collected between the 20th and 25th gestational weeks, were studied. ML models were applied to different combinations of the parameters to predict SGA and severe SGA at birth (defined as 10th and third centile birth weight). RESULTS: Using second-trimester measurements, ML models achieved an accuracy of 70% and 73% in predicting SGA and severe SGA whereas clinical guidelines had accuracies of 64% and 48%. Uterine PI (Ut PI) was found to be an important predictor, corroborating with existing literature, but surprisingly, so was nuchal fold thickness (NF). Logistic regression showed that Ut PI and NF were significant predictors and statistical comparisons showed that these parameters were significantly different in disease. Further, including NF was found to improve ML model performance, and vice versa. CONCLUSION: ML could potentially improve the prediction of SGA at birth from second-trimester measurements, and demonstrated reduced NF to be an important predictor. Early prediction of SGA allows closer clinical monitoring, which provides an opportunity to discover any underlying diseases associated with SGA.

An end-to-end breast tumour classification model using context-based patch modelling - A BiLSTM approach for image classification.

Researchers working on computational analysis of Whole Slide Images (WSIs) in histopathology have primarily resorted to patch-based modelling due to large resolution of each WSI. The large resolution makes WSIs infeasible to be fed directly into the machine learning models due to computational constraints. However, due to patch-based analysis, most of the current methods fail to exploit the underlying spatial relationship among the patches. In our work, we have tried to integrate this relationship along with feature-based correlation among the extracted patches from the particular tumorous region. The tumour regions extracted from WSI have arbitrary dimensions having the range 20,570 to 195 pixels across width and 17,290 to 226 pixels across height. For the given task of classification, we have used BiLSTMs to model both forward and backward contextual relationship. Also, using RNN based model, the limitation of sequence size is eliminated which allows the modelling of variable size images within a deep learning model. We have also incorporated the effect of spatial continuity by exploring different scanning techniques used to sample patches. To establish the efficiency of our approach, we trained and tested our model on two datasets, microscopy images and WSI tumour regions. Both datasets were published by ICIAR BACH Challenge 2018. Finally, we compared our results with top 5 teams who participated in the BACH challenge and achieved the top accuracy of 90% for microscopy image dataset. For WSI tumour region dataset, we compared the classification results with state of the art deep learning networks such as ResNet, DenseNet, and InceptionV3 using maximum voting technique. We achieved the highest performance accuracy of 84%. We found out that BiLSTMs with CNN features have performed much better in modelling patches into an end-to-end Image classification network. Additionally, the variable dimensions of WSI tumour regions were used for classification without the need for resizing. This suggests that our method is independent of tumour image size and can process large dimensional images without losing the resolution details.

Regional registration of whole slide image stacks containing major histological artifacts.

BACKGROUND: High resolution 2D whole slide imaging provides rich information about the tissue structure. This information can be a lot richer if these 2D images can be stacked into a 3D tissue volume. A 3D analysis, however, requires accurate reconstruction of the tissue volume from the 2D image stack. This task is not trivial due to the distortions such as tissue tearing, folding and missing at each slide. Performing registration for the whole tissue slices may be adversely affected by distorted tissue regions. Consequently, regional registration is found to be more effective. In this paper, we propose a new approach to an accurate and robust registration of regions of interest for whole slide images. We introduce the idea of multi-scale attention for registration. RESULTS: Using mean similarity index as the metric, the proposed algorithm (mean ± SD [Formula: see text]) followed by a fine registration algorithm ([Formula: see text]) outperformed the state-of-the-art linear whole tissue registration algorithm ([Formula: see text]) and the regional version of this algorithm ([Formula: see text]). The proposed algorithm also outperforms the state-of-the-art nonlinear registration algorithm (original: [Formula: see text], regional: [Formula: see text]) for whole slide images and a recently proposed patch-based registration algorithm (patch size 256: [Formula: see text] , patch size 512: [Formula: see text]) for medical images. CONCLUSION: Using multi-scale attention mechanism leads to a more robust and accurate solution to the problem of regional registration of whole slide images corrupted in some parts by major histological artifacts in the imaged tissue.

Exploring MRI based radiomics analysis of intratumoral spatial heterogeneity in locally advanced nasopharyngeal carcinoma treated with intensity modulated radiotherapy.

BACKGROUND: We hypothesized that spatial heterogeneity exists between recurrent and non-recurrent regions within a tumor. The aim of this study was to determine if there is a difference between radiomics features derived from recurrent versus non recurrent regions within the tumor based on pre-treatment MRI. METHODS: A total of 14 T4NxM0 NPC patients with histologically proven "in field" recurrence in the post nasal space following curative intent IMRT were included in this study. Pretreatment MRI were co-registered with MRI at the time of recurrence for the delineation of gross tumor volume at diagnosis(GTV) and at recurrence(GTVr). A total of 7 histogram features and 40 texture features were computed from the recurrent(GTVr) and non-recurrent region(GTV-GTVr). Paired t-tests and Wilcoxon signed-rank tests were carried out on the 47 quantified radiomics features. RESULTS: A total of 7 features were significantly different between recurrent and non-recurrent regions. Other than the variance from intensity-based histogram, the remaining six significant features were either from the gray-level size zone matrix (GLSZM) or the neighbourhood gray-tone difference matrix (NGTDM). CONCLUSIONS: The radiomic features extracted from pre-treatment MRI can potentially reflect the difference between recurrent and non-recurrent regions within a tumor and has a potential role in pre-treatment identification of intra-tumoral radio-resistance for selective dose escalation.

Machine-learning study using improved correlation configuration and application to quantum Monte Carlo simulation.

We use the Fortuin-Kasteleyn representation-based improved estimator of the correlation configuration as an alternative to the ordinary correlation configuration in the machine-learning study of the phase classification of spin models. The phases of classical spin models are classified using the improved estimators, and the method is also applied to the quantum Monte Carlo simulation using the loop algorithm. We analyze the Berezinskii-Kosterlitz-Thouless (BKT) transition of the spin-1/2 quantum XY model on the square lattice. We classify the BKT phase and the paramagnetic phase of the quantum XY model using the machine-learning approach. We show that the classification of the quantum XY model can be performed by using the training data of the classical XY model.

Accelerated spin dynamics using deep learning corrections.

Theoretical models capture very precisely the behaviour of magnetic materials at the microscopic level. This makes computer simulations of magnetic materials, such as spin dynamics simulations, accurately mimic experimental results. New approaches to efficient spin dynamics simulations are limited by integration time step barrier to solving the equations-of-motions of many-body problems. Using a short time step leads to an accurate but inefficient simulation regime whereas using a large time step leads to accumulation of numerical errors that render the whole simulation useless. In this paper, we use a Deep Learning method to compute the numerical errors of each large time step and use these computed errors to make corrections to achieve higher accuracy in our spin dynamics. We validate our method on the 3D Ferromagnetic Heisenberg cubic lattice over a range of temperatures. Here we show that the Deep Learning method can accelerate the simulation speed by 10 times while maintaining simulation accuracy and overcome the limitations of requiring small time steps in spin dynamic simulations.

Machine-Learning Studies on Spin Models.

With the recent developments in machine learning, Carrasquilla and Melko have proposed a paradigm that is complementary to the conventional approach for the study of spin models. As an alternative to investigating the thermal average of macroscopic physical quantities, they have used the spin configurations for the classification of the disordered and ordered phases of a phase transition through machine learning. We extend and generalize this method. We focus on the configuration of the long-range correlation function instead of the spin configuration itself, which enables us to provide the same treatment to multi-component systems and the systems with a vector order parameter. We analyze the Berezinskii-Kosterlitz-Thouless (BKT) transition with the same technique to classify three phases: the disordered, the BKT, and the ordered phases. We also present the classification of a model using the training data of a different model.

Automated grading of acne vulgaris by deep learning with convolutional neural networks.

BACKGROUND: The visual assessment and severity grading of acne vulgaris by physicians can be subjective, resulting in inter- and intra-observer variability. OBJECTIVE: To develop and validate an algorithm for the automated calculation of the Investigator's Global Assessment (IGA) scale, to standardize acne severity and outcome measurements. MATERIALS AND METHODS: A total of 472 photographs (retrieved 01/01/2004-04/08/2017) in the frontal view from 416 acne patients were used for training and testing. Photographs were labeled according to the IGA scale in three groups of IGA clear/almost clear (0-1), IGA mild (2), and IGA moderate to severe (3-4). The classification model used a convolutional neural network, and models were separately trained on three image sizes. The photographs were then subjected to analysis by the algorithm, and the generated automated IGA scores were compared to clinical scoring. The prediction accuracy of each IGA grade label and the agreement (Pearson correlation) of the two scores were computed. RESULTS: The best classification accuracy was 67%. Pearson correlation between machine-predicted score and human labels (clinical scoring and researcher scoring) for each model and various image input sizes was 0.77. Correlation of predictions with clinical scores was highest when using Inception v4 on the largest image size of 1200 × 1600. Two sets of human labels showed a high correlation of 0.77, verifying the repeatability of the ground truth labels. Confusion matrices show that the models performed sub-optimally on the IGA 2 label. CONCLUSION: Deep learning techniques harnessing high-resolution images and large datasets will continue to improve, demonstrating growing potential for automated clinical image analysis and grading.

A compact network learning model for distribution regression.

Despite the superior performance of deep learning in many applications, challenges remain in the area of regression on function spaces. In particular, neural networks are unable to encode function inputs compactly as each node encodes just a real value. We propose a novel idea to address this shortcoming: to encode an entire function in a single network node. To that end, we design a compact network representation that encodes and propagates functions in single nodes for the distribution regression task. Our proposed distribution regression network (DRN) achieves higher prediction accuracies while using fewer parameters than traditional neural networks.