Machine learning improves early prediction of small-for-gestational-age births and reveals nuchal fold thickness as unexpected predictor.

OBJECTIVE: To investigate the performance of the machine learning (ML) model in predicting small-for-gestational-age (SGA) at birth, using second-trimester data. METHODS: Retrospective data of 347 patients, consisting of maternal demographics and ultrasound parameters collected between the 20th and 25th gestational weeks, were studied. ML models were applied to different combinations of the parameters to predict SGA and severe SGA at birth (defined as 10th and third centile birth weight). RESULTS: Using second-trimester measurements, ML models achieved an accuracy of 70% and 73% in predicting SGA and severe SGA whereas clinical guidelines had accuracies of 64% and 48%. Uterine PI (Ut PI) was found to be an important predictor, corroborating with existing literature, but surprisingly, so was nuchal fold thickness (NF). Logistic regression showed that Ut PI and NF were significant predictors and statistical comparisons showed that these parameters were significantly different in disease. Further, including NF was found to improve ML model performance, and vice versa. CONCLUSION: ML could potentially improve the prediction of SGA at birth from second-trimester measurements, and demonstrated reduced NF to be an important predictor. Early prediction of SGA allows closer clinical monitoring, which provides an opportunity to discover any underlying diseases associated with SGA.

Regional registration of whole slide image stacks containing major histological artifacts.

BACKGROUND: High resolution 2D whole slide imaging provides rich information about the tissue structure. This information can be a lot richer if these 2D images can be stacked into a 3D tissue volume. A 3D analysis, however, requires accurate reconstruction of the tissue volume from the 2D image stack. This task is not trivial due to the distortions such as tissue tearing, folding and missing at each slide. Performing registration for the whole tissue slices may be adversely affected by distorted tissue regions. Consequently, regional registration is found to be more effective. In this paper, we propose a new approach to an accurate and robust registration of regions of interest for whole slide images. We introduce the idea of multi-scale attention for registration. RESULTS: Using mean similarity index as the metric, the proposed algorithm (mean ± SD [Formula: see text]) followed by a fine registration algorithm ([Formula: see text]) outperformed the state-of-the-art linear whole tissue registration algorithm ([Formula: see text]) and the regional version of this algorithm ([Formula: see text]). The proposed algorithm also outperforms the state-of-the-art nonlinear registration algorithm (original: [Formula: see text], regional: [Formula: see text]) for whole slide images and a recently proposed patch-based registration algorithm (patch size 256: [Formula: see text] , patch size 512: [Formula: see text]) for medical images. CONCLUSION: Using multi-scale attention mechanism leads to a more robust and accurate solution to the problem of regional registration of whole slide images corrupted in some parts by major histological artifacts in the imaged tissue.

Automated grading of acne vulgaris by deep learning with convolutional neural networks.

BACKGROUND: The visual assessment and severity grading of acne vulgaris by physicians can be subjective, resulting in inter- and intra-observer variability. OBJECTIVE: To develop and validate an algorithm for the automated calculation of the Investigator's Global Assessment (IGA) scale, to standardize acne severity and outcome measurements. MATERIALS AND METHODS: A total of 472 photographs (retrieved 01/01/2004-04/08/2017) in the frontal view from 416 acne patients were used for training and testing. Photographs were labeled according to the IGA scale in three groups of IGA clear/almost clear (0-1), IGA mild (2), and IGA moderate to severe (3-4). The classification model used a convolutional neural network, and models were separately trained on three image sizes. The photographs were then subjected to analysis by the algorithm, and the generated automated IGA scores were compared to clinical scoring. The prediction accuracy of each IGA grade label and the agreement (Pearson correlation) of the two scores were computed. RESULTS: The best classification accuracy was 67%. Pearson correlation between machine-predicted score and human labels (clinical scoring and researcher scoring) for each model and various image input sizes was 0.77. Correlation of predictions with clinical scores was highest when using Inception v4 on the largest image size of 1200 × 1600. Two sets of human labels showed a high correlation of 0.77, verifying the repeatability of the ground truth labels. Confusion matrices show that the models performed sub-optimally on the IGA 2 label. CONCLUSION: Deep learning techniques harnessing high-resolution images and large datasets will continue to improve, demonstrating growing potential for automated clinical image analysis and grading.

Semi-automated quantitative Drosophila wings measurements.

BACKGROUND: Drosophila melanogaster is an important organism used in many fields of biological research such as genetics and developmental biology. Drosophila wings have been widely used to study the genetics of development, morphometrics and evolution. Therefore there is much interest in quantifying wing structures of Drosophila. Advancement in technology has increased the ease in which images of Drosophila can be acquired. However such studies have been limited by the slow and tedious process of acquiring phenotypic data. RESULTS: We have developed a system that automatically detects and measures key points and vein segments on a Drosophila wing. Key points are detected by performing image transformations and template matching on Drosophila wing images while vein segments are detected using an Active Contour algorithm. The accuracy of our key point detection was compared against key point annotations of users. We also performed key point detection using different training data sets of Drosophila wing images. We compared our software with an existing automated image analysis system for Drosophila wings and showed that our system performs better than the state of the art. Vein segments were manually measured and compared against the measurements obtained from our system. CONCLUSION: Our system was able to detect specific key points and vein segments from Drosophila wing images with high accuracy.

Large-scale image-based screening and profiling of cellular phenotypes.

Cellular phenotypes are observable characteristics of cells resulting from the interactions of intrinsic and extrinsic chemical or biochemical factors. Image-based phenotypic screens under large numbers of basal or perturbed conditions can be used to study the influences of these factors on cellular phenotypes. Hundreds to thousands of phenotypic descriptors can also be quantified from the images of cells under each of these experimental conditions. Therefore, huge amounts of data can be generated, and the analysis of these data has become a major bottleneck in large-scale phenotypic screens. Here, we review current experimental and computational methods for large-scale image-based phenotypic screens. Our focus is on phenotypic profiling, a computational procedure for constructing quantitative and compact representations of cellular phenotypes based on the images collected in these screens. © 2016 International Society for Advancement of Cytometry.

Optimal processing for gel electrophoresis images: Applying Monte Carlo Tree Search in GelApp.

In biomedical research, gel band size estimation in electrophoresis analysis is a routine process. To facilitate and automate this process, numerous software have been released, notably the GelApp mobile app. However, the band detection accuracy is limited due to a band detection algorithm that cannot adapt to the variations in input images. To address this, we used the Monte Carlo Tree Search with Upper Confidence Bound (MCTS-UCB) method to efficiently search for optimal image processing pipelines for the band detection task, thereby improving the segmentation algorithm. Incorporating this into GelApp, we report a significant enhancement of gel band detection accuracy by 55.9 ± 2.0% for protein polyacrylamide gels, and 35.9 ± 2.5% for DNA SYBR green agarose gels. This implementation is a proof-of-concept in demonstrating MCTS-UCB as a strategy to optimize general image segmentation. The improved version of GelApp-GelApp 2.0-is freely available on both Google Play Store (for Android platform), and Apple App Store (for iOS platform).

Myopia in asian subjects with primary angle closure: implications for glaucoma trends in East Asia.

PURPOSE: To evaluate the occurrence of myopia in Asian subjects with angle closure and to assess the ocular biometric parameters in these subjects. DESIGN: Cross-sectional study. PARTICIPANTS: We prospectively recruited 427 angle-closure subjects (143 primary angle-closure suspects, 75 patients with primary angle closure, 165 patients with primary angle-closure glaucoma, and 44 patients with acute primary angle closure) from a Singapore hospital. METHODS: Refractive status was derived from the spherical equivalent of autorefraction. A-scan biometry (Nidek Echoscan Ultrasound US-800; Nidek Co., Tokyo, Japan) was performed to obtain anterior chamber depth (ACD), axial length (AL), lens thickness, and vitreous cavity length (VL). Anterior segment optical coherence tomography was performed to measure lens vault. MAIN OUTCOME MEASURES: Refractive status was categorized as myopia (≤-0.50 diopter [D]), emmetropia (-0.50 to +0.50 D), and hyperopia (≥+0.50 D). RESULTS: The mean age ± standard deviation of study subjects was 65.6 ± 7.6 years, with most being Chinese (n = 394; 92.3%) and women (n = 275; 64.4%). Overall, myopia was present in 94 subjects (22%), hyperopia was present in 222 subjects (52%), and emmetropia was present in 111 subjects (26%). Of the 94 myopic angle-closure patients, 28 (29.8%) were categorized as having moderate myopia (≤-2.0 to -5.0 D) and 11 (11.7%) were categorized as having high myopia (≤-5.00 D). Although myopic angle-closure subjects had longer ALs (P<0.001) and VLs (P = 0.001) than their emmetropic and hyperopic counterparts, there were no significant differences in ACD (P = 0.77), lens thickness (P = 0.44), or lens vault (P = 0.053). CONCLUSIONS: Almost one quarter of angle-closure patients were myopic. Myopic angle-closure subjects had longer VLs and ALs, but there was no difference in ACD. With the increasing rate of myopia in many East Asian populations, there may be many subjects with axial myopia but shallow ACD and angle closure. The implication is that ophthalmologists should not assume that glaucoma patients who are myopic have open angles.

Light sheet fluorescence microscopy (LSFM): past, present and future.

Light sheet fluorescence microscopy (LSFM) has emerged as an important imaging modality to follow biology in live 3D samples over time with reduced phototoxicity and photobleaching. In particular, LSFM has been instrumental in revealing the detail of early embryonic development of Zebrafish, Drosophila, and C. elegans. Open access projects, DIY-SPIM, OpenSPIM, and OpenSPIN, now allow LSFM to be set-up easily and at low cost. The aim of this paper is to facilitate the set-up and use of LSFM by reviewing and comparing open access projects, image processing tools and future challenges.

BANKSY unifies cell typing and tissue domain segmentation for scalable spatial omics data analysis.

Spatial omics data are clustered to define both cell types and tissue domains. We present Building Aggregates with a Neighborhood Kernel and Spatial Yardstick (BANKSY), an algorithm that unifies these two spatial clustering problems by embedding cells in a product space of their own and the local neighborhood transcriptome, representing cell state and microenvironment, respectively. BANKSY's spatial feature augmentation strategy improved performance on both tasks when tested on diverse RNA (imaging, sequencing) and protein (imaging) datasets. BANKSY revealed unexpected niche-dependent cell states in the mouse brain and outperformed competing methods on domain segmentation and cell typing benchmarks. BANKSY can also be used for quality control of spatial transcriptomics data and for spatially aware batch effect correction. Importantly, it is substantially faster and more scalable than existing methods, enabling the processing of millions of cell datasets. In summary, BANKSY provides an accurate, biologically motivated, scalable and versatile framework for analyzing spatially resolved omics data.

A comprehensive AI model development framework for consistent Gleason grading.

BACKGROUND: Artificial Intelligence(AI)-based solutions for Gleason grading hold promise for pathologists, while image quality inconsistency, continuous data integration needs, and limited generalizability hinder their adoption and scalability. METHODS: We present a comprehensive digital pathology workflow for AI-assisted Gleason grading. It incorporates A!MagQC (image quality control), A!HistoClouds (cloud-based annotation), Pathologist-AI Interaction (PAI) for continuous model improvement, Trained on Akoya-scanned images only, the model utilizes color augmentation and image appearance migration to address scanner variations. We evaluate it on Whole Slide Images (WSI) from another five scanners and conduct validations with pathologists to assess AI efficacy and PAI. RESULTS: Our model achieves an average F1 score of 0.80 on annotations and 0.71 Quadratic Weighted Kappa on WSIs for Akoya-scanned images. Applying our generalization solution increases the average F1 score for Gleason pattern detection from 0.73 to 0.88 on images from other scanners. The model accelerates Gleason scoring time by 43% while maintaining accuracy. Additionally, PAI improve annotation efficiency by 2.5 times and led to further improvements in model performance. CONCLUSIONS: This pipeline represents a notable advancement in AI-assisted Gleason grading for improved consistency, accuracy, and efficiency. Unlike previous methods limited by scanner specificity, our model achieves outstanding performance across diverse scanners. This improvement paves the way for its seamless integration into clinical workflows.

Gleason grading is a well-accepted diagnostic standard to assess the severity of prostate cancer in patients' tissue samples, based on how abnormal the cells in their prostate tumor look under a microscope. This process can be complex and time-consuming. We explore how artificial intelligence (AI) can help pathologists perform Gleason grading more efficiently and consistently. We build an AI-based system which automatically checks image quality, standardizes the appearance of images from different equipment, learns from pathologists' feedback, and constantly improves model performance. Testing shows that our approach achieves consistent results across different equipment and improves efficiency of the grading process. With further testing and implementation in the clinic, our approach could potentially improve prostate cancer diagnosis and management.

Subgrouping of primary angle-closure suspects based on anterior segment optical coherence tomography parameters.

PURPOSE: To identify subgroups of primary angle-closure suspects (PACS) based on anterior segment optical coherence tomography (AS-OCT) and biometric parameters. DESIGN: Cross-sectional study. PARTICIPANTS: We evaluated 243 PACS subjects in the primary group and 165 subjects in the validation group. METHODS: Participants underwent gonioscopy and AS-OCT (Carl Zeiss Meditec, Dublin, CA). Customized software (Zhongshan Angle Assessment Program, Guangzhou, China) was used to measure AS-OCT parameters. An agglomerative hierarchical clustering method was first used to determine the optimum number of parameters to be included in the determination of subgroups. The best number of subgroups was then determined using Akaike Information Criterion (AIC) and Gaussian Mixture Model (GMM) methods. MAIN OUTCOME MEASURES: Subgroups of PACS. RESULTS: The mean age of the subjects was 64.8 years, and 65.02% were female. After hierarchical clustering, 1 or 2 parameters from each cluster were chosen to ensure representativeness of the parameters and yet keep a minimum of redundancy. The parameters included were iris area, anterior chamber depth (ACD), anterior chamber width (ACW), and lens vault (LV). With the use of GMM, the optimal number of subgroups as given by AIC was 3. Subgroup 1 was characterized by a large iris area, subgroup 2 was characterized by a large LV and a shallow ACD, and subgroup 3 was characterized by elements of both subgroups 1 and 2. The results were replicated in a second independent group of 165 PACS subjects. CONCLUSIONS: Clustering analysis identified 3 distinct subgroups of PACS subjects based on AS-OCT and biometric parameters. These findings may be relevant for understanding angle-closure pathogenesis and management.

Improving transparency and representational generalizability through parallel continual learning.

This paper takes a parallel learning approach in continual learning scenarios. We define parallel continual learning as learning a sequence of tasks where the data for the previous tasks, whose distribution may have shifted over time, are also available while learning new tasks. We propose a parallel continual learning method by assigning subnetworks to each task, and simultaneously training only the assigned subnetworks on their corresponding tasks. In doing so, some parts of the network will be shared across multiple tasks. This is unlike the existing literature in continual learning which aims at learning incoming tasks sequentially, with the assumption that the data for the previous tasks have a fixed distribution. Our proposed method offers promises in: (1) Transparency in the network and in the relationship across tasks by enabling examination of the learned representations by independent and shared subnetworks, (2) Representation generalizability through sharing and training subnetworks on multiple tasks simultaneously. Our analysis shows that compared to many competing approaches such as continual learning, neural architecture search, and multi-task learning, parallel continual learning is capable of learning more generalizable representations. Also, (3)Parallel continual learning overcomes the common issue of catastrophic forgetting in continual learning algorithms. This is the first effort to train a neural network on multiple tasks and input domains simultaneously in a continual learning scenario. Our code is available at https://github.com/yours-anonym/PaRT.

Distribution based MIL pooling filters: Experiments on a lymph node metastases dataset.

Histopathology is a crucial diagnostic tool in cancer and involves the analysis of gigapixel slides. Multiple instance learning (MIL) promises success in digital histopathology thanks to its ability to handle gigapixel slides and work with weak labels. MIL is a machine learning paradigm that learns the mapping between bags of instances and bag labels. It represents a slide as a bag of patches and uses the slide's weak label as the bag's label. This paper introduces distribution-based pooling filters that obtain a bag-level representation by estimating marginal distributions of instance features. We formally prove that the distribution-based pooling filters are more expressive than the classical point estimate-based counterparts, like 'max' and 'mean' pooling, in terms of the amount of information captured while obtaining bag-level representations. Moreover, we empirically show that models with distribution-based pooling filters perform equal to or better than those with point estimate-based pooling filters on distinct real-world MIL tasks defined on the CAMELYON16 lymph node metastases dataset. Our model with a distribution pooling filter achieves an area under the receiver operating characteristics curve value of 0.9325 (95% confidence interval: 0.8798 - 0.9743) in the tumor vs. normal slide classification task.

Oner, Sung, and Lee: Researchers in digital pathology for the future of modern medicine.

Oner, an early-career researcher, and Lee and Sung, group leaders, have developed a deep learning model for accurate prediction of the proportion of cancer cells within tumor tissue. This is a necessary step for precision oncology and target therapy in cancer. They talk about their view of data science and the evolution of pathology in the coming years.

An AI-assisted tool for efficient prostate cancer diagnosis in low-grade and low-volume cases.

Pathologists diagnose prostate cancer by core needle biopsy. In low-grade and low-volume cases, they look for a few malignant glands out of hundreds within a core. They may miss a few malignant glands, resulting in repeat biopsies or missed therapeutic opportunities. This study developed a multi-resolution deep-learning pipeline to assist pathologists in detecting malignant glands in core needle biopsies of low-grade and low-volume cases. Analyzing a gland at multiple resolutions, our model exploited morphology and neighborhood information, which were crucial in prostate gland classification. We developed and tested our pipeline on the slides of a local cohort of 99 patients in Singapore. Besides, we made the images publicly available, becoming the first digital histopathology dataset of patients of Asian ancestry with prostatic carcinoma. Our multi-resolution classification model achieved an area under the receiver operating characteristic curve (AUROC) value of 0.992 (95% confidence interval [CI]: 0.985-0.997) in the external validation study, showing the generalizability of our multi-resolution approach.

Obtaining spatially resolved tumor purity maps using deep multiple instance learning in a pan-cancer study.

Tumor purity is the percentage of cancer cells within a tissue section. Pathologists estimate tumor purity to select samples for genomic analysis by manually reading hematoxylin-eosin (H&E)-stained slides, which is tedious, time consuming, and prone to inter-observer variability. Besides, pathologists' estimates do not correlate well with genomic tumor purity values, which are inferred from genomic data and accepted as accurate for downstream analysis. We developed a deep multiple instance learning model predicting tumor purity from H&E-stained digital histopathology slides. Our model successfully predicted tumor purity in eight The Cancer Genome Atlas (TCGA) cohorts and a local Singapore cohort. The predictions were highly consistent with genomic tumor purity values. Thus, our model can be utilized to select samples for genomic analysis, which will help reduce pathologists' workload and decrease inter-observer variability. Furthermore, our model provided tumor purity maps showing the spatial variation within sections. They can help better understand the tumor microenvironment.

Automated interpretation of systolic and diastolic function on the echocardiogram: a multicohort study.

BACKGROUND: Echocardiography is the diagnostic modality for assessing cardiac systolic and diastolic function to diagnose and manage heart failure. However, manual interpretation of echocardiograms can be time consuming and subject to human error. Therefore, we developed a fully automated deep learning workflow to classify, segment, and annotate two-dimensional (2D) videos and Doppler modalities in echocardiograms. METHODS: We developed the workflow using a training dataset of 1145 echocardiograms and an internal test set of 406 echocardiograms from the prospective heart failure research platform (Asian Network for Translational Research and Cardiovascular Trials; ATTRaCT) in Asia, with previous manual tracings by expert sonographers. We validated the workflow against manual measurements in a curated dataset from Canada (Alberta Heart Failure Etiology and Analysis Research Team; HEART; n=1029 echocardiograms), a real-world dataset from Taiwan (n=31 241), the US-based EchoNet-Dynamic dataset (n=10 030), and in an independent prospective assessment of the Asian (ATTRaCT) and Canadian (Alberta HEART) datasets (n=142) with repeated independent measurements by two expert sonographers. FINDINGS: In the ATTRaCT test set, the automated workflow classified 2D videos and Doppler modalities with accuracies (number of correct predictions divided by the total number of predictions) ranging from 0·91 to 0·99. Segmentations of the left ventricle and left atrium were accurate, with a mean Dice similarity coefficient greater than 93% for all. In the external datasets (n=1029 to 10 030 echocardiograms used as input), automated measurements showed good agreement with locally measured values, with a mean absolute error range of 9-25 mL for left ventricular volumes, 6-10% for left ventricular ejection fraction (LVEF), and 1·8-2·2 for the ratio of the mitral inflow E wave to the tissue Doppler e' wave (E/e' ratio); and reliably classified systolic dysfunction (LVEF <40%, area under the receiver operating characteristic curve [AUC] range 0·90-0·92) and diastolic dysfunction (E/e' ratio ≥13, AUC range 0·91-0·91), with narrow 95% CIs for AUC values. Independent prospective evaluation confirmed less variance of automated compared with human expert measurements, with all individual equivalence coefficients being less than 0 for all measurements. INTERPRETATION: Deep learning algorithms can automatically annotate 2D videos and Doppler modalities with similar accuracy to manual measurements by expert sonographers. Use of an automated workflow might accelerate access, improve quality, and reduce costs in diagnosing and managing heart failure globally. FUNDING: A*STAR Biomedical Research Council and A*STAR Exploit Technologies.

Automatic measurement of volume percentage stroma in endometrial images using texture segmentation.

The popularity of digital microscopy and tissue microarrays allow the use of high-throughput imaging for pathology research. To coordinate with this new technique, it is essential to automate the process of extracting information from such high amount of images. In this paper, we present a new model called the Subspace Mumford-Shah model for texture segmentation of microscopic endometrial images. The model incorporates subspace clustering techniques into a Mumford-Shah model to solve texture segmentation problems. The method first uses a supervised procedure to determine several optimal subspaces. These subspaces are then embedded into a Mumford-Shah objective function so that each segment of the optimal partition is homogeneous in its own subspace. The method outperforms a widely used method in bioimaging community called k-means segmentation since it can separate textures which are less separated in the full feature space, which confirm the usefulness of subspace clustering in texture segmentation. Experimental results also show that the proposed method is well performed on diagnosing premalignant endometrial disease and is very practical for segmenting image set sharing similar properties.

Subspace learning for Mumford-Shah-model-based texture segmentation through texture patches.

In this paper, we develop a robust and effective algorithm for texture segmentation and feature selection. The approach is to incorporate a patch-based subspace learning technique into the subspace Mumford-Shah (SMS) model to make the minimization of the SMS model robust and accurate. The proposed method is fully unsupervised in that it removes the need to specify training data, which is required by existing methods for the same model. We further propose a novel (to our knowledge) pairwise dissimilarity measure for pixels. Its novelty lies in the use of the relevance scores of the features of each pixel to improve its discriminating power. Some superior results are obtained compared to existing unsupervised algorithms, which do not use a subspace approach. This confirms the usefulness of the subspace approach and the proposed unsupervised algorithm.

Restoration of uneven illumination in light sheet microscopy images.

Light microscopy images suffer from poor contrast due to light absorption and scattering by the media. The resulting decay in contrast varies exponentially across the image along the incident light path. Classical space invariant deconvolution approaches, while very effective in deblurring, are not designed for the restoration of uneven illumination in microscopy images. In this article, we present a modified radiative transfer theory approach to solve the contrast degradation problem of light sheet microscopy (LSM) images. We confirmed the effectiveness of our approach through simulation as well as real LSM images.