Multi-Modal Ensemble Deep Learning in Head and Neck Cancer HPV Sub-Typing.

Oropharyngeal Squamous Cell Carcinoma (OPSCC) is one of the common forms of heterogeneity in head and neck cancer. Infection with human papillomavirus (HPV) has been identified as a major risk factor for OPSCC. Therefore, differentiating the HPV-positive and negative cases in OPSCC patients is an essential diagnostic factor influencing future treatment decisions. In this study, we investigated the accuracy of a deep learning-based method for image interpretation and automatically detected the HPV status of OPSCC in routinely acquired Computed Tomography (CT) and Positron Emission Tomography (PET) images. We introduce a 3D CNN-based multi-modal feature fusion architecture for HPV status prediction in primary tumor lesions. The architecture is composed of an ensemble of CNN networks and merges image features in a softmax classification layer. The pipeline separately learns the intensity, contrast variation, shape, texture heterogeneity, and metabolic assessment from CT and PET tumor volume regions and fuses those multi-modal features for final HPV status classification. The precision, recall, and AUC scores of the proposed method are computed, and the results are compared with other existing models. The experimental results demonstrate that the multi-modal ensemble model with soft voting outperformed single-modality PET/CT, with an AUC of 0.76 and F1 score of 0.746 on publicly available TCGA and MAASTRO datasets. In the MAASTRO dataset, our model achieved an AUC score of 0.74 over primary tumor volumes of interest (VOIs). In the future, more extensive cohort validation may suffice for better diagnostic accuracy and provide preliminary assessment before the biopsy.

Improving Medical Images Classification With Label Noise Using Dual-Uncertainty Estimation.

Deep neural networks are known to be data-driven and label noise can have a marked impact on model performance. Recent studies have shown great robustness to classic image recognition even under a high noisy rate. In medical applications, learning from datasets with label noise is more challenging since medical imaging datasets tend to have instance-dependent noise (IDN) and suffer from high observer variability. In this paper, we systematically discuss the two common types of label noise in medical images - disagreement label noise from inconsistency expert opinions and single-target label noise from biased aggregation of individual annotations. We then propose an uncertainty estimation-based framework to handle these two label noise amid the medical image classification task. We design a dual-uncertainty estimation approach to measure the disagreement label noise and single-target label noise via improved Direct Uncertainty Prediction and Monte-Carlo-Dropout. A boosting-based curriculum training procedure is later introduced for robust learning. We demonstrate the effectiveness of our method by conducting extensive experiments on three different diseases with synthesized and real-world label noise: skin lesions, prostate cancer, and retinal diseases. We also release a large re-engineered database that consists of annotations from more than ten ophthalmologists with an unbiased golden standard dataset for evaluation and benchmarking. The dataset is available at https://mmai.group/peoples/julie/.

MoNuSAC2020: A Multi-Organ Nuclei Segmentation and Classification Challenge.

Detecting various types of cells in and around the tumor matrix holds a special significance in characterizing the tumor micro-environment for cancer prognostication and research. Automating the tasks of detecting, segmenting, and classifying nuclei can free up the pathologists' time for higher value tasks and reduce errors due to fatigue and subjectivity. To encourage the computer vision research community to develop and test algorithms for these tasks, we prepared a large and diverse dataset of nucleus boundary annotations and class labels. The dataset has over 46,000 nuclei from 37 hospitals, 71 patients, four organs, and four nucleus types. We also organized a challenge around this dataset as a satellite event at the International Symposium on Biomedical Imaging (ISBI) in April 2020. The challenge saw a wide participation from across the world, and the top methods were able to match inter-human concordance for the challenge metric. In this paper, we summarize the dataset and the key findings of the challenge, including the commonalities and differences between the methods developed by various participants. We have released the MoNuSAC2020 dataset to the public.

Active learning based segmentation of Crohns disease from abdominal MRI.

This paper proposes a novel active learning (AL) framework, and combines it with semi supervised learning (SSL) for segmenting Crohns disease (CD) tissues from abdominal magnetic resonance (MR) images. Robust fully supervised learning (FSL) based classifiers require lots of labeled data of different disease severities. Obtaining such data is time consuming and requires considerable expertise. SSL methods use a few labeled samples, and leverage the information from many unlabeled samples to train an accurate classifier. AL queries labels of most informative samples and maximizes gain from the labeling effort. Our primary contribution is in designing a query strategy that combines novel context information with classification uncertainty and feature similarity. Combining SSL and AL gives a robust segmentation method that: (1) optimally uses few labeled samples and many unlabeled samples; and (2) requires lower training time. Experimental results show our method achieves higher segmentation accuracy than FSL methods with fewer samples and reduced training effort.

Image Registration Based on Autocorrelation of Local Structure.

Registration of images in the presence of intra-image signal fluctuations is a challenging task. The definition of an appropriate objective function measuring the similarity between the images is crucial for accurate registration. This paper introduces an objective function that embeds local phase features derived from the monogenic signal in the modality independent neighborhood descriptor (MIND). The image similarity relies on the autocorrelation of local structure (ALOST) which has two important properties: 1) low sensitivity to space-variant intensity distortions (e.g., differences in contrast enhancement in MRI); 2) high distinctiveness for 'salient' image features such as edges. The ALOST method is quantitatively compared to the MIND approach based on three different datasets: thoracic CT images, synthetic and real abdominal MR images. The proposed method outperformed the NMI and MIND similarity measures on these three datasets. The registration of dynamic contrast enhanced and post-contrast MR images of patients with Crohn's disease led to relative contrast enhancement measures with the highest correlation (r=0.56) to the Crohn's disease endoscopic index of severity.

Prostate MRI segmentation using learned semantic knowledge and graph cuts.

We propose a fully automated method for prostate segmentation using random forests (RFs) and graph cuts. A volume of interest (VOI) is automatically selected using supervoxel segmentation, and its subsequent classification using image features and RF classifiers. The VOIs probability map is generated using image and context features, and a second set of RF classifiers. The negative log-likelihood of the probability maps acts as the penalty cost in a second-order Markov random field cost function. Semantic information from the second set of RF classifiers is an important measure of each feature to the classification task, which contributes to formulating the smoothness cost. The cost function is optimized using graph cuts to get the final segmentation of the prostate. With average dice metric (DM) (on the training set) and DM (on the test set), our experimental results show that inclusion of the context and semantic information contributes to higher segmentation accuracy than other methods.

A supervised learning approach for Crohn's disease detection using higher-order image statistics and a novel shape asymmetry measure.

Increasing incidence of Crohn's disease (CD) in the Western world has made its accurate diagnosis an important medical challenge. The current reference standard for diagnosis, colonoscopy, is time-consuming and invasive while magnetic resonance imaging (MRI) has emerged as the preferred noninvasive procedure over colonoscopy. Current MRI approaches assess rate of contrast enhancement and bowel wall thickness, and rely on extensive manual segmentation for accurate analysis. We propose a supervised learning method for the identification and localization of regions in abdominal magnetic resonance images that have been affected by CD. Low-level features like intensity and texture are used with shape asymmetry information to distinguish between diseased and normal regions. Particular emphasis is laid on a novel entropy-based shape asymmetry method and higher-order statistics like skewness and kurtosis. Multi-scale feature extraction renders the method robust. Experiments on real patient data show that our features achieve a high level of accuracy and perform better than two competing methods.

Semi-supervised and active learning for automatic segmentation of Crohn's disease.

Our proposed method combines semi supervised learning (SSL) and active learning (AL) for automatic detection and segmentation of Crohn's disease (CD) from abdominal magnetic resonance (MR) images. Random forest (RF) classifiers are used due to fast SSL classification and capacity to interpret learned knowledge. Query samples for AL are selected by a novel information density weighted approach using context information, semantic knowledge and labeling uncertainty. Experimental results show that our proposed method combines the advantages of SSL and AL, and with fewer samples achieves higher classification and segmentation accuracy over fully supervised methods.

Computational modeling for assessment of IBD: to be or not to be?

The grading of inflammatory bowel disease (IBD) severity is important to determine the proper treatment strategy and to quantify the response to treatment. Traditionally, ileocolonoscopy is considered the reference standard for assessment of IBD. However, the procedure is invasive and requires extensive bowel preparation. Magnetic resonance imaging (MRI) has become an important tool for determining the presence of disease activity. Unfortunately, only moderate interobserver agreement is reported for most of the radiological severity measures. There is a clear demand for automated evaluation of MR images in Crohn's disease (CD). This paper aims to introduce a preliminary suite of fundamental tools for assessment of CD severity. It involves procedures for image analysis, classification and visualization to predict the colonoscopy disease scores.