The Whole Is More Than Its Parts? From Explicit to Implicit Pose Normalization.

Fine-grained classification describes the automated recognition of visually similar object categories like birds species. Previous works were usually based on explicit pose normalization, i.e., the detection and description of object parts. However, recent models based on a final global average or bilinear pooling have achieved a comparable accuracy without this concept. In this paper, we analyze the advantages of these approaches over generic CNNs and explicit pose normalization approaches. We also show how they can achieve an implicit normalization of the object pose. A novel visualization technique called activation flow is introduced to investigate limitations in pose handling in traditional CNNs like AlexNet and VGG. Afterward, we present and compare the explicit pose normalization approach neural activation constellations and a generalized framework for the final global average and bilinear pooling called α-pooling. We observe that the latter often achieves a higher accuracy improving common CNN models by up to 22.9 percent, but lacks the interpretability of the explicit approaches. We present a visualization approach for understanding and analyzing predictions of the model to address this issue. Furthermore, we show that our approaches for fine-grained recognition are beneficial for other fields like action recognition.

Detecting Regions of Maximal Divergence for Spatio-Temporal Anomaly Detection.

Automatic detection of anomalies in space- and time-varying measurements is an important tool in several fields, e.g., fraud detection, climate analysis, or healthcare monitoring. We present an algorithm for detecting anomalous regions in multivariate spatio-temporal time-series, which allows for spotting the interesting parts in large amounts of data, including video and text data. In opposition to existing techniques for detecting isolated anomalous data points, we propose the "Maximally Divergent Intervals" (MDI) framework for unsupervised detection of coherent spatial regions and time intervals characterized by a high Kullback-Leibler divergence compared with all other data given. In this regard, we define an unbiased Kullback-Leibler divergence that allows for ranking regions of different size and show how to enable the algorithm to run on large-scale data sets in reasonable time using an interval proposal technique. Experiments on both synthetic and real data from various domains, such as climate analysis, video surveillance, and text forensics, demonstrate that our method is widely applicable and a valuable tool for finding interesting events in different types of data.

Fully convolutional networks in multimodal nonlinear microscopy images for automated detection of head and neck carcinoma: Pilot study.

BACKGROUND: A fully convolutional neural networks (FCN)-based automated image analysis algorithm to discriminate between head and neck cancer and noncancerous epithelium based on nonlinear microscopic images was developed. METHODS: Head and neck cancer sections were used for standard histopathology and co-registered with multimodal images from the same sections using the combination of coherent anti-Stokes Raman scattering, two-photon excited fluorescence, and second harmonic generation microscopy. The images analyzed with semantic segmentation using a FCN for four classes: cancer, normal epithelium, background, and other tissue types. RESULTS: A total of 114 images of 12 patients were analyzed. Using a patch score aggregation, the average recognition rate and an overall recognition rate or the four classes were 88.9% and 86.7%, respectively. A total of 113 seconds were needed to process a whole-slice image in the dataset. CONCLUSION: Multimodal nonlinear microscopy in combination with automated image analysis using FCN seems to be a promising technique for objective differentiation between head and neck cancer and noncancerous epithelium.

HER2 challenge contest: a detailed assessment of automated HER2 scoring algorithms in whole slide images of breast cancer tissues.

AIMS: Evaluating expression of the human epidermal growth factor receptor 2 (HER2) by visual examination of immunohistochemistry (IHC) on invasive breast cancer (BCa) is a key part of the diagnostic assessment of BCa due to its recognized importance as a predictive and prognostic marker in clinical practice. However, visual scoring of HER2 is subjective, and consequently prone to interobserver variability. Given the prognostic and therapeutic implications of HER2 scoring, a more objective method is required. In this paper, we report on a recent automated HER2 scoring contest, held in conjunction with the annual PathSoc meeting held in Nottingham in June 2016, aimed at systematically comparing and advancing the state-of-the-art artificial intelligence (AI)-based automated methods for HER2 scoring. METHODS AND RESULTS: The contest data set comprised digitized whole slide images (WSI) of sections from 86 cases of invasive breast carcinoma stained with both haematoxylin and eosin (H&E) and IHC for HER2. The contesting algorithms predicted scores of the IHC slides automatically for an unseen subset of the data set and the predicted scores were compared with the 'ground truth' (a consensus score from at least two experts). We also report on a simple 'Man versus Machine' contest for the scoring of HER2 and show that the automated methods could beat the pathology experts on this contest data set. CONCLUSIONS: This paper presents a benchmark for comparing the performance of automated algorithms for scoring of HER2. It also demonstrates the enormous potential of automated algorithms in assisting the pathologist with objective IHC scoring.

Automatic Classification of Cancerous Tissue in Laserendomicroscopy Images of the Oral Cavity using Deep Learning.

Oral Squamous Cell Carcinoma (OSCC) is a common type of cancer of the oral epithelium. Despite their high impact on mortality, sufficient screening methods for early diagnosis of OSCC often lack accuracy and thus OSCCs are mostly diagnosed at a late stage. Early detection and accurate outline estimation of OSCCs would lead to a better curative outcome and a reduction in recurrence rates after surgical treatment. Confocal Laser Endomicroscopy (CLE) records sub-surface micro-anatomical images for in vivo cell structure analysis. Recent CLE studies showed great prospects for a reliable, real-time ultrastructural imaging of OSCC in situ. We present and evaluate a novel automatic approach for OSCC diagnosis using deep learning technologies on CLE images. The method is compared against textural feature-based machine learning approaches that represent the current state of the art. For this work, CLE image sequences (7894 images) from patients diagnosed with OSCC were obtained from 4 specific locations in the oral cavity, including the OSCC lesion. The present approach is found to outperform the state of the art in CLE image recognition with an area under the curve (AUC) of 0.96 and a mean accuracy of 88.3% (sensitivity 86.6%, specificity 90%).

Automated analysis of confocal laser endomicroscopy images to detect head and neck cancer.

BACKGROUND: The purpose of this study was to develop an automated image analysis algorithm to discriminate between head and neck cancer and nonneoplastic epithelium in confocal laser endomicroscopy (CLE) images. METHODS: CLE was applied to image head and neck cancer epithelium in vivo. Histopathologic diagnosis from biopsies was used to classify the CLE images offline as cancer or noncancer tissue. The classified images were used to train automated software based on distance map histograms. The performance of the final algorithm was confirmed by "leave 2 patients out" cross-validation and area under the curve (AUC)/receiver operating characteristic (ROC) analysis. RESULTS: Ninety-two CLE videos and 92 biopsies were analyzed from 12 patients. One hundred two frames of classified neoplastic tissue and 52 frames of nonneoplastic tissue were used for cross-validation of the developed algorithm. AUC varied from 0.52 to 0.92. CONCLUSION: The proposed software allows an objective classification of CLE images of head and neck cancer and adjacent nonneoplastic epithelium. © 2015 Wiley Periodicals, Inc. Head Neck 38: E1419-E1426, 2016.

Automatic identification of novel bacteria using Raman spectroscopy and Gaussian processes.

Raman spectroscopy is successfully used for the reliable classification of complex biological samples. Much effort concentrates on the accurate prediction of known categories for highly relevant tasks in a wide area of applications such as cancer detection and bacteria recognition. However, the resulting recognition systems cannot always be directly used in practice since unseen samples might not belong to classes present in the training set. Our work aims to tackle this problem of novelty detection using a recently proposed approach based on Gaussian processes. By learning novelty scores for a large bacteria Raman dataset comprising 50 different strains, we analyze the behavior of this method on an independent dataset which includes known as well as unknown categories. Our experiment reveals that non-parametric methods such as Gaussian processes can be successfully applied to the task of finding unknown bacterial strains, leading to encouraging results motivating their further utilization in this area.