Prediction of molecular subclasses of uveal melanoma by deep learning using routine haematoxylin-eosin-stained tissue slides.

AIMS: Uveal melanoma has a high propensity to metastasize. Prognosis is associated with specific driver mutations and copy number variations, and these can only be obtained after genetic testing. In this study we evaluated the efficacy of patient outcome prediction using deep learning on haematoxylin and eosin (HE)-stained primary uveal melanoma slides in comparison to molecular testing. METHODS: In this retrospective study of patients with uveal melanoma, 113 patients from the Erasmus Medical Centre who underwent enucleation had tumour tissue analysed for molecular classification between 1993 and 2020. Routine HE-stained slides were scanned to obtain whole-slide images (WSI). After annotation of regions of interest, tiles of 1024 × 1024 pixels were extracted at a magnification of 40×. An ablation study to select the best-performing deep-learning model was carried out using three state-of-the-art deep-learning models (EfficientNet, Vision Transformer, and Swin Transformer). RESULTS: Deep-learning models were subjected to a training cohort (n = 40), followed by a validation cohort (n = 20), and finally underwent a test cohort (n = 48). A k-fold cross-validation (k = 3) of validation and test cohorts (n = 113 of three classes: BAP1, SF3B1, EIF1AX) demonstrated Swin Transformer as the best-performing deep-learning model to predict molecular subclasses based on HE stains. The model achieved an accuracy of 0.83 ± 0.09 on the validation cohort and 0.75 ± 0.04 on the test cohort. Within the subclasses, this model correctly predicted 70% BAP1-mutated, 61% SF3B1-mutated and 80% EIF1AX-mutated UM in the test set. CONCLUSIONS: This study showcases the potential of the deep-learning methodology for predicting molecular subclasses in a multiclass manner using HE-stained WSI. This development holds promise for advanced prognostication of UM patients without the need of molecular or immunohistochemical testing. Additionally, this study suggests there are distinct histopathological features per subclass; mainly utilizing epithelioid cellular morphology for BAP1-classification, but an unknown feature distinguishes EIF1AX and SF3B1.

Artificial Intelligence in Digital Pathology for Bladder Cancer: Hype or Hope? A Systematic Review.

Bladder cancer (BC) diagnosis and prediction of prognosis are hindered by subjective pathological evaluation, which may cause misdiagnosis and under-/over-treatment. Computational pathology (CPATH) can identify clinical outcome predictors, offering an objective approach to improve prognosis. However, a systematic review of CPATH in BC literature is lacking. Therefore, we present a comprehensive overview of studies that used CPATH in BC, analyzing 33 out of 2285 identified studies. Most studies analyzed regions of interest to distinguish normal versus tumor tissue and identify tumor grade/stage and tissue types (e.g., urothelium, stroma, and muscle). The cell's nuclear area, shape irregularity, and roundness were the most promising markers to predict recurrence and survival based on selected regions of interest, with >80% accuracy. CPATH identified molecular subtypes by detecting features, e.g., papillary structures, hyperchromatic, and pleomorphic nuclei. Combining clinicopathological and image-derived features improved recurrence and survival prediction. However, due to the lack of outcome interpretability and independent test datasets, robustness and clinical applicability could not be ensured. The current literature demonstrates that CPATH holds the potential to improve BC diagnosis and prediction of prognosis. However, more robust, interpretable, accurate models and larger datasets-representative of clinical scenarios-are needed to address artificial intelligence's reliability, robustness, and black box challenge.

Self-initialized active contours for microscopic cell image segmentation.

Level set models are suitable for processing topological changes in different regions of images while performing segmentation. Active contour models require an empirical setting for initial parameters, which is tedious for the end-user. This study proposes an incremental level set model with the automatic initialization of contours based on local and global fitting energies that enable it to capture image regions containing intensity corruption or other light artifacts. The region-based area and the region-based length terms use signed pressure force (SPF) to strengthen the balloon force. SPF helps to achieve a smooth version of the gradient descent flow in terms of energy minimization. The proposed model is tested on multiple synthetic and real images. Our model has four advantages: first, there is no need for the end user to initialize the parameters; instead, the model is self-initialized. Second, it is more accurate than other methods. Third, it shows lower computational complexity. Fourth, it does not depend on the starting position of the contour. Finally, we evaluated the performance of our model on microscopic cell images (Coelho et al., in: 2009 IEEE international symposium on biomedical imaging: from nano to macro, IEEE, 2009) to confirm that its performance is superior to that of other state-of-the-art models.

Efficient Breast Cancer Classification Network with Dual Squeeze and Excitation in Histopathological Images.

Medical image analysis methods for mammograms, ultrasound, and magnetic resonance imaging (MRI) cannot provide the underline features on the cellular level to understand the cancer microenvironment which makes them unsuitable for breast cancer subtype classification study. In this paper, we propose a convolutional neural network (CNN)-based breast cancer classification method for hematoxylin and eosin (H&E) whole slide images (WSIs). The proposed method incorporates fused mobile inverted bottleneck convolutions (FMB-Conv) and mobile inverted bottleneck convolutions (MBConv) with a dual squeeze and excitation (DSE) network to accurately classify breast cancer tissue into binary (benign and malignant) and eight subtypes using histopathology images. For that, a pre-trained EfficientNetV2 network is used as a backbone with a modified DSE block that combines the spatial and channel-wise squeeze and excitation layers to highlight important low-level and high-level abstract features. Our method outperformed ResNet101, InceptionResNetV2, and EfficientNetV2 networks on the publicly available BreakHis dataset for the binary and multi-class breast cancer classification in terms of precision, recall, and F1-score on multiple magnification levels.

Exploring MRI based radiomics analysis of intratumoral spatial heterogeneity in locally advanced nasopharyngeal carcinoma treated with intensity modulated radiotherapy.

BACKGROUND: We hypothesized that spatial heterogeneity exists between recurrent and non-recurrent regions within a tumor. The aim of this study was to determine if there is a difference between radiomics features derived from recurrent versus non recurrent regions within the tumor based on pre-treatment MRI. METHODS: A total of 14 T4NxM0 NPC patients with histologically proven "in field" recurrence in the post nasal space following curative intent IMRT were included in this study. Pretreatment MRI were co-registered with MRI at the time of recurrence for the delineation of gross tumor volume at diagnosis(GTV) and at recurrence(GTVr). A total of 7 histogram features and 40 texture features were computed from the recurrent(GTVr) and non-recurrent region(GTV-GTVr). Paired t-tests and Wilcoxon signed-rank tests were carried out on the 47 quantified radiomics features. RESULTS: A total of 7 features were significantly different between recurrent and non-recurrent regions. Other than the variance from intensity-based histogram, the remaining six significant features were either from the gray-level size zone matrix (GLSZM) or the neighbourhood gray-tone difference matrix (NGTDM). CONCLUSIONS: The radiomic features extracted from pre-treatment MRI can potentially reflect the difference between recurrent and non-recurrent regions within a tumor and has a potential role in pre-treatment identification of intra-tumoral radio-resistance for selective dose escalation.

Automated grading of acne vulgaris by deep learning with convolutional neural networks.

BACKGROUND: The visual assessment and severity grading of acne vulgaris by physicians can be subjective, resulting in inter- and intra-observer variability. OBJECTIVE: To develop and validate an algorithm for the automated calculation of the Investigator's Global Assessment (IGA) scale, to standardize acne severity and outcome measurements. MATERIALS AND METHODS: A total of 472 photographs (retrieved 01/01/2004-04/08/2017) in the frontal view from 416 acne patients were used for training and testing. Photographs were labeled according to the IGA scale in three groups of IGA clear/almost clear (0-1), IGA mild (2), and IGA moderate to severe (3-4). The classification model used a convolutional neural network, and models were separately trained on three image sizes. The photographs were then subjected to analysis by the algorithm, and the generated automated IGA scores were compared to clinical scoring. The prediction accuracy of each IGA grade label and the agreement (Pearson correlation) of the two scores were computed. RESULTS: The best classification accuracy was 67%. Pearson correlation between machine-predicted score and human labels (clinical scoring and researcher scoring) for each model and various image input sizes was 0.77. Correlation of predictions with clinical scores was highest when using Inception v4 on the largest image size of 1200 × 1600. Two sets of human labels showed a high correlation of 0.77, verifying the repeatability of the ground truth labels. Confusion matrices show that the models performed sub-optimally on the IGA 2 label. CONCLUSION: Deep learning techniques harnessing high-resolution images and large datasets will continue to improve, demonstrating growing potential for automated clinical image analysis and grading.

Active contours driven by difference of Gaussians.

In this paper, a novel edge-based active contour method is proposed based on the difference of Gaussians (DoG) to segment intensity inhomogeneous images. DoG is known as a feature enhancement tool, which can enhance the edges of an image. However, in the proposed energy functional it is used as an edge-indicator parameter, which acts like a balloon force during the level-set curve evolution process. In the proposed formulation, the internal energy term penalizes the deviation of the level-set function from a signed distance function and external energy term evolves the contour towards the boundaries of the objects. There are three main advantages of the proposed method. First, image difference computed using the DoG function provides the global structure of an image, which helps to segment the image globally that the traditional edge-based methods are unable to do. Second, it has a low time complexity compared to the state-of-the-art active contours developed in the context of intensity inhomogeneity. Third, it is not sensitive to the initial position of contour. Experimental results using both synthetic and real brain magnetic resonance (MR) images show that the proposed method yields better segmentation results compared to the state-of-the-art.

Segmentation of Left and Right Ventricles in Cardiac MRI Using Active Contours.

Segmentation of left and right ventricles plays a crucial role in quantitatively analyzing the global and regional information in the cardiac magnetic resonance imaging (MRI). In MRI, the intensity inhomogeneity and weak or blurred object boundaries are the problems, which makes it difficult for the intensity-based segmentation methods to properly delineate the regions of interests (ROI). In this paper, a hybrid signed pressure force function (SPF) is proposed, which yields both local and global image fitted differences in an additive fashion. A characteristic term is also introduced in the SPF function to restrict the contour within the ROI. The overlapping dice index and Hausdorff-Distance metrics have been used over cardiac datasets for quantitative validation. Using 2009 LV MICCAI validation dataset, the proposed method yields DSC values of 0.95 and 0.97 for endocardial and epicardial contours, respectively. Using 2012 RV MICCAI dataset, for the endocardial region, the proposed method yields DSC values of 0.97 and 0.90 and HD values of 8.51 and 7.67 for ED and ES, respectively. For the epicardial region, it yields DSC values of 0.92 and 0.91 and HD values of 6.47 and 9.34 for ED and ES, respectively. Results show its robustness in the segmentation application of the cardiac MRI.

Active contours driven by local and global fitted image models for image segmentation robust to intensity inhomogeneity.

This paper presents a region-based active contour method for the segmentation of intensity inhomogeneous images using an energy functional based on local and global fitted images. A square image fitted model is defined by using both local and global fitted differences. Moreover, local and global signed pressure force functions are introduced in the solution of the energy functional to stabilize the gradient descent flow. In the final gradient descent solution, the local fitted term helps extract regions with intensity inhomogeneity, whereas the global fitted term targets homogeneous regions. A Gaussian kernel is applied to regularize the contour at each step, which not only smoothes it but also avoids the computationally expensive re-initialization. Intensity inhomogeneous images contain undesired smooth intensity variations (bias field) that alter the results of intensity-based segmentation methods. The bias field is approximated with a Gaussian distribution and the bias of intensity inhomogeneous regions is corrected by dividing the original image by the approximated bias field. In this paper, a two-phase model is first derived and then extended to a four-phase model to segment brain magnetic resonance (MR) images into the desired regions of interest. Experimental results with both synthetic and real brain MR images are used for a quantitative and qualitative comparison with state-of-the-art active contour methods to show the advantages of the proposed segmentation technique in practical terms.

Active Contours Using Additive Local and Global Intensity Fitting Models for Intensity Inhomogeneous Image Segmentation.

This paper introduces an improved region based active contour method with a level set formulation. The proposed energy functional integrates both local and global intensity fitting terms in an additive formulation. Local intensity fitting term influences local force to pull the contour and confine it to object boundaries. In turn, the global intensity fitting term drives the movement of contour at a distance from the object boundaries. The global intensity term is based on the global division algorithm, which can better capture intensity information of an image than Chan-Vese (CV) model. Both local and global terms are mutually assimilated to construct an energy function based on a level set formulation to segment images with intensity inhomogeneity. Experimental results show that the proposed method performs better both qualitatively and quantitatively compared to other state-of-the-art-methods.

Segmentation of Regions of Interest Using Active Contours with SPF Function.

Segmentation of regions of interest is a well-known problem in image segmentation. This paper presents a region-based image segmentation technique using active contours with signed pressure force (SPF) function. The proposed algorithm contemporaneously traces high intensity or dense regions in an image by evolving the contour inwards. In medical image modalities these high intensity or dense regions refer to tumor, masses, or dense tissues. The proposed method partitions an image into an arbitrary number of subregions and tracks down salient regions step by step. It is implemented by enforcing a new region-based SPF function in a traditional edge-based level set model. It partitions an image into subregions and then discards outer subregion and partitions inner region into two more subregions; this continues iteratively until a stopping condition is fulfilled. A Gaussian kernel is used to regularize the level set function, which not only regularizes it but also removes the need of computationally expensive reinitialization. The proposed segmentation algorithm has been applied to different images in order to demonstrate the accuracy, effectiveness, and robustness of the algorithm.

Segmentation of intensity inhomogeneous brain MR images using active contours.

Segmentation of intensity inhomogeneous regions is a well-known problem in image analysis applications. This paper presents a region-based active contour method for image segmentation, which properly works in the context of intensity inhomogeneity problem. The proposed region-based active contour method embeds both region and gradient information unlike traditional methods. It contains mainly two terms, area and length, in which the area term practices a new region-based signed pressure force (SPF) function, which utilizes mean values from a certain neighborhood using the local binary fitted (LBF) energy model. In turn, the length term uses gradient information. The novelty of our method is to locally compute new SPF function, which uses local mean values and is able to detect boundaries of the homogenous regions. Finally, a truncated Gaussian kernel is used to regularize the level set function, which not only regularizes it but also removes the need of computationally expensive reinitialization. The proposed method targets the segmentation problem of intensity inhomogeneous images and reduces the time complexity among locally computed active contour methods. The experimental results show that the proposed method yields better segmentation result as well as less time complexity compared with the state-of-the-art active contour methods.

Multipass active contours for an adaptive contour map.

Isocontour mapping is efficient for extracting meaningful information from a biomedical image in a topographic analysis. Isocontour extraction from real world medical images is difficult due to noise and other factors. As such, adaptive selection of contour generation parameters is needed. This paper proposes an algorithm for generating an adaptive contour map that is spatially adjusted. It is based on the modified active contour model, which imposes successive spatial constraints on the image domain. The adaptability of the proposed algorithm is governed by the energy term of the model. This work focuses on mammograms and the analysis of their intensity. Our algorithm employs the Mumford-Shah energy functional, which considers an image's intensity distribution. In mammograms, the brighter regions generally contain significant information. Our approach exploits this characteristic to address the initialization and local optimum problems of the active contour model. Our algorithm starts from the darkest region; therefore, local optima encountered during the evolution of contours are populated in less important regions, and the important brighter regions are reserved for later stages. For an unrestricted initial contour, our algorithm adopts an existing technique without re-initialization. To assess its effectiveness and robustness, the proposed algorithm was tested on a set of mammograms.