Comprehensive analysis of clinical images contributions for melanoma classification using convolutional neural networks.

BACKGROUND: Timely diagnosis plays a critical role in determining melanoma prognosis, prompting the development of deep learning models to aid clinicians. Questions persist regarding the efficacy of clinical images alone or in conjunction with dermoscopy images for model training. This study aims to compare the classification performance for melanoma of three types of CNN models: those trained on clinical images, dermoscopy images, and a combination of paired clinical and dermoscopy images from the same lesion. MATERIALS AND METHODS: We divided 914 image pairs into training, validation, and test sets. Models were built using pre-trained Inception-ResNetV2 convolutional layers for feature extraction, followed by binary classification. Training comprised 20 models per CNN type using sets of random hyperparameters. Best models were chosen based on validation AUC-ROC. RESULTS: Significant AUC-ROC differences were found between clinical versus dermoscopy models (0.661 vs. 0.869, p < 0.001) and clinical versus clinical + dermoscopy models (0.661 vs. 0.822, p = 0.001). Significant sensitivity differences were found between clinical and dermoscopy models (0.513 vs. 0.799, p = 0.01), dermoscopy versus clinical + dermoscopy models (0.799 vs. 1.000, p = 0.02), and clinical versus clinical + dermoscopy models (0.513 vs. 1.000, p < 0.001). Significant specificity differences were found between dermoscopy versus clinical + dermoscopy models (0.800 vs. 0.288, p < 0.001) and clinical versus clinical + dermoscopy models (0.650 vs. 0.288, p < 0.001). CONCLUSION: CNN models trained on dermoscopy images outperformed those relying solely on clinical images under our study conditions. The potential advantages of incorporating paired clinical and dermoscopy images for CNN-based melanoma classification appear less clear based on our findings.

Hair removal in dermoscopy images using variational autoencoders.

BACKGROUND: In recent years, melanoma is rising at a faster rate compared to other cancers. Although it is the most serious type of skin cancer, the diagnosis at early stages makes it curable. Dermoscopy is a reliable medical technique used to detect melanoma by using a dermoscope to examine the skin. In the last few decades, digital imaging devices have made great progress which allowed capturing and storing high-quality images from these examinations. The stored images are now being standardized and used for the automatic detection of melanoma. However, when the hair covers the skin, this makes the task challenging. Therefore, it is important to eliminate the hair to get accurate results. METHODS: In this paper, we propose a simple yet efficient method for hair removal using a variational autoencoder without the need for paired samples. The encoder takes as input a dermoscopy image and builds a latent distribution that ignores hair as it is considered noise, while the decoder reconstructs a hair-free image. Both encoder and decoder use a decent convolutional neural networks architecture that provides high performance. The construction of our model comprises two stages of training. In the first stage, the model has trained on hair-occluded images to output hair-free images, and in the second stage, it is optimized using hair-free images to preserve the image textures. Although the variational autoencoder produces hair-free images, it does not maintain the quality of the generated images. Thus, we explored the use of three-loss functions including the structural similarity index (SSIM), L1-norm, and L2-norm to improve the visual quality of the generated images. RESULTS: The evaluation of the hair-free reconstructed images is carried out using t-distributed stochastic neighbor embedding (SNE) feature mapping by visualizing the distribution of the real hair-free images and the synthesized hair-free images. The conducted experiments on the publicly available dataset HAM10000 show that our method is very efficient.

Using content-based image retrieval of dermoscopic images for interpretation and education: A pilot study.

BACKGROUND: Dermoscopic content-based image retrieval (CBIR) systems provide a set of visually similar dermoscopic (magnified and illuminated) skin images with a pathology-confirmed diagnosis for a given dermoscopic query image of a skin lesion. Although recent advances in machine learning have spurred novel CBIR algorithms, we have few insights into how end users interact with CBIRs and to what extent CBIRs can be useful for education and image interpretation. MATERIALS AND METHODS: We developed an interactive user interface for a CBIR system with dermoscopic images as a decision support tool and investigated users' interactions and decisions with the system. We performed a pilot experiment with 14 non-medically trained users for a given set of annotated dermoscopic images. RESULTS: Our pilot showed that the number of correct classifications and users' confidence levels significantly increased with the CBIR interface compared with a non-CBIR interface, although the timing also increased significantly. The users found the CBIR interface of high educational value, engaging and easy to use. CONCLUSION: Overall, users became more accurate, found the CBIR approach provided a useful decision aid, and had educational value for learning about skin conditions.

Improving Skin Lesion Segmentation with Self-Training.

Skin lesion segmentation plays a key role in the diagnosis of skin cancer; it can be a component in both traditional algorithms and end-to-end approaches. The quality of segmentation directly impacts the accuracy of classification; however, attaining optimal segmentation necessitates a substantial amount of labeled data. Semi-supervised learning allows for employing unlabeled data to enhance the results of the machine learning model. In the case of medical image segmentation, acquiring detailed annotation is time-consuming and costly and requires skilled individuals so the utilization of unlabeled data allows for a significant mitigation of manual segmentation efforts. This study proposes a novel approach to semi-supervised skin lesion segmentation using self-training with a Noisy Student. This approach allows for utilizing large amounts of available unlabeled images. It consists of four steps-first, training the teacher model on labeled data only, then generating pseudo-labels with the teacher model, training the student model on both labeled and pseudo-labeled data, and lastly, training the student* model on pseudo-labels generated with the student model. In this work, we implemented DeepLabV3 architecture as both teacher and student models. As a final result, we achieved a mIoU of 88.0% on the ISIC 2018 dataset and a mIoU of 87.54% on the PH2 dataset. The evaluation of the proposed approach shows that Noisy Student training improves the segmentation performance of neural networks in a skin lesion segmentation task while using only small amounts of labeled data.

An effective hair removal algorithm for dermoscopy images.

BACKGROUND/PURPOSE: Dermoscopy is one of the major imaging modalities used in the diagnosis of pigmented skin lesions. Due to the difficulty and subjectivity of human interpretation, computerized image analysis techniques have become important tools in this research area. Hair removal from skin lesion images is one of the key problems for the precise segmentation and analysis of the skin lesions. In this study, we present a new scheme that automatically detects and removes hairs from dermoscopy images. METHODS: The proposed algorithm includes two steps: firstly, light and dark hairs and ruler marking are segmented through adaptive canny edge detector and refinement by morphological operators. Secondly, the hairs are repaired based on multi-resolution coherence transport inpainting. RESULTS: The algorithm was applied to 50 dermoscopy images. To estimate the accuracy of the proposed hair detection algorithm, quantitative analysis was performed using TDR, FPR, and DA metrics. Moreover, to evaluate the performance of the proposed hair repaired algorithm, three statistical metrics namely entropy, standard deviation, and co-occurrence matrix were used. CONCLUSION: The results demonstrate that the proposed algorithm is highly accurate and able to detect and repair the hair pixels with few errors. In addition, the segmentation veracity of the skin lesion is effectively improved after our proposed hair removal algorithm.

Evaluation of denoising techniques to remove speckle and Gaussian noise from dermoscopy images.

Computerized methods provide analyses of skin lesions from dermoscopy images automatically. However, the images acquired from dermoscopy devices are noisy and cause low accuracy in automated methods. Therefore, various methods have been applied for denoising in the literature. There are some review-type papers about these methods. However, their authors have focused on either denoising with a specific approach or denoising from other images rather than dermoscopy images, which have a different characteristic. It is not possible to determine which method is the most suitable for denoising from dermoscopy images according to the results presented in them. Therefore, a review on the denoising approaches applied with dermoscopy images is required and, according to our knowledge, there is no such a review-type paper. To fill this gap in the literature, the required review has been performed in this work. Also, in this work, the methods in the literature have been implemented using the same data sets containing images with speckle or Gaussian types of noise. The results have been analyzed not only visually but also quantitatively to compare capabilities of the techniques. Our experiments indicated that each denoising technique has its own disadvantages and advantages. The main contributions of this paper are three-fold: (i) A comprehensive review on the denoising approaches applied with dermoscopy images has been presented. (ii) The denoising techniques have been implemented with the same images for meaningful comparisons. (iii) Both visual and quantitative analyses with different metrics have been performed and comparative performance evaluations have been presented.

Multi-features extraction based on deep learning for skin lesion classification.

For various forms of skin lesion, many different feature extraction methods have been investigated so far. Indeed, feature extraction is a crucial step in machine learning processes. In general, we can distinct handcrafted and deep learning features. In this paper, we investigate the efficiency of using 17 commonly pre-trained convolutional neural networks (CNN) architectures as feature extractors and of 24 machine learning classifiers to evaluate the classification of skin lesions from two different datasets: ISIC 2019 and PH2. In this research, we find out that a DenseNet201 combined with Fine KNN or Cubic SVM achieved the best results in accuracy (92.34% and 91.71%) for the ISIC 2019 dataset. The results also show that the suggested method outperforms others approaches with an accuracy of 99% on the PH2 dataset.

SL-HarDNet: Skin lesion segmentation with HarDNet.

Automatic segmentation of skin lesions from dermoscopy is of great significance for the early diagnosis of skin cancer. However, due to the complexity and fuzzy boundary of skin lesions, automatic segmentation of skin lesions is a challenging task. In this paper, we present a novel skin lesion segmentation network based on HarDNet (SL-HarDNet). We adopt HarDNet as the backbone, which can learn more robust feature representation. Furthermore, we introduce three powerful modules, including: cascaded fusion module (CFM), spatial channel attention module (SCAM) and feature aggregation module (FAM). Among them, CFM combines the features of different levels and effectively aggregates the semantic and location information of skin lesions. SCAM realizes the capture of key spatial information. The cross-level features are effectively fused through FAM, and the obtained high-level semantic position information features are reintegrated with the features from CFM to improve the segmentation performance of the model. We apply the challenge dataset ISIC-2016&PH2 and ISIC-2018, and extensively evaluate and compare the state-of-the-art skin lesion segmentation methods. Experiments show that our SL-HarDNet performance is always superior to other segmentation methods and achieves the latest performance.

Interpretability of a Deep Learning Based Approach for the Classification of Skin Lesions into Main Anatomic Body Sites.

Over the past few decades, different clinical diagnostic algorithms have been proposed to diagnose malignant melanoma in its early stages. Furthermore, the detection of skin moles driven by current deep learning based approaches yields impressive results in the classification of malignant melanoma. However, in all these approaches, the researchers do not take into account the origin of the skin lesion. It has been observed that the specific criteria for in situ and early invasive melanoma highly depend on the anatomic site of the body. To address this problem, we propose a deep learning architecture based framework to classify skin lesions into the three most important anatomic sites, including the face, trunk and extremities, and acral lesions. In this study, we take advantage of pretrained networks, including VGG19, ResNet50, Xception, DenseNet121, and EfficientNetB0, to calculate the features with an adjusted and densely connected classifier. Furthermore, we perform in depth analysis on database, architecture, and result regarding the effectiveness of the proposed framework. Experiments confirm the ability of the developed algorithms to classify skin lesions into the most important anatomical sites with 91.45% overall accuracy for the EfficientNetB0 architecture, which is a state-of-the-art result in this domain.

Two-Stage Deep Neural Network via Ensemble Learning for Melanoma Classification.

Melanoma is a skin disease with a high fatality rate. Early diagnosis of melanoma can effectively increase the survival rate of patients. There are three types of dermoscopy images, malignant melanoma, benign nevis, and seborrheic keratosis, so using dermoscopy images to classify melanoma is an indispensable task in diagnosis. However, early melanoma classification works can only use the low-level information of images, so the melanoma cannot be classified efficiently; the recent deep learning methods mainly depend on a single network, although it can extract high-level features, the poor scale and type of the features limited the results of the classification. Therefore, we need an automatic classification method for melanoma, which can make full use of the rich and deep feature information of images for classification. In this study, we propose an ensemble method that can integrate different types of classification networks for melanoma classification. Specifically, we first use U-net to segment the lesion area of images to generate a lesion mask, thus resize images to focus on the lesion; then, we use five excellent classification models to classify dermoscopy images, and adding squeeze-excitation block (SE block) to models to emphasize the more informative features; finally, we use our proposed new ensemble network to integrate five different classification results. The experimental results prove the validity of our results. We test our method on the ISIC 2017 challenge dataset and obtain excellent results on multiple metrics; especially, we get 0.909 on accuracy. Our classification framework can provide an efficient and accurate way for melanoma classification using dermoscopy images, laying the foundation for early diagnosis and later treatment of melanoma.

Convolutional neural networks for the automatic diagnosis of melanoma: An extensive experimental study.

Melanoma is the type of skin cancer with the highest levels of mortality, and it is more dangerous because it can spread to other parts of the body if not caught and treated early. Melanoma diagnosis is a complex task, even for expert dermatologists, mainly due to the great variety of morphologies in moles of patients. Accordingly, the automatic diagnosis of melanoma is a task that poses the challenge of developing efficient computational methods that ease the diagnostic and, therefore, aid dermatologists in decision-making. In this work, an extensive analysis was conducted, aiming at assessing and illustrating the effectiveness of convolutional neural networks in coping with this complex task. To achieve this objective, twelve well-known convolutional network models were evaluated on eleven public image datasets. The experimental study comprised five phases, where first it was analyzed the sensitivity of the models regarding the optimization algorithm used for their training, and then it was analyzed the impact in performance when using different techniques such as cost-sensitive learning, data augmentation and transfer learning. The conducted study confirmed the usefulness, effectiveness and robustness of different convolutional architectures in solving melanoma diagnosis problem. Also, important guidelines to researchers working on this area were provided, easing the selection of both the proper convolutional model and technique according the characteristics of data.

Automatic segmentation of dermoscopy images using saliency combined with Otsu threshold.

Segmentation is one of the crucial steps for the computer-aided diagnosis (CAD) of skin cancer with dermoscopy images. To accurately extract lesion borders from dermoscopy images, a novel automatic segmentation algorithm using saliency combined with Otsu threshold is proposed in this paper, which includes enhancement and segmentation stages. In the enhancement stage, prior information on healthy skin is extracted, and the color saliency map and brightness saliency map are constructed respectively. By fusing the two saliency maps, the final enhanced image is obtained. In the segmentation stage, according to the histogram distribution of the enhanced image, an optimization function is designed to adjust the traditional Otsu threshold method to obtain more accurate lesion borders. The proposed model is validated from enhancement effectiveness and segmentation accuracy. Experimental results demonstrate that our method is robust and performs better than other state-of-the-art methods.

Skin lesion image segmentation using Delaunay Triangulation for melanoma detection.

Developing automatic diagnostic tools for the early detection of skin cancer lesions in dermoscopic images can help to reduce melanoma-induced mortality. Image segmentation is a key step in the automated skin lesion diagnosis pipeline. In this paper, a fast and fully-automatic algorithm for skin lesion segmentation in dermoscopic images is presented. Delaunay Triangulation is used to extract a binary mask of the lesion region, without the need of any training stage. A quantitative experimental evaluation has been conducted on a publicly available database, by taking into account six well-known state-of-the-art segmentation methods for comparison. The results of the experimental analysis demonstrate that the proposed approach is highly accurate when dealing with benign lesions, while the segmentation accuracy significantly decreases when melanoma images are processed. This behavior led us to consider geometrical and color features extracted from the binary masks generated by our algorithm for classification, achieving promising results for melanoma detection.

Enhancing classification accuracy utilizing globules and dots features in digital dermoscopy.

The interest in image dermoscopy has been significantly increased recently and skin lesion images are nowadays routinely acquired for a number of skin disorders. An important finding in the assessment of a skin lesion severity is the existence of dark dots and globules, which are hard to locate and count using existing image software tools. In this work we present a novel methodology for detecting/segmenting and count dark dots and globules from dermoscopy images. Segmentation is performed using a multi-resolution approach based on inverse non-linear diffusion. Subsequently, a number of features are extracted from the segmented dots/globules and their diagnostic value in automatic classification of dermoscopy images of skin lesions into melanoma and non-malignant nevus is evaluated. The proposed algorithm is applied to a number of images with skin lesions with known histo-pathology. Results show that the proposed algorithm is very effective in automatically segmenting dark dots and globules. Furthermore, it was found that the features extracted from the segmented dots/globules can enhance the performance of classification algorithms that discriminate between malignant and benign skin lesions, when they are combined with other region-based descriptors.