Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images.

BACKGROUND: Computer vision may aid in melanoma detection. OBJECTIVE: We sought to compare melanoma diagnostic accuracy of computer algorithms to dermatologists using dermoscopic images. METHODS: We conducted a cross-sectional study using 100 randomly selected dermoscopic images (50 melanomas, 44 nevi, and 6 lentigines) from an international computer vision melanoma challenge dataset (n = 379), along with individual algorithm results from 25 teams. We used 5 methods (nonlearned and machine learning) to combine individual automated predictions into "fusion" algorithms. In a companion study, 8 dermatologists classified the lesions in the 100 images as either benign or malignant. RESULTS: The average sensitivity and specificity of dermatologists in classification was 82% and 59%. At 82% sensitivity, dermatologist specificity was similar to the top challenge algorithm (59% vs. 62%, P = .68) but lower than the best-performing fusion algorithm (59% vs. 76%, P = .02). Receiver operating characteristic area of the top fusion algorithm was greater than the mean receiver operating characteristic area of dermatologists (0.86 vs. 0.71, P = .001). LIMITATIONS: The dataset lacked the full spectrum of skin lesions encountered in clinical practice, particularly banal lesions. Readers and algorithms were not provided clinical data (eg, age or lesion history/symptoms). Results obtained using our study design cannot be extrapolated to clinical practice. CONCLUSION: Deep learning computer vision systems classified melanoma dermoscopy images with accuracy that exceeded some but not all dermatologists.

A survey on deep learning for skin lesion segmentation.

Skin cancer is a major public health problem that could benefit from computer-aided diagnosis to reduce the burden of this common disease. Skin lesion segmentation from images is an important step toward achieving this goal. However, the presence of natural and artificial artifacts (e.g., hair and air bubbles), intrinsic factors (e.g., lesion shape and contrast), and variations in image acquisition conditions make skin lesion segmentation a challenging task. Recently, various researchers have explored the applicability of deep learning models to skin lesion segmentation. In this survey, we cross-examine 177 research papers that deal with deep learning-based segmentation of skin lesions. We analyze these works along several dimensions, including input data (datasets, preprocessing, and synthetic data generation), model design (architecture, modules, and losses), and evaluation aspects (data annotation requirements and segmentation performance). We discuss these dimensions both from the viewpoint of select seminal works, and from a systematic viewpoint, examining how those choices have influenced current trends, and how their limitations should be addressed. To facilitate comparisons, we summarize all examined works in a comprehensive table as well as an interactive table available online3.

An ensemble-based deep learning model for detection of mutation causing cutaneous melanoma.

When the mutation affects the melanocytes of the body, a condition called melanoma results which is one of the deadliest skin cancers. Early detection of cutaneous melanoma is vital for raising the chances of survival. Melanoma can be due to inherited defective genes or due to environmental factors such as excessive sun exposure. The accuracy of the state-of-the-art computer-aided diagnosis systems is unsatisfactory. Moreover, the major drawback of medical imaging is the shortage of labeled data. Generalized classifiers are required to diagnose melanoma to avoid overfitting the dataset. To address these issues, blending ensemble-based deep learning (BEDLM-CMS) model is proposed to detect mutation of cutaneous melanoma by integrating long short-term memory (LSTM), Bi-directional LSTM (BLSTM) and gated recurrent unit (GRU) architectures. The dataset used in the proposed study contains 2608 human samples and 6778 mutations in total along with 75 types of genes. The most prominent genes that function as biomarkers for early diagnosis and prognosis are utilized. Multiple extraction techniques are used in this study to extract the most-prominent features. Afterwards, we applied different DL models optimized through grid search technique to diagnose melanoma. The validity of the results is confirmed using several techniques, including tenfold cross validation (10-FCVT), independent set (IST), and self-consistency (SCT). For validation of the results multiple metrics are used which include accuracy, specificity, sensitivity, and Matthews's correlation coefficient. BEDLM gives the highest accuracy of 97% in the independent set test whereas in self-consistency test and tenfold cross validation test it gives 94% and 93% accuracy, respectively. Accuracy of in self-consistency test, independent set test, and tenfold cross validation test is LSTM (96%, 94%, 92%), GRU (93%, 94%, 91%), and BLSTM (99%, 98%, 93%), respectively. The findings demonstrate that the proposed BEDLM-CMS can be used effectively applied for early diagnosis and treatment efficacy evaluation of cutaneous melanoma.

Computer-aided pattern classification system for dermoscopy images.

BACKGROUND: Computer-aided pattern classification of melanoma and other pigmented skin lesions is one of the most important tasks for clinical diagnosis. To differentiate between benign and malignant lesions, the extraction of color, architectural order, symmetry of pattern and homogeneity (CASH) is a challenging task. METHODS: In this article, a novel pattern classification system (PCS) based on the clinical CASH rule is presented to classify among six classes of patterns. The PCS system consists of the following five steps: transformation to the CIE L*a*b* color space, pre-processing to enhance the tumor region and removal of hairs, tumor-area segmentation, color and texture feature extraction, and finally, classification based on a multiclass support vector machine. RESULTS: The PCS system is tested on a total of 180 dermoscopic images. To test the performance of the PCS diagnostic classifier, the area under the receiver operating characteristics curve (AUC) is utilized. The proposed classifier achieved a sensitivity of 91.64%, specificity of 94.14%, and AUC of 0.948. CONCLUSION: The experimental results demonstrate that the proposed pattern classifier is highly accurate and classify between benign and malignant lesions into some extend. The PCS method is fully automatic and can accurately detect different patterns from dermoscopy images using color and texture properties. Additional pattern features can be included to investigate the impact of pattern classification based on the CASH rule.

Three-phase general border detection method for dermoscopy images using non-uniform illumination correction.

BACKGROUND: Computer-aided diagnosis of dermoscopy images has shown great promise in developing a quantitative, objective way of classifying skin lesions. An important step in the classification process is lesion segmentation. Many studies have been successful in segmenting melanocytic skin lesions (MSLs), but few have focused on non-melanocytic skin lesions (NoMSLs), as the wide variety of lesions makes accurate segmentation difficult. METHODS: We developed an automatic segmentation program for detecting borders of skin lesions in dermoscopy images. The method consists of a pre-processing phase, general lesion segmentation phase, including illumination correction, and bright region segmentation phase. RESULTS: We tested our method on a set of 107 NoMSLs and a set of 319 MSLs. Our method achieved precision/recall scores of 84.5% and 88.5% for NoMSLs, and 93.9% and 93.8% for MSLs, in comparison with manual extractions from four or five dermatologists. CONCLUSION: The accuracy of our method was competitive or better than five recently published methods. Our new method is the first method for detecting borders of both non-melanocytic and melanocytic skin lesions.

Private Facial Prediagnosis as an Edge Service for Parkinson's DBS Treatment Valuation.

Facial phenotyping for medical prediagnosis has recently been successfully exploited as a novel way for the preclinical assessment of a range of rare genetic diseases, where facial biometrics is revealed to have rich links to underlying genetic or medical causes. In this paper, we aim to extend this facial prediagnosis technology for a more general disease, Parkinson's Diseases (PD), and proposed an Artificial-Intelligence-of-Things (AIoT) edge-oriented privacy-preserving facial prediagnosis framework to analyze the treatment of Deep Brain Stimulation (DBS) on PD patients. In the proposed framework, a novel edge-based privacy-preserving framework is proposed to implement private deep facial diagnosis as a service over an AIoT-oriented information theoretically secure multi-party communication scheme, while data privacy has been a primary concern toward a wider exploitation of Electronic Health and Medical Records (EHR/EMR) over cloud-based medical services. In our experiments with a collected facial dataset from PD patients, for the first time, we proved that facial patterns could be used to evaluate the facial difference of PD patients undergoing DBS treatment. We further implemented a privacy-preserving information theoretical secure deep facial prediagnosis framework that can achieve the same accuracy as the non-encrypted one, showing the potential of our facial prediagnosis as a trustworthy edge service for grading the severity of PD in patients.

Checklist for Evaluation of Image-Based Artificial Intelligence Reports in Dermatology: CLEAR Derm Consensus Guidelines From the International Skin Imaging Collaboration Artificial Intelligence Working Group.

IMPORTANCE: The use of artificial intelligence (AI) is accelerating in all aspects of medicine and has the potential to transform clinical care and dermatology workflows. However, to develop image-based algorithms for dermatology applications, comprehensive criteria establishing development and performance evaluation standards are required to ensure product fairness, reliability, and safety. OBJECTIVE: To consolidate limited existing literature with expert opinion to guide developers and reviewers of dermatology AI. EVIDENCE REVIEW: In this consensus statement, the 19 members of the International Skin Imaging Collaboration AI working group volunteered to provide a consensus statement. A systematic PubMed search was performed of English-language articles published between December 1, 2008, and August 24, 2021, for "artificial intelligence" and "reporting guidelines," as well as other pertinent studies identified by the expert panel. Factors that were viewed as critical to AI development and performance evaluation were included and underwent 2 rounds of electronic discussion to achieve consensus. FINDINGS: A checklist of items was developed that outlines best practices of image-based AI development and assessment in dermatology. CONCLUSIONS AND RELEVANCE: Clinically effective AI needs to be fair, reliable, and safe; this checklist of best practices will help both developers and reviewers achieve this goal.

Lesion border detection in dermoscopy images using dynamic programming.

BACKGROUND/PURPOSE: Automated border detection is an important and challenging task in the computerized analysis of dermoscopy images. However, dermoscopic images often contain artifacts such as illumination, dermoscopic gel, and outline (hair, skin lines, ruler markings, and blood vessels). As a result, there is a need for robust methods to remove artifacts and detect lesion borders in dermoscopy images. METHODS: This automated method consists of three main steps: (1) preprocessing, (2) edge candidate point detection, and (3) tumor outline delineation. First, algorithms to reduce artifacts were used. Second, a least-squares method (LSM) was performed to acquire edge points. Third, dynamic programming (DP) technique was used to find the optimal boundary of the lesion. Statistical measures based on dermatologist-drawn borders were utilized as ground-truth to evaluate the performance of the proposed method. RESULTS: The method is tested on a total of 240 dermoscopic images: 30 benign melanocytic, 50 malignant melanomas, 50 basal cell carcinomas, 20 Merkel cell carcinomas, 60 seborrheic keratosis, and 30 atypical naevi. We obtained mean border detection error of 8.6%, 5.04%, 9.0%, 7.02%, 2.01%, and 3.24%, respectively. CONCLUSIONS: The results demonstrate that border detection combined with artifact removal increases sensitivity and specificity for segmentation of lesions in dermoscopy images.

Skin Melanoma Detection in Microscopic Images Using HMM-Based Asymmetric Analysis and Expectation Maximization.

Melanoma is one of the deadliest types of skin cancer with increasing incidence. The most definitive diagnosis method is the histopathological examination of the tissue sample. In this paper, a melanoma detection algorithm is proposed based on decision-level fusion and a Hidden Markov Model (HMM), whose parameters are optimized using Expectation Maximization (EM) and asymmetric analysis. The texture heterogeneity of the samples is determined using asymmetric analysis. A fusion-based HMM classifier trained using EM is introduced. For this purpose, a novel texture feature is extracted based on two local binary patterns, namely local difference pattern (LDP) and statistical histogram features of the microscopic image. Extensive experiments demonstrate that the proposed melanoma detection algorithm yields a total error of less than 0.04%.

Detection of atypical texture features in early malignant melanoma.

BACKGROUND: The presence of an atypical (irregular) pigment network (APN) can indicate a diagnosis of melanoma. This study sought to analyze the APN with texture measures. METHODS: For 106 dermoscopy images including 28 melanomas and 78 benign dysplastic nevi, the areas of APN were selected manually. Ten texture measures in the CVIPtools image analysis system were applied. RESULTS: Of the 10 texture measures used, correlation average provided the highest discrimination accuracy, an average of 95.4%. Discrimination of melanomas was optimal at a pixel distance of 20 for the 768 x 512 images, consistent with a melanocytic lesion texel size estimate of 4-5 texels per mm. CONCLUSION: Texture analysis, in particular correlation average at an optimized pixel spacing, may afford automatic detection of an irregular pigment network in early malignant melanoma.

Watershed segmentation of dermoscopy images using a watershed technique.

BACKGROUND/PURPOSE: Automatic lesion segmentation is an important part of computer-based image analysis of pigmented skin lesions. In this research, a watershed algorithm is developed and investigated for adequacy of skin lesion segmentation in dermoscopy images. METHODS: Hair, black border and vignette removal methods are introduced as preprocessing steps. The flooding variant of the watershed segmentation algorithm was implemented with novel features adapted to this domain. An outer bounding box, determined by a difference function derived from horizontal and vertical projection functions, is added to estimate the lesion area, and the lesion area error is reduced by a linear estimation function. As a post-processing step, a second-order B-Spline smoothing method is introduced to smooth the watershed border. RESULTS: Using the average of three sets of dermatologist-drawn borders as the ground truth, an overall error of 15.98% was obtained using the watershed technique. CONCLUSION: The implementation of the flooding variant of the watershed algorithm presented here allows satisfactory automatic segmentation of pigmented skin lesions.

Fast color quantization using weighted sort-means clustering.

Color quantization is an important operation with numerous applications in graphics and image processing. Most quantization methods are essentially based on data clustering algorithms. However, despite its popularity as a general purpose clustering algorithm, K-means has not received much respect in the color quantization literature because of its high computational requirements and sensitivity to initialization. In this paper, a fast color quantization method based on K-means is presented. The method involves several modifications to the conventional (batch) K-means algorithm, including data reduction, sample weighting, and the use of the triangle inequality to speed up the nearest-neighbor search. Experiments on a diverse set of images demonstrate that, with the proposed modifications, K-means becomes very competitive with state-of-the-art color quantization methods in terms of both effectiveness and efficiency.

Approximate lesion localization in dermoscopy images.

BACKGROUND: Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Because of the difficulty and subjectivity of human interpretation, automated analysis of dermoscopy images has become an important research area. Border detection is often the first step in this analysis. METHODS: In this article, we present an approximate lesion localization method that serves as a preprocessing step for detecting borders in dermoscopy images. In this method, first the black frame around the image is removed using an iterative algorithm. The approximate location of the lesion is then determined using an ensemble of thresholding algorithms. RESULTS: The method is tested on a set of 428 dermoscopy images. The localization error is quantified by a metric that uses dermatologist-determined borders as the ground truth. CONCLUSION: The results demonstrate that the method presented here achieves both fast and accurate localization of lesions in dermoscopy images.

An improved objective evaluation measure for border detection in dermoscopy images.

BACKGROUND: Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Owing to the difficulty and subjectivity of human interpretation, dermoscopy image analysis has become an important research area. One of the most important steps in dermoscopy image analysis is the automated detection of lesion borders. Although numerous methods have been developed for the detection of lesion borders, very few studies were comprehensive in the evaluation of their results. METHODS: In this paper, we evaluate five recent border detection methods on a set of 90 dermoscopy images using three sets of dermatologist-drawn borders as the ground truth. In contrast to previous work, we utilize an objective measure, the normalized probabilistic rand index, which takes into account the variations in the ground-truth images. CONCLUSION: The results demonstrate that the differences between four of the evaluated border detection methods are in fact smaller than those predicted by the commonly used exclusive-OR measure.

Detection of basal cell carcinoma using color and histogram measures of semitranslucent areas.

BACKGROUND: Semitranslucency, defined as a smooth, jelly-like area with varied, near-skin-tone color, can indicate a diagnosis of basal cell carcinoma (BCC) with high specificity. This study sought to analyze potential areas of semitranslucency with histogram-derived texture and color measures to discriminate BCC from non-semitranslucent areas in non-BCC skin lesions. METHODS: For 210 dermoscopy images, the areas of semitranslucency in 42 BCCs and comparable areas of smoothness and color in 168 non-BCCs were selected manually. Six color measures and six texture measures were applied to the semitranslucent areas of the BCC and the comparable areas in the non-BCC images. RESULTS: Receiver operating characteristic (ROC) curve analysis showed that the texture measures alone provided greater separation of BCC from non-BCC than the color measures alone. Statistical analysis showed that the four most important measures of semitranslucency are three histogram measures: contrast, smoothness, and entropy, and one color measure: blue chromaticity. Smoothness is the single most important measure. The combined 12 measures achieved a diagnostic accuracy of 95.05% based on area under the ROC curve. CONCLUSION: Texture and color analysis measures, especially smoothness, may afford automatic detection of BCC images with semitranslucency.

Dermoscopy Image Analysis: Overview and Future Directions.

Dermoscopy is a non-invasive skin imaging technique that permits visualization of features of pigmented melanocytic neoplasms that are not discernable by examination with the naked eye. While studies on the automated analysis of dermoscopy images date back to the late 1990s, because of various factors (lack of publicly available datasets, open-source software, computational power, etc.), the field progressed rather slowly in its first two decades. With the release of a large public dataset by the International Skin Imaging Collaboration in 2016, development of open-source software for convolutional neural networks, and the availability of inexpensive graphics processing units, dermoscopy image analysis has recently become a very active research field. In this paper, we present a brief overview of this exciting subfield of medical image analysis, primarily focusing on three aspects of it, namely, segmentation, feature extraction, and classification. We then provide future directions for researchers.

A Survey of Feature Extraction in Dermoscopy Image Analysis of Skin Cancer.

Dermoscopy image analysis (DIA) is a growing field, with works being published every week. This makes it difficult not only to keep track of all the contributions, but also for new researchers to identify relevant information and new directions to be explored. Several surveys have been written in the past decade, but these tend to cover all of the steps of a CAD system, which can be overwhelming. Moreover, in these works, each of the steps is briefly discussed due to lack of space. Among the different blocks of the CAD system, the most relevant is the one devoted to feature extraction. This is also the block where existing works exhibit the most variability. Therefore, we believe that it is important to review the state-of-the-art on this matter. This work thoroughly explores the several types of features that have been used in DIA. A discussion on their relevance and limitations, as well as suggestions for future research are provided.

Border detection in dermoscopy images using statistical region merging.

BACKGROUND: As a result of advances in skin imaging technology and the development of suitable image processing techniques, during the last decade, there has been a significant increase of interest in the computer-aided diagnosis of melanoma. Automated border detection is one of the most important steps in this procedure, because the accuracy of the subsequent steps crucially depends on it. METHODS: In this article, we present a fast and unsupervised approach to border detection in dermoscopy images of pigmented skin lesions based on the statistical region merging algorithm. RESULTS: The method is tested on a set of 90 dermoscopy images. The border detection error is quantified by a metric in which three sets of dermatologist-determined borders are used as the ground-truth. The proposed method is compared with four state-of-the-art automated methods (orientation-sensitive fuzzy c-means, dermatologist-like tumor extraction algorithm, meanshift clustering, and the modified JSEG method). CONCLUSION: The results demonstrate that the method presented here achieves both fast and accurate border detection in dermoscopy images.

An improved Internet-based melanoma screening system with dermatologist-like tumor area extraction algorithm.

In this paper, we present an Internet-based melanoma screening system. Our web server is accessible from all over the world and performs the following procedures when a remote user uploads a dermoscopy image: separates the tumor area from the surrounding skin using highly accurate dermatologist-like tumor area extraction algorithm, calculates a total of 428 features for the characterization of the tumor, classifies the tumor as melanoma or nevus using a neural network classifier, and presents the diagnosis. Our system achieves a sensitivity of 85.9% and a specificity of 86.0% on a set of 1258 dermoscopy images using cross-validation.