An ensemble-based deep learning model for detection of mutation causing cutaneous melanoma.

When the mutation affects the melanocytes of the body, a condition called melanoma results which is one of the deadliest skin cancers. Early detection of cutaneous melanoma is vital for raising the chances of survival. Melanoma can be due to inherited defective genes or due to environmental factors such as excessive sun exposure. The accuracy of the state-of-the-art computer-aided diagnosis systems is unsatisfactory. Moreover, the major drawback of medical imaging is the shortage of labeled data. Generalized classifiers are required to diagnose melanoma to avoid overfitting the dataset. To address these issues, blending ensemble-based deep learning (BEDLM-CMS) model is proposed to detect mutation of cutaneous melanoma by integrating long short-term memory (LSTM), Bi-directional LSTM (BLSTM) and gated recurrent unit (GRU) architectures. The dataset used in the proposed study contains 2608 human samples and 6778 mutations in total along with 75 types of genes. The most prominent genes that function as biomarkers for early diagnosis and prognosis are utilized. Multiple extraction techniques are used in this study to extract the most-prominent features. Afterwards, we applied different DL models optimized through grid search technique to diagnose melanoma. The validity of the results is confirmed using several techniques, including tenfold cross validation (10-FCVT), independent set (IST), and self-consistency (SCT). For validation of the results multiple metrics are used which include accuracy, specificity, sensitivity, and Matthews's correlation coefficient. BEDLM gives the highest accuracy of 97% in the independent set test whereas in self-consistency test and tenfold cross validation test it gives 94% and 93% accuracy, respectively. Accuracy of in self-consistency test, independent set test, and tenfold cross validation test is LSTM (96%, 94%, 92%), GRU (93%, 94%, 91%), and BLSTM (99%, 98%, 93%), respectively. The findings demonstrate that the proposed BEDLM-CMS can be used effectively applied for early diagnosis and treatment efficacy evaluation of cutaneous melanoma.

A survey on deep learning for skin lesion segmentation.

Skin cancer is a major public health problem that could benefit from computer-aided diagnosis to reduce the burden of this common disease. Skin lesion segmentation from images is an important step toward achieving this goal. However, the presence of natural and artificial artifacts (e.g., hair and air bubbles), intrinsic factors (e.g., lesion shape and contrast), and variations in image acquisition conditions make skin lesion segmentation a challenging task. Recently, various researchers have explored the applicability of deep learning models to skin lesion segmentation. In this survey, we cross-examine 177 research papers that deal with deep learning-based segmentation of skin lesions. We analyze these works along several dimensions, including input data (datasets, preprocessing, and synthetic data generation), model design (architecture, modules, and losses), and evaluation aspects (data annotation requirements and segmentation performance). We discuss these dimensions both from the viewpoint of select seminal works, and from a systematic viewpoint, examining how those choices have influenced current trends, and how their limitations should be addressed. To facilitate comparisons, we summarize all examined works in a comprehensive table as well as an interactive table available online3.

Private Facial Prediagnosis as an Edge Service for Parkinson's DBS Treatment Valuation.

Facial phenotyping for medical prediagnosis has recently been successfully exploited as a novel way for the preclinical assessment of a range of rare genetic diseases, where facial biometrics is revealed to have rich links to underlying genetic or medical causes. In this paper, we aim to extend this facial prediagnosis technology for a more general disease, Parkinson's Diseases (PD), and proposed an Artificial-Intelligence-of-Things (AIoT) edge-oriented privacy-preserving facial prediagnosis framework to analyze the treatment of Deep Brain Stimulation (DBS) on PD patients. In the proposed framework, a novel edge-based privacy-preserving framework is proposed to implement private deep facial diagnosis as a service over an AIoT-oriented information theoretically secure multi-party communication scheme, while data privacy has been a primary concern toward a wider exploitation of Electronic Health and Medical Records (EHR/EMR) over cloud-based medical services. In our experiments with a collected facial dataset from PD patients, for the first time, we proved that facial patterns could be used to evaluate the facial difference of PD patients undergoing DBS treatment. We further implemented a privacy-preserving information theoretical secure deep facial prediagnosis framework that can achieve the same accuracy as the non-encrypted one, showing the potential of our facial prediagnosis as a trustworthy edge service for grading the severity of PD in patients.

Checklist for Evaluation of Image-Based Artificial Intelligence Reports in Dermatology: CLEAR Derm Consensus Guidelines From the International Skin Imaging Collaboration Artificial Intelligence Working Group.

IMPORTANCE: The use of artificial intelligence (AI) is accelerating in all aspects of medicine and has the potential to transform clinical care and dermatology workflows. However, to develop image-based algorithms for dermatology applications, comprehensive criteria establishing development and performance evaluation standards are required to ensure product fairness, reliability, and safety. OBJECTIVE: To consolidate limited existing literature with expert opinion to guide developers and reviewers of dermatology AI. EVIDENCE REVIEW: In this consensus statement, the 19 members of the International Skin Imaging Collaboration AI working group volunteered to provide a consensus statement. A systematic PubMed search was performed of English-language articles published between December 1, 2008, and August 24, 2021, for "artificial intelligence" and "reporting guidelines," as well as other pertinent studies identified by the expert panel. Factors that were viewed as critical to AI development and performance evaluation were included and underwent 2 rounds of electronic discussion to achieve consensus. FINDINGS: A checklist of items was developed that outlines best practices of image-based AI development and assessment in dermatology. CONCLUSIONS AND RELEVANCE: Clinically effective AI needs to be fair, reliable, and safe; this checklist of best practices will help both developers and reviewers achieve this goal.

Skin Melanoma Detection in Microscopic Images Using HMM-Based Asymmetric Analysis and Expectation Maximization.

Melanoma is one of the deadliest types of skin cancer with increasing incidence. The most definitive diagnosis method is the histopathological examination of the tissue sample. In this paper, a melanoma detection algorithm is proposed based on decision-level fusion and a Hidden Markov Model (HMM), whose parameters are optimized using Expectation Maximization (EM) and asymmetric analysis. The texture heterogeneity of the samples is determined using asymmetric analysis. A fusion-based HMM classifier trained using EM is introduced. For this purpose, a novel texture feature is extracted based on two local binary patterns, namely local difference pattern (LDP) and statistical histogram features of the microscopic image. Extensive experiments demonstrate that the proposed melanoma detection algorithm yields a total error of less than 0.04%.

Dermoscopy Image Analysis: Overview and Future Directions.

Dermoscopy is a non-invasive skin imaging technique that permits visualization of features of pigmented melanocytic neoplasms that are not discernable by examination with the naked eye. While studies on the automated analysis of dermoscopy images date back to the late 1990s, because of various factors (lack of publicly available datasets, open-source software, computational power, etc.), the field progressed rather slowly in its first two decades. With the release of a large public dataset by the International Skin Imaging Collaboration in 2016, development of open-source software for convolutional neural networks, and the availability of inexpensive graphics processing units, dermoscopy image analysis has recently become a very active research field. In this paper, we present a brief overview of this exciting subfield of medical image analysis, primarily focusing on three aspects of it, namely, segmentation, feature extraction, and classification. We then provide future directions for researchers.

A Survey of Feature Extraction in Dermoscopy Image Analysis of Skin Cancer.

Dermoscopy image analysis (DIA) is a growing field, with works being published every week. This makes it difficult not only to keep track of all the contributions, but also for new researchers to identify relevant information and new directions to be explored. Several surveys have been written in the past decade, but these tend to cover all of the steps of a CAD system, which can be overwhelming. Moreover, in these works, each of the steps is briefly discussed due to lack of space. Among the different blocks of the CAD system, the most relevant is the one devoted to feature extraction. This is also the block where existing works exhibit the most variability. Therefore, we believe that it is important to review the state-of-the-art on this matter. This work thoroughly explores the several types of features that have been used in DIA. A discussion on their relevance and limitations, as well as suggestions for future research are provided.

Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images.

BACKGROUND: Computer vision may aid in melanoma detection. OBJECTIVE: We sought to compare melanoma diagnostic accuracy of computer algorithms to dermatologists using dermoscopic images. METHODS: We conducted a cross-sectional study using 100 randomly selected dermoscopic images (50 melanomas, 44 nevi, and 6 lentigines) from an international computer vision melanoma challenge dataset (n = 379), along with individual algorithm results from 25 teams. We used 5 methods (nonlearned and machine learning) to combine individual automated predictions into "fusion" algorithms. In a companion study, 8 dermatologists classified the lesions in the 100 images as either benign or malignant. RESULTS: The average sensitivity and specificity of dermatologists in classification was 82% and 59%. At 82% sensitivity, dermatologist specificity was similar to the top challenge algorithm (59% vs. 62%, P = .68) but lower than the best-performing fusion algorithm (59% vs. 76%, P = .02). Receiver operating characteristic area of the top fusion algorithm was greater than the mean receiver operating characteristic area of dermatologists (0.86 vs. 0.71, P = .001). LIMITATIONS: The dataset lacked the full spectrum of skin lesions encountered in clinical practice, particularly banal lesions. Readers and algorithms were not provided clinical data (eg, age or lesion history/symptoms). Results obtained using our study design cannot be extrapolated to clinical practice. CONCLUSION: Deep learning computer vision systems classified melanoma dermoscopy images with accuracy that exceeded some but not all dermatologists.

Improving dermoscopy image classification using color constancy.

Robustness is one of the most important characteristics of computer-aided diagnosis systems designed for dermoscopy images. However, it is difficult to ensure this characteristic if the systems operate with multisource images acquired under different setups. Changes in the illumination and acquisition devices alter the color of images and often reduce the performance of the systems. Thus, it is important to normalize the colors of dermoscopy images before training and testing any system. In this paper, we investigate four color constancy algorithms: Gray World, max-RGB, Shades of Gray, and General Gray World. Our results show that color constancy improves the classification of multisource images, increasing the sensitivity of a bag-of-features system from 71.0% to 79.7% and the specificity from 55.2% to 76% using only 1-D RGB histograms as features.

Four-class classification of skin lesions with task decomposition strategy.

This paper proposes a new computer-aided method for the skin lesion classification applicable to both melanocytic skin lesions (MSLs) and nonmelanocytic skin lesions (NoMSLs). The computer-aided skin lesion classification has drawn attention as an aid for detection of skin cancers. Several researchers have developed methods to distinguish between melanoma and nevus, which are both categorized as MSL. However, most of these studies did not focus on NoMSLs such as basal cell carcinoma (BCC), the most common skin cancer and seborrheic keratosis (SK) despite their high incidence rates. It is preferable to deal with these NoMSLs as well as MSLs especially for the potential users who are not enough capable of diagnosing pigmented skin lesions on their own such as dermatologists in training and physicians with different expertise. We developed a new method to distinguish among melanomas, nevi, BCCs, and SKs. Our method calculates 828 candidate features grouped into three categories: color, subregion, and texture. We introduced two types of classification models: a layered model that uses a task decomposition strategy and flat models to serve as performance baselines. We tested our methods on 964 dermoscopy images: 105 melanomas, 692 nevi, 69 BCCs, and 98 SKs. The layered model outperformed the flat models, achieving detection rates of 90.48%, 82.51%, 82.61%, and 80.61% for melanomas, nevi, BCCs, and SKs, respectively. We also identified specific features effective for the classification task including irregularity of color distribution. The results show promise for enhancing the capability of the computer-aided skin lesion classification.

Three-phase general border detection method for dermoscopy images using non-uniform illumination correction.

BACKGROUND: Computer-aided diagnosis of dermoscopy images has shown great promise in developing a quantitative, objective way of classifying skin lesions. An important step in the classification process is lesion segmentation. Many studies have been successful in segmenting melanocytic skin lesions (MSLs), but few have focused on non-melanocytic skin lesions (NoMSLs), as the wide variety of lesions makes accurate segmentation difficult. METHODS: We developed an automatic segmentation program for detecting borders of skin lesions in dermoscopy images. The method consists of a pre-processing phase, general lesion segmentation phase, including illumination correction, and bright region segmentation phase. RESULTS: We tested our method on a set of 107 NoMSLs and a set of 319 MSLs. Our method achieved precision/recall scores of 84.5% and 88.5% for NoMSLs, and 93.9% and 93.8% for MSLs, in comparison with manual extractions from four or five dermatologists. CONCLUSION: The accuracy of our method was competitive or better than five recently published methods. Our new method is the first method for detecting borders of both non-melanocytic and melanocytic skin lesions.

Computer-aided pattern classification system for dermoscopy images.

BACKGROUND: Computer-aided pattern classification of melanoma and other pigmented skin lesions is one of the most important tasks for clinical diagnosis. To differentiate between benign and malignant lesions, the extraction of color, architectural order, symmetry of pattern and homogeneity (CASH) is a challenging task. METHODS: In this article, a novel pattern classification system (PCS) based on the clinical CASH rule is presented to classify among six classes of patterns. The PCS system consists of the following five steps: transformation to the CIE L*a*b* color space, pre-processing to enhance the tumor region and removal of hairs, tumor-area segmentation, color and texture feature extraction, and finally, classification based on a multiclass support vector machine. RESULTS: The PCS system is tested on a total of 180 dermoscopic images. To test the performance of the PCS diagnostic classifier, the area under the receiver operating characteristics curve (AUC) is utilized. The proposed classifier achieved a sensitivity of 91.64%, specificity of 94.14%, and AUC of 0.948. CONCLUSION: The experimental results demonstrate that the proposed pattern classifier is highly accurate and classify between benign and malignant lesions into some extend. The PCS method is fully automatic and can accurately detect different patterns from dermoscopy images using color and texture properties. Additional pattern features can be included to investigate the impact of pattern classification based on the CASH rule.

Lesion border detection in dermoscopy images using dynamic programming.

BACKGROUND/PURPOSE: Automated border detection is an important and challenging task in the computerized analysis of dermoscopy images. However, dermoscopic images often contain artifacts such as illumination, dermoscopic gel, and outline (hair, skin lines, ruler markings, and blood vessels). As a result, there is a need for robust methods to remove artifacts and detect lesion borders in dermoscopy images. METHODS: This automated method consists of three main steps: (1) preprocessing, (2) edge candidate point detection, and (3) tumor outline delineation. First, algorithms to reduce artifacts were used. Second, a least-squares method (LSM) was performed to acquire edge points. Third, dynamic programming (DP) technique was used to find the optimal boundary of the lesion. Statistical measures based on dermatologist-drawn borders were utilized as ground-truth to evaluate the performance of the proposed method. RESULTS: The method is tested on a total of 240 dermoscopic images: 30 benign melanocytic, 50 malignant melanomas, 50 basal cell carcinomas, 20 Merkel cell carcinomas, 60 seborrheic keratosis, and 30 atypical naevi. We obtained mean border detection error of 8.6%, 5.04%, 9.0%, 7.02%, 2.01%, and 3.24%, respectively. CONCLUSIONS: The results demonstrate that border detection combined with artifact removal increases sensitivity and specificity for segmentation of lesions in dermoscopy images.

Automated color calibration method for dermoscopy images.

Accurate color information in dermoscopy images is very important for melanoma diagnosis since inappropriate white balance or brightness in the images adversely affects the diagnostic performance. In this paper, we present an automated color calibration method for dermoscopy images of skin lesions. On a set of 319 dermoscopy images, we develop color calibration filters based on the HSV color system. We determined that the color characteristics of the peripheral part of the tumors have significant influence on the color calibration filters and confirmed that the presented filters achieved satisfactory calibration performance as evaluated by cross-validation. We also confirmed that our method successfully modifies the color distribution of a given image to make it closer to the color distribution of the training image set.

Gradient vector flow with mean shift for skin lesion segmentation.

Image segmentation is an important task in the analysis of dermoscopy images since the extraction of skin lesion borders provides important cues for accurate diagnosis. In recent years, gradient vector flow based algorithms have demonstrated their merits in image segmentation. However, due to the compromise of internal and external energy forces within the partial differential equation these methods commonly lead to under- or over-segmentation problems. In this paper, we introduce a new mean shift based gradient vector flow (GVF) algorithm that drives the internal/external energies towards the correct direction. The proposed segmentation method incorporates a mean shift operation within the standard GVF cost function. Theoretical analysis proves that the proposed algorithm converges rapidly, while experimental results on a large set of diverse dermoscopy images demonstrate that the presented method accurately determines skin lesion borders in dermoscopy images.

Border detection in dermoscopy images using hybrid thresholding on optimized color channels.

Automated border detection is one of the most important steps in dermoscopy image analysis. Although numerous border detection methods have been developed, few studies have focused on determining the optimal color channels for border detection in dermoscopy images. This paper proposes an automatic border detection method which determines the optimal color channels and performs hybrid thresholding to detect the lesion borders. The color optimization process is tested on a set of 30 dermoscopy images with four sets of dermatologist-drawn borders used as the ground truth. The hybrid border detection method is tested on a set of 85 dermoscopy images with two sets of ground truth using various metrics including accuracy, precision, sensitivity, specificity, and border error. The proposed method, which is comprised of two stages, is designed to increase specificity in the first stage and sensitivity in the second stage. It is shown to be highly competitive with three state-of-the-art border detection methods and potentially faster, since it mainly involves scalar processing as opposed to vector processing performed in the other methods. Furthermore, it is shown that our method is as good as, and in some cases more effective than a dermatology registrar.

Modified watershed technique and post-processing for segmentation of skin lesions in dermoscopy images.

In previous research, a watershed-based algorithm was shown to be useful for automatic lesion segmentation in dermoscopy images, and was tested on a set of 100 benign and malignant melanoma images with the average of three sets of dermatologist-drawn borders used as the ground truth, resulting in an overall error of 15.98%. In this study, to reduce the border detection errors, a neural network classifier was utilized to improve the first-pass watershed segmentation; a novel "edge object value (EOV) threshold" method was used to remove large light blobs near the lesion boundary; and a noise removal procedure was applied to reduce the peninsula-shaped false-positive areas. As a result, an overall error of 11.09% was achieved.