Observational study investigating the level of support from a convolutional neural network in face and scalp lesions deemed diagnostically 'unclear' by dermatologists.

BACKGROUND: The clinical diagnosis of face and scalp lesions (FSL) is challenging due to overlapping features. Dermatologists encountering diagnostically 'unclear' lesions may benefit from artificial intelligence support via convolutional neural networks (CNN). METHODS: In a web-based classification task, dermatologists (n = 64) diagnosed a convenience sample of 100 FSL as 'benign', 'malignant', or 'unclear' and indicated their management decisions ('no action', 'follow-up', 'treatment/excision'). A market-approved CNN (Moleanalyzer-Pro®, FotoFinder Systems, Germany) was applied for binary classifications (benign/malignant) of dermoscopic images. RESULTS: After reviewing one dermoscopic image per case, dermatologists labelled 562 of 6400 diagnoses (8.8%) as 'unclear' and mostly managed these by follow-up examinations (57.3%, n = 322) or excisions (42.5%, n = 239). Management was incorrect in 58.8% of 291 truly malignant cases (171 'follow-up' or 'no action') and 43.9% of 271 truly benign cases (119 'excision'). Accepting CNN classifications in unclear cases would have reduced false management decisions to 4.1% in truly malignant and 31.7% in truly benign lesions (both p < 0.01). After receiving full case information 239 diagnoses (3.7%) remained 'unclear' to dermatologists, now triggering more excisions (72.0%) than follow-up examinations (28.0%). These management decisions were incorrect in 32.8% of 116 truly malignant cases and 76.4% of 123 truly benign cases. Accepting CNN classifications would have reduced false management decisions to 6.9% in truly malignant lesions and to 38.2% in truly benign cases (both p < 0.01). CONCLUSIONS: Dermatologists mostly managed diagnostically 'unclear' FSL by treatment/excision or follow-up examination. Following CNN classifications as guidance in unclear cases seems suitable to significantly reduce incorrect decisions.

Does sex matter? Analysis of sex-related differences in the diagnostic performance of a market-approved convolutional neural network for skin cancer detection.

BACKGROUND: Advances in biomedical artificial intelligence may introduce or perpetuate sex and gender discriminations. Convolutional neural networks (CNN) have proven a dermatologist-level performance in image classification tasks but have not been assessed for sex and gender biases that may affect training data and diagnostic performance. In this study, we investigated sex-related imbalances in training data and diagnostic performance of a market-approved CNN for skin cancer classification (Moleanalyzer Pro®, Fotofinder Systems GmbH, Bad Birnbach, Germany). METHODS: We screened open-access dermoscopic image repositories widely used for CNN training for distribution of sex. Moreover, the sex-related diagnostic performance of the market-approved CNN was tested in 1549 dermoscopic images stratified by sex (female n = 773; male n = 776). RESULTS: Most open-access repositories showed a marked under-representation of images originating from female (40%) versus male (60%) patients. Despite these imbalances and well-known sex-related differences in skin anatomy or skin-directed behaviour, the tested CNN achieved a comparable sensitivity of 87.0% [80.9%-91.3%] versus 87.1% [81.1%-91.4%], specificity of 98.7% [97.4%-99.3%] versus 96.9% [95.2%-98.0%] and ROC-AUC of 0.984 [0.975-0.993] versus 0.979 [0.969-0.988] in dermoscopic images of female versus male origin, respectively. In the sample at hand, sex-related differences in ROC-AUCs were not statistically significant in the per-image analysis nor in an additional per-individual analysis (p ≥ 0.59). CONCLUSION: Design and training of artificial intelligence algorithms for medical applications should generally acknowledge sex and gender dimensions. Despite sex-related imbalances in open-access training data, the diagnostic performance of the tested CNN showed no sex-related bias in the classification of skin lesions.

Dark corner artefact and diagnostic performance of a market-approved neural network for skin cancer classification.

BACKGROUND AND OBJECTIVES: Convolutional neural networks (CNN) have proven dermatologist-level performance in skin lesion classification. Prior to a broader clinical application, an assessment of limitations is crucial. Therefore, the influence of a dark tubular periphery in dermatoscopic images (also called dark corner artefact [DCA]) on the diagnostic performance of a market-approved CNN for skin lesion classification was investigated. PATIENTS AND METHODS: A prospective image set of 233 skin lesions (60 malignant, 173 benign) without DCA (control-set) was modified to show small, medium or large DCA. All 932 images were analyzed by a market-approved CNN (Moleanalyzer-Pro® , FotoFinder Systems), providing malignancy scores (range 0-1) with the cut-off > 0.5 indicating malignancy. RESULTS: In the control-set the CNN achieved a sensitivity of 90.0 % (79.9 % - 95.3 %), a specificity of 96.5 % (92.6 % - 98.4 %), and an area under the curve (AUC) of receiver operating characteristics (ROC) of 0.961 (0.932 - 0.989). Comparable diagnostic performance was observed in the DCAsmall-set and DCAmedium-set. Conversely, in the DCAlarge-set significantly increased malignancy scores triggered a significantly decreased specificity (87.9 % [82.2 % - 91.9 %], P < 0.001), non-significantly increased sensitivity (96.7 % [88.6 % - 99.1 %]) and unchanged ROC-AUC of 0.962 (0.935 - 0.989). CONCLUSIONS: Convolutional neural network classification was robust in images with small and medium DCA, but impaired in images with large DCA. Physicians should be aware of this limitation when submitting images to CNN classification.

Past and present of computer-assisted dermoscopic diagnosis: performance of a conventional image analyser versus a convolutional neural network in a prospective data set of 1,981 skin lesions.

BACKGROUND: Convolutional neural networks (CNNs) have shown a dermatologist-level performance in the classification of skin lesions. We aimed to deliver a head-to-head comparison of a conventional image analyser (CIA), which depends on segmentation and weighting of handcrafted features, to a CNN trained by deep learning. METHODS: Cross-sectional study using a real-world, prospectively acquired, dermoscopic dataset of 1981 skin lesions to compare the diagnostic performance of a market-approved CNN (Moleanalyzer-Pro™, developed in 2018) to a CIA (Moleanalyzer-3™/Dynamole™; developed in 2004, all FotoFinder Systems Inc, Germany). As a reference standard, we used histopathological diagnoses (n = 785) or, in non-excised benign lesions (n = 1196), expert consensus plus an uneventful follow-up by sequential digital dermoscopy for at least 2 years. RESULTS: A total of 281 malignant lesions and 1700 benign lesions from 435 patients (62.2% male, mean age: 52 years) were prospectively imaged. The CNN showed a sensitivity of 77.6% (95% confidence interval [CI]: [72.4%-82.1%]), specificity of 95.3% (95% CI: [94.2%-96.2%]), and receiver operating characteristic (ROC)-area under the curve (AUC) of 0.945 (95% CI: [0.930-0.961]). In contrast, the CIA achieved a sensitivity of 53.4% (95% CI: [47.5%-59.1%]), specificity of 86.6% (95% CI: [84.9%-88.1%]) and ROC-AUC of 0.738 (95% CI: [0.701-0.774]). The data set included melanomas originally diagnosed by dynamic changes during sequential digital dermoscopy (52 of 201, 20.6%), which reduced the sensitivities of both classifiers. Pairwise comparisons of sensitivities, specificities, and ROC-AUCs indicated a clear outperformance by the CNN (all p < 0.001). CONCLUSIONS: The superior diagnostic performance of the CNN argues against a continued application of former CIAs as an aide to physicians' clinical management decisions.