A protocol for annotation of total body photography for machine learning to analyze skin phenotype and lesion classification.

Introduction: Artificial Intelligence (AI) has proven effective in classifying skin cancers using dermoscopy images. In experimental settings, algorithms have outperformed expert dermatologists in classifying melanoma and keratinocyte cancers. However, clinical application is limited when algorithms are presented with 'untrained' or out-of-distribution lesion categories, often misclassifying benign lesions as malignant, or misclassifying malignant lesions as benign. Another limitation often raised is the lack of clinical context (e.g., medical history) used as input for the AI decision process. The increasing use of Total Body Photography (TBP) in clinical examinations presents new opportunities for AI to perform holistic analysis of the whole patient, rather than a single lesion. Currently there is a lack of existing literature or standards for image annotation of TBP, or on preserving patient privacy during the machine learning process. Methods: This protocol describes the methods for the acquisition of patient data, including TBP, medical history, and genetic risk factors, to create a comprehensive dataset for machine learning. 500 patients of various risk profiles will be recruited from two clinical sites (Australia and Spain), to undergo temporal total body imaging, complete surveys on sun behaviors and medical history, and provide a DNA sample. This patient-level metadata is applied to image datasets using DICOM labels. Anonymization and masking methods are applied to preserve patient privacy. A two-step annotation process is followed to label skin images for lesion detection and classification using deep learning models. Skin phenotype characteristics are extracted from images, including innate and facultative skin color, nevi distribution, and UV damage. Several algorithms will be developed relating to skin lesion detection, segmentation and classification, 3D mapping, change detection, and risk profiling. Simultaneously, explainable AI (XAI) methods will be incorporated to foster clinician and patient trust. Additionally, a publicly released dataset of anonymized annotated TBP images will be released for an international challenge to advance the development of new algorithms using this type of data. Conclusion: The anticipated results from this protocol are validated AI-based tools to provide holistic risk assessment for individual lesions, and risk stratification of patients to assist clinicians in monitoring for skin cancer.

An Improved Skin Lesion Matching Scheme in Total Body Photography.

Total body photography is used for early detection of malignant melanoma, primarily as a means of temporal skin surface monitoring. In a prior work, we presented a scanner with a set of algorithms to map and detect changes in pigmented skin lesions, thus demonstrating that it is possible to fully automate the process of total body image acquisition and processing. The key procedure in these algorithms is skin lesion matching that determines whether two images depict the same real lesion. In this paper, we aim to improve it with respect to false positive and negative outcomes. To this end, we developed two novel methods: one based on successive rigid transformations of three-dimensional point clouds and one based on nonrigid coordinate plane deformations in regions of interest around the lesions. In both approaches, we applied a robust outlier rejection procedure based on progressive graph matching. Using the images obtained from the scanner, we created a ground truth dataset tailored to diversify false positive match scenarios. The algorithms were evaluated according to their precision and recall values, and the results demonstrated the superiority of the second approach in all the tests. In the complete interpositional matching experiment, it reached a precision and recall as high as 99.92% and 81.65%, respectively, showing a significant improvement over our original method.

A new total body scanning system for automatic change detection in multiple pigmented skin lesions.

The detection of newly appearing and changing pigmented skin lesions (PSLs) is essential for timely diagnosis of cutaneous melanoma. Total body skin examination (TBSE) procedures, currently practiced for this purpose, can be extremely time-consuming for patients with numerous lesions. In addition, these procedures are prone to subjectivity when selecting PSLs for baseline image comparison, increasing the risk of missing a developing cancer. To address this issue, we propose a new photogrammetry-based total body scanning system allowing for skin surface image acquisition using cross-polarized light. Equipped with 21 high-resolution cameras and a turntable, this scanner automatically acquires a set of overlapping images, covering 85%-90% of the patient's skin surface. These images are used for the automated mapping of PSLs and their change estimation between explorations. The maps produced relate images of individual lesions with their locations on the patient's body, solving the body-to-image and image-to-image correspondence problem in TBSEs. Currently, the scanner is limited to patients with sparse body hair and, for a complete skin examination, the scalp, palms, soles and inner arms should be photographed manually. The initial tests of the scanner showed that it can be successfully applied for automated mapping and temporal monitoring of multiple lesions: PSLs relevant for follow-up were repeatedly mapped in several explorations. Moreover, during the baseline image comparison, all lesions with artificially induced changes were correctly identified as "evolved."

A novel method for color correction in epiluminescence microscopy.

This paper proposes a new color correction pipeline to improve the dermoscopy image quality. Images acquired with different cameras or different dermoscopes often present problems of faithful color reproduction. The colors of these images are often far different the ones observed with the naked eye, and usually vary from one camera to another. Nowadays digital cameras perform "black-box" color corrections taking into account the color temperature of the imaged scene, which may result in some cases in unrealistic color rendering. For this reason, it is necessary to calibrate the imaging system (the camera and a specific dermoscope). The calibration process requires finding a relationship between a device-dependent color space and a standard color space depending only on the human eye. This relation is obtained acquiring known color patches of a color checker and relating them with the pixel values obtained by the camera. In our approach we model the color calibration problem using a new formulation that takes into account the spectral distribution of the dermoscope lighting system and conveys a solution for both RAW and JPEG images. When comparing images captured with different cameras, this new method improves the results between 0.1 and 0.9 ΔE with respect to previous approaches.