A protocol for annotation of total body photography for machine learning to analyze skin phenotype and lesion classification.

Introduction: Artificial Intelligence (AI) has proven effective in classifying skin cancers using dermoscopy images. In experimental settings, algorithms have outperformed expert dermatologists in classifying melanoma and keratinocyte cancers. However, clinical application is limited when algorithms are presented with 'untrained' or out-of-distribution lesion categories, often misclassifying benign lesions as malignant, or misclassifying malignant lesions as benign. Another limitation often raised is the lack of clinical context (e.g., medical history) used as input for the AI decision process. The increasing use of Total Body Photography (TBP) in clinical examinations presents new opportunities for AI to perform holistic analysis of the whole patient, rather than a single lesion. Currently there is a lack of existing literature or standards for image annotation of TBP, or on preserving patient privacy during the machine learning process. Methods: This protocol describes the methods for the acquisition of patient data, including TBP, medical history, and genetic risk factors, to create a comprehensive dataset for machine learning. 500 patients of various risk profiles will be recruited from two clinical sites (Australia and Spain), to undergo temporal total body imaging, complete surveys on sun behaviors and medical history, and provide a DNA sample. This patient-level metadata is applied to image datasets using DICOM labels. Anonymization and masking methods are applied to preserve patient privacy. A two-step annotation process is followed to label skin images for lesion detection and classification using deep learning models. Skin phenotype characteristics are extracted from images, including innate and facultative skin color, nevi distribution, and UV damage. Several algorithms will be developed relating to skin lesion detection, segmentation and classification, 3D mapping, change detection, and risk profiling. Simultaneously, explainable AI (XAI) methods will be incorporated to foster clinician and patient trust. Additionally, a publicly released dataset of anonymized annotated TBP images will be released for an international challenge to advance the development of new algorithms using this type of data. Conclusion: The anticipated results from this protocol are validated AI-based tools to provide holistic risk assessment for individual lesions, and risk stratification of patients to assist clinicians in monitoring for skin cancer.

Validating the Reliability and Clinical Relevance of a Nasolabial Fold Photonumeric Scale.

BACKGROUND: The use of tissue fillers to treat age-related deepening of the nasolabial fold (NLF) has increased and become the standard clinical approach, creating a need for evidence-based, objective evaluation for pre- and post-procedure assessment of the NLF. METHODS: A 5-point rating scale was developed to assess the NLF, specifically the presence of depression and shadowing. Live validation of the scale was performed with a total of 73 participants representing the full range of NLF severities. Physicians board-certified in a core aesthetic specialty (3 trained raters) performed the scale validation over 2 rounds, 2 weeks apart. Training was carried out, and test-retest reliability was quantitated through the determination of intra- and inter-rater reliability by percentage of agreement, weighted kappa statistic with 95% confidence interval (CI), and intraclass correlation coefficient with 95% CI. To evaluate the clinical relevance of a 1-grade difference, rater assessments of 90 photo pairs were compared with previous designations of clinically different or not clinically different. RESULTS: The NLF scale achieved near-perfect intra- and inter-rater reliability when utilized by trained raters to assess a diverse group of live participants. Furthermore, clinically relevant differences between grades were established, and a 1-point difference was detectable by trained evaluators using the NLF scale. CONCLUSION: The clinically relevant and highly reliable validated NLF scale provides a standardized grading system with a user-friendly design for objectively assessing NLF in clinical practice and as a research tool for clinical approval studies of new aesthetic products and technologies. J Drugs Dermatol. 2024;23(1):1284-1291.   doi:10.36849/JDD.7316.

Validating the Reliability and Clinical Relevance of an Infraorbital Hollow Photonumeric Scale.

Background: Infraorbital hollowing is a facial aesthetic issue for which a broad age range of patients seek treatment. Expanding treatment options for this region warrants the development of validated tools to objectively assess infraorbital hollow (IOH) severity before and after treatment. Objectives: To validate a 4-point rating scale to assess depression of IOH, depression relative to the mid-pupillary line, and visibility of the lateral orbital rim. Methods: The IOH scale described herein was developed and subjected to live validation with a total of 73 patients representing the full range of IOH severities. Scale validation was performed by board-certified plastic surgeons and dermatologists (3 raters) over 2 rounds, 2 weeks apart. Intrarater and interrater reliabilities were used to demonstrate test-retest reliability as quantitated with percentage of agreement, weighted kappa statistic with 95% confidence interval (CI), and intraclass correlation coefficient with 95% CI. The clinical relevance of a 1-grade difference was evaluated by comparing rater assessments of 77 photo pairs with their previously determined designation as "clinically different" or "not clinically different." Results: The IOH scale demonstrated substantial to near-perfect intrarater and interrater reliabilities when utilized by trained raters to assess a diverse group of live patients. Furthermore, clinically relevant differences between grades were established, and detection of a 1-point difference could be achieved by trained evaluators using the IOH scale. Conclusions: This highly reliable, clinically relevant, and validated IOH scale provides a user-friendly, standardized grading system to objectively evaluate and track changes in infraorbital hollowing in clinical practice and research.

Reproducible Naevus Counts Using 3D Total Body Photography and Convolutional Neural Networks.

BACKGROUND: The number of naevi on a person is the strongest risk factor for melanoma; however, naevus counting is highly variable due to lack of consistent methodology and lack of inter-rater agreement. Machine learning has been shown to be a valuable tool for image classification in dermatology. OBJECTIVES: To test whether automated, reproducible naevus counts are possible through the combination of convolutional neural networks (CNN) and three-dimensional (3D) total body imaging. METHODS: Total body images from a study of naevi in the general population were used for the training (82 subjects, 57,742 lesions) and testing (10 subjects; 4,868 lesions) datasets for the development of a CNN. Lesions were labelled as naevi, or not ("non-naevi"), by a senior dermatologist as the gold standard. Performance of the CNN was assessed using sensitivity, specificity, and Cohen's kappa, and evaluated at the lesion level and person level. RESULTS: Lesion-level analysis comparing the automated counts to the gold standard showed a sensitivity and specificity of 79% (76-83%) and 91% (90-92%), respectively, for lesions ≥2 mm, and 84% (75-91%) and 91% (88-94%) for lesions ≥5 mm. Cohen's kappa was 0.56 (0.53-0.59) indicating moderate agreement for naevi ≥2 mm, and substantial agreement (0.72, 0.63-0.80) for naevi ≥5 mm. For the 10 individuals in the test set, person-level agreement was assessed as categories with 70% agreement between the automated and gold standard counts. Agreement was lower in subjects with numerous seborrhoeic keratoses. CONCLUSION: Automated naevus counts with reasonable agreement to those of an expert clinician are possible through the combination of 3D total body photography and CNNs. Such an algorithm may provide a faster, reproducible method over the traditional in person total body naevus counts.

A New Model to Simulate Human Facial Appearance, Accurately, and Realistically, across Age and Ethnicity.

BACKGROUND: The human population is aging globally, and there is significant, growing interest in modeling and simulating facial appearance. METHODS: The authors describe a new means to model and simulate aging in facial images, using an approach based entirely on 3D whole-face data collected from 1250 female subjects, across 5 ethnicities, ages 10-80. RESULTS: Three models were built, each describing changes with age within each ethnic group, namely shape, color, and topography. These three models were used to build a simulation able to age or de-age a 2D image of a female subject's face, with a degree of accuracy and realism not achievable with previous approaches. Simulated images were validated by a cloud-based age estimator. CONCLUSIONS: The authors have developed a new facial age simulation model, where the use of three submodels (shape, color and topography), built from acquired 3D data, provides both scientifically robust and realistic output. As the data were acquired across five of the world's major ethnicities, this new model allows valuable insight into changes in the facial appearance of our aging global population.

Development and qualification of a machine learning algorithm for automated hair counting.

OBJECTIVE: Determining the amount of hair on the scalp has always been an important metric of patient satisfaction for hair growth and hair retention technologies. While simple in concept, this measurement is a difficult, resource intensive task for the dermatologist and the research scientist. Specifically, counting and measuring hair in phototrichogram images is very time consuming and labour intensive. Due to cost, often only a fraction of available images is manually analysed. There is a need for an automated method that can significantly increase speed and throughput while reducing the cost of counting and measuring hair in phototrichogram images. METHODS: Recent advances in machine learning and deep convolutional neural networks (deep learning) have led to a revolution in the analysis of image, video, speech, text and other sensor data. Image diagnostics have seen remarkable improvements with completely automated methods outperforming both human experts and human-engineered analysis methods. Deep learning methods can also provide speed and cost benefits. To enable use of a deep learning, we created a data set of 288 manually annotated phototrichogram images with marked location and length of each hair (the training dataset). We designed a custom neural network architecture and custom image processing algorithms to best utilize the available training data and to maximize performance for hair counting and length measurement. The performance of the algorithm was qualified by comparing hair count and length measurements to an independent ground truth method, the semi-manual Canfield's Hair Metrix method. RESULTS: Leveraging deep neural networks, we have developed capability to apply machine learning to reduce the time needed to acquire data from phototrichograms of patients' scalp from months to seconds. Our algorithm enables fast and fully automated hair counting and length measurement. The algorithm shows high agreement with human manually assisted analysis (ground truth). CONCLUSIONS: We have trained and deployed an algorithm utilizing this technology and have demonstrated the reproducibility, accuracy and speed of this algorithm that, once deployed, requires little to no recurring cost or manual intervention for its operation. The method allows fast analysis of large number of images, reducing study cost and significantly reducing study analysis time.

Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification.

Recent research has helped to cultivate growing awareness that machine-learning systems fueled by big data can create or exacerbate troubling disparities in society. Much of this research comes from outside of the practicing data science community, leaving its members with little concrete guidance to proactively address these concerns. This article introduces issues of discrimination to the data science community on its own terms. In it, we tour the familiar data-mining process while providing a taxonomy of common practices that have the potential to produce unintended discrimination. We also survey how discrimination is commonly measured, and suggest how familiar development processes can be augmented to mitigate systems' discriminatory potential. We advocate that data scientists should be intentional about modeling and reducing discriminatory outcomes. Without doing so, their efforts will result in perpetuating any systemic discrimination that may exist, but under a misleading veil of data-driven objectivity.

Effects of intrinsic aging and photodamage on skin dyspigmentation: an explorative study.

Photoaging is associated with increasing pigmentary heterogeneity and darkening of skin color. However, little is known about age-related changes in skin pigmentation on sun-protected areas. The aim of this explorative study was to measure skin color and dyspigmentation using image processing and to evaluate the reliability of these parameters. Twenty-four volunteers of three age-groups were included in this explorative study. Measurements were conducted at sun-exposed and sun-protected areas. Overall skin-color estimates were similar among age groups. The hyper- and hypopigmentation indices differed significantly by age groups and their correlations with age ranged between 0.61 and 0.74. Dorsal forearm skin differed from the other investigational areas (p<0.001). We observed an increase in dyspigmentation at all skin areas, including sun-protected skin areas, already in young adulthood. Associations between age and dyspigmentation estimates were higher compared to color parameters. All color and dyspigmentation estimates showed high reliability. Dyspigmentation parameters seem to be better biomarkers for UV damage than the overall color measurements.

3-D volume reconstruction of skin lesions for melanin and blood volume estimation and lesion severity analysis.

Subsurface information about skin lesions, such as the blood volume beneath the lesion, is important for the analysis of lesion severity towards early detection of skin cancer such as malignant melanoma. Depth information can be obtained from diffuse reflectance based multispectral transillumination images of the skin. An inverse volume reconstruction method is presented which uses a genetic algorithm optimization procedure with a novel population initialization routine and nudge operator based on the multispectral images to reconstruct the melanin and blood layer volume components. Forward model evaluation for fitness calculation is performed using a parallel processing voxel-based Monte Carlo simulation of light in skin. Reconstruction results for simulated lesions show excellent volume accuracy. Preliminary validation is also done using a set of 14 clinical lesions, categorized into lesion severity by an expert dermatologist. Using two features, the average blood layer thickness and the ratio of blood volume to total lesion volume, the lesions can be classified into mild and moderate/severe classes with 100% accuracy. The method therefore has excellent potential for detection and analysis of pre-malignant lesions.

Transillumination imaging for blood oxygen saturation estimation of skin lesions.

Detecting the early stages of melanoma can be greatly assisted by an accurate estimate of subsurface blood volume and blood oxygen saturation, indicative of angiogenesis. Visualization of this blood volume present beneath a skin lesion can be achieved through the transillumination of the skin. As the absorption of major chromophores in the skin is wavelength dependent, multispectral imaging can provide the needed information to separate out relative amounts of each chromophore. However, a critical challenge to this strategy is relating the pixel intensities observed in a given image to the wavelength-dependent total absorption existing at each spatial location. Consequently, in this paper, we develop an extension to Beer's law, estimated through a novel voxel-based, parallel processing Monte Carlo simulation of light propagation in skin which takes into account the specific geometry of our transillumination imaging apparatus. We then use this relation in a linear mixing model, solved using a multispectral image set, for chromophore separation and oxygen saturation estimation of an absorbing object located at a given depth within the medium. Validation is performed through the Monte Carlo simulation, as well as by imaging on a skin phantom. Results show that subsurface oxygen saturation can be reasonably estimated with good implications for the reconstruction of 3-D skin lesion volumes using transillumination toward early detection of malignancy.

Synthetic positron emission tomography-computed tomography images for use in perceptual studies.

To better understand fundamental issues, perception studies of the fusion display would best be performed with a panel of lesions of variable location, size, intensity, and background. There are compelling reasons to use synthetic images that contain artificial lesions for perception research. A consideration of how to obtain this panel of lesions is the nucleus of the present review. This article is a conjoint effort of 3 groups that have joined together to review results from work that they and others have performed. The techniques we review include (1) substitution of lesions into a preexisting image matrix (either using actual prior patient-derived lesions or mathematically modeled artificial lesions), (2) addition of images (either in the attenuation-corrected image space or at an earlier stage before image reconstruction), and (3) simulation of the entire patient image. A judicious combination of the techniques discussed in this review may represent the most efficient pathway of simulating statistically varied but realistic appearing lesions.

Computer aided analysis of epi-illumination and transillumination images of skin lesions for diagnosis of skin cancers.

Skin lesion pigmentation area from surface, or, epi-illumination (ELM) images and blood volume area from transillumination (TLM) images are useful features to aid a dermatologist in the diagnosis of melanoma and other skin cancers in early curable stages. However, segmentation of these areas is difficult. In this work, we present an automatic segmentation tool for ELM and TLM images that also provides additional choices for user selection and interaction with adaptive learning. Our tool uses a combination of k-means clustering, wavelet analysis, and morphological operations to segment the lesion and blood volume, and then presents the user with six segmentation suggestions for both ELM and TLM images. The final selection of segmentation boundary may then be iteratively improved through scoring by multiple users. The ratio of TLM to ELM segmented areas is an indicator of dysplasia in skin lesions for detection of skin cancers, and this ratio is found to show a statistically significant trend in association with lesion dysplasia on a set of 81 pathologically validated lesions (p = 0.0058). We then present a support vector machine classifier using the results from the interactive segmentation method along with ratio, color, texture, and shape features to characterize skin lesions into three degrees of dysplasia with promising accuracy.

Voxel-based, parallel simulation of light in skin tissue for the reconstruction of subsurface skin lesion volumes.

Early detection and diagnosis of skin cancer is essential to treating the malignancy and preventing death. Subsurface features and depth information are critical in evaluating a skin lesion for this early malignancy screening. We present a novel voxel-based Monte Carlo simulation of light propagation in skin tissue which runs in a highly parallel environment on desktop graphics processors, resulting in an extremely fast simulation of millions of photons in less than one second. We then use this model in a genetic algorithm for the inverse 3D volume reconstruction of a skin lesion, given a set of multispectral images obtained using non-invasive transillumination imaging. Our method demonstrates improved accuracy at a superior resolution to existing methods.

Multispectral transillumination imaging of skin lesions for oxygenated and deoxygenated hemoglobin measurement.

The early detection of melanoma is critical for patient survival. One of the indentifying features of new malignancy is increased blood flow to the lesion. Multispectral transillumination using the Nevoscope has been demonstrated to be an effective tool for imaging the sub-surface vascular architecture of skin lesions. Using multispectral images obtained from this tool in the visible and near-infrared range, as well as the relative difference in spectral absorption due to oxyhemoglobin and deoxyhemoglobin, we propose an empirical method to estimate the blood flow volume within a skin lesion. From the images, estimates of the distribution of both Hb and HbO(2) are calculated along with a ratiometric feature describing the relative oxygen saturation level in the blood. We validate our proposed method through the imaging of a skin phantom with embedded capillaries which can be filled with either an artificial Hb or HbO(2) liquid. Our near-IR, multispectral computations nicely differentiate the Hb filled phantom versus the HbO(2) filled phantom, demonstrating that these chromophores can be successfully separated and individually characterized for use in estimating the relative oxygen saturation of skin tissue.

Depth-dependent hemoglobin analysis from multispectral transillumination images.

Multispectral transillumination imaging is a promising modality for noninvasive imaging of living tissue. Multispectral Nevoscope imaging is directed toward the imaging of skin lesions for the detection and characterization of skin cancers through the volumetric analysis of selected chromophores, such as melanin, oxy-, and deoxyhemoglobin. In this letter, we present a novel method of recovering depth-dependent measurements from transillumination images obtained through the Nevoscope. A method for estimating the depth-dependent point spread function is presented and used in recovering multispectral transillumination images of a skin phantom or lesion through blind deconvolution. A method for ratiometric analysis for the quantification of oxy- and deoxyhemoglobin is then presented and evaluated on a skin phantom. The presented methods would allow reliable quantitative analysis of multispectral Nevoscope images for early detection of angiogenesis leading to early diagnosis of skin cancers.

Optical imaging modalities for biomedical applications.

Optical photographic imaging is a well known imaging method that has been successfully translated into biomedical applications such as microscopy and endoscopy. Although several advanced medical imaging modalities are used today to acquire anatomical, physiological, metabolic, and functional information from the human body, optical imaging modalities including optical coherence tomography, confocal microscopy, multiphoton microscopy, multispectral endoscopy, and diffuse reflectance imaging have recently emerged with significant potential for non-invasive, portable, and cost-effective imaging for biomedical applications spanning tissue, cellular, and molecular levels. This paper reviews methods for modeling the propagation of light photons in a biological medium, as well as optical imaging from organ to cellular levels using visible and near-infrared wavelengths for biomedical and clinical applications.

Multispectral optical imaging of skin-lesions for detection of malignant melanomas.

Optical imaging of skin-lesions for early detection and management of the most fatal skin-cancer malignant melanoma is of significant interest in mass screening of skin-lesions with high-risk population. Surface illumination based optical imaging methods such as epiluminescence light microscopy (ELM) through "Dermascopy" has shown a significant potential in improving early diagnosis of malignant melanomas. Limitations of surface reflectance based imaging systems have been realized in analyzing images for important vascular and depth dependent information. We have developed a novel optical imaging system, the Nevoscope, that uses multispectral transillumination as to provide images of skin-lesions showing sub-surface pigmentation as well as vascular architecture based blood volume information. This paper presents multispectral Nevoscope transillumination method to compare and analyze ratiometric measurements to epiluminescence imaging for its ability to discriminate malignant melanomas from dysplastic nevi and other normal skin-lesions.