Deep learning-based skin care product recommendation: A focus on cosmetic ingredient analysis and facial skin conditions.

BACKGROUND: Recommendations for cosmetics are gaining popularity, but they are not being made with consideration of the analysis of cosmetic ingredients, which customers consider important when selecting cosmetics. AIMS: This article aims to propose a method for estimating the efficacy of cosmetics based on their ingredients and introduces a system that recommends personalized products for consumers, combined with AI skin analysis. METHODS: We constructed a deep neural network architecture to analyze sequentially arranged cosmetic ingredients in the product and incorporated skin analysis models to get the precise skin status of users from frontal face images. Our recommendation system makes decisions based on the results optimized for the individual. RESULTS: Our cosmetic recommendation system has shown its effectiveness through reliable evaluation metrics, and numerous examples have demonstrated its ability to make reasonable recommendations for various skin problems. CONCLUSION: The result shows that deep learning methods can be used to predict the effects of products based on their cosmetic ingredients and are available for use in personalized cosmetic recommendations.

Facial wrinkle segmentation using weighted deep supervision and semi-automatic labeling.

Facial wrinkles are important indicators of human aging. Recently, a method using deep learning and a semi-automatic labeling was proposed to segment facial wrinkles, which showed much better performance than conventional image-processing-based methods. However, the difficulty of wrinkle segmentation remains challenging due to the thinness of wrinkles and their small proportion in the entire image. Therefore, performance improvement in wrinkle segmentation is still necessary. To address this issue, we propose a novel loss function that takes into account the thickness of wrinkles based on the semi-automatic labeling approach. First, considering the different spatial dimensions of the decoder in the U-Net architecture, we generated weighted wrinkle maps from ground truth. These weighted wrinkle maps were used to calculate the training losses more accurately than the existing deep supervision approach. This new loss computation approach is defined as weighted deep supervision in our study. The proposed method was evaluated using an image dataset obtained from a professional skin analysis device and labeled using semi-automatic labeling. In our experiment, the proposed weighted deep supervision showed higher Jaccard Similarity Index (JSI) performance for wrinkle segmentation compared to conventional deep supervision and traditional image processing methods. Additionally, we conducted experiments on the labeling using a semi-automatic labeling approach, which had not been explored in previous research, and compared it with human labeling. The semi-automatic labeling technology showed more consistent wrinkle labels than human-made labels. Furthermore, to assess the scalability of the proposed method to other domains, we applied it to retinal vessel segmentation. The results demonstrated superior performance of the proposed method compared to existing retinal vessel segmentation approaches. In conclusion, the proposed method offers high performance and can be easily applied to various biomedical domains and U-Net-based architectures. Therefore, the proposed approach will be beneficial for various biomedical imaging approaches. To facilitate this, we have made the source code of the proposed method publicly available at: https://github.com/resemin/WeightedDeepSupervision.

Generation of skin tone and pigmented region-modified images using a pigment discrimination model trained with an optical approach.

BACKGROUND: Skin tone and pigmented regions, associated with melanin and hemoglobin, are critical indicators of skin condition. While most prior research focuses on pigment analysis, the capability to simulate diverse pigmentation conditions could greatly broaden the range of applications. However, current methodologies have limitations in terms of numerical control and versatility. METHODS: We introduce a hybrid technique that integrates optical methods with deep learning to produce skin tone and pigmented region-modified images with numerical control. The pigment discrimination model produces melanin, hemoglobin, and shading maps from skin images. The outputs are reconstructed into skin images using a forward problem-solving approach, with model training aimed at minimizing the discrepancy between the reconstructed and input images. By adjusting the melanin and hemoglobin maps, we create pigment-modified images, allowing precise control over changes in melanin and hemoglobin levels. Changes in pigmentation are quantified using the individual typology angle (ITA) for skin tone and melanin and erythema indices for pigmented regions, validating the intended modifications. RESULTS: The pigment discrimination model achieved correlation coefficients with clinical equipment of 0.915 for melanin and 0.931 for hemoglobin. The alterations in the melanin and hemoglobin maps exhibit a proportional correlation with the ITA and pigment indices in both quantitative and qualitative assessments. Additionally, regions overlaying melanin and hemoglobin are demonstrated to verify independent adjustments. CONCLUSION: The proposed method offers an approach to generate modified images of skin tone and pigmented regions. Potential applications include visualizing alterations for clinical assessments, simulating the effects of skincare products, and generating datasets for deep learning.

Deep learning-based pigment analysis model trained with optical approach and ground truth assistance.

This study introduces an integrated training method combining the optical approach with ground truth for skin pigment analysis. Deep learning is increasingly applied to skin pigment analysis, primarily melanin and hemoglobin. While regression analysis is a widely used training method to predict ground truth-like outputs, the input image resolution is restricted by computational resources. The optical approach-based regression method can alleviate this problem, but compromises performance. We propose a strategy to overcome the limitation of image resolution while preserving performance by incorporating ground truth within the optical approach-based learning structure. The proposed model decomposes skin images into melanin, hemoglobin, and shading maps, reconstructing them by solving the forward problem with reference to the ground truth for pigments. Evaluation against the VISIA system, a professional diagnostic equipment, yields correlation coefficients of 0.978 for melanin and 0.975 for hemoglobin. Furthermore, our model can produce pigment-modified images for applications like simulating treatment effects.

Deep-Learning-Based Morphological Feature Segmentation for Facial Skin Image Analysis.

Facial skin analysis has attracted considerable attention in the skin health domain. The results of facial skin analysis can be used to provide skin care and cosmetic recommendations in aesthetic dermatology. Because of the existence of several skin features, grouping similar features and processing them together can improve skin analysis. In this study, a deep-learning-based method of simultaneous segmentation of wrinkles and pores is proposed. Unlike color-based skin analysis, this method is based on the analysis of the morphological structures of the skin. Although multiclass segmentation is widely used in computer vision, this segmentation was first used in facial skin analysis. The architecture of the model is U-Net, which has an encoder-decoder structure. We added two types of attention schemes to the network to focus on important areas. Attention in deep learning refers to the process by which a neural network focuses on specific parts of its input to improve its performance. Second, a method to enhance the learning capability of positional information is added to the network based on the fact that the locations of wrinkles and pores are fixed. Finally, a novel ground truth generation scheme suitable for the resolution of each skin feature (wrinkle and pore) was proposed. The experimental results revealed that the proposed unified method achieved excellent localization of wrinkles and pores and outperformed both conventional image-processing-based approaches and one of the recent successful deep-learning-based approaches. The proposed method should be expanded to applications such as age estimation and the prediction of potential diseases.

Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution.

Significance: Melanin and hemoglobin have been measured as important diagnostic indicators of facial skin conditions for aesthetic and diagnostic purposes. Commercial clinical equipment provides reliable analysis results, but it has several drawbacks: exclusive to the acquisition system, expensive, and computationally intensive. Aim: We propose an approach to alleviate those drawbacks using a deep learning model trained to solve the forward problem of light-tissue interactions. The model is structurally extensible for various light sources and cameras and maintains the input image resolution for medical applications. Approach: A facial image is divided into multiple patches and decomposed into melanin, hemoglobin, shading, and specular maps. The outputs are reconstructed into a facial image by solving the forward problem over skin areas. As learning progresses, the difference between the reconstructed image and input image is reduced, resulting in the melanin and hemoglobin maps becoming closer to their distribution of the input image. Results: The proposed approach was evaluated on 30 subjects using the professional clinical system, VISIA VAESTRO. The correlation coefficients for melanin and hemoglobin were found to be 0.932 and 0.857, respectively. Additionally, this approach was applied to simulated images with varying amounts of melanin and hemoglobin. Conclusion: The proposed approach showed high correlation with the clinical system for analyzing melanin and hemoglobin distribution, indicating its potential for accurate diagnosis. Further calibration studies using clinical equipment can enhance its diagnostic ability. The structurally extensible model makes it a promising tool for various image acquisition conditions.

Enhanced electrogenerated chemiluminescence of phenylethynylpyrene derivatives: use of weakly electron-donating group as a substituent.

A weakly donating group (n-propyl) has been used as a substituent at the para-position of the phenyl group for a series of phenylethynylpyrene derivatives where the number of phenylethynyl peripheral arms appended to the pyrene core is varied. This system markedly improved the concurrent stability of both cation and anion radicals and consequently greatly improved electrogenerated chemiluminescence (ECL). Density functional theory (DFT)-based theoretical calculations supported the associated photophysical and electrochemical properties of the series compounds.