Weakly Supervised Deep Learning for Tooth-Marked Tongue Recognition.

The recognition of tooth-marked tongues has important value for clinical diagnosis of traditional Chinese medicine. Tooth-marked tongue is often related to spleen deficiency, cold dampness, sputum, effusion, and blood stasis. The clinical manifestations of patients with tooth-marked tongue include loss of appetite, borborygmus, gastric distention, and loose stool. Traditional clinical tooth-marked tongue recognition is conducted subjectively based on the doctor's visual observation, and its performance is affected by the doctor's subjectivity, experience, and environmental lighting changes. In addition, the tooth marks typically have various shapes and colors on the tongue, which make it very challenging for doctors to identify tooth marks. The existing methods based on deep learning have made great progress for tooth-marked tongue recognition, but there are still shortcomings such as requiring a large amount of manual labeling of tooth marks, inability to detect and locate the tooth marks, and not conducive to clinical diagnosis and interpretation. In this study, we propose an end-to-end deep neural network for tooth-marked tongue recognition based on weakly supervised learning. Note that the deep neural network only requires image-level annotations of tooth-marked or non-tooth marked tongues. In this method, a deep neural network is trained to classify tooth-marked tongues with the image-level annotations. Then, a weakly supervised tooth-mark detection network (WSTDN) as an architecture variant of the pre-trained deep neural network is proposed for the tooth-marked region detection. Finally, the WSTDN is re-trained and fine-tuned using only the image-level annotations to simultaneously realize the classification of the tooth-marked tongue and the positioning of the tooth-marked region. Experimental results of clinical tongue images demonstrate the superiority of the proposed method compared with previously reported deep learning methods for tooth-marked tongue recognition. The proposed tooth-marked tongue recognition model may provide important syndrome diagnosis and efficacy evaluation methods, and contribute to the understanding of ethnopharmacological mechanisms.

Constructing tongue coating recognition model using deep transfer learning to assist syndrome diagnosis and its potential in noninvasive ethnopharmacological evaluation.

ETHNOPHARMACOLOGICAL RELEVANCE: Tongue coating has been used as an effective signature of health in traditional Chinese medicine (TCM). The level of greasy coating closely relates to the strength of dampness or pathogenic qi in TCM theory. Previous empirical studies and our systematic review have shown the relation between greasy coating and various diseases, including gastroenteropathy, coronary heart disease, and coronavirus disease 2019 (COVID-19). However, the objective and intelligent greasy coating and related diseases recognition methods are still lacking. The construction of the artificial intelligent tongue recognition models may provide important syndrome diagnosis and efficacy evaluation methods, and contribute to the understanding of ethnopharmacological mechanisms based on TCM theory. AIM OF THE STUDY: The present study aimed to develop an artificial intelligent model for greasy tongue coating recognition and explore its application in COVID-19. MATERIALS AND METHODS: Herein, we developed greasy tongue coating recognition networks (GreasyCoatNet) using convolutional neural network technique and a relatively large (N = 1486) set of tongue images from standard devices. Tests were performed using both cross-validation procedures and a new dataset (N = 50) captured by common cameras. Besides, the accuracy and time efficiency comparisons between the GreasyCoatNet and doctors were also conducted. Finally, the model was transferred to recognize the greasy coating level of COVID-19. RESULTS: The overall accuracy in 3-level greasy coating classification with cross-validation was 88.8% and accuracy on new dataset was 82.0%, indicating that GreasyCoatNet can obtain robust greasy coating estimates from diverse datasets. In addition, we conducted user study to confirm that our GreasyCoatNet outperforms TCM practitioners, yet only consuming roughly 1% of doctors' examination time. Critically, we demonstrated that GreasyCoatNet, along with transfer learning, can construct more proper classifier of COVID-19, compared to directly training classifier on patient versus control datasets. We, therefore, derived a disease-specific deep learning network by finetuning the generic GreasyCoatNet. CONCLUSIONS: Our framework may provide an important research paradigm for differentiating tongue characteristics, diagnosing TCM syndrome, tracking disease progression, and evaluating intervention efficacy, exhibiting its unique potential in clinical applications.

Artificial intelligence in tongue diagnosis: Using deep convolutional neural network for recognizing unhealthy tongue with tooth-mark.

Tongue diagnosis plays a pivotal role in traditional Chinese medicine (TCM) for thousands of years. As one of the most important tongue characteristics, tooth-marked tongue is related to spleen deficiency and can greatly contribute to the symptoms differentiation and treatment selection. Yet, the tooth-marked tongue recognition for TCM practitioners is subjective and challenging. Most of the previous studies have concentrated on subjectively selected features of the tooth-marked region and gained accuracy under 80%. In the present study, we proposed an artificial intelligence framework using deep convolutional neural network (CNN) for the recognition of tooth-marked tongue. First, we constructed relatively large datasets with 1548 tongue images captured by different equipments. Then, we used ResNet34 CNN architecture to extract features and perform classifications. The overall accuracy of the models was over 90%. Interestingly, the models can be successfully generalized to images captured by other devices with different illuminations. The good effectiveness and generalization of our framework may provide objective and convenient computer-aided tongue diagnostic method on tracking disease progression and evaluating pharmacological effect from a informatics perspective.

Long-term efficacy of Chinese medicine Bushen Capsule on cognition and brain activity in patients with amnestic mild cognitive impairment.

Mild cognitive impairment (MCI), regarded as the prodromal stage before the clinical phase of Alzheimer's disease (AD), has been considered for early intervention. Unfortunately, many trials in this stage with drugs with single-target turned out to be little or no effect. Multi-targeting in nature based on the theory of Traditional Chinese Medicine (TCM) offers another prospect for intervention. Together with advanced functional magnetic resonance imaging (fMRI) technique for more sensitive and objective evaluation, we investigated the long-term therapeutic effects of a TCM compound on cognition and task-related neuronal activity. Sixty amnestic MCI (aMCI) participants from randomly divided into drug (30 with Bushen capsules (BSC)) and placebo (30 with placebo capsules) groups for this 2-years trial. Neuropsychological and N-back task-fMRI data were acquired at baseline and two follow-ups to assess, via linear mixed effect models, the changes of cognitive ability and brain activation over treatments. The drug group, compared with placebo group, exhibited improvement or stabilization in memory measures over time. Analyses of fMRI revealed that the placebo group exhibited higher activation magnitude and spatial extents at left superior parietal lobule; importantly, the greater activation identified in placebo group was related to more decline in the digit span. BSC showed long-term ameliorative effects on cognitive performances in aMCI patients, which might result from the regulation of abnormal brain activities. Our study provided evidence for the potential of TCM in early prevention of AD, as well as the feasibility of neuroimaging biomarkers in clinical trials.

3D craniofacial registration using thin-plate spline transform and cylindrical surface projection.

Craniofacial registration is used to establish the point-to-point correspondence in a unified coordinate system among human craniofacial models. It is the foundation of craniofacial reconstruction and other craniofacial statistical analysis research. In this paper, a non-rigid 3D craniofacial registration method using thin-plate spline transform and cylindrical surface projection is proposed. First, the gradient descent optimization is utilized to improve a cylindrical surface fitting (CSF) for the reference craniofacial model. Second, the thin-plate spline transform (TPST) is applied to deform a target craniofacial model to the reference model. Finally, the cylindrical surface projection (CSP) is used to derive the point correspondence between the reference and deformed target models. To accelerate the procedure, the iterative closest point ICP algorithm is used to obtain a rough correspondence, which can provide a possible intersection area of the CSP. Finally, the inverse TPST is used to map the obtained corresponding points from the deformed target craniofacial model to the original model, and it can be realized directly by the correspondence between the original target model and the deformed target model. Three types of registration, namely, reflexive, involutive and transitive registration, are carried out to verify the effectiveness of the proposed craniofacial registration algorithm. Comparison with the methods in the literature shows that the proposed method is more accurate.

A PCA-Based method for determining craniofacial relationship and sexual dimorphism of facial shapes.

Previous studies have used principal component analysis (PCA) to investigate the craniofacial relationship, as well as sex determination using facial factors. However, few studies have investigated the extent to which the choice of principal components (PCs) affects the analysis of craniofacial relationship and sexual dimorphism. In this paper, we propose a PCA-based method for visual and quantitative analysis, using 140 samples of 3D heads (70 male and 70 female), produced from computed tomography (CT) images. There are two parts to the method. First, skull and facial landmarks are manually marked to guide the model's registration so that dense corresponding vertices occupy the same relative position in every sample. Statistical shape spaces of the skull and face in dense corresponding vertices are constructed using PCA. Variations in these vertices, captured in every principal component (PC), are visualized to observe shape variability. The correlations of skull- and face-based PC scores are analysed, and linear regression is used to fit the craniofacial relationship. We compute the PC coefficients of a face based on this craniofacial relationship and the PC scores of a skull, and apply the coefficients to estimate a 3D face for the skull. To evaluate the accuracy of the computed craniofacial relationship, the mean and standard deviation of every vertex between the two models are computed, where these models are reconstructed using real PC scores and coefficients. Second, each PC in facial space is analysed for sex determination, for which support vector machines (SVMs) are used. We examined the correlation between PCs and sex, and explored the extent to which the choice of PCs affects the expression of sexual dimorphism. Our results suggest that skull- and face-based PCs can be used to describe the craniofacial relationship and that the accuracy of the method can be improved by using an increased number of face-based PCs. The results show that the accuracy of the sex classification is related to the choice of PCs. The highest sex classification rate is 91.43% using our method.

Craniofacial similarity analysis through sparse principal component analysis.

The computer-aided craniofacial reconstruction (CFR) technique has been widely used in the fields of criminal investigation, archaeology, anthropology and cosmetic surgery. The evaluation of craniofacial reconstruction results is important for improving the effect of craniofacial reconstruction. Here, we used the sparse principal component analysis (SPCA) method to evaluate the similarity between two sets of craniofacial data. Compared with principal component analysis (PCA), SPCA can effectively reduce the dimensionality and simultaneously produce sparse principal components with sparse loadings, thus making it easy to explain the results. The experimental results indicated that the evaluation results of PCA and SPCA are consistent to a large extent. To compare the inconsistent results, we performed a subjective test, which indicated that the result of SPCA is superior to that of PCA. Most importantly, SPCA can not only compare the similarity of two craniofacial datasets but also locate regions of high similarity, which is important for improving the craniofacial reconstruction effect. In addition, the areas or features that are important for craniofacial similarity measurements can be determined from a large amount of data. We conclude that the craniofacial contour is the most important factor in craniofacial similarity evaluation. This conclusion is consistent with the conclusions of psychological experiments on face recognition and our subjective test. The results may provide important guidance for three- or two-dimensional face similarity evaluation, analysis and face recognition.

The production of digital and printed resources from multiple modalities using visualization and three-dimensional printing techniques.

PURPOSE: Virtual digital resources and printed models have become indispensable tools for medical training and surgical planning. Nevertheless, printed models of soft tissue organs are still challenging to reproduce. This study adopts open source packages and a low-cost desktop 3D printer to convert multiple modalities of medical images to digital resources (volume rendering images and digital models) and lifelike printed models, which are useful to enhance our understanding of the geometric structure and complex spatial nature of anatomical organs. MATERIALS AND METHODS: Neuroimaging technologies such as CT, CTA, MRI, and TOF-MRA collect serial medical images. The procedures for producing digital resources can be divided into volume rendering and medical image reconstruction. To verify the accuracy of reconstruction, this study presents qualitative and quantitative assessments. Subsequently, digital models are archived as stereolithography format files and imported to the bundled software of the 3D printer. The printed models are produced using polylactide filament materials. RESULTS: We have successfully converted multiple modalities of medical images to digital resources and printed models for both hard organs (cranial base and tooth) and soft tissue organs (brain, blood vessels of the brain, the heart chambers and vessel lumen, and pituitary tumor). Multiple digital resources and printed models were provided to illustrate the anatomical relationship between organs and complicated surrounding structures. Three-dimensional printing (3DP) is a powerful tool to produce lifelike and tangible models. CONCLUSIONS: We present an available and cost-effective method for producing both digital resources and printed models. The choice of modality in medical images and the processing approach is important when reproducing soft tissue organs models. The accuracy of the printed model is determined by the quality of organ models and 3DP. With the ongoing improvement of printing techniques and the variety of materials available, 3DP will become an indispensable tool in medical training and surgical planning.

Densely calculated facial soft tissue thickness for craniofacial reconstruction in Chinese adults.

Craniofacial reconstruction (CFR) is used to recreate a likeness of original facial appearance for an unidentified skull; this technique has been applied in both forensics and archeology. Many CFR techniques rely on the average facial soft tissue thickness (FSTT) of anatomical landmarks, related to ethnicity, age, sex, body mass index (BMI), etc. Previous studies typically employed FSTT at sparsely distributed anatomical landmarks, where different landmark definitions may affect the contrasting results. In the present study, a total of 90,198 one-to-one correspondence skull vertices are established on 171 head CT-scans and the FSTT of each corresponding vertex is calculated (hereafter referred to as densely calculated FSTT) for statistical analysis and CFR. Basic descriptive statistics (i.e., mean and standard deviation) for densely calculated FSTT are reported separately according to sex and age. Results show that 76.12% of overall vertices indicate that the FSTT is greater in males than females, with the exception of vertices around the zygoma, zygomatic arch and mid-lateral orbit. These sex-related significant differences are found at 55.12% of all vertices and the statistically age-related significant differences are depicted between the three age groups at a majority of all vertices (73.31% for males and 63.43% for females). Five non-overlapping categories are given and the descriptive statistics (i.e., mean, standard deviation, local standard deviation and percentage) are reported. Multiple appearances are produced using the densely calculated FSTT of various age and sex groups, and a quantitative assessment is provided to examine how relevant the choice of FSTT is to increasing the accuracy of CFR. In conclusion, this study provides a new perspective in understanding the distribution of FSTT and the construction of a new densely calculated FSTT database for craniofacial reconstruction.

A regional method for craniofacial reconstruction based on coordinate adjustments and a new fusion strategy.

Craniofacial reconstruction recreates a facial outlook from the cranium based on the relationship between the face and the skull to assist identification. But craniofacial structures are very complex, and this relationship is not the same in different craniofacial regions. Several regional methods have recently been proposed, these methods segmented the face and skull into regions, and the relationship of each region is then learned independently, after that, facial regions for a given skull are estimated and finally glued together to generate a face. Most of these regional methods use vertex coordinates to represent the regions, and they define a uniform coordinate system for all of the regions. Consequently, the inconsistence in the positions of regions between different individuals is not eliminated before learning the relationships between the face and skull regions, and this reduces the accuracy of the craniofacial reconstruction. In order to solve this problem, an improved regional method is proposed in this paper involving two types of coordinate adjustments. One is the global coordinate adjustment performed on the skulls and faces with the purpose to eliminate the inconsistence of position and pose of the heads; the other is the local coordinate adjustment performed on the skull and face regions with the purpose to eliminate the inconsistence of position of these regions. After these two coordinate adjustments, partial least squares regression (PLSR) is used to estimate the relationship between the face region and the skull region. In order to obtain a more accurate reconstruction, a new fusion strategy is also proposed in the paper to maintain the reconstructed feature regions when gluing the facial regions together. This is based on the observation that the feature regions usually have less reconstruction errors compared to rest of the face. The results demonstrate that the coordinate adjustments and the new fusion strategy can significantly improve the craniofacial reconstructions.

A network-based approach to investigate the pattern of syndrome in depression.

In Traditional Chinese Medicine theory, syndrome is essential to diagnose diseases and treat patients, and symptom is the foundation of syndrome differentiation. Thus the combination and interaction between symptoms represent the pattern of syndrome at phenotypic level, which can be modeled and analyzed using complex network. At first, we collected inquiry information of 364 depression patients from 2007 to 2009. Next, we learned classification models for 7 syndromes in depression using naïve Bayes, Bayes network, support vector machine (SVM), and C4.5. Among them, SVM achieves the highest accuracies larger than 0.9 except for Yin deficiency. Besides, Bayes network outperforms naïve Bayes for all 7 syndromes. Then key symptoms for each syndrome were selected using Fisher's score. Based on these key symptoms, symptom networks for 7 syndromes as well as a global network for depression were constructed through weighted mutual information. Finally, we employed permutation test to discover dynamic symptom interactions, in order to investigate the difference between syndromes from the perspective of symptom network. As a result, significant dynamic interactions were quite different for 7 syndromes. Therefore, symptom networks could facilitate our understanding of the pattern of syndrome and further the improvement of syndrome differentiation in depression.

A novel skull registration based on global and local deformations for craniofacial reconstruction.

Craniofacial reconstruction is important in forensic identification. It aims to estimate a facial appearance for human skeletal remains using the relationship between the soft tissue and the underlying bone structure. Various computerized methods have been developed in recent decades. An effective way is to deform a reference skull to the discovered skull, and then apply the same deformation to the skin associated with the reference skull to provide an approximate face for the discovered skull. For this method, the better the two skulls match each other, the more face-like the reconstructed skin surface will be. In this paper, we present a novel skull registration method that can match the two skulls closely, so as to improve the accuracy of the reconstruction. It combines both global and local deformations. A generic thin-plate spline (TPS)-based deformation, which is global, is applied first to roughly align the two skulls based on two groups of manually defined landmarks. Afterwards, the two skulls are largely matched, except some regions, on which some new landmarks are automatically marked. A compact support radial basis functions (CSRBF)-based deformation, which is local, will then be performed on these regions to adjust the initial alignment of the two skulls. Such adjustment can be repeatedly implemented until the two skulls have optimal alignment. In addition, all the skulls and face involved in the registration are represented by their single outer surfaces to facilitate the reconstruction procedure. The experiments demonstrate that our method can create a plausible face even when the reference skull is very different from the discovered skull. As a result, we can make full use of our database to provide multiple estimates for a principle components analysis (PCA) for the final reconstruction.