[Automated cephalometric landmark identification and location based on convolutional neural network].

Objective: To develop an automated landmark location system applicable to the case of landmark missing. Methods: Four and eighty-one lateral cephalograms, which contained 240 males and 241 females, with an average age of (24.5±5.6) years, taken from January 2015 to January 2021 in the Department of Orthodontics, Capital Medical University School of Stomatology, and met the inclusion criteria were collected. Five postgraduate orthodontic students were the annotators to manually locate 61 possible landmarks in 481 lateral cephalograms. Two assistant professors in the department as reviewers performed calibration. Two professors as arbitrators, made final decision. Data sets were established (341 were used as training set, 40 as validation set, and 100 as test set). In this paper, an automatic landmarks identification and location model based on convolutional neural networks (CNN), CephaNET, was developed. The model was trained by feeding the original image into the feature extraction module and convolutional pose machine (CPM) module to locate landmarks with high accuracy using deep supervision. Training set was enhanced to 1 684 images by histogram equalization, cropping, and adjustment of brightness. The model was trained to compare the Gaussian heat maps output from the network with the set threshold to identify landmark missing cases. Test set of 100 lateral cephalograms was used to test the accuracy of the model. The evaluation criteria used were success detection rate of missing landmark, mean radial error (MRE) and success detection rate (SDR) in the range of 2.0, 2.5, 3.0, 3.5 and 4.0 mm. Results: The model identified and located 61 commonly used landmarks in 0.13 seconds on average. It had an average accuracy of 93.5% in identifying missing landmarks. The MRE of our testing set was (1.19±0.91) mm. SDR of 2.0, 2.5, 3.0, 3.5 and 4.0 mm were 85.4%, 90.2%, 93.5%, 95.4%, 97.0% respectively. Conclusions: The model proposed in this paper could adapt to the absence of landmark in lateral cephalograms and locate 61 commonly used landmarks with high accuracy to meet the requirements of different cephalometric analysis methods.

[Automated diagnostic classification with lateral cephalograms based on deep learning network model].

Objective: To establish a comprehensive diagnostic classification model of lateral cephalograms based on artificial intelligence (AI) to provide reference for orthodontic diagnosis. Methods: A total of 2 894 lateral cephalograms were collected in Department of Orthodontics, Capital Medical University School of Stomatology from January 2015 to December 2021 to construct a data set, including 1 351 males and 1 543 females with a mean age of (26.4± 7.4) years. Firstly, 2 orthodontists (with 5 and 8 years of orthodontic experience, respectively) performed manual annotation and calculated measurement for primary classification, and then 2 senior orthodontists (with more than 20 years of orthodontic experience) verified the 8 diagnostic classifications including skeletal and dental indices. The data were randomly divided into training, validation, and test sets in the ratio of 7∶2∶1. The open source DenseNet121 was used to construct the model. The performance of the model was evaluated by classification accuracy, precision rate, sensitivity, specificity and area under the curve (AUC). Visualization of model regions of interest through class activation heatmaps. Results: The automatic classification model of lateral cephalograms was successfully established. It took 0.012 s on average to make 8 diagnoses on a lateral cephalogram. The accuracy of 5 classifications was 80%-90%, including sagittal and vertical skeletal facial pattern, mandibular growth, inclination of upper incisors, and protrusion of lower incisors. The acuracy rate of 3 classifications was 70%-80%, including maxillary growth, inclination of lower incisors and protrusion of upper incisors. The average AUC of each classification was ≥0.90. The class activation heat map of successfully classified lateral cephalograms showed that the AI model activation regions were distributed in the relevant structural regions. Conclusions: In this study, an automatic classification model for lateral cephalograms was established based on the DenseNet121 to achieve rapid classification of eight commonly used clinical diagnostic items.

[Research on multi-class orthodontic image recognition system based on deep learning network model].

Objective: To develop a multi-classification orthodontic image recognition system using the SqueezeNet deep learning model for automatic classification of orthodontic image data. Methods: A total of 35 000 clinical orthodontic images were collected in the Department of Orthodontics, Capital Medical University School of Stomatology, from October to November 2020 and June to July 2021. The images were from 490 orthodontic patients with a male-to-female ratio of 49∶51 and the age range of 4 to 45 years. After data cleaning based on inclusion and exclusion criteria, the final image dataset included 17 453 face images (frontal, smiling, 90° right, 90° left, 45° right, and 45° left), 8 026 intraoral images [frontal occlusion, right occlusion, left occlusion, upper occlusal view (original and flipped), lower occlusal view (original and flipped) and coverage of occlusal relationship], 4 115 X-ray images [lateral skull X-ray from the left side, lateral skull X-ray from the right side, frontal skull X-ray, cone-beam CT (CBCT), and wrist bone X-ray] and 684 other non-orthodontic images. A labeling team composed of orthodontic doctoral students, associate professors, and professors used image labeling tools to classify the orthodontic images into 20 categories, including 6 face image categories, 8 intraoral image categories, 5 X-ray image categories, and other images. The data for each label were randomly divided into training, validation, and testing sets in an 8∶1∶1 ratio using the random function in the Python programming language. The improved SqueezeNet deep learning model was used for training, and 13 000 natural images from the ImageNet open-source dataset were used as additional non-orthodontic images for algorithm optimization of anomaly data processing. A multi-classification orthodontic image recognition system based on deep learning models was constructed. The accuracy of the orthodontic image classification was evaluated using precision, recall, F1 score, and confusion matrix based on the prediction results of the test set. The reliability of the model's image classification judgment logic was verified using the gradient-weighted class activation mapping (Grad-CAM) method to generate heat maps. Results: After data cleaning and labeling, a total of 30 278 orthodontic images were included in the dataset. The test set classification results showed that the precision, recall, and F1 scores of most classification labels were 100%, with only 5 misclassified images out of 3 047, resulting in a system accuracy of 99.84%(3 042/3 047). The precision of anomaly data processing was 100% (10 500/10 500). The heat map showed that the judgment basis of the SqueezeNet deep learning model in the image classification process was basically consistent with that of humans. Conclusions: This study developed a multi-classification orthodontic image recognition system for automatic classification of 20 types of orthodontic images based on the improved SqueezeNet deep learning model. The system exhibitted good accuracy in orthodontic image classification.