Network analysis of three-dimensional hard-soft tissue relationships in the lower 1/3 of the face: skeletal Class I-normodivergent malocclusion versus Class II-hyperdivergent malocclusion.

BACKGROUND: The determining effect of facial hard tissues on soft tissue morphology in orthodontic patients has yet to be explained. The aim of this study was to clarify the hard-soft tissue relationships of the lower 1/3 of the face in skeletal Class II-hyperdivergent patients compared with those in Class I-normodivergent patients using network analysis. METHODS: Fifty-two adult patients (42 females, 10 males; age, 26.58 ± 5.80 years) were divided into two groups: Group 1, 25 subjects, skeletal Class I normodivergent pattern with straight profile; Group 2, 27 subjects, skeletal Class II hyperdivergent pattern with convex profile. Pretreatment cone-beam computed tomography and three-dimensional facial scans were taken and superimposed, on which landmarks were identified manually, and their coordinate values were used for network analysis. RESULTS: (1) In sagittal direction, Group 2 correlations were generally weaker than Group 1. In both the vertical and sagittal directions of Group 1, the most influential hard tissue landmarks to soft tissues were located between the level of cemento-enamel junction of upper teeth and root apex of lower teeth. In Group 2, the hard tissue landmarks with the greatest influence in vertical direction were distributed more forward and downward than in Group 1. (2) In Group 1, all the correlations for vertical-hard tissue to sagittal-soft tissue position and sagittal-hard tissue to vertical-soft tissue position were positive. However, Group 2 correlations between vertical-hard tissue and sagittal-soft tissue positions were mostly negative. Between sagittal-hard tissue and vertical-soft tissue positions, Group 2 correlations were negative for mandible, and were positive for maxilla and teeth. CONCLUSION: Compared with Class I normodivergent patients with straight profile, Class II hyperdivergent patients with convex profile had more variations in soft tissue morphology in sagittal direction. In vertical direction, the most relevant hard tissue landmarks on which soft tissue predictions should be based were distributed more forward and downward in Class II hyperdivergent patients with convex profile. Class II hyperdivergent pattern with convex profile was an imbalanced phenotype concerning sagittal and vertical positions of maxillofacial hard and soft tissues.

Natural reference structures for three-dimensional maxillary regional superimposition in growing patients.

BACKGROUND: Assessment of growth-related or treatment-related changes in the maxilla requires a reliable method of superimposition. Such methods are well established for two-dimensional (2D) cephalometric images but not yet for three-dimensions (3D). The aims of this study were to identify natural reference structures (NRS) for the maxilla in growing patients in 3D, opportunistically using orthodontic mini-screws as reference; and to test the applicability of the proposed NRS for maxillary superimposition by assessing the concordance of this approach with Björk's 'stable reference structures' in lateral projection. METHODS: The stability of the mini-screws was tested on longitudinal pairs of pre- and post-orthodontic cone-beam computed tomography (CBCT) images by measuring the distance changes between screws. After verifying the stability of the mini-screws, rigid registration was performed for aligning the stable mini-screws. Then, non-rigid registration was used to establish the dense voxel-correspondence among CBCT images and calculate the displacement of each voxel belonging to the maxilla relative to the mini-screws. The displacement vectors were transformed to a standardized maxillary template to categorize the stability of the internal structures statistically. Those voxels that displaced less relative to the mini-screws were considered as the natural reference structures (NRS) for the maxilla. Test samples included another dataset of longitudinal CBCT scans. They were used to evaluate the applicability of the proposed NRS for maxillary superimposition. We assessed whether aligning the maxilla with proposed NRS is in concordance with the maxillary internal reference structures superimposition in the traditional 2D lateral view as suggested by Björk. This was quantitively assessed by comparing the mean sagittal and vertical tooth movements for both superimposition methods. RESULTS: The stability of the mini-screws was tested on 10 pairs of pre- and post-orthodontic cone-beam computed tomography (CBCT) images (T1: 12.9 ± 0.8 yrs, T2: 14.8 ± 0.7 yrs). Both the loaded and the unloaded mini-screws were shown to be stable during orthodontic treatment, which indicates that they can be used as reference points. By analyzing the deformation map of the maxilla, we confirmed that the infraorbital rims, maxilla around the piriform foramen, the infrazygomatic crest and the hard palate (palatal vault more than  1 cm distal to incisor foramen except the palatal suture) were stable during growth. Another dataset of longitudinal CBCT scans (T1: 12.2 ± 0.63 yrs, T2: 15.2 ± 0.96 yrs) was used to assess the concordance of this approach with Björk's 'stable reference structures'. The movement of the maxillary first molar and central incisor showed no statistically significant difference when superimposing the test images with the proposed NRS or with the classic Björk maxillary superimposition in the lateral view. CONCLUSIONS: The infraorbital rims, maxilla around the piriform foramen, the infrazygomatic crest and the hard palate (palatal vault more than 1 cm posterior to incisal foramen except the palatal suture) were identified as stable regions in the maxilla. These stable structures can be used for maxillary superimposition in 3D and generate comparable results to Björk superimposition in the lateral view.

Automated assessment of mandibular shape asymmetry in 3-dimensions.

INTRODUCTION: This study aimed to develop an automatic pipeline for analyzing mandibular shape asymmetry in 3-dimensions. METHODS: Forty patients with skeletal Class I pattern and 80 patients with skeletal Class III pattern were used. The mandible was automatically segmented from the cone-beam computed tomography images using a U-net deep learning network. A total of 17,415 uniformly sampled quasi-landmarks were automatically identified on the mandibular surface via a template mapping technique. After alignment with the robust Procrustes superimposition, the pointwise surface-to-surface distance between original and reflected mandibles was visualized in a color-coded map, indicating the location of asymmetry. The degree of overall mandibular asymmetry and the asymmetry of subskeletal units were scored using the root-mean-squared-error between the left and right sides. These asymmetry parameters were compared between the skeletal Class I and skeletal Class III groups. RESULTS: The mandible shape was significantly more asymmetrical in patients with skeletal Class III pattern with positional asymmetry. The condyles were identified as the most asymmetric region in all groups, followed by the coronoid process and the ramus. CONCLUSIONS: This automated approach to quantify mandibular shape asymmetry will facilitate high-throughput image processing for big data analysis. The spatially-dense landmarks allow for evaluating mandibular asymmetry over the entire surface, which overcomes the information loss inherent in conventional linear distance or angular measurements. Precise quantification of the asymmetry can provide important information for individualized diagnosis and treatment planning in orthodontics and orthognathic surgery.

Thickness change of masseter muscles and the surrounding soft tissues in female patients during orthodontic treatment: a retrospective study.

BACKGROUND: Facial esthetics is a major concern of orthodontic patients. This study aims to evaluate orthodontic treatment-related thickness changes of the masseter muscles and surrounding soft tissues and the potential factors that would influence these changes during orthodontic treatment in female adults. METHODS: Forty-two female adult patients were included in this retrospective study and were divided into extraction (n = 22) and nonextraction (n = 20) groups. Pretreatment and posttreatment cone-beam computed tomography (CBCT) images were superimposed and reconstructed. The thickness changes of the masseter area of facial soft tissue (MAS), masseter muscles (MM) and surrounding fat tissue (FT) were measured. Pretreatment age, treatment duration, sagittal relationship (ANB), and vertical relationship (Frankfort-mandibular plane angle, FMA)-related MAS, MM and FT changes were compared between extraction and nonextraction groups. Spearman's correlation coefficient was calculated between the above variables. Regression analysis was conducted to confirm the causal relations of the variables. RESULTS: The thickness of MAS and MM significantly decreased in both groups, with larger decreases (> 1 mm) in the extraction group. There were strong correlations (r > 0.7) between the thickness decrease in MAS and MM in both groups and moderate correlations (r > 0.4) between MAS and FT in the nonextraction group. A significantly greater decrease of MAS and MM were found to be moderately correlated with a smaller FMA (r > 0.4) in the extraction group. Scatter plots and regression analysis confirmed these correlations. CONCLUSIONS: Masseter muscles and the surrounding soft tissue exhibited a significant decrease in thickness during orthodontic treatment in female adults. Low-angle patients experienced a greater decrease in soft tissue thickness in the masseter area in the extraction case. But the thickness changes were clinically very small in most patients.

Adaptable cascaded registration for personalized maxilla completion and cleft defect volume estimation.

BACKGROUND: Cone-beam computed tomography (CBCT) images provide high-resolution insights into the underlying craniofacial anomaly in patients with cleft lip and palate (CLP), requiring non-negligible annotation costs to measure the cleft defect for the guidance of the clinical secondary alveolar bone graft procedures. Considering the cumbersome volumetric image acquisition, there is a lack of paired CLP CBCTs and normal CBCTs for learning-based anatomical structure restoration models. Nowadays, the registration-based method relieves the annotation burden, though one-shot registration and the regular mask are limited to handling fine-grained shape variations and harmony between restored bony tissues and the defected maxilla. PURPOSE: This study aimed to design and evaluate a novel method for deformable partial registration of the CLP CBCTs and normal CBCTs, enabling personalized maxilla completion and cleft defect volume prediction from CLP CBCTs. METHODS: We proposed an adaptable deep registration framework for personalized maxilla completion and cleft defect volume prediction from CLP CBCTs. The key ingredient was a cascaded partial registration to exploit the maxillary morphology prior and attribute transfer. Cascaded registration with coarse-to-fine registration fields handled morphological variations of cleft defects and fine-grained maxillary restoration. We designed an adaptable cleft defect mask and volumetric Boolean operators for reliable voxel filling of the defected maxilla. A total of 36 clinically obtained CLP CBCTs were used to train and validate the proposed model, among which 22 CLP CBCTs were used to generate a training dataset with 440 synthetic CBCTs by B-spline deformation-based data augmentation and the remaining for testing. The proposed model was evaluated on maxilla completion and cleft defect volume prediction from clinically obtained unilateral and bilateral CLP CBCTs. RESULTS: Extensive experiments demonstrated the effectiveness of the adaptable cleft defect mask and the cascaded partial registration on maxilla completion and cleft defect volume prediction. The proposed method achieved state-of-the-art performances with the Dice similarity coefficient of 0.90 ± $\pm$ 0.02 on the restored maxilla and 0.84 ± $\pm$ 0.04 on the estimated cleft defect, respectively. The average Hausdorff distance between the estimated cleft defect and the manually annotated ground truth was 0.30 ± $\pm$ 0.08 mm. The relative volume error of the cleft defect was 0.09 ± $0.09\pm$ 0.08. The proposed model allowed for the prediction of cleft defect maps that were in line with the ground truth in the challenging unilateral and bilateral CLP CBCTs. CONCLUSIONS: The results suggest that the proposed adaptable deep registration model enables patient-specific maxilla completion and automatic annotation of cleft defects, relieving tedious voxel-wise annotation and image acquisition burdens.

Deformable registration of lateral cephalogram and cone-beam computed tomography image.

PURPOSE: This study aimed to design and evaluate a novel method for the registration of 2D lateral cephalograms and 3D craniofacial cone-beam computed tomography (CBCT) images, providing patient-specific 3D structures from a 2D lateral cephalogram without additional radiation exposure. METHODS: We developed a cross-modal deformable registration model based on a deep convolutional neural network. Our approach took advantage of a low-dimensional deformation field encoding and an iterative feedback scheme to infer coarse-to-fine volumetric deformations. In particular, we constructed a statistical subspace of deformation fields and parameterized the nonlinear mapping function from an image pair, consisting of the target 2D lateral cephalogram and the reference volumetric CBCT, to a latent encoding of the deformation field. Instead of the one-shot registration by the learned mapping function, a feedback scheme was introduced to progressively update the reference volumetric image and to infer coarse-to-fine deformations fields, accounting for the shape variations of anatomical structures. A total of 220 clinically obtained CBCTs were used to train and validate the proposed model, among which 120 CBCTs were used to generate a training dataset with 24k paired synthetic lateral cephalograms and CBCTs. The proposed approach was evaluated on the deformable 2D-3D registration of clinically obtained lateral cephalograms and CBCTs from growing and adult orthodontic patients. RESULTS: Strong structural consistencies were observed between the deformed CBCT and the target lateral cephalogram in all criteria. The proposed method achieved state-of-the-art performances with the mean contour deviation of 0.41 ± 0.12 mm on the anterior cranial base, 0.48 ± 0.17 mm on the mandible, and 0.35 ± 0.08 mm on the maxilla, respectively. The mean surface mesh ranged from 0.78 to 0.97 mm on various craniofacial structures, and the LREs ranged from 0.83 to 1.24 mm on the growing datasets regarding 14 landmarks. The proposed iterative feedback scheme handled the structural details and improved the registration. The resultant deformed volumetric image was consistent with the target lateral cephalogram in both 2D projective planes and 3D volumetric space regarding the multicategory craniofacial structures. CONCLUSIONS: The results suggest that the deep learning-based 2D-3D registration model enables the deformable alignment of 2D lateral cephalograms and CBCTs and estimates patient-specific 3D craniofacial structures.

Spatially Consistent Supervoxel Correspondences of Cone-Beam Computed Tomography Images.

Establishing dense correspondences of cone-beam computed tomography (CBCT) images is a crucial step for the attribute transfer and morphological variation assessment in clinical orthodontics. In this paper, a novel method, unsupervised spatially consistent clustering forest, is proposed to tackle the challenges for automatic supervoxel-wise correspondences of CBCT images. A complexity analysis of the proposed method with respect to the clustering hypotheses is provided with a data-dependent learning guarantee. The learning bound considers both the sequential tree traversals determined by questions stored in branch nodes and the clustering compactness of leaf nodes. A novel tree-pruning algorithm, guided by the learning bound, is also proposed to remove locally inconsistent leaf nodes. The resulting forest yields spatially consistent affinity estimations, thanks to the pruning penalizing trees with inconsistent leaf assignments and the combinational contextual feature channels used to learn the forest. A forest-based metric is utilized to derive the pairwise affinities and dense correspondences of CBCT images. The proposed method has been applied to the label propagation of clinically captured CBCT images. In the experiments, the method outperforms variants of both supervised and unsupervised forest-based methods and state-of-the-art label-propagation methods, achieving the mean dice similarity coefficients of 0.92, 0.89, 0.94, and 0.93 for the mandible, the maxilla, the zygoma arch, and the teeth data, respectively.

Transferring of speech movements from video to 3D face space.

We present a novel method for transferring speech animation recorded in low quality videos to high resolution 3D face models. The basic idea is to synthesize the animated faces by an interpolation based on a small set of 3D key face shapes which span a 3D face space. The 3D key shapes are extracted by an unsupervised learning process in 2D video space to form a set of 2D visemes which are then mapped to the 3D face space. The learning process consists of two main phases: 1) Isomap-based nonlinear dimensionality reduction to embed the video speech movements into a low-dimensional manifold and 2) K-means clustering in the low-dimensional space to extract 2D key viseme frames. Our main contribution is that we use the Isomap-based learning method to extract intrinsic geometry of the speech video space and thus to make it possible to define the 3D key viseme shapes. To do so, we need only to capture a limited number of 3D key face models by using a general 3D scanner. Moreover, we also develop a skull movement recovery method based on simple anatomical structures to enhance 3D realism in local mouth movements. Experimental results show that our method can achieve realistic 3D animation effects with a small number of 3D key face models.

Superimposition of Cone-Beam Computed Tomography Images by Joint Embedding.

OBJECTIVE: The superimposition of cone-beam computed tomography (CBCT) images is an essential step to evaluate shape variations of pre and postorthodontic operations due to pose variations and the bony growth. The aim of this paper is to present and discuss the latest accomplishments in voxel-based craniofacial CBCT superimpositions along with structure discriminations. METHODS: We propose a CBCT superimposition method based on joint embedding of subsets extracted from CBCT images. The subset is defined at local extremes of the first-order difference of Gaussian-smoothed volume images to reduce the data involved in the computation. A rotation-invariant integral operator is proposed as the context-aware textural descriptor of subsets. We cope with subset correspondences by joint embedding with matching identifications in manifolds, which take into account the structure of subsets as a whole to avoid mapping ambiguities. Once given subset correspondences, the rigid transformations, as well as the superimposition of volume images, are obtained. Our system allows users to specify the structure-of-interest based on a semisupervised label propagation technique. RESULTS: The performance of the proposed method is evaluated on ten pairs of pre and postoperative CBCT images of adult patients and ten pairs of growing patients, respectively. The experiments demonstrate that the craniofacial CBCT superimposition can be performed effectively, and outperform state of the arts. CONCLUSION: The integration of sparse subsets with context-aware spherical intensity integral descriptors and correspondence establishment by joint embedding enables the reliable and efficient CBCT superimposition. SIGNIFICANCE: The potential of CBCT superimposition techniques discussed in this paper is highlighted and related challenges are addressed.

3D exemplar-based random walks for tooth segmentation from cone-beam computed tomography images.

PURPOSE: Tooth segmentation is an essential step in acquiring patient-specific dental geometries from cone-beam computed tomography (CBCT) images. Tooth segmentation from CBCT images is still a challenging task considering the comparatively low image quality caused by the limited radiation dose, as well as structural ambiguities from intercuspation and nearby alveolar bones. The goal of this paper is to present and discuss the latest accomplishments in semisupervised tooth segmentation with adaptive 3D shape constraints. METHODS: The authors propose a 3D exemplar-based random walk method of tooth segmentation from CBCT images. The proposed method integrates semisupervised label propagation and regularization by 3D exemplar registration. To begin with, the pure random walk method is to get an initial segmentation of the teeth, which tends to be erroneous because of the structural ambiguity of CBCT images. And then, as an iterative refinement, the authors conduct a regularization by using 3D exemplar registration, as well as label propagation by random walks with soft constraints, to improve the tooth segmentation. In the first stage of the iteration, 3D exemplars with well-defined topologies are adapted to fit the tooth contours, which are obtained from the random walks based segmentation. The soft constraints on voxel labeling are defined by shape-based foreground dentine probability acquired by the exemplar registration, as well as the appearance-based probability from a support vector machine (SVM) classifier. In the second stage, the labels of the volume-of-interest (VOI) are updated by the random walks with soft constraints. The two stages are optimized iteratively. Instead of the one-shot label propagation in the VOI, an iterative refinement process can achieve a reliable tooth segmentation by virtue of exemplar-based random walks with adaptive soft constraints. RESULTS: The proposed method was applied for tooth segmentation of twenty clinically captured CBCT images. Three metrics, including the Dice similarity coefficient (DSC), the Jaccard similarity coefficient (JSC), and the mean surface deviation (MSD), were used to quantitatively analyze the segmentation of anterior teeth including incisors and canines, premolars, and molars. The segmentation of the anterior teeth achieved a DSC up to 98%, a JSC of 97%, and an MSD of 0.11 mm compared with manual segmentation. For the premolars, the average values of DSC, JSC, and MSD were 98%, 96%, and 0.12 mm, respectively. The proposed method yielded a DSC of 95%, a JSC of 89%, and an MSD of 0.26 mm for molars. Aside from the interactive definition of label priors by the user, automatic tooth segmentation can be achieved in an average of 1.18 min. CONCLUSIONS: The proposed technique enables an efficient and reliable tooth segmentation from CBCT images. This study makes it clinically practical to segment teeth from CBCT images, thus facilitating pre- and interoperative uses of dental morphologies in maxillofacial and orthodontic treatments.

Evaluation of facial attractiveness for patients with malocclusion: a machine-learning technique employing Procrustes.

OBJECTIVE: To establish an objective method for evaluating facial attractiveness from a set of orthodontic photographs. MATERIALS AND METHODS: One hundred eight malocclusion patients randomly selected from six universities in China were randomly divided into nine groups, with each group containing an equal number of patients with Class I, II, and III malocclusions. Sixty-nine expert Chinese orthodontists ranked photographs of the patients (frontal, lateral, and frontal smiling photos) before and after orthodontic treatment from "most attractive" to "least attractive" in each group. A weighted mean ranking was then calculated for each patient, based on which a three-point scale was created. Procrustes superimposition was conducted on 101 landmarks identified on the photographs. A support vector regression (SVR) function was set up according to the coordinate values of identified landmarks of each photographic set and its corresponding grading. Its predictive ability was tested for each group in turn. RESULTS: The average coincidence rate obtained for comparisons of the subjective ratings with the SVR evaluation was 71.8% according to 18 verification tests. CONCLUSIONS: Geometric morphometrics combined with SVR may be a prospective method for objective comprehensive evaluation of facial attractiveness in the near future.

Personalized tooth shape estimation from radiograph and cast.

Three-dimensional geometric information of teeth is usually needed in pre- and postoperative diagnoses of orthodontic dentistry. The computerized tomography can provide comprehensive 3-D teeth geometries. However, there is still a discussion on computed tomography (CT) as a routine in orthodontic dentistry due to radiation dose. Moreover, the CT is useless when a dentist needs to extract 3-D structures from old archive files with only radiographs and casts, where patient's teeth changed ever since. In this paper, we propose a reconstruction framework for patient-specific teeth based on an integration of 2-D radiographs and digitized casts. The reconstruction is under a template-fitting framework. The shape and orientation of teeth templates are tuned in accordance with patient's radiographs. Specially, the tooth root morphology is controlled by 2-D contours in radiographs. With ray tracing and a contour plane assumption, 2-D root contours in radiographs are projected back to 3-D space, and guide tooth root deformations. Moreover, the template's crown is deformed nonrigidly to fit digitized casts that bear patient's crown details. The system allows 3-D tooth reconstruction with patient-specific geometric details from just casts and 2-D radiographs.