Correspondence attention for facial appearance simulation.

In orthognathic surgical planning for patients with jaw deformities, it is crucial to accurately simulate the changes in facial appearance that follow the bony movement. Compared with the traditional biomechanics-based methods like the finite-element method (FEM), which are both labor-intensive and computationally inefficient, deep learning-based methods offer an efficient and robust modeling alternative. However, current methods do not account for the physical relationship between facial soft tissue and bony structure, causing them to fall short in accuracy compared to FEM. In this work, we propose an Attentive Correspondence assisted Movement Transformation network (ACMT-Net) to predict facial changes by correlating facial soft tissue changes with bony movement through a point-to-point attentive correspondence matrix. To ensure efficient training, we also introduce a contrastive loss for self-supervised pre-training of the ACMT-Net with a k-Nearest Neighbors (k-NN) based clustering. Experimental results on patients with jaw deformities show that our proposed solution can achieve significantly improved computational efficiency over the state-of-the-art FEM-based method with comparable facial change prediction accuracy.

Machine Learning Effectively Diagnoses Mandibular Deformity Using Three-Dimensional Landmarks.

BACKGROUND: Jaw deformity diagnosis requires objective tests. Current methods, like cephalometry, have limitations. However, recent studies have shown that machine learning can diagnose jaw deformities in two dimensions. Therefore, we hypothesized that a multilayer perceptron (MLP) could accurately diagnose jaw deformities in three dimensions (3D). PURPOSE: Examine the hypothesis by focusing on anomalous mandibular position. We aimed to: (1) create a machine learning model to diagnose mandibular retrognathism and prognathism; and (2) compare its performance with traditional cephalometric methods. STUDY DESIGN, SETTING, SAMPLE: An in-silico experiment on deidentified retrospective data. The study was conducted at the Houston Methodist Research Institute and Rensselaer Polytechnic Institute. Included were patient records with jaw deformities and preoperative 3D facial models. Patients with significant jaw asymmetry were excluded. PREDICTOR VARIABLES: The tests used to diagnose mandibular anteroposterior position are: (1) SNB angle; (2) facial angle; (3) mandibular unit length (MdUL); and (4) MLP model. MAIN OUTCOME VARIABLE: The resultant diagnoses: normal, prognathic, or retrognathic. COVARIATES: None. ANALYSES: A senior surgeon labeled the patients' mandibles as prognathic, normal, or retrognathic, creating a gold standard. Scientists at Rensselaer Polytechnic Institute developed an MLP model to diagnose mandibular prognathism and retrognathism using the 3D coordinates of 50 landmarks. The performance of the MLP model was compared with three traditional cephalometric measurements: (1) SNB, (2) facial angle, and (3) MdUL. The primary metric used to assess the performance was diagnostic accuracy. McNemar's exact test tested the difference between traditional cephalometric measurement and MLP. Cohen's Kappa measured inter-rater agreement between each method and the gold standard. RESULTS: The sample included 101 patients. The diagnostic accuracy of SNB, facial angle, MdUL, and MLP were 74.3, 74.3, 75.3, and 85.2%, respectively. McNemar's test shows that our MLP performs significantly better than the SNB (P = .027), facial angle (P = .019), and MdUL (P = .031). The agreement between the traditional cephalometric measurements and the surgeon's diagnosis was fair. In contrast, the agreement between the MLP and the surgeon was moderate. CONCLUSION AND RELEVANCE: The performance of the MLP is significantly better than that of the traditional cephalometric measurements.

Simulation of Postoperative Facial Appearances via Geometric Deep Learning for Efficient Orthognathic Surgical Planning.

Orthognathic surgery corrects jaw deformities to improve aesthetics and functions. Due to the complexity of the craniomaxillofacial (CMF) anatomy, orthognathic surgery requires precise surgical planning, which involves predicting postoperative changes in facial appearance. To this end, most conventional methods involve simulation with biomechanical modeling methods, which are labor intensive and computationally expensive. Here we introduce a learning-based framework to speed up the simulation of postoperative facial appearances. Specifically, we introduce a facial shape change prediction network (FSC-Net) to learn the nonlinear mapping from bony shape changes to facial shape changes. FSC-Net is a point transform network weakly-supervised by paired preoperative and postoperative data without point-wise correspondence. In FSC-Net, a distance-guided shape loss places more emphasis on the jaw region. A local point constraint loss restricts point displacements to preserve the topology and smoothness of the surface mesh after point transformation. Evaluation results indicate that FSC-Net achieves 15× speedup with accuracy comparable to a state-of-the-art (SOTA) finite-element modeling (FEM) method.

Bidirectional prediction of facial and bony shapes for orthognathic surgical planning.

This paper proposes a deep learning framework to encode subject-specific transformations between facial and bony shapes for orthognathic surgical planning. Our framework involves a bidirectional point-to-point convolutional network (P2P-Conv) to predict the transformations between facial and bony shapes. P2P-Conv is an extension of the state-of-the-art P2P-Net and leverages dynamic point-wise convolution (i.e., PointConv) to capture local-to-global spatial information. Data augmentation is carried out in the training of P2P-Conv with multiple point subsets from the facial and bony shapes. During inference, network outputs generated for multiple point subsets are combined into a dense transformation. Finally, non-rigid registration using the coherent point drift (CPD) algorithm is applied to generate surface meshes based on the predicted point sets. Experimental results on real-subject data demonstrate that our method substantially improves the prediction of facial and bony shapes over state-of-the-art methods.

Predicting Optimal Patient-Specific Postoperative Facial Landmarks for Patients with Craniomaxillofacial Deformities.

Orthognathic surgery traditionally focuses on correcting skeletal abnormalities and malocclusion, with the expectation that an optimal facial appearance will naturally follow. However, this skeletal-driven approach can lead to undesirable facial aesthetics and residual asymmetry. To address these issues, a soft-tissue-driven planning method has been proposed. This innovative method bases bone movement estimates on the targeted ideal facial appearance, thus increasing the surgical plan's accuracy and effectiveness. This study explores the initial phase of implementing a soft-tissue-driven approach, simulating the patient's optimal facial look by repositioning deformed facial landmarks to an ideal state. The algorithm incorporates symmetrization and weighted optimization strategies, aligning projected optimal landmarks with standard cephalometric values for both facial symmetry and form, which are integral to facial aesthetics in orthognathic surgery. It also includes regularization to preserve the patient's original facial characteristics. Validated using retrospective analysis of data from both preoperative patients and normal subjects, this approach effectively achieves not only facial symmetry, particularly in the lower face, but also a more natural and normalized facial form. This novel approach, aligning with soft-tissue-driven planning principles, shows promise in surpassing traditional methods, potentially leading to enhanced facial outcomes and patient satisfaction in orthognathic surgery.

Identification and Characterization of Cancer-Associated Fibroblast Subpopulations in Lung Adenocarcinoma.

Cancer-associated fibroblasts (CAFs) reside within the tumor microenvironment, facilitating cancer progression and metastasis via direct and indirect interactions with cancer cells and other stromal cell types. CAFs are composed of heterogeneous subpopulations of activated fibroblasts, including myofibroblastic, inflammatory, and immunosuppressive CAFs. In this study, we sought to identify subpopulations of CAFs isolated from human lung adenocarcinomas and describe their transcriptomic and functional characteristics through single-cell RNA sequencing (scRNA-seq) and subsequent bioinformatics analyses. Cell trajectory analysis of combined total and THY1 + CAFs revealed two branching points with five distinct branches. Based on Gene Ontology analysis, we denoted Branch 1 as "immunosuppressive", Branch 2 as "neoantigen presenting", Branch 4 as "myofibroblastic", and Branch 5 as "proliferative" CAFs. We selected representative branch-specific markers and measured their expression levels in total and THY1 + CAFs. We also investigated the effects of these markers on CAF activity under coculture with lung cancer cells. This study describes novel subpopulations of CAFs in lung adenocarcinoma, highlighting their potential value as therapeutic targets.

Phase retrieval based on deep learning in grating interferometer.

Grating interferometry is a promising technique to obtain differential phase contrast images with illumination source of low intrinsic transverse coherence. However, retrieving the phase contrast image from the differential phase contrast image is difficult due to the accumulated noise and artifacts from the differential phase contrast image (DPCI) reconstruction. In this paper, we implemented a deep learning-based phase retrieval method to suppress these artifacts. Conventional deep learning based denoising requires noise/clean image pair, but it is not feasible to obtain sufficient number of clean images for grating interferometry. In this paper, we apply a recently developed neural network called Noise2Noise (N2N) that uses noise/noise image pairs for training. We obtained many DPCIs through combination of phase stepping images, and these were used as input/target pairs for N2N training. The application of the N2N network to simulated and measured DPCI showed that the phase contrast images were retrieved with strongly suppressed phase retrieval artifacts. These results can be used in grating interferometer applications which uses phase stepping method.

Deep learning for biomechanical modeling of facial tissue deformation in orthognathic surgical planning.

PURPOSE: Orthognathic surgery requires an accurate surgical plan of how bony segments are moved and how the face passively responds to the bony movement. Currently, finite element method (FEM) is the standard for predicting facial deformation. Deep learning models have recently been used to approximate FEM because of their faster simulation speed. However, current solutions are not compatible with detailed facial meshes and often do not explicitly provide the network with known boundary type information. Therefore, the purpose of this proof-of-concept study is to develop a biomechanics-informed deep neural network that accepts point cloud data and explicit boundary types as inputs to the network for fast prediction of soft-tissue deformation. METHODS: A deep learning network was developed based on the PointNet++ architecture. The network accepts the starting facial mesh, input displacement, and explicit boundary type information and predicts the final facial mesh deformation. RESULTS: We trained and tested our deep learning model on datasets created from FEM simulations of facial meshes. Our model achieved a mean error between 0.159 and 0.642 mm on five subjects. Including explicit boundary types had mixed results, improving performance in simulations with large deformations but decreasing performance in simulations with small deformations. These results suggest that including explicit boundary types may not be necessary to improve network performance. CONCLUSION: Our deep learning method can approximate FEM for facial change prediction in orthognathic surgical planning by accepting geometrically detailed meshes and explicit boundary types while significantly reducing simulation time.

Analysis of a silicon comb structure using an inverse Talbot-Lau neutron grating interferometer.

We describe an inverse Talbot-Lau neutron grating interferometer that provides an extended autocorrelation length range for quantitative dark-field imaging. To our knowledge, this is the first report of a Talbot-Lau neutron grating interferometer (nTLI) with inverse geometry. We demonstrate a range of autocorrelation lengths (ACL) starting at low tens of nanometers, which is significantly extended compared to the ranges of conventional and symmetric setups. ACLs from a minimum of 44 nm to the maximum of 3.5 µm were presented for the designed wavelength of 4.4 Å in experiments. Additionally, the inverse nTLI has neutron-absorbing gratings with an optically thick gadolinium oxysulfide (Gadox) structure, allowing it to provide a visibility of up to 52% while maintaining a large field of view of approximately 100 mm × 100 mm. We demonstrate the application of our interferometer to quantitative dark-field imaging by using diluted polystyrene particles in an aqueous solution and silicon comb structures. We obtain quantitative structural information of the sphere size and concentration of diluted polystyrene particles and the period, height, and duty cycle of the silicon comb structures. The optically thick Gadox structure of the analyzer grating also provides improved characteristics for the correction of incoherent neutron scattering in an aqueous solution compared to the symmetric nTLI.

Midsagittal Plane First: Building a Strong Facial Reference Frame for Computer-Aided Surgical Simulation.

PURPOSE: A facial reference frame is a 3-dimensional Cartesian coordinate system that includes 3 perpendicular planes: midsagittal, axial, and coronal. The order in which one defines the planes matters. The purposes of this study are to determine the following: 1) what sequence (axial-midsagittal-coronal vs midsagittal-axial-coronal) produced more appropriate reference frames and 2) whether orbital or auricular dystopia influenced the outcomes. METHODS: This study is an ambispective cross-sectional study. Fifty-four subjects with facial asymmetry were included. The facial reference frames of each subject (outcome variable) were constructed using 2 methods (independent variable): axial plane first and midsagittal plane first. Two board-certified orthodontists together blindly evaluated the results using a 3-point categorical scale based on their careful inspection and expert intuition. The covariant for stratification was the existence of orbital or auricular dystopia. Finally, Wilcoxon signed rank tests were performed. RESULTS: The facial reference frames defined by the midsagittal plane first method was statistically significantly different from ones defined by the axial plane first method (P = .001). Using the midsagittal plane first method, the reference frames were more appropriately defined in 22 (40.7%) subjects, equivalent in 26 (48.1%) and less appropriately defined in 6 (11.1%). After stratified by orbital or auricular dystopia, the results also showed that the reference frame computed using midsagittal plane first method was statistically significantly more appropriate in both subject groups regardless of the existence of orbital or auricular dystopia (27 with orbital or auricular dystopia and 27 without, both P < .05). CONCLUSIONS: The midsagittal plane first sequence improves the facial reference frames compared with the traditional axial plane first approach. However, regardless of the sequence used, clinicians need to judge the correctness of the reference frame before diagnosis or surgical planning.

A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning.

Virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. Due to the lack of necessary guidance, the planning procedure is highly experience-dependent and the planning results are often suboptimal. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy. Therefore, we propose a self-supervised deep framework to automatically estimate reference facial bony shape models. Our framework is an end-to-end trainable network, consisting of a simulator and a corrector. In the training stage, the simulator maps jaw deformities of a patient bone to a normal bone to generate a simulated deformed bone. The corrector then restores the simulated deformed bone back to normal. In the inference stage, the trained corrector is applied to generate a patient-specific normal-looking reference bone from a real deformed bone. The proposed framework was evaluated using a clinical dataset and compared with a state-of-the-art method that is based on a supervised point-cloud network. Experimental results show that the estimated shape models given by our approach are clinically acceptable and significantly more accurate than that of the competing method.

Deep Simulation of Facial Appearance Changes Following Craniomaxillofacial Bony Movements in Orthognathic Surgical Planning.

Facial appearance changes with the movements of bony segments in orthognathic surgery of patients with craniomaxillofacial (CMF) deformities. Conventional bio-mechanical methods, such as finite element modeling (FEM), for simulating such changes, are labor intensive and computationally expensive, preventing them from being used in clinical settings. To overcome these limitations, we propose a deep learning framework to predict post-operative facial changes. Specifically, FC-Net, a facial appearance change simulation network, is developed to predict the point displacement vectors associated with a facial point cloud. FC-Net learns the point displacements of a pre-operative facial point cloud from the bony movement vectors between pre-operative and simulated post-operative bony models. FC-Net is a weakly-supervised point displacement network trained using paired data with strict point-to-point correspondence. To preserve the topology of the facial model during point transform, we employ a local-point-transform loss to constrain the local movements of points. Experimental results on real patient data reveal that the proposed framework can predict post-operative facial appearance changes remarkably faster than a state-of-the-art FEM method with comparable prediction accuracy.

Unsupervised learning of reference bony shapes for orthognathic surgical planning with a surface deformation network.

PURPOSE: The purpose of this study was to reduce the experience dependence during the orthognathic surgical planning that involves virtually simulating the corrective procedure for jaw deformities. METHODS: We introduce a geometric deep learning framework for generating reference facial bone shape models for objective guidance in surgical planning. First, we propose a surface deformation network to warp a patient's deformed bone to a set of normal bones for generating a dictionary of patient-specific normal bony shapes. Subsequently, sparse representation learning is employed to estimate a reference shape model based on the dictionary. RESULTS: We evaluated our method on a clinical dataset containing 24 patients, and compared it with a state-of-the-art method that relies on landmark-based sparse representation. Our method yields significantly higher accuracy than the competing method for estimating normal jaws and maintains the midfaces of patients' facial bones as well as the conventional way. CONCLUSIONS: Experimental results indicate that our method generates accurate shape models that meet clinical standards.

A novel incremental simulation of facial changes following orthognathic surgery using FEM with realistic lip sliding effect.

Accurate prediction of facial soft-tissue changes following orthognathic surgery is crucial for surgical outcome improvement. We developed a novel incremental simulation approach using finite element method (FEM) with a realistic lip sliding effect to improve the prediction accuracy in the lip region. First, a lip-detailed mesh is generated based on accurately digitized lip surface points. Second, an improved facial soft-tissue change simulation method is developed by applying a lip sliding effect along with the mucosa sliding effect. Finally, the orthognathic surgery initiated soft-tissue change is simulated incrementally to facilitate a natural transition of the facial change and improve the effectiveness of the sliding effects. Our method was quantitatively validated using 35 retrospective clinical data sets by comparing it to the traditional FEM simulation method and the FEM simulation method with mucosa sliding effect only. The surface deviation error of our method showed significant improvement in the upper and lower lips over the other two prior methods. In addition, the evaluation results using our lip-shape analysis, which reflects clinician's qualitative evaluation, also proved significant improvement of the lip prediction accuracy of our method for the lower lip and both upper and lower lips as a whole compared to the other two methods. In conclusion, the prediction accuracy in the clinically critical region, i.e., the lips, significantly improved after applying incremental simulation with realistic lip sliding effect compared with the FEM simulation methods without the lip sliding effect.

Estimating Reference Bony Shape Models for Orthognathic Surgical Planning Using 3D Point-Cloud Deep Learning.

Orthognathic surgical outcomes rely heavily on the quality of surgical planning. Automatic estimation of a reference facial bone shape significantly reduces experience-dependent variability and improves planning accuracy and efficiency. We propose an end-to-end deep learning framework to estimate patient-specific reference bony shape models for patients with orthognathic deformities. Specifically, we apply a point-cloud network to learn a vertex-wise deformation field from a patient's deformed bony shape, represented as a point cloud. The estimated deformation field is then used to correct the deformed bony shape to output a patient-specific reference bony surface model. To train our network effectively, we introduce a simulation strategy to synthesize deformed bones from any given normal bone, producing a relatively large and diverse dataset of shapes for training. Our method was evaluated using both synthetic and real patient data. Experimental results show that our framework estimates realistic reference bony shape models for patients with varying deformities. The performance of our method is consistently better than an existing method and several deep point-cloud networks. Our end-to-end estimation framework based on geometric deep learning shows great potential for improving clinical workflows.

Study on dark-field imaging with a laboratory x-ray source: Random stress variation analysis based on x-ray grating interferometry.

The dark-field image (DFI) in a grating interferometer involves the small-angle scattering properties of a material. The microstructure of the material can be characterized by an analysis of the auto-correlation length and the DFI. The feasibility of a DFI in a laboratory x-ray source with grating interferometry has been reported, but a follow-up study is needed. In this study, the random stress distribution was measured in the laboratory environment as an applied study. SiO2 mono-spheres as a cohesive powder with a 0.5 µm particle size were used as the sample. The microstructural changes according to the stresses on the particles were observed by acquiring a DFI along the auto-correlation length. In x-rays, a random two-phase media model was first used to analyze the characteristics of cohesive powder. This study showed that the microstructure of materials and x-ray images could be analyzed in a laboratory environment.

Efficacy and safety of transvaginal high-intensity focused ultrasound therapy in women with symptomatic uterine leiomyomas: A clinical trial.

OBJECTIVE: To evaluate the efficacy and safety of transvaginal high-intensity focused ultrasound (vHIFU) therapy in women with symptomatic uterine leiomyomas. METHODS: This first-in-human, two-center, prospective, unblinded, single-arm trial was performed in the Republic of Korea from December 2017 to February 2019. Premenopausal women with symptomatic, contrast-enhanced uterine leiomyomas with a diameter ≤5 cm were eligible. Under sedation or monitored anesthesia, leiomyomas were ablated with vHIFU under ultrasound guidance. The primary endpoint was the non-perfused volume (NPV) ratio measured immediately after therapy. Secondary endpoints were changes in Uterine Fibroid Symptom-Quality of Life (UFS-QOL) scores, dysmenorrhea visual analog scale (VAS), uterine leiomyoma volume, rate of subsequent therapy, and treatment-emergent adverse events (TEAE). RESULTS: Thirty-five women were screened; 13 women were enrolled and underwent vHIFU therapy for 33 uterine leiomyomas. NPV ratios were 0.76 ± 0.27 (mean ± SD); the lower limit of a one-sided 97.5 % confidence interval was 0.67, surpassing the non-inferiority cut-off of 0.50. UFS-QOL scores (symptom severity score, median, baseline: 66.60, 3-month follow-up: 32.85; p = 0.0010; health related quality of life score, median, baseline: 41.40, 3-month follow-up: 73.30; p = 0.0010) and dysmenorrhea VAS (mean, baseline: 50.92, 3-month follow-up: 20.67; p = 0.0019) improved significantly. Volume of uterine leiomyoma was reduced (median, baseline: 8.10 cm3, 3-month follow=-up: 5.30 cm3; p < 0.0001), and none received subsequent therapy. Twenty-six TEAEs from 8 participants were observed, and all TEAEs were resolved without sequelae. CONCLUSION: vHIFU therapy exhibited promising efficacy and safety and might be considered as a treatment option for women with symptomatic uterine leiomyomas. Registration: This trial was registered at: www.clinicaltrial.gov (NCT03328260).

Neutron grating interferometer with an analyzer grating based on a light blocker.

We study an analyzer grating based on a scintillation light blocker for a Talbot-Lau grating interferometer. This is an alternative way to analyze the Talbot self-image without the need for an often difficult to fabricate absorption grating for the incident radiation. The feasibility of this approach using a neutron beam has been evaluated and experiments have been conducted at the cold neutron imaging facility of the NIST center for Neutron Research. The neutron grating interferometer with the proposed analyzer grating successfully produced attenuation, differential phase, and dark-field contrast images. In addition, numerical simulations were performed to simulate the Talbot pattern and visibility using scintillation screens of different thicknesses and there is good agreement with the experimental measurements. The results show potential for reducing the difficulty of fabricating analyzer grating, and a possibility for the so-called shadow effect to be eliminated and large-area gratings to be produced, especially when applied to X-rays. We report the performance of the analyzer grating based on a light blocker and evaluate its feasibility for the grating interferometer.

Deep learning for high-resolution and high-sensitivity interferometric phase contrast imaging.

In Talbot-Lau interferometry, the sample position yielding the highest phase sensitivity suffers from strong geometric blur. This trade-off between phase-sensitivity and spatial resolution is a fundamental challenge in such interferometric imaging applications with either neutron or conventional x-ray sources due to their relatively large beam-defining apertures or focal spots. In this study, a deep learning method is introduced to estimate a high phase-sensitive and high spatial resolution image from a trained neural network to attempt to avoid the trade-off for both high phase-sensitivity and high resolution. To realize this, the training data sets of the differential phase contrast images at a pair of sample positions, one of which is close to the phase grating and the other close to the detector, are numerically generated and are used as the inputs for the training data set of a generative adversarial network. The trained network has been applied to the real experimental data sets from a neutron grating interferometer and we have obtained improved images both in phase-sensitivity and spatial resolution.