Non-destructive Characterization of Skeletal Muscle Extracellular Matrix Morphology by Combining Optical Coherence Tomography (OCT) Imaging with Tissue Clearing.

Histology is an essential step to visualize and analyze the microstructure of any biological tissue; however, histological processing is often irreversible, and histological samples are unable to be imaged or tested further. In this work, a novel non-destructive protocol is proposed for morphological analysis of skeletal muscles, combining Optical Coherence Tomography (OCT) imaging with Tissue Clearing. Imaging combining OCT and Propylene Glycol (PG) as a tissue-clearing agent, was performed on rat tail and extensor digitorum longus (EDL) muscle. The results show that the extracellular matrix morphology of skeletal muscles, including muscular fibers and the whole microstructure architecture were clearly identified. PG improved OCT imaging as measured by image quality metric Contrast Per Pixel CPP (increases by 3.9%), Naturalness Image Quality Evaluator NIQE (decreases by 23%), and Volume of Interest VOI size (higher for CPP and lower for NIQE values). The tendon microstructure was observed with less precision, as collagen fibers could not be clearly detected. The reversibility of the optical effects of the PG on the immersed tissue (in a Phosphate-Buffered Saline solution) was studied comparing native and rehydrated OCT image acquisition from a single EDL sample. Optical properties and microstructure visibility (CPP and NIQE) have been recovered to 99% of the native sample values. Moreover, clearing process caused shrinkage of the tissue recovered to 86% of the original width. Future work will aim to employ the proposed experimental protocol to identify the local mechanical properties of biological tissues.

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients.

The 3D reconstruction of an accurate face model is essential for delivering reliable feedback for clinical decision support. Medical imaging and specific depth sensors are accurate but not suitable for an easy-to-use and portable tool. The recent development of deep learning (DL) models opens new challenges for 3D shape reconstruction from a single image. However, the 3D face shape reconstruction of facial palsy patients is still a challenge, and this has not been investigated. The contribution of the present study is to apply these state-of-the-art methods to reconstruct the 3D face shape models of facial palsy patients in natural and mimic postures from one single image. Three different methods (3D Basel Morphable model and two 3D Deep Pre-trained models) were applied to the dataset of two healthy subjects and two facial palsy patients. The reconstructed outcomes were compared to the 3D shapes reconstructed using Kinect-driven and MRI-based information. As a result, the best mean error of the reconstructed face according to the Kinect-driven reconstructed shape is 1.5±1.1 mm. The best error range is 1.9±1.4 mm when compared to the MRI-based shapes. Before using the procedure to reconstruct the 3D faces of patients with facial palsy or other facial disorders, several ideas for increasing the accuracy of the reconstruction can be discussed based on the results. This present study opens new avenues for the fast reconstruction of the 3D face shapes of facial palsy patients from a single image. As perspectives, the best DL method will be implemented into our computer-aided decision support system for facial disorders.

Reinforcement learning coupled with finite element modeling for facial motion learning.

BACKGROUND AND OBJECTIVE: Facial palsy patients or patients with facial transplantation have abnormal facial motion due to altered facial muscle functions and nerve damage. Computer-aided system and physics-based models have been developed to provide objective and quantitative information. However, the predictive capacity of these solutions is still limited to explore the facial motion patterns with emerging properties. The present study aims to couple the reinforcement learning and the finite element modeling for facial motion learning and prediction. METHODS: A novel modeling workflow for learning facial motion was developed. A physically-based model of the face within the Artisynth modeling platform was used. Information exchange protocol was proposed to link reinforcement learning and rigid multi-bodies dynamics outcomes. Two reinforcement learning algorithms (deep deterministic policy gradient (DDPG) and Twin-delayed DDPG (TD3)) were used and implemented to drive the simulations of symmetry-oriented and smile movements. Numerical outcomes were compared to experimental observations (Bosphorus database) for evaluation and validation purposes. RESULTS: As result, after more than 100 episodes of exploring the environment, the agent starts to learn from previous trials and can find the optimal policy after more than 300 episodes of training. Regarding the symmetry-oriented motion, the muscle excitations predicted by the trained agent help to increase the value of reward from R = -2.06 to R = -0.23, which counts for ∼89% improvement of the symmetry value of the face. For smile-oriented motion, two points at the edge of the mouth move up 0.35 cm, which is within the range of movements estimated from the Bosphorus database (0.4 ± 0.32 cm). CONCLUSIONS: The present study explored the muscle excitation patterns by coupling reinforcement learning with a detailed finite element model of the face. We developed, for the first time, a novel coupling scheme to integrate the finite element simulation into the reinforcement learning process for facial motion learning. As perspectives, this present workflow will be applied for facial palsy and facial transplantation patients to guide and optimize the functional rehabilitation program.

A simple and effective 1D-element discrete-based method for computational bone remodeling.

In-silico models applied to bone remodeling are widely used to investigate bone mechanics, bone diseases, bone-implant interactions, and also the effect of treatments of bone pathologies. This article proposes a new methodology to solve the bone remodeling problem using one-dimensional (1D) elements to discretize trabecular structures more efficiently for 2D and 3D domains. An Euler integration scheme is coupled with the momentum equations to obtain the evolution of material density at each step. For the simulations, the equations were solved by using the finite element method, and two benchmark tests were solved varying mesh parameters. Proximal femur and calcaneus bone were selected as study cases given the vast research available on the topology of these bones, and compared with the anatomical features of trabecular bone reported in the literature. The presented methodology has proven to be efficient in optimizing topologies of lattice structures; It can predict the trend of formation patterns of the main trabecular groups from two different cancellous bones (femur and calcaneus) using domains set up by discrete elements as a starting point. Preliminary results confirm that the proposed approach is suitable and useful in bone remodeling problems leading to a considerable computational cost reduction. Characteristics similar to those encountered in topological optimization algorithms were identified in the benchmark tests as well, showing the viability of the proposed approach in other applications such as bio-inspired design.

Enhanced facial expression recognition using 3D point sets and geometric deep learning.

Facial expression recognition plays an essential role in human conversation and human-computer interaction. Previous research studies have recognized facial expressions mainly based on 2D image processing requiring sensitive feature engineering and conventional machine learning approaches. The purpose of the present study was to recognize facial expressions by applying a new class of deep learning called geometric deep learning directly on 3D point cloud data. Two databases (Bosphorus and SIAT-3DFE) were used. The Bosphorus database includes sixty-five subjects with seven basic expressions (i.e., anger, disgust, fearness, happiness, sadness, surprise, and neutral). The SIAT-3DFE database has 150 subjects and 4 basic facial expressions (neutral, happiness, sadness, and surprise). First, preprocessing procedures such as face center cropping, data augmentation, and point cloud denoising were applied on 3D face scans. Then, a geometric deep learning model called PointNet++ was applied. A hyperparameter tuning process was performed to find the optimal model parameters. Finally, the developed model was evaluated using the recognition rate and confusion matrix. The facial expression recognition accuracy on the Bosphorus database was 69.01% for 7 expressions and could reach 85.85% when recognizing five specific expressions (anger, disgust, happiness, surprise, and neutral). The recognition rate was 78.70% with the SIAT-3DFE database. The present study suggested that 3D point cloud could be directly processed for facial expression recognition by using geometric deep learning approach. In perspectives, the developed model will be applied for facial palsy patients to guide and optimize the functional rehabilitation program.

Kinect-driven Patient-specific Head, Skull, and Muscle Network Modelling for Facial Palsy Patients.

BACKGROUND AND OBJECTIVE: Facial palsy negatively affects both professional and personal life qualities of involved patients. Classical facial rehabilitation strategies can recover facial mimics into their normal and symmetrical movements and appearances. However, there is a lack of objective, quantitative, and in-vivo facial texture and muscle activation bio-feedbacks for personalizing rehabilitation programs and diagnosing recovering progresses. Consequently, this study proposed a novel patient-specific modelling method for generating a full patient specific head model from a visual sensor and then computing the facial texture and muscle activation in real-time for further clinical decision making. METHODS: The modeling workflow includes (1) Kinect-to-head, (2) head-to-skull, and (3) muscle network definition & generation processes. In the Kinect-to-head process, subject-specific data acquired from a new user in neutral mimic were used for generating his/her geometrical head model with facial texture. In particular, a template head model was deformed to optimally fit with high-definition facial points acquired by the Kinect sensor. Moreover, the facial texture was also merged from his/her facial images in left, right, and center points of view. In the head-to-skull process, a generic skull model was deformed so that its shape was statistically fitted with his/her geometrical head model. In the muscle network definition & generation process, a muscle network was defined from the head and skull models for computing muscle strains during facial movements. Muscle insertion points and muscle attachment points were defined as vertex positions on the head model and the skull model respectively based on the standard facial anatomy. Three healthy subjects and two facial palsy patients were selected for validating the proposed method. In neutral positions, magnetic resonance imaging (MRI)-based head and skull models were compared with Kinect-based head and skull models. In mimic positions, infrared depth-based head models in smiling and [u]-pronouncing mimics were compared with appropriate animated Kinect-driven head models. The Hausdorff distance metric was used for these comparisons. Moreover, computed muscle lengths and strains in the tested facial mimics were validated with reported values in literature. RESULTS: With the current hardware configuration, the patient-specific head model with skull and muscle network could be fast generated within 17.16±0.37s and animated in real-time with the framerate of 40 fps. In neutral positions, the best mean error was 1.91 mm for the head models and 3.21 mm for the skull models. On facial regions, the best mean errors were 1.53 mm and 2.82 mm for head and skull models respectively. On muscle insertion/attachment point regions, the best mean errors were 1.09 mm and 2.16 mm for head and skull models respectively. In mimic positions, these errors were 2.02 mm in smiling mimics and 2.00 mm in [u]-pronouncing mimics for the head models on facial regions. All above error values were computed on a one-time validation procedure. Facial muscles exhibited muscle shortening and muscle elongating for smiling and pronunciation of sound [u] respectively. Extracted muscle features (i.e. muscle length and strain) are in agreement with experimental and literature data. CONCLUSIONS: This study proposed a novel modeling method for fast generating and animating patient-specific biomechanical head model with facial texture and muscle activation bio-feedbacks. The Kinect-driven muscle strains could be applied for further real-time muscle-oriented facial paralysis grading and other facial analysis applications.

Real-time computer vision system for tracking simultaneously subject-specific rigid head and non-rigid facial mimic movements using a contactless sensor and system of systems approach.

BACKGROUND AND OBJECTIVE: Head and facial mimic animations play important roles in various fields such as human-machine interactions, internet communications, multimedia applications, and facial mimic analysis. Numerous studies have been trying to simulate these animations. However, they hardly achieved all requirements of full rigid head and non-rigid facial mimic animations in a subject-specific manner with real-time framerates. Consequently, this present study aimed to develop a real-time computer vision system for tracking simultaneously rigid head and non-rigid facial mimic movements. METHODS: Our system was developed using the system of systems approach. A data acquisition sub-system was implemented using a contactless Kinect sensor. A subject-specific model generation sub-system was designed to create the geometrical model from the Kinect sensor without texture information. A subject-specific texture generation sub-system was designed for enhancing the reality of the generated model with texture information. A head animation sub-system with graphical user interfaces was also developed. Model accuracy and system performances were analyzed. RESULTS: The comparison with MRI-based model shows a very good accuracy level (distance deviation of ~1 mm in neutral position and an error range of [2-3 mm] for different facial mimic positions) for the generated model from our system. Moreover, the system speed can be optimized to reach a high framerate (up to 60 fps) during different head and facial mimic animations. CONCLUSIONS: This study presents a novel computer vision system for tracking simultaneously subject-specific rigid head and non-rigid facial mimic movements in real time. In perspectives, serious game technology will be integrated into this system towards a full computer-aided decision support system for facial rehabilitation.

A Systematic Review of Real-Time Medical Simulations with Soft-Tissue Deformation: Computational Approaches, Interaction Devices, System Architectures, and Clinical Validations.

Simulating deformations of soft tissues is a complex engineering task, and it is even more difficult when facing the constraint between computation speed and system accuracy. However, literature lacks of a holistic review of all necessary aspects (computational approaches, interaction devices, system architectures, and clinical validations) for developing an effective system of soft-tissue simulations. This paper summarizes and analyses recent achievements of resolving these issues to estimate general trends and weakness for future developments. A systematic review process was conducted using the PRISMA protocol with three reliable scientific search engines (ScienceDirect, PubMed, and IEEE). Fifty-five relevant papers were finally selected and included into the review process, and a quality assessment procedure was also performed on them. The computational approaches were categorized into mesh, meshfree, and hybrid approaches. The interaction devices concerned about combination between virtual surgical instruments and force-feedback devices, 3D scanners, biomechanical sensors, human interface devices, 3D viewers, and 2D/3D optical cameras. System architectures were analysed based on the concepts of system execution schemes and system frameworks. In particular, system execution schemes included distribution-based, multithread-based, and multimodel-based executions. System frameworks are grouped into the input and output interaction frameworks, the graphic interaction frameworks, the modelling frameworks, and the hybrid frameworks. Clinical validation procedures are ordered as three levels: geometrical validation, model behavior validation, and user acceptability/safety validation. The present review paper provides useful information to characterize how real-time medical simulation systems with soft-tissue deformations have been developed. By clearly analysing advantages and drawbacks in each system development aspect, this review can be used as a reference guideline for developing systems of soft-tissue simulations.

Comparison of patient-specific computational models vs. clinical follow-up, for adjacent segment disc degeneration and bone remodelling after spinal fusion.

Spinal fusion is a standard surgical treatment for patients suffering from low back pain attributed to disc degeneration. However, results are somewhat variable and unpredictable. With fusion the kinematic behaviour of the spine is altered. Fusion and/or stabilizing implants carrying considerable load and prevent rotation of the fused segments. Associated with these changes, a risk for accelerated disc degeneration at the adjacent levels to fusion has been demonstrated. However, there is yet no method to predict the effect of fusion surgery on the adjacent tissue levels, i.e. bone and disc. The aim of this study was to develop a coupled and patient-specific mechanoregulated model to predict disc generation and changes in bone density after spinal fusion and to validate the results relative to patient follow-up data. To do so, a multiscale disc mechanoregulation adaptation framework was developed and coupled with a previously developed bone remodelling algorithm. This made it possible to determine extra cellular matrix changes in the intervertebral disc and bone density changes simultaneously based on changes in loading due to fusion surgery. It was shown that for 10 cases the predicted change in bone density and degeneration grade conforms reasonable well to clinical follow-up data. This approach helps us to understand the effect of surgical intervention on the adjacent tissue remodelling. Thereby, providing the first insight for a spine surgeon as to which patient could potentially be treated successfully by spinal fusion and in which patient has a high risk for adjacent tissue changes.

Rehabilitation-Oriented Serious Game Development and Evaluation Guidelines for Musculoskeletal Disorders.

BACKGROUND: The progress in information and communication technology (ICT) led to the development of a new rehabilitation technique called "serious game for functional rehabilitation." Previous works have shown that serious games can be used for general health and specific disease management. However, there is still lack of consensus on development and evaluation guidelines. It is important to note that the game performance depends on the designed scenario. OBJECTIVE: The objective of this work was to develop specific game scenarios and evaluate them with a panel of musculoskeletal patients to propose game development and evaluation guidelines. METHODS: A two-stage workflow was proposed using determinant framework. The development guideline includes the selection of three-dimensional (3D) computer graphics technologies and tools, the modeling of physical aspects, the design of rehabilitation scenarios, and the implementation of the proposed scenarios. The evaluation guideline consists of the definition of evaluation metrics, the execution of the evaluation campaign, the analysis of user results and feedbacks, and the improvement of the designed game. RESULTS: The case study for musculoskeletal disorders on the healthy control and patient groups showed the usefulness of these guidelines and associated games. All participants enjoyed the 2 developed games (football and object manipulation), and found them challenging and amusing. In particular, some healthy subjects increased their score when enhancing the level of difficulty. Furthermore, there were no risks and accidents associated with the execution of these games. CONCLUSIONS: It is expected that with the proven effectiveness of the proposed guidelines and associated games, this new rehabilitation game may be translated into clinical routine practice for the benefit of patients with musculoskeletal disorders.