Simultaneous automated ascertainment of prevalent vertebral fracture and abdominal aortic calcification in clinical practice: role in fracture risk assessment.

Whether simultaneous automated ascertainments of prevalent vertebral fracture (auto-PVFx) and abdominal aortic calcification (auto-AAC) on vertebral fracture assessment (VFA) lateral spine bone density (BMD) images jointly predict incident fractures in routine clinical practice is unclear. We estimated the independent associations of auto-PVFx and auto-AAC primarily with incident major osteoporotic and secondarily with incident hip and any clinical fractures in 11 013 individuals (mean [SD] age 75.8 [6.8] years, 93.3% female) who had a BMD test combined with VFA between March 2010 and December 2017. Auto-PVFx and auto-AAC were ascertained using convolutional neural networks (CNNs). Proportional hazards models were used to estimate the associations of auto-PVFx and auto-AAC with incident fractures over a mean (SD) follow-up of 3.7 (2.2) years, adjusted for each other and other risk factors. At baseline, 17% (n = 1881) had auto-PVFx and 27% (n = 2974) had a high level of auto-AAC (≥ 6 on scale of 0 to 24). Multivariable-adjusted hazard ratios (HR) for incident major osteoporotic fracture (95% C.I.) were 1.85 (1.59, 2.15) for those with compared to those without auto-PVFx, and 1.36 (1.14, 1.62) for those with high compared to low auto-AAC. The multivariable-adjusted HRs for incident hip fracture were 1.62 (95% C.I. 1.26 to 2.07) for those with compared to those without auto-PVFx, and 1.55 (95% C.I. 1.15 to 2.09) for those high auto-AAC compared to low auto-AAC. The 5-year cumulative incidence of major osteoporotic fracture was 7.1% in those with no auto-PVFx and low auto-AAC, 10.1% in those with no auto-PVFx and high auto-AAC, 13.4% in those with auto-PVFx and low auto-AAC, and 18.0% in those with auto-PVFx and high auto-AAC. While physician manual review of images in clinical practice will still be needed to confirm image quality and provide clinical context for interpretation, simultaneous automated ascertainment of auto-PVFx and auto-AAC can aid fracture risk assessment.

Individuals with calcification of their abdominal aorta (AAC) and vertebral fractures seen on lateral spine bone density images (easily obtained as part of a bone density test) are much more likely to have subsequent fractures. Prior studies have not shown if both AAC and prior vertebral fracture both contribute to fracture prediction in routine clinical practice. Additionally, a barrier to using these images to aid fracture risk assessment at the time of bone density testing has been the need for expert readers to be able to accurately detect both AAC and vertebral fractures. We have developed automated computer methods (using artificial intelligence) to accurately detect vertebral fracture (auto-PVFx) and auto-AAC on lateral spine bone density images for 11 013 older individuals having a bone density test in routine clinical practice. Over a 5-year follow-up period, 7.1% of those with no auto-PVFx and low auto-AAC, 10.1% of those with no auto-PVFx and high auto-AAC, 13.4% of those with auto-PVFx and low auto-AAC, and 18.0% of those with auto-PVFx and high auto-AAC had a major osteoporotic fracture. Auto-PVFx and auto-AAC, ascertained simultaneously on lateral spine bone density images, both contribute to the risk of subsequent major osteoporotic fractures in routine clinical practice settings.

Machine learning for abdominal aortic calcification assessment from bone density machine-derived lateral spine images.

BACKGROUND: Lateral spine images for vertebral fracture assessment can be easily obtained on modern bone density machines. Abdominal aortic calcification (AAC) can be scored on these images by trained imaging specialists to assess cardiovascular disease risk. However, this process is laborious and requires careful training. METHODS: Training and testing of model performance of the convolutional neural network (CNN) algorithm for automated AAC-24 scoring utilised 5012 lateral spine images (2 manufacturers, 4 models of bone density machines), with trained imaging specialist AAC scores. Validation occurred in a registry-based cohort study of 8565 older men and women with images captured as part of routine clinical practice for fracture risk assessment. Cox proportional hazards models were used to estimate the association between machine-learning AAC (ML-AAC-24) scores with future incident Major Adverse Cardiovascular Events (MACE) that including death, hospitalised acute myocardial infarction or ischemic cerebrovascular disease ascertained from linked healthcare data. FINDINGS: The average intraclass correlation coefficient between imaging specialist and ML-AAC-24 scores for 5012 images was 0.84 (95% CI 0.83, 0.84) with classification accuracy of 80% for established AAC groups. During a mean follow-up 4 years in the registry-based cohort, MACE outcomes were reported in 1177 people (13.7%). With increasing ML-AAC-24 scores there was an increasing proportion of people with MACE (low 7.9%, moderate 14.5%, high 21.2%), as well as individual MACE components (all p-trend <0.001). After multivariable adjustment, moderate and high ML-AAC-24 groups remained significantly associated with MACE (HR 1.54, 95% CI 1.31-1.80 & HR 2.06, 95% CI 1.75-2.42, respectively), compared to those with low ML-AAC-24. INTERPRETATION: The ML-AAC-24 scores had substantial levels of agreement with trained imaging specialists, and was associated with a substantial gradient of risk for cardiovascular events in a real-world setting. This approach could be readily implemented into these clinical settings to improve identification of people at high CVD risk. FUNDING: The study was supported by a National Health and Medical Research Council of Australia Ideas grant and the Rady Innovation Fund, Rady Faculty of Health Sciences, University of Manitoba.

Unsupervised Learning for Maximum Consensus Robust Fitting: A Reinforcement Learning Approach.

Robust model fitting is a core algorithm in several computer vision applications. Despite being studied for decades, solving this problem efficiently for datasets that are heavily contaminated by outliers is still challenging: due to the underlying computational complexity. A recent focus has been on learning-based algorithms. However, most of these approaches are supervised (which require a large amount of labelled training data). In this paper, we introduce a novel unsupervised learning framework: that learns to directly (without labelled data) solve robust model fitting. Moreover, unlike other learning-based methods, our work is agnostic to the underlying input features, and can be easily generalized to a wide variety of LP-type problems with quasi-convex residuals. We empirically show that our method outperforms existing (un)supervised learning approaches, and also achieves competitive results compared to traditional (non-learning-based) methods. Our approach is designed to try to maximise consensus (MaxCon), similar to the popular RANSAC. The basis of our approach, is to adopt a Reinforcement Learning framework. This requires designing appropriate reward functions, and state encodings. We provide a family of reward functions, tunable by choice of a parameter. We also investigate the application of different basic and enhanced Q-learning components.

An investigation of a novel broad autism phenotype: increased facial masculinity among parents of children on the autism spectrum.

The broad autism phenotype commonly refers to sub-clinical levels of autistic-like behaviour and cognition presented in biological relatives of autistic people. In a recent study, we reported findings suggesting that the broad autism phenotype may also be expressed in facial morphology, specifically increased facial masculinity. Increased facial masculinity has been reported among autistic children, as well as their non-autistic siblings. The present study builds on our previous findings by investigating the presence of increased facial masculinity among non-autistic parents of autistic children. Using a previously established method, a 'facial masculinity score' and several facial distances were calculated for each three-dimensional facial image of 192 parents of autistic children (58 males, 134 females) and 163 age-matched parents of non-autistic children (50 males, 113 females). While controlling for facial area and age, significantly higher masculinity scores and larger (more masculine) facial distances were observed in parents of autistic children relative to the comparison group, with effect sizes ranging from small to medium (0.16 ≤ d ≤ .41), regardless of sex. These findings add to an accumulating evidence base that the broad autism phenotype is expressed in physical characteristics and suggest that both maternal and paternal pathways are implicated in masculinized facial morphology.

Facial asymmetry in parents of children on the autism spectrum.

Greater facial asymmetry has been consistently found in children with autism spectrum disorder (ASD) relative to children without ASD. There is substantial evidence that both facial structure and the recurrence of ASD diagnosis are highly heritable within a nuclear family. Furthermore, sub-clinical levels of autistic-like behavioural characteristics have also been reported in first-degree relatives of individuals with ASD, commonly known as the 'broad autism phenotype'. Therefore, the aim of the current study was to examine whether a broad autism phenotype expresses as facial asymmetry among 192 biological parents of autistic individuals (134 mothers) compared to those of 163 age-matched adults without a family history of ASD (113 females). Using dense surface-modelling techniques on three dimensional facial images, we found evidence for greater facial asymmetry in parents of autistic individuals compared to age-matched adults in the comparison group (p = 0.046, d = 0.21 [0.002, 0.42]). Considering previous findings and the current results, we conclude that facial asymmetry expressed in the facial morphology of autistic children may be related to heritability factors. LAY ABSTRACT: In a previous study, we showed that autistic children presented with greater facial asymmetry than non-autistic children. In the current study, we examined the amount of facial asymmetry shown on three-dimensional facial images of 192 parents of autistic children compared to a control group consisting of 163 similarly aged adults with no known history of autism. Although parents did show greater levels of facial asymmetry than those in the control group, this effect is statistically small. We concluded that the facial asymmetry previously found in autistic children may be related to genetic factors.

Relation Graph Network for 3D Object Detection in Point Clouds.

Convolutional Neural Networks (CNNs) have emerged as a powerful tool for object detection in 2D images. However, their power has not been fully realised for detecting 3D objects directly in point clouds without conversion to regular grids. Moreover, existing state-of-the-art 3D object detection methods aim to recognize objects individually without exploiting their relationships during learning or inference. In this article, we first propose a strategy that associates the predictions of direction vectors with pseudo geometric centers, leading to a win-win solution for 3D bounding box candidates regression. Secondly, we propose point attention pooling to extract uniform appearance features for each 3D object proposal, benefiting from the learned direction features, semantic features and spatial coordinates of the object points. Finally, the appearance features are used together with the position features to build 3D object-object relationship graphs for all proposals to model their co-existence. We explore the effect of relation graphs on proposals' appearance feature enhancement under supervised and unsupervised settings. The proposed relation graph network comprises a 3D object proposal generation module and a 3D relation module, making it an end-to-end trainable network for detecting 3D objects in point clouds. Experiments on challenging benchmark point cloud datasets (SunRGB-D, ScanNet and KITTI) show that our algorithm performs better than existing state-of-the-art.

Self-Supervised Learning to Detect Key Frames in Videos.

Detecting key frames in videos is a common problem in many applications such as video classification, action recognition and video summarization. These tasks can be performed more efficiently using only a handful of key frames rather than the full video. Existing key frame detection approaches are mostly designed for supervised learning and require manual labelling of key frames in a large corpus of training data to train the models. Labelling requires human annotators from different backgrounds to annotate key frames in videos which is not only expensive and time consuming but also prone to subjective errors and inconsistencies between the labelers. To overcome these problems, we propose an automatic self-supervised method for detecting key frames in a video. Our method comprises a two-stream ConvNet and a novel automatic annotation architecture able to reliably annotate key frames in a video for self-supervised learning of the ConvNet. The proposed ConvNet learns deep appearance and motion features to detect frames that are unique. The trained network is then able to detect key frames in test videos. Extensive experiments on UCF101 human action and video summarization VSUMM datasets demonstrates the effectiveness of our proposed method.

Structural Similarity Loss for Learning to Fuse Multi-Focus Images.

Convolutional neural networks have recently been used for multi-focus image fusion. However, some existing methods have resorted to adding Gaussian blur to focused images, to simulate defocus, thereby generating data (with ground-truth) for supervised learning. Moreover, they classify pixels as 'focused' or 'defocused', and use the classified results to construct the fusion weight maps. This then necessitates a series of post-processing steps. In this paper, we present an end-to-end learning approach for directly predicting the fully focused output image from multi-focus input image pairs. The suggested approach uses a CNN architecture trained to perform fusion, without the need for ground truth fused images. The CNN exploits the image structural similarity (SSIM) to calculate the loss, a metric that is widely accepted for fused image quality evaluation. What is more, we also use the standard deviation of a local window of the image to automatically estimate the importance of the source images in the final fused image when designing the loss function. Our network can accept images of variable sizes and hence, we are able to utilize real benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluation on benchmark datasets show that our method outperforms, or is comparable with, existing state-of-the-art techniques on both objective and subjective benchmarks.

A broad autism phenotype expressed in facial morphology.

Autism spectrum disorder is a heritable neurodevelopmental condition diagnosed based on social and communication differences. There is strong evidence that cognitive and behavioural changes associated with clinical autism aggregate with biological relatives but in milder form, commonly referred to as the 'broad autism phenotype'. The present study builds on our previous findings of increased facial masculinity in autistic children (Sci. Rep., 7:9348, 2017) by examining whether facial masculinity represents as a broad autism phenotype in 55 non-autistic siblings (25 girls) of autistic children. Using 3D facial photogrammetry and age-matched control groups of children without a family history of ASD, we found that facial features of male siblings were more masculine than those of male controls (n = 69; p < 0.001, d = 0.81 [0.36, 1.26]). Facial features of female siblings were also more masculine than the features of female controls (n = 60; p = 0.005, d = 0.63 [0.16, 1.10]). Overall, we demonstrated for males and females that facial masculinity in non-autistic siblings is increased compared to same-sex comparison groups. These data provide the first evidence for a broad autism phenotype expressed in a physical characteristic, which has wider implications for our understanding of the interplay between physical and cognitive development in humans.

Predicting sleep apnea from three-dimensional face photography.

STUDY OBJECTIVES: Craniofacial anatomy is recognized as an important predisposing factor in the pathogenesis of obstructive sleep apnea (OSA). This study used three-dimensional (3D) facial surface analysis of linear and geodesic (shortest line between points over a curved surface) distances to determine the combination of measurements that best predicts presence and severity of OSA. METHODS: 3D face photographs were obtained in 100 adults without OSA (apnea-hypopnea index [AHI] < 5 events/h), 100 with mild OSA (AHI 5 to < 15 events/h), 100 with moderate OSA (AHI 15 to < 30 events/h), and 100 with severe OSA (AHI ≥ 30 events/h). Measurements of linear distances and angles, and geodesic distances were obtained between 24 anatomical landmarks from the 3D photographs. The accuracy with which different combinations of measurements could classify an individual as having OSA or not was assessed using linear discriminant analyses and receiver operating characteristic analyses. These analyses were repeated using different AHI thresholds to define presence of OSA. RESULTS: Relative to linear measurements, geodesic measurements of craniofacial anatomy improved the ability to identify individuals with and without OSA (classification accuracy 86% and 89% respectively, P < .01). A maximum classification accuracy of 91% was achieved when linear and geodesic measurements were combined into a single predictive algorithm. Accuracy decreased when using AHI thresholds ≥ 10 events/h and ≥ 15 events/h to define OSA although greatest accuracy was always achieved using a combination of linear and geodesic distances. CONCLUSIONS: This study suggests that 3D photographs of the face have predictive value for OSA and that geodesic measurements enhance this capacity.

Increased facial asymmetry in autism spectrum conditions is associated with symptom presentation.

A key research priority in the study of autism spectrum conditions (ASC) is the discovery of biological markers that may help to identify and elucidate etiologically distinct subgroups. One physical marker that has received increasing research attention is facial structure. Although there remains little consensus in the field, findings relating to greater facial asymmetry (FA) in ASC exhibit some consistency. As there is growing recognition of the importance of replicatory studies in ASC research, the aim of this study was to investigate the replicability of increased FA in autistic children compared to nonautistic peers. Using three-dimensional photogrammetry, this study examined FA in 84 autistic children, 110 typically developing children with no family history of the condition, and 49 full siblings of autistic children. In support of previous literature, significantly greater depth-wise FA was identified in autistic children relative to the two comparison groups. As a further investigation, increased lateral FA in autistic children was found to be associated with greater severity of ASC symptoms on the Autism Diagnostic Observation Schedule, second edition, specifically related to repetitive and restrictive behaviors. These outcomes provide an important and independent replication of increased FA in ASC, as well as a novel contribution to the field. Having confirmed the direction and areas of increased FA in ASC, these findings could motivate a search for potential underlying brain dysmorphogenesis. Autism Res 2019, 12: 1774-1783. © 2019 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: This study looked at the amount of facial asymmetry (FA) in autistic children compared to typically developing children and children who have siblings with autism. The study found that autistic children, compared to the other two groups, had greater FA, and that increased FA was related to greater severity of autistic symptoms. The face and brain grow together during the earliest stages of development, and so findings of facial differences in autism might inform future studies of early brain differences associated with the condition.

Dense 3D Face Correspondence.

We present an algorithm that automatically establishes dense correspondences between a large number of 3D faces. Starting from automatically detected sparse correspondences on the outer boundary of 3D faces, the algorithm triangulates existing correspondences and expands them iteratively by matching points of distinctive surface curvature along the triangle edges. After exhausting keypoint matches, further correspondences are established by generating evenly distributed points within triangles by evolving level set geodesic curves from the centroids of large triangles. A deformable model (K3DM) is constructed from the dense corresponded faces and an algorithm is proposed for morphing the K3DM to fit unseen faces. This algorithm iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model. We have extensively evaluated the proposed algorithms on synthetic data and real 3D faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using quantitative and qualitative benchmarks. Our algorithm achieved dense correspondences with a mean localisation error of 1.28 mm on synthetic faces and detected 14 anthropometric landmarks on unseen real faces from the FRGCv2 database with 3 mm precision. Furthermore, our deformable model fitting algorithm achieved 98.5 percent face recognition accuracy on the FRGCv2 and 98.6 percent on Bosphorus database. Our dense model is also able to generalize to unseen datasets.

Hypermasculinised facial morphology in boys and girls with Autism Spectrum Disorder and its association with symptomatology.

Elevated prenatal testosterone exposure has been associated with Autism Spectrum Disorder (ASD) and facial masculinity. By employing three-dimensional (3D) photogrammetry, the current study investigated whether prepubescent boys and girls with ASD present increased facial masculinity compared to typically-developing controls. There were two phases to this research. 3D facial images were obtained from a normative sample of 48 boys and 53 girls (3.01-12.44 years old) to determine typical facial masculinity/femininity. The sexually dimorphic features were used to create a continuous 'gender score', indexing degree of facial masculinity. Gender scores based on 3D facial images were then compared for 54 autistic and 54 control boys (3.01-12.52 years old), and also for 20 autistic and 60 control girls (4.24-11.78 years). For each sex, increased facial masculinity was observed in the ASD group relative to control group. Further analyses revealed that increased facial masculinity in the ASD group correlated with more social-communication difficulties based on the Social Affect score derived from the Autism Diagnostic Observation Scale-Generic (ADOS-G). There was no association between facial masculinity and the derived Restricted and Repetitive Behaviours score. This is the first study demonstrating facial hypermasculinisation in ASD and its relationship to social-communication difficulties in prepubescent children.

Prenatal testosterone exposure is related to sexually dimorphic facial morphology in adulthood.

Prenatal testosterone may have a powerful masculinizing effect on postnatal physical characteristics. However, no study has directly tested this hypothesis. Here, we report a 20-year follow-up study that measured testosterone concentrations from the umbilical cord blood of 97 male and 86 female newborns, and procured three-dimensional facial images on these participants in adulthood (range: 21-24 years). Twenty-three Euclidean and geodesic distances were measured from the facial images and an algorithm identified a set of six distances that most effectively distinguished adult males from females. From these distances, a 'gender score' was calculated for each face, indicating the degree of masculinity or femininity. Higher cord testosterone levels were associated with masculinized facial features when males and females were analysed together (n = 183; r = -0.59), as well as when males (n = 86; r = -0.55) and females (n = 97; r = -0.48) were examined separately (p-values < 0.001). The relationships remained significant and substantial after adjusting for potentially confounding variables. Adult circulating testosterone concentrations were available for males but showed no statistically significant relationship with gendered facial morphology (n = 85, r = 0.01, p = 0.93). This study provides the first direct evidence of a link between prenatal testosterone exposure and human facial structure.

Sexually dimorphic facial features vary according to level of autistic-like traits in the general population.

BACKGROUND: In a recent study, Bejerot et al. observed that several physical features (including faces) of individuals with an autism spectrum disorder (ASD) were more androgynous than those of their typically developed counterparts, suggesting that ASD may be understood as a 'gender defiant' disorder. These findings are difficult to reconcile with the hypermasculinisation account, which proposes that ASD may be an exaggerated form of cognitive and biological masculinity. The current study extended these data by first identifying six facial features that best distinguished males and females from the general population and then examining these features in typically developing groups selected for high and low levels of autistic-like traits. METHODS: In study 1, three-dimensional (3D) facial images were collected from 208 young adult males and females recruited from the general population. Twenty-three facial distances were measured from these images and a gender classification and scoring algorithm was employed to identify a set of six facial features that most effectively distinguished male from female faces. In study 2, measurements of these six features were compared for groups of young adults selected for high (n = 46) or low (n = 66) levels of autistic-like traits. RESULTS: For each sex, four of the six sexually dimorphic facial distances significantly differentiated participants with high levels of autistic-like traits from those with low trait levels. All four features were less masculinised for high-trait males compared to low-trait males. Three of four features were less feminised for high-trait females compared to low-trait females. One feature was, however, not consistent with the general pattern of findings and was more feminised among females who reported more autistic-like traits. Based on the four significantly different facial distances for each sex, discriminant function analysis correctly classified 89.7% of the males and 88.9% of the females into their respective high- and low-trait groups. CONCLUSIONS: The current data provide support for Bejerot et al.'s androgyny account since males and females with high levels of autistic-like traits generally showed less sex-typical facial features than individuals with low levels of autistic-like traits.

Geometric facial gender scoring: objectivity of perception.

Gender score is the cognitive judgement of the degree of masculinity or femininity of a face which is considered to be a continuum. Gender scores have long been used in psychological studies to understand the complex psychosocial relationships between people. Perceptual scores for gender and attractiveness have been employed for quality assessment and planning of cosmetic facial surgery. Various neurological disorders have been linked to the facial structure in general and the facial gender perception in particular. While, subjective gender scoring by human raters has been a tool of choice for psychological studies for many years, the process is both time and resource consuming. In this study, we investigate the geometric features used by the human cognitive system in perceiving the degree of masculinity/femininity of a 3D face. We then propose a mathematical model that can mimic the human gender perception. For our experiments, we obtained 3D face scans of 64 subjects using the 3dMDface scanner. The textureless 3D face scans of the subjects were then observed in different poses and assigned a gender score by 75 raters of a similar background. Our results suggest that the human cognitive system employs a combination of Euclidean and geodesic distances between biologically significant landmarks of the face for gender scoring. We propose a mathematical model that is able to automatically assign an objective gender score to a 3D face with a correlation of up to 0.895 with the human subjective scores.