The definitions of three-dimensional landmarks on the human face: an interdisciplinary view.

The analysis of shape is a key part of anatomical research and in the large majority of cases landmarks provide a standard starting point. However, while the technology of image capture has developed rapidly and in particular three-dimensional imaging is widely available, the definitions of anatomical landmarks remain rooted in their two-dimensional origins. In the important case of the human face, standard definitions often require careful orientation of the subject. This paper considers the definitions of facial landmarks from an interdisciplinary perspective, including biological and clinical motivations, issues associated with imaging and subsequent analysis, and the mathematical definition of surface shape using differential geometry. This last perspective provides a route to definitions of landmarks based on surface curvature, often making use of ridge and valley curves, which is genuinely three-dimensional and is independent of orientation. Specific definitions based on curvature are proposed. These are evaluated, along with traditional definitions, in a study that uses a hierarchical (random effects) model to estimate the error variation that is present at several different levels within the image capture process. The estimates of variation at these different levels are of interest in their own right but, in addition, evidence is provided that variation is reduced at the observer level when the new landmark definitions are used.

Accuracy and repeatability of fetal facial measurements in 3D ultrasound: A longitudinal study.

OBJECTIVE: Fetal face measurements in prenatal ultrasound can aid in identifying craniofacial abnormalities in the developing fetus. However, the accuracy and reliability of ultrasound measurements can be affected by factors such as fetal position, image quality, and the sonographer's expertise. This study assesses the accuracy and reliability of fetal facial measurements in prenatal ultrasound. Additionally, the temporal evolution of measurements is studied, comparing prenatal and postnatal measurements. METHODS: Three different experts located up to 23 facial landmarks in 49 prenatal 3D ultrasound scans from normal Caucasian fetuses at weeks 20, 26, and 35 of gestation. Intra- and inter-observer variability was obtained. Postnatal facial measurements were also obtained at 15 days and 1 month postpartum. RESULTS: Most facial landmarks exhibited low errors, with overall intra- and inter-observer errors of 1.01 mm and 1.60 mm, respectively. Landmarks on the nose were found to be the most reliable, while the most challenging ones were those located on the ears and eyes. Overall, scans obtained at 26 weeks of gestation presented the best trade-off between observer variability and landmark visibility. The temporal evolution of the measurements revealed that the lower face area had the highest rate of growth throughout the latest stages of pregnancy. CONCLUSIONS: Craniofacial landmarks can be evaluated using 3D fetal ultrasound, especially those located on the nose, mouth, and chin. Despite its limitations, this study provides valuable insights into prenatal and postnatal biometric changes over time, which could aid in developing predictive models for postnatal measurements based on prenatal data.

Reconstruction of the fetus face from three-dimensional ultrasound using a newborn face statistical shape model.

BACKGROUND AND OBJECTIVE: The fetal face is an essential source of information in the assessment of congenital malformations and neurological anomalies. Disturbance in early stages of development can lead to a wide range of effects, from subtle changes in facial and neurological features to characteristic facial shapes observed in craniofacial syndromes. Three-dimensional ultrasound (3D US) can provide more detailed information about the facial morphology of the fetus than the conventional 2D US, but its use for pre-natal diagnosis is challenging due to imaging noise, fetal movements, limited field-of-view, low soft-tissue contrast, and occlusions. METHODS: In this paper, we propose the use of a novel statistical morphable model of newborn faces, the BabyFM, for fetal face reconstruction from 3D US images. We test the feasibility of using newborn statistics to accurately reconstruct fetal faces by fitting the regularized morphable model to the noisy 3D US images. RESULTS: The results indicate that the reconstructions are quite accurate in the central-face and less reliable in the lateral regions (mean point-to-surface error of 2.35 mm vs 4.86 mm). The algorithm is able to reconstruct the whole facial morphology of babies from US scans while handle adverse conditions (e.g. missing parts, noisy data). CONCLUSIONS: The proposed algorithm has the potential to aid in-utero diagnosis for conditions that involve facial dysmorphology.

Look-alike humans identified by facial recognition algorithms show genetic similarities.

The human face is one of the most visible features of our unique identity as individuals. Interestingly, monozygotic twins share almost identical facial traits and the same DNA sequence but could exhibit differences in other biometrical parameters. The expansion of the world wide web and the possibility to exchange pictures of humans across the planet has increased the number of people identified online as virtual twins or doubles that are not family related. Herein, we have characterized in detail a set of "look-alike" humans, defined by facial recognition algorithms, for their multiomics landscape. We report that these individuals share similar genotypes and differ in their DNA methylation and microbiome landscape. These results not only provide insights about the genetics that determine our face but also might have implications for the establishment of other human anthropometric properties and even personality characteristics.

Refining the resolution of craniofacial dysmorphology in bipolar disorder as an index of brain dysmorphogenesis.

As understanding of the genetics of bipolar disorder increases, controversy endures regarding whether the origins of this illness include early maldevelopment. Clarification would be facilitated by a 'hard' biological index of fetal developmental abnormality, among which craniofacial dysmorphology bears the closest embryological relationship to brain dysmorphogenesis. Therefore, 3D laser surface imaging was used to capture the facial surface of 21 patients with bipolar disorder and 45 control subjects; 21 patients with schizophrenia were also studied. Surface images were subjected to geometric morphometric analysis in non-affine space for more incisive resolution of subtle, localised dysmorphologies that might distinguish patients from controls. Complex and more biologically informative, non-linear changes distinguished bipolar patients from control subjects. On a background of minor dysmorphology of the upper face, maxilla, midface and periorbital regions, bipolar disorder was characterised primarily by the following dysmorphologies: (a) retrusion and shortening of the premaxilla, nose, philtrum, lips and mouth (the frontonasal prominences), with (b) some protrusion and widening of the mandible-chin. The topography of facial dysmorphology in bipolar disorder indicates disruption to early development in the frontonasal process and, on embryological grounds, cerebral dysmorphogenesis in the forebrain, most likely between the 10th and 15th week of fetal life.

Reliability estimation for statistical shape models.

One of the drawbacks of statistical shape models is their occasional failure to converge. Although visually this fact is usually easy to recognize, there is no automatic way to detect it. In this paper, we introduce a generic reliability measure for statistical shape models. It is based on a probabilistic framework and uses information extracted by the model itself during the matching process. The proposed method was validated with two variants of Active Shape Models in the context facial image analysis. Experimental results on more than 3700 facial images showed a high degree of correlation between the segmentation accuracy and the estimated reliability metric.

Weighted regularized statistical shape space projection for breast 3D model reconstruction.

The use of 3D imaging has increased as a practical and useful tool for plastic and aesthetic surgery planning. Specifically, the possibility of representing the patient breast anatomy in a 3D shape and simulate aesthetic or plastic procedures is a great tool for communication between surgeon and patient during surgery planning. For the purpose of obtaining the specific 3D model of the breast of a patient, model-based reconstruction methods can be used. In particular, 3D morphable models (3DMM) are a robust and widely used method to perform 3D reconstruction. However, if additional prior information (i.e., known landmarks) is combined with the 3DMM statistical model, shape constraints can be imposed to improve the 3DMM fitting accuracy. In this paper, we present a framework to fit a 3DMM of the breast to two possible inputs: 2D photos and 3D point clouds (scans). Our method consists in a Weighted Regularized (WR) projection into the shape space. The contribution of each point in the 3DMM shape is weighted allowing to assign more relevance to those points that we want to impose as constraints. Our method is applied at multiple stages of the 3D reconstruction process. Firstly, it can be used to obtain a 3DMM initialization from a sparse set of 3D points. Additionally, we embed our method in the 3DMM fitting process in which more reliable or already known 3D points or regions of points, can be weighted in order to preserve their shape information. The proposed method has been tested in two different input settings: scans and 2D pictures assessing both reconstruction frameworks with very positive results.

Active shape models with invariant optimal features: application to facial analysis.

This work is framed in the field of statistical face analysis. In particular, the problem of accurate segmentation of prominent features of the face in frontal view images is addressed. We propose a method that generalizes linear Active Shape Models (ASMs), which have already been used for this task. The technique is built upon the development of a nonlinear intensity model, incorporating a reduced set of differential invariant features as local image descriptors. These features are invariant to rigid transformations, and a subset of them is chosen by Sequential Feature Selection for each landmark and resolution level. The new approach overcomes the unimodality and Gaussianity assumptions of classical ASMs regarding the distribution of the intensity values across the training set. Our methodology has demonstrated a significant improvement in segmentation precision as compared to the linear ASM and Optimal Features ASM (a nonlinear extension of the pioneer algorithm) in the tests performed on AR, XM2VTS, and EQUINOX databases.

3-D Facial Landmark Localization With Asymmetry Patterns and Shape Regression from Incomplete Local Features.

We present a method for the automatic localization of facial landmarks that integrates nonrigid deformation with the ability to handle missing points. The algorithm generates sets of candidate locations from feature detectors and performs combinatorial search constrained by a flexible shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing, so that the probability of the flexible model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, drastically reducing the number of combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in the face recognition grand challenge database, where we obtain average errors of approximately 3.5 mm when targeting 14 prominent facial landmarks. For the majority of these our method produces the most accurate results reported to date in this database. Handling of occlusions and surfaces with missing parts is demonstrated with tests on the Bosphorus database, where we achieve an overall error of 4.81 and 4.25 mm for data with and without occlusions, respectively. To investigate potential limits in the accuracy that could be reached, we also report experiments on a database of 144 facial scans acquired in the context of clinical research, with manual annotations performed by experts, where we obtain an overall error of 2.3 mm, with averages per landmark below 3.4 mm for all 14 targeted points and within 2 mm for half of them. The coordinates of automatically located landmarks are made available on-line.

A high-resolution atlas and statistical model of the human heart from multislice CT.

Atlases and statistical models play important roles in the personalization and simulation of cardiac physiology. For the study of the heart, however, the construction of comprehensive atlases and spatio-temporal models is faced with a number of challenges, in particular the need to handle large and highly variable image datasets, the multi-region nature of the heart, and the presence of complex as well as small cardiovascular structures. In this paper, we present a detailed atlas and spatio-temporal statistical model of the human heart based on a large population of 3D+time multi-slice computed tomography sequences, and the framework for its construction. It uses spatial normalization based on nonrigid image registration to synthesize a population mean image and establish the spatial relationships between the mean and the subjects in the population. Temporal image registration is then applied to resolve each subject-specific cardiac motion and the resulting transformations are used to warp a surface mesh representation of the atlas to fit the images of the remaining cardiac phases in each subject. Subsequently, we demonstrate the construction of a spatio-temporal statistical model of shape such that the inter-subject and dynamic sources of variation are suitably separated. The framework is applied to a 3D+time data set of 138 subjects. The data is drawn from a variety of pathologies, which benefits its generalization to new subjects and physiological studies. The obtained level of detail and the extendability of the atlas present an advantage over most cardiac models published previously.

Automatic training and reliability estimation for 3D ASM applied to cardiac MRI segmentation.

Training active shape models requires collecting manual ground-truth meshes in a large image database. While shape information can be reused across multiple imaging modalities, intensity information needs to be imaging modality and protocol specific. In this context, this study has two main purposes: (1) to test the potential of using intensity models learned from MRI simulated datasets and (2) to test the potential of including a measure of reliability during the matching process to increase robustness. We used a population of 400 virtual subjects (XCAT phantom), and two clinical populations of 40 and 45 subjects. Virtual subjects were used to generate simulated datasets (MRISIM simulator). Intensity models were trained both on simulated and real datasets. The trained models were used to segment the left ventricle (LV) and right ventricle (RV) from real datasets. Segmentations were also obtained with and without reliability information. Performance was evaluated with point-to-surface and volume errors. Simulated intensity models obtained average accuracy comparable to inter-observer variability for LV segmentation. The inclusion of reliability information reduced volume errors in hypertrophic patients (EF errors from 17 ± 57% to 10 ± 18%; LV MASS errors from -27 ± 22 g to -14 ± 25 g), and in heart failure patients (EF errors from -8 ± 42% to -5 ± 14%). The RV model of the simulated images needs further improvement to better resemble image intensities around the myocardial edges. Both for real and simulated models, reliability information increased segmentation robustness without penalizing accuracy.

Automatic cardiac MRI segmentation using a biventricular deformable medial model.

We present a novel approach for automatic segmentation of the myocardium in short-axis MRI using deformable medial models with an explicit representation of thickness. Segmentation is constrained by a Markov prior on myocardial thickness. Best practices from Active Shape Modeling (global PCA shape prior, statistical appearance model, local search) are adapted to the medial model. Segmentation performance is evaluated by comparing to manual segmentation in a heterogeneous adult MRI dataset. Average boundary displacement error is under 1.4 mm for left and right ventricles, comparing favorably with published work.

The multiscenario multienvironment BioSecure Multimodal Database (BMDB).

A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1) over the Internet, 2) in an office environment with desktop PC, and 3) in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008.

Cardiac medial modeling and time-course heart wall thickness analysis.

The medial model is a powerful shape representation method that models a 3D object by explicitly defining its skeleton (medial axis) and deriving the boundary geometry according to medial geometry. It has been recently extended to model complex shapes with multi-figures, i.e., shapes whose skeletons can not be described by a single sheet in 3D. This paper applied the medial model to a 2-chamber heart data set consisting of 428 cardiac shapes from 90 subjects. The results show that the medial model can capture the heart shape accurately. To demonstrate the usage of the medial model, the changes of the heart wall thickness over time are analyzed. We calculated the mean heart wall thickness map of 90 subjects for different phases of the cardiac cycle, as well as the mean thickness change between phases.

Automatic construction of 3D-ASM intensity models by simulating image acquisition: application to myocardial gated SPECT studies.

Active shape models bear a great promise for model-based medical image analysis. Their practical use, though, is undermined due to the need to train such models on large image databases. Automatic building of point distribution models (PDMs) has been successfully addressed and a number of autolandmarking techniques are currently available. However, the need for strategies to automatically build intensity models around each landmark has been largely overlooked in the literature. This work demonstrates the potential of creating intensity models automatically by simulating image generation. We show that it is possible to reuse a 3D PDM built from computed tomography (CT) to segment gated single photon emission computed tomography (gSPECT) studies. Training is performed on a realistic virtual population where image acquisition and formation have been modeled using the SIMIND Monte Carlo simulator and ASPIRE image reconstruction software, respectively. The dataset comprised 208 digital phantoms (4D-NCAT) and 20 clinical studies. The evaluation is accomplished by comparing point-to-surface and volume errors against a proper gold standard. Results show that gSPECT studies can be successfully segmented by models trained under this scheme with subvoxel accuracy. The accuracy in estimated LV function parameters, such as end diastolic volume, end systolic volume, and ejection fraction, ranged from 90.0% to 94.5% for the virtual population and from 87.0% to 89.5% for the clinical population.