Automated syndrome detection in a set of clinical facial photographs.

Computer systems play an important role in clinical genetics and are a routine part of finding clinical diagnoses but make it difficult to fully exploit information derived from facial appearance. So far, automated syndrome diagnosis based on digital, facial photographs has been demonstrated under study conditions but has not been applied in clinical practice. We have therefore investigated how well statistical classifiers trained on study data comprising 202 individuals affected by one of 14 syndromes could classify a set of 91 patients for whom pictures were taken under regular, less controlled conditions in clinical practice. We found a classification accuracy of 21% percent in the clinical sample representing a ratio of 3.0 over a random choice. This contrasts with a 60% accuracy or 8.5 ratio in the training data. Producing average images in both groups from sets of pictures for each syndrome demonstrates that the groups exhibit large phenotypic differences explaining discrepancies in accuracy. A broadening of the data set is suggested in order to improve accuracy in clinical practice. In order to further this goal, a software package is made available that allows application of the procedures and contributions toward an improved data set.

Computer Vision Technology in the Differential Diagnosis of Cushing's Syndrome.

OBJECTIVE: Cushing's syndrome is a rare disease characterized by clinical features that show morphological similarity with the metabolic syndrome. Distinguishing these diseases in clinical practice is challenging. We have previously shown that computer vision technology can be a potentially useful diagnostic tool in Cushing's syndrome. In this follow-up study, we addressed the described problem by increasing the sample size and including controls matched by body mass index. METHODS: We enrolled 82 patients (22 male, 60 female) and 98 control subjects (32 male, 66 female) matched by age, gender and body-mass-index. The control group consisted of patients with initially suspected, but biochemically excluded Cushing's syndrome. Standardized frontal and profile facial digital photographs were acquired. The images were analyzed using specialized computer vision and classification software. A grid of nodes was semi-automatically placed on disease-relevant facial structures for analysis of texture and geometry. Classification accuracy was calculated using a leave-one-out cross-validation procedure with a maximum likelihood classifier. RESULTS: The overall correct classification rates were 10/22 (45.5%) for male patients and 26/32 (81.3%) for male controls, and 34/60 (56.7%) for female patients and 43/66 (65.2%) for female controls. In subgroup analyses, correct classification rates were higher for iatrogenic than for endogenous Cushing's syndrome. CONCLUSION: Regarding the advanced problem of detecting Cushing's syndrome within a study sample matched by body mass index, we found moderate classification accuracy by facial image analysis. Classification accuracy is most likely higher in a larger sample with healthy control subjects. Further studies might pursue a more advanced analysis and classification algorithm.

Impact of geometry and viewing angle on classification accuracy of 2D based analysis of dysmorphic faces.

Digital image analysis of faces has been demonstrated to be effective in a small number of syndromes. In this paper we investigate several aspects that help bringing these methods closer to clinical application. First, we investigate the impact of increasing the number of syndromes from 10 to 14 as compared to an earlier study. Second, we include a side-view pose into the analysis and third, we scrutinize the effect of geometry information. Picture analysis uses a Gabor wavelet transform, standardization of landmark coordinates and subsequent statistical analysis. We can demonstrate that classification accuracy drops from 76% for 10 syndromes to 70% for 14 syndromes for frontal images. Including side-views achieves an accuracy of 76% again. Geometry performs excellently with 85% for combined poses. Combination of wavelets and geometry for both poses increases accuracy to 93%. In conclusion, a larger number of syndromes can be handled effectively by means of image analysis.

Syndrome identification based on 2D analysis software.

Clinical evaluation of children with developmental delay continues to present a challenge to the clinicians. In many cases, the face provides important information to diagnose a condition. However, database support with respect to facial traits is limited at present. Computer-based analyses of 2D and 3D representations of faces have been developed, but it is unclear how well a larger number of conditions can be handled by such systems. We have therefore analysed 2D pictures of patients each being affected with one of 10 syndromes (fragile X syndrome; Cornelia de Lange syndrome; Williams-Beuren syndrome; Prader-Willi syndrome; Mucopolysaccharidosis type III; Cri-du-chat syndrome; Smith-Lemli-Opitz syndrome; Sotos syndrome; Microdeletion 22q11.2; Noonan syndrome). We can show that a classification accuracy of >75% can be achieved for a computer-based diagnosis among the 10 syndromes, which is about the same accuracy achieved for five syndromes in a previous study. Pairwise discrimination of syndromes ranges from 80 to 99%. Furthermore, we can demonstrate that the criteria used by the computer decisions match clinical observations in many cases. These findings indicate that computer-based picture analysis might be a helpful addition to existing database systems, which are meant to assist in syndrome diagnosis, especially as data acquisition is straightforward and involves off-the-shelf digital camera equipment.

Computer-based recognition of dysmorphic faces.

Genetic syndromes often involve craniofacial malformations. We have investigated whether a computer can recognize disease-specific facial patterns in unrelated individuals. For this, 55 photographs (256 x 256 pixel) of patients with mucopolysaccharidosis type III (n=6), Cornelia de Lange (n=12), fragile X (n=12), Prader-Willi (n=12), and Williams-Beuren (n=13) syndromes were preprocessed by a Gabor wavelet transformation. By comparing the feature vectors at 32 facial nodes, 42/55 (76%) of the patients were correctly classified. In another four patients (7%), the correct and an incorrect diagnosis scored equally well. Clinical geneticists who were shown the same photographs achieved a recognition rate of 62%. Our results prove that certain syndromes are associated with a specific facial pattern and that this pattern can be described in mathematical terms.

Learning invariant object recognition from temporal correlation in a hierarchical network.

Invariant object recognition, which means the recognition of object categories independent of conditions like viewing angle, scale and illumination, is a task of great interest that humans can fulfill much better than artificial systems. During the last years several basic principles were derived from neurophysiological observations and careful consideration: (1) Developing invariance to possible transformations of the object by learning temporal sequences of visual features that occur during the respective alterations. (2) Learning in a hierarchical structure, so basic level (visual) knowledge can be reused for different kinds of objects. (3) Using feedback to compare predicted input with the current one for choosing an interpretation in the case of ambiguous signals. In this paper we propose a network which implements all of these concepts in a computationally efficient manner which gives very good results on standard object datasets. By dynamically switching off weakly active neurons and pruning weights computation is sped up and thus handling of large databases with several thousands of images and a number of categories in a similar order becomes possible. The involved parameters allow flexible adaptation to the information content of training data and allow tuning to different databases relatively easily. Precondition for successful learning is that training images are presented in an order assuring that images of the same object under similar viewing conditions follow each other. Through an implementation with sparse data structures the system has moderate memory demands and still yields very good recognition rates.

Classification and visualization based on derived image features: application to genetic syndromes.

Data transformations prior to analysis may be beneficial in classification tasks. In this article we investigate a set of such transformations on 2D graph-data derived from facial images and their effect on classification accuracy in a high-dimensional setting. These transformations are low-variance in the sense that each involves only a fixed small number of input features. We show that classification accuracy can be improved when penalized regression techniques are employed, as compared to a principal component analysis (PCA) pre-processing step. In our data example classification accuracy improves from 47% to 62% when switching from PCA to penalized regression. A second goal is to visualize the resulting classifiers. We develop importance plots highlighting the influence of coordinates in the original 2D space. Features used for classification are mapped to coordinates in the original images and combined into an importance measure for each pixel. These plots assist in assessing plausibility of classifiers, interpretation of classifiers, and determination of the relative importance of different features.

Learning invariant face recognition from examples.

Autonomous learning is demonstrated by living beings that learn visual invariances during their visual experience. Standard neural network models do not show this sort of learning. On the example of face recognition in different situations we propose a learning process that separates learning of the invariance proper from learning new instances of individuals. The invariance is learned by a set of examples called model, which contains instances of all situations. New instances are compared with these on the basis of rank lists, which allow generalization across situations. The result is also implemented as a spike-time-based neural network, which is shown to be robust against disturbances. The learning capability is demonstrated by recognition experiments on a set of standard face databases.

A novel approach to the detection of acromegaly: accuracy of diagnosis by automatic face classification.

CONTEXT: The delay between onset of first symptoms and diagnosis of the acromegaly is 6-10 yr. Acromegaly causes typical changes of the face that might be recognized by face classification software. OBJECTIVE: The objective of the study was to assess classification accuracy of acromegaly by face-classification software. DESIGN: This was a diagnostic study. SETTING: The study was conducted in specialized care. PARTICIPANTS: Participants in the study included 57 patients with acromegaly (29 women, 28 men) and 60 sex- and age-matched controls. INTERVENTIONS: We took frontal and side photographs of the faces and grouped patients into subjects with mild, moderate, and severe facial features of acromegaly by overall impression. We then analyzed all pictures using computerized similarity analysis based on Gabor jets and geometry functions. We used the leave-one-out cross-validation method to classify subjects by the software. Additionally, all subjects were classified by visual impression by three acromegaly experts and three general internists. MAIN OUTCOME MEASURE: Classification accuracy by software, experts, and internists was measured. FINDINGS: The software correctly classified 71.9% of patients and 91.5% of controls. Classification accuracy for patients by visual analysis was 63.2 and 42.1% by experts and general internists, respectively. Classification accuracy for controls was 80.8 and 87.0% by experts and internists, respectively. The highest differences in accuracy between software and experts and internists were present for patients with mild acromegaly. CONCLUSIONS: Acromegaly can be detected by computer software using photographs of the face. Classification accuracy by software is higher than by medical experts or general internists, particularly in patients with mild features of acromegaly. This is a promising tool to help detecting acromegaly.

Genetic determination of human facial morphology: links between cleft-lips and normal variation.

Recent genome-wide association studies have identified single nucleotide polymorphisms (SNPs) associated with non-syndromic cleft lip with or without cleft palate (NSCL/P), and other previous studies showed distinctly differing facial distance measurements when comparing unaffected relatives of NSCL/P patients with normal controls. Here, we test the hypothesis that genetic loci involved in NSCL/P also influence normal variation in facial morphology. We tested 11 SNPs from 10 genomic regions previously showing replicated evidence of association with NSCL/P for association with normal variation of nose width and bizygomatic distance in two cohorts from Germany (N=529) and the Netherlands (N=2497). The two most significant associations found were between nose width and SNP rs1258763 near the GREM1 gene in the German cohort (P=6 × 10(-4)), and between bizygomatic distance and SNP rs987525 at 8q24.21 near the CCDC26 gene (P=0.017) in the Dutch sample. A genetic prediction model explained 2% of phenotype variation in nose width in the German and 0.5% of bizygomatic distance variation in the Dutch cohort. Although preliminary, our data provide a first link between genetic loci involved in a pathological facial trait such as NSCL/P and variation of normal facial morphology. Moreover, we present a first approach for understanding the genetic basis of human facial appearance, a highly intriguing trait with implications on clinical practice, clinical genetics, forensic intelligence, social interactions and personal identity.

Combining feature- and correspondence-based methods for visual object recognition.

We present an object recognition system built on a combination of feature- and correspondence-based pattern recognizers. The feature-based part, called preselection network, is a single-layer feedforward network weighted with the amount of information contributed by each feature to the decision at hand. For processing arbitrary objects, we employ small, regular graphs whose nodes are attributed with Gabor amplitudes, termed parquet graphs. The preselection network can quickly rule out most irrelevant matches and leaves only the ambiguous cases, so-called model candidates, to be verified by a rudimentary version of elastic graph matching, a standard correspondence-based technique for face and object recognition. According to the model, graphs are constructed that describe the object in the input image well. We report the results of experiments on standard databases for object recognition. The method achieved high recognition rates on identity and pose. Unlike many other models, it can also cope with varying background, multiple objects, and partial occlusion.

Learning the Gestalt rule of collinearity from object motion.

The Gestalt principle of collinearity (and curvilinearity) is widely regarded as being mediated by the long-range connection structure in primary visual cortex. We review the neurophysiological and psychophysical literature to argue that these connections are developed from visual experience after birth, relying on coherent object motion. We then present a neural network model that learns these connections in an unsupervised Hebbian fashion with input from real camera sequences. The model uses spatiotemporal retinal filtering, which is very sensitive to changes in the visual input. We show that it is crucial for successful learning to use the correlation of the transient responses instead of the sustained ones. As a consequence, learning works best with video sequences of moving objects. The model addresses a special case of the fundamental question of what represents the necessary a priori knowledge the brain is equipped with at birth so that the self-organized process of structuring by experience can be successful.

On the performance of neuronal matching algorithms.

For a solution of the visual correspondence problem we have modified the Self Organizing Map (SOM) to map image planes onto another in a neighborhood- and feature-preserving way. We have investigated the convergence speed of this SOM and Dynamic Link Matching (DLM) on a benchmark problem for the solution of which both algorithms are good candidates. We show that even after careful parameter adjustment the SOM needs a large number of simple update steps and DLM a small number of complicated ones. The results are consistent with an exponential vs. polynomial scaling behavior with increased pattern size. Finally, we present and motivate a rule for adjusting the parameters of DLM for all problem sizes, which we could not find for SOM.

Image representation by complex cell responses.

We present an analysis of the representation of images as the magnitudes of their transform with complex-valued Gabor wavelets. Such a representation is a model for complex cells in the early stage of visual processing and of high technical usefulness for image understanding, because it makes the representation insensitive to small local shifts. We show that if the images are band limited and of zero mean, then reconstruction from the magnitudes is unique up to the sign for almost all images.