A domain-agnostic approach for characterization of lifelong learning systems.

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.

Early diagnosis of acromegaly: computers vs clinicians.

BACKGROUND: Early diagnosis of a number of endocrine diseases is theoretically possible by the examination of facial photographs. One of these is acromegaly. If acromegaly were found, early in the course of the disease, morbidity would be lessened and cures more likely. OBJECTIVES, DESIGN, PATIENTS, MEASUREMENTS: Our objective was to develop a computer program which would separate 24 facial photographs, of patients with acromegaly, from those of 25 normal subjects. The key to doing this was to use a previously developed database that consisted of three-dimensional representations of 200 normal person's heads (SIGGRAPH '99 Conference Proceedings, 1999). We transformed our 49, two-dimensional photos into three-dimensional constructs and then, using the computer program, attempted to separate them into those with and without the features of acromegaly. We compared the accuracy of the computer to that of 10 generalist physicians. A second objective was to examine, by a subjective analysis, the features of acromegaly in the normal subjects of our photographic database. RESULTS: The accuracy of the computer model was 86%; the average of the 10 physicians was 26%. The worst individual physician, 16%, the best, 90%. The faces of 200 normal subjects, the original faces in the database, could be divided into four groups, averaged by computer, from those with fewer to those with more features of acromegaly. CONCLUSIONS: The present computer model can sort photographs of patients with acromegaly from photographs of normal subjects and is much more accurate than the sorting by practicing generalists. Even normal subjects have some of the features of acromegaly. Screening with this approach can be improved with automation of the procedure, software development and the identification of target populations in which the prevalence of acromegaly may be increased over that in the general population.

Image registration: Maximum likelihood, minimum entropy and deep learning.

In this work, we propose a theoretical framework based on maximum profile likelihood for pairwise and groupwise registration. By an asymptotic analysis, we demonstrate that maximum profile likelihood registration minimizes an upper bound on the joint entropy of the distribution that generates the joint image data. Further, we derive the congealing method for groupwise registration by optimizing the profile likelihood in closed form, and using coordinate ascent, or iterative model refinement. We also describe a method for feature based registration in the same framework and demonstrate it on groupwise tractographic registration. In the second part of the article, we propose an approach to deep metric registration that implements maximum likelihood registration using deep discriminative classifiers. We show further that this approach can be used for maximum profile likelihood registration to discharge the need for well-registered training data, using iterative model refinement. We demonstrate that the method succeeds on a challenging registration problem where the standard mutual information approach does not perform well.

Scene text recognition using similarity and a lexicon with sparse belief propagation.

Scene text recognition (STR) is the recognition of text anywhere in the environment, such as signs and storefronts. Relative to document recognition, it is challenging because of font variability, minimal language context, and uncontrolled conditions. Much information available to solve this problem is frequently ignored or used sequentially. Similarity between character images is often overlooked as useful information. Because of language priors, a recognizer may assign different labels to identical characters. Directly comparing characters to each other, rather than only a model, helps ensure that similar instances receive the same label. Lexicons improve recognition accuracy but are used post hoc. We introduce a probabilistic model for STR that integrates similarity, language properties, and lexical decision. Inference is accelerated with sparse belief propagation, a bottom-up method for shortening messages by reducing the dependency between weakly supported hypotheses. By fusing information sources in one model, we eliminate unrecoverable errors that result from sequential processing, improving accuracy. In experimental results recognizing text from images of signs in outdoor scenes, incorporating similarity reduces character recognition error by 19 percent, the lexicon reduces word recognition error by 35 percent, and sparse belief propagation reduces the lexicon words considered by 99.9 percent with a 12X speedup and no loss in accuracy.

Analyzing in situ gene expression in the mouse brain with image registration, feature extraction and block clustering.

BACKGROUND: Many important high throughput projects use in situ hybridization and may require the analysis of images of spatial cross sections of organisms taken with cellular level resolution. Projects creating gene expression atlases at unprecedented scales for the embryonic fruit fly as well as the embryonic and adult mouse already involve the analysis of hundreds of thousands of high resolution experimental images mapping mRNA expression patterns. Challenges include accurate registration of highly deformed tissues, associating cells with known anatomical regions, and identifying groups of genes whose expression is coordinately regulated with respect to both concentration and spatial location. Solutions to these and other challenges will lead to a richer understanding of the complex system aspects of gene regulation in heterogeneous tissue. RESULTS: We present an end-to-end approach for processing raw in situ expression imagery and performing subsequent analysis. We use a non-linear, information theoretic based image registration technique specifically adapted for mapping expression images to anatomical annotations and a method for extracting expression information within an anatomical region. Our method consists of coarse registration, fine registration, and expression feature extraction steps. From this we obtain a matrix for expression characteristics with rows corresponding to genes and columns corresponding to anatomical sub-structures. We perform matrix block cluster analysis using a novel row-column mixture model and we relate clustered patterns to Gene Ontology (GO) annotations. CONCLUSION: Resulting registrations suggest that our method is robust over intensity levels and shape variations in ISH imagery. Functional enrichment studies from both simple analysis and block clustering indicate that gene relationships consistent with biological knowledge of neuronal gene functions can be extracted from large ISH image databases such as the Allen Brain Atlas 1 and the Max-Planck Institute 2 using our method. While we focus here on imagery and experiments of the mouse brain our approach should be applicable to a variety of in situ experiments.

Imaging cellulose synthase motility during primary cell wall synthesis in the grass Brachypodium distachyon.

The mechanism of cellulose synthesis has been studied by characterizing the motility of cellulose synthase complexes tagged with a fluorescent protein; however, this approach has been used exclusively on the hypocotyl of Arabidopsis thaliana. Here we characterize cellulose synthase motility in the model grass, Brachypodium distachyon. We generated lines in which mEGFP is fused N-terminal to BdCESA3 or BdCESA6 and which grew indistinguishably from the wild type (Bd21-3) and had dense fluorescent puncta at or near the plasma membrane. Measured with a particle tracking algorithm, the average speed of GFP-BdCESA3 particles in the mesocotyl was 164 ± 78 nm min-1 (error gives standard deviation [SD], n = 1451 particles). Mean speed in the root appeared similar. For comparison, average speed in the A. thaliana hypocotyl expressing GFP-AtCESA6 was 184 ± 86 nm min-1 (n = 2755). For B. distachyon, we quantified root diameter and elongation rate in response to inhibitors of cellulose (dichlorobenylnitrile; DCB), microtubules (oryzalin), or actin (latrunculin B). Neither oryzalin nor latrunculin affected the speed of CESA complexes; whereas, DCB reduced average speed by about 50% in B. distachyon and by about 35% in A. thaliana. Evidently, between these species, CESA motility is well conserved.

Data driven image models through continuous joint alignment.

This paper presents a family of techniques that we call congealing for modeling image classes from data. The idea is to start with a set of images and make them appear as similar as possible by removing variability along the known axes of variation. This technique can be used to eliminate "nuisance" variables such as affine deformations from handwritten digits or unwanted bias fields from magnetic resonance images. In addition to separating and modeling the latent images-i.e., the images without the nuisance variables-we can model the nuisance variables themselves, leading to factorized generative image models. When nuisance variable distributions are shared between classes, one can share the knowledge learned in one task with another task, leading to efficient learning. We demonstrate this process by building a handwritten digit classifier from just a single example of each class. In addition to applications in handwritten character recognition, we describe in detail the application of bias removal from magnetic resonance images. Unlike previous methods, we use a separate, nonparametric model for the intensity values at each pixel. This allows us to leverage the data from the MR images of different patients to remove bias from each other. Only very weak assumptions are made about the distributions of intensity values in the images. In addition to the digit and MR applications, we discuss a number of other uses of congealing and describe experiments about the robustness and consistency of the method.

DeTEXT: A Database for Evaluating Text Extraction from Biomedical Literature Figures.

Hundreds of millions of figures are available in biomedical literature, representing important biomedical experimental evidence. Since text is a rich source of information in figures, automatically extracting such text may assist in the task of mining figure information. A high-quality ground truth standard can greatly facilitate the development of an automated system. This article describes DeTEXT: A database for evaluating text extraction from biomedical literature figures. It is the first publicly available, human-annotated, high quality, and large-scale figure-text dataset with 288 full-text articles, 500 biomedical figures, and 9308 text regions. This article describes how figures were selected from open-access full-text biomedical articles and how annotation guidelines and annotation tools were developed. We also discuss the inter-annotator agreement and the reliability of the annotations. We summarize the statistics of the DeTEXT data and make available evaluation protocols for DeTEXT. Finally we lay out challenges we observed in the automated detection and recognition of figure text and discuss research directions in this area. DeTEXT is publicly available for downloading at http://prir.ustb.edu.cn/DeTEXT/.

Detecting acromegaly: screening for disease with a morphable model.

Acromegaly is a rare disorder which affects about 50 of every million people. The disease typically causes swelling of the hands, feet, and face, and eventually permanent changes to areas such as the jaw, brow ridge, and cheek bones. The disease is often missed by physicians and progresses beyond where it might if it were identified and treated earlier. We consider a semi-automated approach to detecting acromegaly, using a novel combination of support vector machines (SVMs) and a morphable model. Our training set consists of 24 frontal photographs of acromegalic patients and 25 of disease-free subjects. We modelled each subject's face in an analysis-by-synthesis loop using the three-dimensional morphable face model of Blanz and Vetter. The model parameters capture many features of the 3D shape of the subject's head from just a single photograph, and are used directly for classification. We report encouraging results of a classifier built from the training set of real human subjects.

Many heads are better than one: jointly removing bias from multiple MRIs using nonparametric maximum likelihood.

The correction of multiplicative bias in magnetic resonance images is an important problem in medical image processing, especially as a preprocessing step for quantitative measurements and other numerical procedures. Most previous approaches have used a maximum likelihood method to increase the probability of the pixels in a single image by adaptively estimating a correction to the unknown image bias field. The pixel probabilities are defined either in terms of a pre-existing tissue model, or nonparametrically in terms of the image's own pixel values. In both cases, the specific location of a pixel in the image does not influence the probability calculation. Our approach, similar to methods of joint registration, simultaneously eliminates the bias from a set of images of the same anatomy, but from different patients. We use the statistics from the same location across different patients' images, rather than within an image, to eliminate bias fields from all of the images simultaneously. Evaluating the likelihood of a particular voxel in one patient's scan with respect to voxels in the same location in a set of other patients' scans disambiguates effects that might be due to either bias fields or anatomy. We present a variety of "two-dimensional" experimental results (working with one image from each patient) showing how our method overcomes serious problems experienced by other methods. We also present preliminary results on full three-dimensional volume correction across patients.