Trifocal Relative Pose From Lines at Points.

We present a method for solving two minimal problems for relative camera pose estimation from three views, which are based on three view correspondences of (i) three points and one line and the novel case of (ii) three points and two lines through two of the points. These problems are too difficult to be efficiently solved by the state of the art Gröbner basis methods. Our method is based on a new efficient homotopy continuation (HC) solver framework MINUS, which dramatically speeds up previous HC solving by specializing hc methods to generic cases of our problems. We characterize their number of solutions and show with simulated experiments that our solvers are numerically robust and stable under image noise, a key contribution given the borderline intractable degree of nonlinearity of trinocular constraints. We show in real experiments that (i) sift feature location and orientation provide good enough point-and-line correspondences for three-view reconstruction and (ii) that we can solve difficult cases with too few or too noisy tentative matches, where the state of the art structure from motion initialization fails.

Camera Pose Estimation Using First-Order Curve Differential Geometry.

This paper considers and solves the problem of estimating camera pose given a pair of point-tangent correspondences between a 3D scene and a projected image. The problem arises when considering curve geometry as the basis of forming correspondences, computation of structure and calibration, which in its simplest form is a point augmented with the curve tangent. We show that while the resectioning problem is solved with a minimum of three points given the intrinsic parameters, when points are augmented with tangent information only two points are required, leading to substantial robustness and computational savings, e.g., as a minimal engine within ransac. In addition, algorithms are developed to find a practical solution shown to effectively recover camera pose using synthetic and real datasets. This technology is intended as a building block of curve-based structure from motion systems, allowing new views to be incrementally registered to a core set of views for which relative pose has been computed.

Discriminating between sleep and exercise-induced fatigue using computer vision and behavioral genetics.

Following prolonged swimming, Caenorhabditis elegans cycle between active swimming bouts and inactive quiescent bouts. Swimming is exercise for C. elegans and here we suggest that inactive bouts are a recovery state akin to fatigue. It is known that cGMP-dependent kinase (PKG) activity plays a conserved role in sleep, rest, and arousal. Using C. elegans EGL-4 PKG, we first validate a novel learning-based computer vision approach to automatically analyze C. elegans locomotory behavior and an edge detection program that is able to distinguish between activity and inactivity during swimming for long periods of time. We find that C. elegans EGL-4 PKG function impacts timing of exercise-induced quiescent (EIQ) bout onset, fractional quiescence, bout number, and bout duration, suggesting that previously described pathways are engaged during EIQ bouts. However, EIQ bouts are likely not sleep as animals are feeding during the majority of EIQ bouts. We find that genetic perturbation of neurons required for other C. elegans sleep states also does not alter EIQ dynamics. Additionally, we find that EIQ onset is sensitive to age and DAF-16 FOXO function. In summary, we have validated behavioral analysis software that enables a quantitative and detailed assessment of swimming behavior, including EIQ. We found novel EIQ defects in aged animals and animals with mutations in a gene involved in stress tolerance. We anticipate that further use of this software will facilitate the analysis of genes and pathways critical for fatigue and other C. elegans behaviors.

Ablation Therapy for Advanced Stage Non-Small Cell Lung Cancer: A National Cancer Database Study.

PURPOSE: To compare overall survival (OS) of ablation with no treatment for patients with advanced stage non-small cell lung cancer. METHODS: Patients with clinical stage IIIB (T1-4N3M0, T4N2M0) and stage IV (T1-4N0-3M1) non-small cell lung cancer, in accordance with the American Joint Committee on Cancer, 7th edition, who did not receive treatment or who received ablation as their sole primary treatment besides chemotherapy from 2004 to 2014, were identified from the National Cancer Data Base. OS was estimated using the Kaplan-Meier method and evaluated by log-rank test, univariate and multivariate Cox proportional hazard regression, and propensity score-matched analysis. Relative survival analyses comparing age- and sex-matched United States populations were performed. RESULTS: A total of 140,819 patients were included. The 1-, 2-, 3- and 5-year survival rates relative to age- and sex-matched United States population were 28%, 18%, 12%, and 10%, respectively, for ablation (n = 249); and 30%, 15%, 9%, and 5%, respectively for no treatment (n = 140,570). Propensity score matching resulted in 249 patients in the ablation group versus 498 patients in the no-treatment group. After matching, ablation was associated with longer OS than that in the no-treatment group (median, 5.9 vs 4.7 months, respectively; hazard ratio, 0.844; 95% confidence interval, 0.719-0.990; P = .037). These results persisted in patients with an initial tumor size of ≤3 cm. CONCLUSIONS: Preliminary results suggest ablation may be associated with longer OS in patients with late-stage non-small cell lung cancer than survival in those who received no treatment.

Differential Geometry in Edge Detection: Accurate Estimation of Position, Orientation and Curvature.

The vast majority of edge detection literature has aimed at improving edge recall and precision, with relatively few addressing the accuracy of edge orientation estimates which are often based on gradient. We show that first-order estimates of orientation can have significant error and this can be remedied by employing Third-Order estimates. This paper aims at estimating differential geometry attributes of an edge, namely, localization, orientation, and curvature, as well as edge topology, and develop robust numerical techniques in gray-scale and color images, applicable to a variety of popular edge detectors, such as gradient-based, gPb and SE. Second, a combinatorial model of edge grouping in a small neighborhood is developed to capture all geometrically consistent grouping called curvels, which establish: (i) edge topology in the form of potential links between an edge and other edges; (ii) an accurate curvature estimate for each possible grouping, whose performance is comparable to methods which use global and multi-scale methods; (iii) a more accurate localization of an edge. These have been evaluated using four distinct methodologies (i) traditional human annotated datasets; (ii) using coherence measure; (iii) stability analysis under visual perturbation, and (iv) utilitarian evaluation, and show meaningful improvements.

Transitions of the 3D medial axis under a one-parameter family of deformations.

The instabilities of the medial axis of a shape under deformations have long been recognized as a major obstacle to its use in recognition and other applications. These instabilities, or transitions, occur when the structure of the medial axis graph changes abruptly under deformations of shape. The recent classification of these transitions in 2D for the medial axis and for the shock graph was a key factor in the development of an object recognition system where the classified instabilities were utilized to represent deformation paths. The classification of generic transitions of the 3D medial axis could likewise potentially lead to a similar representation in 3D. In this paper, these transitions are classified by examining the order of contact of spheres with the surface, leading to an enumeration of possible transitions which are then examined on a case-by-case basis. Some cases are ruled out as never occurring in any family of deformations, while others are shown to be nongeneric in a one-parameter family of deformations. Finally, the remaining cases are shown to be viable by developing a specific example for each. Our work is inspired by that of Bogaevsky, who obtained the transitions as part of an investigation of viscosity solutions of Hamilton-Jacobi equations. Our contribution is to give a more down-to-earth approach, bringing this work to the attention of the computer vision community, and to provide explicit constructions for the various transitions using simple surfaces. We believe that the classification of these transitions is vital to the successful regularization of the medial axis in its use in real applications.

The medial scaffold of 3D unorganized point clouds.

We introduce the notion of the medial scaffold, a hierarchical organization of the medial axis of a 3D shape in the form of a graph constructed from special medial curves connecting special medial points. A key advantage of the scaffold is that it captures the qualitative aspects of shape in a hierarchical and tightly condensed representation. We propose an efficient and exact method for computing the medial scaffold based on a notion of propagation along the scaffold itself, starting from initial sources of the flow and constructing the scaffold during the propagation. We examine this method specifically in the context of an unorganized cloud of points in 3D, e.g., as obtained from laser range finders, which typically involve hundreds of thousands of points, but the ideas are generalizable to data arising from geometrically described surface patches. The computational bottleneck in the propagation-based scheme is in finding the initial sources of the flow. We thus present several ideas to avoid the unnecessary consideration of pairs of points which cannot possibly form a medial point source, such as the "visibility" of a point from another given a third point and the interaction of clusters of points. An application of using the medial scaffold for the representation of point samplings of real-life objects is also illustrated.

Recognition of shapes by editing their shock graphs.

This paper presents a novel framework for the recognition of objects based on their silhouettes. The main idea is to measure the distance between two shapes as the minimum extent of deformation necessary for one shape to match the other. Since the space of deformations is very high-dimensional, three steps are taken to make the search practical: 1) define an equivalence class for shapes based on shock-graph topology, 2) define an equivalence class for deformation paths based on shock-graph transitions, and 3) avoid complexity-increasing deformation paths by moving toward shock-graph degeneracy. Despite these steps, which tremendously reduce the search requirement, there still remain numerous deformation paths to consider. To that end, we employ an edit-distance algorithm for shock graphs that finds the optimal deformation path in polynomial time. The proposed approach gives intuitive correspondences for a variety of shapes and is robust in the presence of a wide range of visual transformations. The recognition rates on two distinct databases of 99 and 216 shapes each indicate highly successful within category matches (100 percent in top three matches), which render the framework potentially usable in a range of shape-based recognition applications.

A formal classification of 3D medial axis points and their local geometry.

This paper proposes a novel hypergraph skeletal representation for 3D shape based on a formal derivation of the generic structure of its medial axis. By classifying each skeletal point by its order of contact, we show that, generically, the medial axis consists of five types of points, which are then organized into sheets, curves, and points: 1) sheets (manifolds with boundary) which are the locus of bitangent spheres with regular tangency A1(2) (Ak(n) notation means n distinct k-fold tangencies of the sphere of contact, as explained in the text); two types of curves, 2) the intersection curve of three sheets and the locus of centers of tritangent spheres, A1(3), and 3) the boundary of sheets, which are the locus of centers of spheres whose radius equals the larger principal curvature, i.e., higher order contact A3 points; and two types of points, 4) centers of quad-tangent spheres, A1(4), and 5) centers of spheres with one regular tangency and one higher order tangency, A1A3. The geometry of the 3D medial axis thus consists of sheets (A1(2)) bounded by one type of curve (A3) on their free end, which corresponds to ridges on the surface, and attached to two other sheets at another type of curve (A1(3)), which support a generalized cylinder description. The A3 curves can only end in A1A3 points where they must meet an A1(3) curve. The A1(3) curves meet together in fours at an A1(4) point. This formal result leads to a compact representation for 3D shape, referred to as the medial axis hypergraph representation consisting of nodes (A1(4) and A1A3 points), links between pairs of nodes (A1(3) and A3 curves) and hyperlinks between groups of links (A1(2) sheets). The description of the local geometry at nodes by itself is sufficient to capture qualitative aspects of shapes, in analogy to 2D. We derive a pointwise reconstruction formula to reconstruct a surface from this medial axis hypergraph together with the radius function. Thus, this information completely characterizes 3D shape and lays the theoretical foundation for its use in recognition, morphing, design, and manipulation of shapes.

On the role of medial geometry in human vision.

A key challenge underlying theories of vision is how the spatially restricted, retinotopically represented feature analysis can be integrated to form abstract, coordinate-free object models. A resolution likely depends on the use of intermediate-level representations which can on the one hand be populated by local features and on the other hand be used as atomic units underlying the formation of, and interaction with, object hypotheses. The precise structure of this intermediate representation derives from the varied requirements of a range of visual tasks which motivate a significant role for incorporating a geometry of visual form. The need to integrate input from features capturing surface properties such as texture, shading, motion, color, etc., as well as from features capturing surface discontinuities such as silhouettes, T-junctions, etc., implies a geometry which captures both regional and boundary aspects. Curves, as a geometric model of boundaries, have been extensively used as an intermediate representation in computational, perceptual, and physiological studies, while the use of the medial axis (MA) has been popular mainly in computer vision as a geometric region-based model of the interior of closed boundaries. We extend the traditional model of the MA to represent images, where each MA segment represents a region of the image which we call a visual fragment. We present a unified theory of perceptual grouping and object recognition where through various sequences of transformations of the MA representation, visual fragments are grouped in various configurations to form object hypotheses, and are related to stored models. The mechanisms underlying both the computation and the transformation of the MA is a lateral wave propagation model. Recent psychophysical experiments depicting contrast sensitivity map peaks at the medial axes of stimuli, and experiments on perceptual filling-in, and brightness induction and modulation, are consistent with both the use of an MA representation and a propagation-based scheme. Also, recent neurophysiological recordings in V1 correlate with the MA hypothesis and a horizontal propagation scheme. This evidence supports a geometric computational paradigm for processing sensory data where both dynamic in-plane propagation and feedforward-feedback connections play an integral role.

Segmentation of carpal bones from CT images using skeletally coupled deformable models.

The in vivo investigation of joint kinematics in normal and injured wrist requires the segmentation of carpal bones from 3D (CT) images, and their registration over time. The non-uniformity of bone tissue, ranging from dense cortical bone to textured spongy bone, the irregular shape of closely packed carpal bones, small inter-bone spaces compared to the resolution of CT images, along with the presence of blood vessels, and the inherent blurring of CT imaging render the segmentation of carpal bones a challenging task. We review the performance of statistical classification, deformable models (active contours), region growing, region competition, and morphological operations for this application. We then propose a model which combines several of these approaches in a unified framework. Specifically, our approach is to use a curve evolution implementation of region growing from initialized seeds, where growth is modulated by a skeletally-mediated competition between neighboring regions. The inter-seed skeleton, which we interpret as the predicted boundary of collision between two regions, is used to couple the growth of seeds and to mediate long-range competition between them. The implementation requires subpixel representations of each growing region as well as the inter-region skeleton. This method combines the advantages of active contour models, region growing, and both local and global region competition methods. We demonstrate the effectiveness of this approach for our application where many of the difficulties presented above are overcome as illustrated by synthetic and real examples. Since this segmentation method does not rely on domain-specific knowledge, it should be applicable to a range of other medical imaging segmentation tasks.