Contour tracking using a knowledge-based snake algorithm to construct three-dimensional pharyngeal bolus movement.

Videofluorography (VFG) using a barium-mixed bolus is in wide clinical use for assessing patients with swallowing disorders. VFG is usually done with both lateral (LA) and anterior-posterior (AP) views, most commonly in two separate sittings. A real-time, three-dimensional (3-D) representation of the evolution of a pharyngeal bolus and its volumetric information can potentially help clinicians analyze and visualize the kinematics of swallowing, dysphagia, and compensatory therapeutic strategies. Active contour models, also known as "Snakes," have been used to solve various image analysis and computer vision problems. We applied a Snake algorithm to automate in part the contour tracking and reconstruction of VFG images to visualize and quantitatively analyze the 3-D evolution of a pharyngeal bolus. To improve the accuracy of the Snake search, we provided the additional "knowledge" of the pharyngeal image itself, which served as an extra constraint to push the Snake curve toward the desired contour. VFG of pharyngeal bolus transport in a normal subject was recorded by using barium-mixed boluses (viscosity: 185 centipoise, density: 2.84 g/cc) with volumes of 5, 10, and 20 ml. The resulting LA and AP video images were digitally captured and matched frame by frame. The knowledge-based Snake search algorithm was used to generate Snake points to satisfy both internal (i.e., smoothness) and external (i.e., boundary fitting) constraints. Using these Snake points, we traced the 3-D bolus movement at each time instant, assuming elliptic geometry in the cross-section of the pharyngeal bolus. By concentrating the 3-D images for each time instant, we developed a 3-D movie representing pharyngeal bolus movement. The efficiency, reproducibility, and accuracy of this algorithm in tracing pharyngeal bolus boundaries and estimating front/tail velocities were assessed and found satisfactory. We conclude that 3-D pharyngeal bolus movement can be traced both accurately and efficiently by using a knowledge-based Snake search algorithm.

Closed contour edge detection of blood vessel lumen and outer wall boundaries in black-blood MR images.

Quantitative measurements of the blood vessel wall area may provide useful information of atherosclerotic plaque burden, progression and/or regression. Magnetic resonance imaging is a promising technique for identifying both luminal and outer wall boundaries of the human blood vessels. Currently these boundaries are primarily defined manually, a process viewed as labor intensive and subject to significant operator bias. Fully automated post-processing techniques used for identifying the lumen and wall boundaries, on the other hand, are also problematic due to the complexity of signal features in the vicinity of the blood vessels. The goals of this study were to develop a robust, automated closed contour edge detection algorithm, apply this algorithm to high resolution human carotid artery images, and assess its accuracy, and reproducibility. Our algorithm has proven to be sensitive to various contrast situations and is reasonably accurate and highly reproducible.

Three-dimensional object representation and invariant recognition using continuous distance transform neural networks.

3D object recognition under partial object viewing is a difficult pattern recognition task. In this paper, we introduce a neural-network solution that is robust to partial viewing of objects and noise corruption. This method directly utilizes the acquired 3D data and requires no feature extraction. The object is first parametrically represented by a continuous distance transform neural network (CDTNN) trained by the surface points of the exemplar object. The CDTNN maps any 3D coordinate into a value that corresponds to the distance between the point and the nearest surface point of the object. Therefore, a mismatch between the exemplar object and an unknown object can be easily computed. When encountered with deformed objects, this mismatch information can be backpropagated through the CDTNN to iteratively determine the deformation in terms of affine transform. Application to 3D heart contour delineation and invariant recognition of 3D rigid-body objects is presented.

Robust speech recognition based on joint model and feature space optimization of hidden Markov models.

The hidden Markov model (HMM) inversion algorithm, based on either the gradient search or the Baum-Welch reestimation of input speech features, is proposed and applied to the robust speech recognition tasks under general types of mismatch conditions. This algorithm stems from the gradient-based inversion algorithm of an artificial neural network (ANN) by viewing an HMM as a special type of ANN. Given input speech features s, the forward training of an HMM finds the model parameters lambda subject to an optimization criterion. On the other hand, the inversion of an HMM finds speech features, s, subject to an optimization criterion with given model parameters lambda. The gradient-based HMM inversion and the Baum-Welch HMM inversion algorithms can be successfully integrated with the model space optimization techniques, such as the robust MINIMAX technique, to compensate the mismatch in the joint model and feature space. The joint space mismatch compensation technique achieves better performance than the single space, i.e. either the model space or the feature space alone, mismatch compensation techniques. It is also demonstrated that approximately 10-dB signal-to-noise ratio (SNR) gain is obtained in the low SNR environments when the joint model and feature space mismatch compensation technique is used.

Lipreading from color video.

We have designed and implemented a lipreading system that recognizes isolated words using only color video of human lips (without acoustic data). The system performs video recognition using "snakes" to extract visual features of geometric space, Karhunen-Loeve transform (KLT) to extract principal components in the color eigenspace, and hidden Markov models (HMM's) to recognize the combined visual features sequences. With the visual information alone, we were able to achieve 94% accuracy for ten isolated words.

The cascade-correlation learning: a projection pursuit learning perspective.

Cascade-correlation (Cascor) is a popular supervised learning architecture that dynamically grows layers of hidden neurons of fixed nonlinear activations (e.g., sigmoids), so that the network topology (size, depth) can be efficiently determined. Similar to a cascade-correlation learning network (CCLN), a projection pursuit learning network (PPLN) also dynamically grows the hidden neurons. Unlike a CCLN where cascaded connections from the existing hidden units to the new candidate hidden unit are required to establish high-order nonlinearity in approximating the residual error, a PPLN approximates the high-order nonlinearity by using trainable parametric or semi-parametric nonlinear smooth activations based on minimum mean squared error criterion. An analysis is provided to show that the maximum correlation training criterion used in a CCLN tends to produce hidden units that saturate and thus makes it more suitable for classification tasks instead of regression tasks as evidenced in the simulation results. It is also observed that this critical weakness in CCLN can also potentially carry over to classification tasks, such as the two-spiral benchmark used in the original CCLN paper.

A neural network-based stochastic active contour model (NNS-SNAKE) for contour finding of distinct features.

Contour finding of distinct features in 2-D/3-D images is essential for image analysis and computer vision. To overcome the potential problems associated with existing contour finding algorithms, we propose a framework, called the neural network-based stochastic active contour model (NNS-SNAKE), which integrates a neural network classifier for systematic knowledge building, an active contour model (also known as the "Snake") for automated contour finding using energy functions, and the Gibbs sampler to help the snake to find the most probable contour using a stochastic decision mechanism. Successful application of the NNS-SNAKE to extraction of several types of contours on magnetic resonance (MR) images is presented.

Regression modeling in back-propagation and projection pursuit learning.

We study and compare two types of connectionist learning methods for model-free regression problems: 1) the backpropagation learning (BPL); and 2) the projection pursuit learning (PPL) emerged in recent years in the statistical estimation literature. Both the BPL and the PPL are based on projections of the data in directions determined from interconnection weights. However, unlike the use of fixed nonlinear activations (usually sigmoidal) for the hidden neurons in BPL, the PPL systematically approximates the unknown nonlinear activations. Moreover, the BPL estimates all the weights simultaneously at each iteration, while the PPL estimates the weights cyclically (neuron-by-neuron and layer-by-layer) at each iteration. Although the BPL and the PPL have comparable training speed when based on a Gauss-Newton optimization algorithm, the PPL proves more parsimonious in that the PPL requires a fewer hidden neurons to approximate the true function. To further improve the statistical performance of the PPL, an orthogonal polynomial approximation is used in place of the supersmoother method originally proposed for nonlinear activation approximation in the PPL.

Iterative inversion of neural networks and its application to adaptive control.

An iterative constrained inversion technique is used to find the control inputs to the plant. That is, rather than training a controller network and placing this network directly in the feedback or feedforward paths, the forward model of the plant is learned, and iterative inversion is performed on line to generate control commands. The control approach allows the controllers to respond online to changes in the plant dynamics. This approach also attempts to avoid the difficulty of analysis introduced by most current neural network controllers, which place the highly nonlinear neural network directly in the feedback path. A neural network-based model reference adaptive controller is also proposed for systems having significant dynamics between the control inputs and the observed (or desired) outputs and is demonstrated on a simple linear control system. These results are interpreted in terms of the need for a dither signal for on-line identification of dynamic systems.

Query-based learning applied to partially trained multilayer perceptrons.

An approach is presented for query-based neural network learning. A layered perceptron partially trained for binary classification is considered. The single-output neuron is trained to be either a zero or a one. A test decision is made by thresholding the output at, for example, one-half. The set of inputs that produce an output of one-half forms the classification boundary. The authors adopted an inversion algorithm for the neural network that allows generation of this boundary. For each boundary point, the classification gradient can be generated. The gradient provides a useful measure of the steepness of the multidimensional decision surfaces. Conjugate input pairs are generated using the boundary point and gradient information and presented to an oracle for proper classification. These data are used to refine further the classification boundary, thereby increasing the classification accuracy. The result can be a significant reduction in the training set cardinality in comparison with, for example, randomly generated data points. An application example to power system security assessment is given.