Learning to Discriminate Information for Online Action Detection: Analysis and Application.

Online action detection, which aims to identify an ongoing action from a streaming video, is an important subject in real-world applications. For this task, previous methods use recurrent neural networks for modeling temporal relations in an input sequence. However, these methods overlook the fact that the input image sequence includes not only the action of interest but background and irrelevant actions. This would induce recurrent units to accumulate unnecessary information for encoding features on the action of interest. To overcome this problem, we propose a novel recurrent unit, named Information Discrimination Unit (IDU), which explicitly discriminates the information relevancy between an ongoing action and others to decide whether to accumulate the input information. This enables learning more discriminative representations for identifying an ongoing action. In this paper, we further present a new recurrent unit, called Information Integration Unit (IIU), for action anticipation. Our IIU exploits the outputs from IDN as pseudo action labels as well as RGB frames to learn enriched features of observed actions effectively. In experiments on TVSeries and THUMOS-14, the proposed methods outperform state-of-the-art methods by a significant margin in online action detection and action anticipation. Moreover, we demonstrate the effectiveness of the proposed units by conducting comprehensive ablation studies.

Blind Deblurring of Text Images Using a Text-Specific Hybrid Dictionary.

In this paper, we propose a blind text image deblurring algorithm by using a text-specific hybrid dictionary. After careful analysis, we find that the text-specific hybrid dictionary has the great ability of providing powerful contextual information for text image deblurring. Here, it is worth noting that our proposed method is inspired by our observation that an intermediate latent image contains not only sharp regions, but also multiple types of small blurred regions. Based upon our discovery, we propose a prior for text images based on sparse representation, which models the relationship between an intermediate latent image and a desired sharp image. To this end, we carefully collect three different image patch pairs, which are 1) Gaussian blur-sharp, 2) motion blur-sharp, and 3) sharp-sharp, in order to construct the text-specific hybrid dictionary. We also propose a new optimization framework suitable for the task of text image deblurring in this paper. Extensive experiments have been conducted on a challenging dataset of synthetic and real-world text images. Our results demonstrate that the proposed method outperforms the state-of-the-art image deblurring methods both quantitatively and qualitatively.

A Quad Edge-Based Grid Encoding Model for Content-Aware Image Retargeting.

In this paper, we present a novel grid encoding model for content-aware image retargeting. In contrast to previous approaches such as vertex-based and axis-aligned grid encoding models, our approach takes each horizontal/vertical distance between two adjacent vertices as an optimization variable. Upon this difference-based encoding scheme, every vertex position of a target grid is subsequently determined after optimizing the one-dimensional values. Our quad edge-based grid model has two major advantages for image retargeting. First, the model enables a grid optimization problem to be developed in a simple quadratic program while ensuring the global convexity of objective functions. Second, due to the independency of variables, spatial regularizations can be applied in a locally adaptive manner to preserve structural components. Based on this model, we propose three quadratic objective functions. Note that, in our work, their linear combination guides a grid deformation process to obtain a visually comfortable retargeting result by preserving salient regions and structural components of an input image. Comparative evaluations have been conducted with ten existing state-of-the-art image retargeting methods, and the results show that our method built upon the quad edge-based model consistently outperforms other previous methods both on qualitative and quantitative perspectives.

Single-view 2D CNNs with fully automatic non-nodule categorization for false positive reduction in pulmonary nodule detection.

BACKGROUND AND OBJECTIVE: In pulmonary nodule detection, the first stage, candidate detection, aims to detect suspicious pulmonary nodules. However, detected candidates include many false positives and thus in the following stage, false positive reduction, such false positives are reliably reduced. Note that this task is challenging due to 1) the imbalance between the numbers of nodules and non-nodules and 2) the intra-class diversity of non-nodules. Although techniques using 3D convolutional neural networks (CNNs) have shown promising performance, they suffer from high computational complexity which hinders constructing deep networks. To efficiently address these problems, we propose a novel framework using the ensemble of 2D CNNs using single views, which outperforms existing 3D CNN-based methods. METHODS: Our ensemble of 2D CNNs utilizes single-view 2D patches to improve both computational and memory efficiency compared to previous techniques exploiting 3D CNNs. We first categorize non-nodules on the basis of features encoded by an autoencoder. Then, all 2D CNNs are trained by using the same nodule samples, but with different types of non-nodules. By extending the learning capability, this training scheme resolves difficulties of extracting representative features from non-nodules with large appearance variations. Note that, instead of manual categorization requiring the heavy workload of radiologists, we propose to automatically categorize non-nodules based on the autoencoder and k-means clustering. RESULTS: We performed extensive experiments to validate the effectiveness of our framework based on the database of the lung nodule analysis 2016 challenge. The superiority of our framework is demonstrated through comparing the performance of five frameworks trained with differently constructed training sets. Our proposed framework achieved state-of-the-art performance (0.922 of the competition performance metric score) with low computational demands (789K of parameters and 1024M of floating point operations per second). CONCLUSION: We presented a novel false positive reduction framework, the ensemble of single-view 2D CNNs with fully automatic non-nodule categorization, for pulmonary nodule detection. Unlike previous 3D CNN-based frameworks, we utilized 2D CNNs using 2D single views to improve computational efficiency. Also, our training scheme using categorized non-nodules, extends the learning capability of representative features of different non-nodules. Our framework achieved state-of-the-art performance with low computational complexity.

Evaluation of nuclear chromatin using grayscale intensity and thresholded percentage area in liquid-based cervical cytology.

OBJECTIVES: Development of computerized image analysis techniques has opened up the possibility for the quantitative analysis of nuclear chromatin in pathology. We hypothesized that the features extracted from digital images could be used to determine specific cytomorphological findings for nuclear chromatin that may be applicable for establishing a medical diagnosis. METHODS: Three parameters were evaluated from nuclear chromatin images obtained from the liquid-based cervical cytology samples of patients with biopsy-proven high-grade squamous intraepithelial lesion (HGSIL), and compared between non-neoplastic squamous epithelia and dysplastic epithelia groups: (1) standard deviation (SD) of the grayscale intensity; (2) difference between the maximum and minimum grayscale intensity (M-M); and (3) thresholded area percentage. Each parameter was evaluated at the mean, mean-1SD, and mean-2SD thresholding intensity levels. RESULTS: Between the mean and mean-1SD levels, the thresholded nuclear chromatin pattern was most similar to the chromatin granularity of the unthresholded grayscale images. The SD of the gray intensity and the thresholded area percentage differed significantly between the non-neoplastic squamous epithelia and dysplastic epithelia of HGSIL images at all three thresholding intensity levels (mean, mean-1SD, and mean-2SD). However, the M-M significantly differed between the two sample types for only two of the thresholding intensity levels (mean-1SD and mean-2SD). CONCLUSIONS: The digital parameters SD and M-M of the grayscale intensity, along with the thresholded area percentage could be useful in automated cytological evaluations. Further studies are needed to identify more valuable parameters for clinical application.

Nanohole-Structured and Palladium-Embedded 3D Porous Graphene for Ultrahigh Hydrogen Storage and CO Oxidation Multifunctionalities.

Atomic-scale defects on carbon nanostructures have been considered as detrimental factors and critical problems to be eliminated in order to fully utilize their intrinsic material properties such as ultrahigh mechanical stiffness and electrical conductivity. However, defects that can be intentionally controlled through chemical and physical treatments are reasonably expected to bring benefits in various practical engineering applications such as desalination thin membranes, photochemical catalysts, and energy storage materials. Herein, we report a defect-engineered self-assembly procedure to produce a three-dimensionally nanohole-structured and palladium-embedded porous graphene hetero-nanostructure having ultrahigh hydrogen storage and CO oxidation multifunctionalities. Under multistep microwave reactions, agglomerated palladium nanoparticles having diameters of ∼10 nm produce physical nanoholes in the basal-plane structure of graphene sheets, while much smaller palladium nanoparticles are readily impregnated inside graphene layers and bonded on graphene surfaces. The present results show that the defect-engineered hetero-nanostructure has a ∼5.4 wt % hydrogen storage capacity under 7.5 MPa and CO oxidation catalytic activity at 190 °C. The defect-laden graphene can be highly functionalized for multipurpose applications such as molecule absorption, electrochemical energy storage, and catalytic activity, resulting in a pathway to nanoengineering based on underlying atomic scale and physical defects.

Impact of the accuracy of automatic segmentation of cell nuclei clusters on classification of thyroid follicular lesions.

Automatic segmentation of cell nuclei clusters is a key building block in systems for quantitative analysis of microscopy cell images. For that reason, it has received a great attention over the last decade, and diverse automatic approaches to segment clustered nuclei with varying levels of performance under different test conditions have been proposed in literature. To the best of our knowledge, however, so far there is no comparative study on the methods. This study is a first attempt to fill this research gap. More precisely, the purpose of this study is to present an objective performance comparison of existing state-of-the-art segmentation methods. Particularly, the impact of their accuracy on classification of thyroid follicular lesions is also investigated "quantitatively" under the same experimental condition, to evaluate the applicability of the methods. Thirteen different segmentation approaches are compared in terms of not only errors in nuclei segmentation and delineation, but also their impact on the performance of system to classify thyroid follicular lesions using different metrics (e.g., diagnostic accuracy, sensitivity, specificity, etc.). Extensive experiments have been conducted on a total of 204 digitized thyroid biopsy specimens. Our study demonstrates that significant diagnostic errors can be avoided using more advanced segmentation approaches. We believe that this comprehensive comparative study serves as a reference point and guide for developers and practitioners in choosing an appropriate automatic segmentation technique adopted for building automated systems for specifically classifying follicular thyroid lesions.

A unified spectral-domain approach for saliency detection and its application to automatic object segmentation.

In this paper, a visual attention model is incorporated for efficient saliency detection, and the salient regions are employed as object seeds for our automatic object segmentation system. In contrast with existing interactive segmentation approaches that require considerable user interaction, the proposed method does not require it, i.e., the segmentation task is fulfilled in a fully automatic manner. First, we introduce a novel unified spectral-domain approach for saliency detection. Our visual attention model originates from a well-known property of the human visual system that the human visual perception is highly adaptive and sensitive to structural information in images rather than nonstructural information. Then, based on the saliency map, we propose an iterative self-adaptive segmentation framework for more accurate object segmentation. Extensive tests on a variety of cluttered natural images show that the proposed algorithm is an efficient indicator for characterizing the human perception and it can provide satisfying segmentation performance.

Detecting shadows from a single image.

We present a novel (to our best knowledge) optical recognition technique for detecting shadows from a single image. Most prior approaches definitely depend on explicit physical computational models, but physics-based approaches have the critical problem that they may fail severely even with slight perturbations. Unlike traditional approaches, our method does not rely on any explicit physical models. This breakthrough originates from a discovery of a new modeling mechanism, derived from a biological vision principle, the so-called retinex theory, which is well suited for single-image shadow detection. Experimental results demonstrate that the proposed method outperforms the previous optical recognition techniques and gives robust results even in real-world complex scenes.

Unsupervised segmentation of overlapped nuclei using Bayesian classification.

In a fully automatic cell extraction process, one of the main issues to overcome is the problem related to extracting overlapped nuclei since such nuclei will often affect the quantitative analysis of cell images. In this paper, we present an unsupervised Bayesian classification scheme for separating overlapped nuclei. The proposed approach first involves applying the distance transform to overlapped nuclei. The topographic surface generated by distance transform is viewed as a mixture of Gaussians in the proposed algorithm. In order to learn the distribution of the topographic surface, the parametric expectation-maximization (EM) algorithm is employed. Cluster validation is performed to determine how many nuclei are overlapped. Our segmentation approach incorporates a priori knowledge about the regular shape of clumped nuclei to yield more accurate segmentation results. Experimental results show that the proposed method yields superior segmentation performance, compared to those produced by conventional schemes.

Segmenting clustered nuclei using H-minima transform-based marker extraction and contour parameterization.

In this letter, we present a novel watershed-based method for segmentation of cervical and breast cell images. We formulate the segmentation of clustered nuclei as an optimization problem. A hypothesis concerning the nuclei, which involves a priori knowledge with respect to the shape of nuclei, is tested to solve the optimization problem. We first apply the distance transform to the clustered nuclei. A marker extraction scheme based on the H-minima transform is introduced to obtain the optimal segmentation result from the distance map. In order to estimate the optimal h-value, a size-invariant segmentation distortion evaluation function is defined based on the fitting residuals between the segmented region boundaries and fitted models. Ellipsoidal modeling of contours is introduced to adjust nuclei contours for more effective analysis. Experiments on a variety of real microscopic cell images show that the proposed method yields more accurate segmentation results than the state-of-the-art watershed-based methods.