Highly Efficient and Unsupervised Framework for Moving Object Detection in Satellite Videos.

Moving object detection in satellite videos (SVMOD) is a challenging task due to the extremely dim and small target characteristics. Current learning-based methods extract spatio-temporal information from multi-frame dense representation with labor-intensive manual labels to tackle SVMOD, which needs high annotation costs and contains tremendous computational redundancy due to the severe imbalance between foreground and background regions. In this paper, we propose a highly efficient unsupervised framework for SVMOD. Specifically, we propose a generic unsupervised framework for SVMOD, in which pseudo labels generated by a traditional method can evolve with the training process to promote detection performance. Furthermore, we propose a highly efficient and effective sparse convolutional anchor-free detection network by sampling the dense multi-frame image form into a sparse spatio-temporal point cloud representation and skipping the redundant computation on background regions. Coping these two designs, we can achieve both high efficiency (label and computation efficiency) and effectiveness. Extensive experiments demonstrate that our method can not only process 98.8 frames per second on 1024 ×1024 images but also achieve state-of-the-art performance.

Deep Learning for Visual Speech Analysis: A Survey.

Visual speech, referring to the visual domain of speech, has attracted increasing attention due to its wide applications, such as public security, medical treatment, military defense, and film entertainment. As a powerful AI strategy, deep learning techniques have extensively promoted the development of visual speech learning. Over the past five years, numerous deep learning based methods have been proposed to address various problems in this area, especially automatic visual speech recognition and generation. To push forward future research on visual speech, this paper will present a comprehensive review of recent progress in deep learning methods on visual speech analysis. We cover different aspects of visual speech, including fundamental problems, challenges, benchmark datasets, a taxonomy of existing methods, and state-of-the-art performance. Besides, we also identify gaps in current research and discuss inspiring future research directions.

Boosting Convolutional Neural Networks With Middle Spectrum Grouped Convolution.

This article proposes a novel module called middle spectrum grouped convolution (MSGC) for efficient deep convolutional neural networks (DCNNs) with the mechanism of grouped convolution. It explores the broad "middle spectrum" area between channel pruning and conventional grouped convolution. Compared with channel pruning, MSGC can retain most of the information from the input feature maps due to the group mechanism; compared with grouped convolution, MSGC benefits from the learnability, the core of channel pruning, for constructing its group topology, leading to better channel division. The middle spectrum area is unfolded along four dimensions: groupwise, layerwise, samplewise, and attentionwise, making it possible to reveal more powerful and interpretable structures. As a result, the proposed module acts as a booster that can reduce the computational cost of the host backbones for general image recognition with even improved predictive accuracy. For example, in the experiments on the ImageNet dataset for image classification, MSGC can reduce the multiply-accumulates (MACs) of ResNet-18 and ResNet-50 by half but still increase the Top-1 accuracy by more than 1% . With a 35% reduction of MACs, MSGC can also increase the Top-1 accuracy of the MobileNetV2 backbone. Results on the MS COCO dataset for object detection show similar observations. Our code and trained models are available at https://github.com/hellozhuo/msgc.

Socioeconomic status, cardiovascular risk factors, and subclinical atherosclerosis in young adults: the cardiovascular risk in Young Finns Study.

OBJECTIVE: The goal of this study was to investigate the extent to which socioeconomic status (SES) in young adults is associated with cardiovascular risk factor levels and carotid intima-media thickness (IMT) and their changes over a 6-year follow-up period. METHODS AND RESULTS: The study population included 1813 subjects participating in the 21- and 27-year follow-ups of the Cardiovascular Risk in Young Finns Study (baseline age 24-39 years in 2001). At baseline, SES (indexed with education) was inversely associated with body mass index (P=0.0002), waist circumference (P<0.0001), glucose (P=0.01), and insulin (P=0.0009) concentrations; inversely associated with alcohol consumption (P=0.02) and cigarette smoking (P<0.0001); and directly associated with high-density lipoprotein cholesterol levels (P=0.05) and physical activity (P=0.006). Higher SES was associated with a smaller 6-year increase in body mass index (P=0.001). Education level and IMT were not associated (P=0.58) at baseline, but an inverse association was observed at follow-up among men (P=0.004). This became nonsignificant after adjustment with conventional risk factors (P=0.11). In all subjects, higher education was associated with a smaller increase in IMT during the follow-up (P=0.002), and this association remained after adjustments for conventional risk factors (P=0.04). CONCLUSION: This study shows that high education in young adults is associated with favorable cardiovascular risk factor profile and 6-year change of risk factors. Most importantly, the progression of carotid atherosclerosis was slower among individuals with higher educational level.

Lightweight Pixel Difference Networks for Efficient Visual Representation Learning.

Recently, there have been tremendous efforts in developing lightweight Deep Neural Networks (DNNs) with satisfactory accuracy, which can enable the ubiquitous deployment of DNNs in edge devices. The core challenge of developing compact and efficient DNNs lies in how to balance the competing goals of achieving high accuracy and high efficiency. In this paper we propose two novel types of convolutions, dubbed Pixel Difference Convolution (PDC) and Binary PDC (Bi-PDC) which enjoy the following benefits: capturing higher-order local differential information, computationally efficient, and able to be integrated with existing DNNs. With PDC and Bi-PDC, we further present two lightweight deep networks named Pixel Difference Networks (PiDiNet) and Binary PiDiNet (Bi-PiDiNet) respectively to learn highly efficient yet more accurate representations for visual tasks including edge detection and object recognition. Extensive experiments on popular datasets (BSDS500, ImageNet, LFW, YTF, etc.) show that PiDiNet and Bi-PiDiNet achieve the best accuracy-efficiency trade-off. For edge detection, PiDiNet is the first network that can be trained without ImageNet, and can achieve the human-level performance on BSDS500 at 100 FPS and with 1 M parameters. For object recognition, among existing Binary DNNs, Bi-PiDiNet achieves the best accuracy and a nearly 2× reduction of computational cost on ResNet18.

Deep Ladder-Suppression Network for Unsupervised Domain Adaptation.

Unsupervised domain adaptation (UDA) aims at learning a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain with a related but different distribution. Most existing approaches learn domain-invariant features by adapting the entire information of the images. However, forcing adaptation of domain-specific variations undermines the effectiveness of the learned features. To address this problem, we propose a novel, yet elegant module, called the deep ladder-suppression network (DLSN), which is designed to better learn the cross-domain shared content by suppressing domain-specific variations. Our proposed DLSN is an autoencoder with lateral connections from the encoder to the decoder. By this design, the domain-specific details, which are only necessary for reconstructing the unlabeled target data, are directly fed to the decoder to complete the reconstruction task, relieving the pressure of learning domain-specific variations at the later layers of the shared encoder. As a result, DLSN allows the shared encoder to focus on learning cross-domain shared content and ignores the domain-specific variations. Notably, the proposed DLSN can be used as a standard module to be integrated with various existing UDA frameworks to further boost performance. Without whistles and bells, extensive experimental results on four gold-standard domain adaptation datasets, for example: 1) Digits; 2) Office31; 3) Office-Home; and 4) VisDA-C, demonstrate that the proposed DLSN can consistently and significantly improve the performance of various popular UDA frameworks.

Tracking of serum lipid levels, blood pressure, and body mass index from childhood to adulthood: the Cardiovascular Risk in Young Finns Study.

OBJECTIVES: To examine tracking and predictiveness of childhood lipid levels, blood pressure, and body mass index for risk profile in adulthood and the best age to measure the childhood risk factor levels. STUDY DESIGN: Study subjects were participants of the longitudinal Cardiovascular Risk in Young Finns Study, started in 1980 (age 3, 6, 9, 12, 15, and 18 years). A total of 2204 subjects participated to the 27-year follow-up in 2007 (age, 30 to 45 years). RESULTS: In both sex groups and in all age groups, childhood risk factors were significantly correlated with levels in adulthood. The correlation coefficients for cholesterol levels and body mass index were 0.43 to 0.56 (P < .0001), and for blood pressure and triglyceride levels, they were 0.21 to 0.32 (P < .0001). To recognize children with abnormal adult levels, the National Cholesterol Education Program and the National High Blood Pressure Education Program cutoff points for lipid and blood pressure values and international cutoff points for overweight and obesity were used. Age seemed to affect associations. The best sensitivity and specificity rates were observed in 12- to 18-year-old subjects. CONCLUSIONS: Childhood blood pressure, serum lipid levels, and body mass index correlate strongly with values measured in middle age. These associations seemed to be stronger with increased age at measurements.

Life-time risk factors and progression of carotid atherosclerosis in young adults: the Cardiovascular Risk in Young Finns study.

AIMS: The aim of this study is to evaluate whether childhood risk factors are associated with a 6-year change in carotid intima-media thickness (IMT) in young adulthood independent of the current risk factors. METHODS AND RESULTS: The Cardiovascular Risk in Young Finns cohort consisted of 1809 subjects who were followed-up for 27 years since baseline (1980, age 3-18 years) and having carotid IMT measured both in 2001 and 2007. Cardiovascular risk factors were assessed repeatedly since childhood. A genotype risk score was calculated using 17 newly identified genetic variants associating with cardiovascular morbidity. The number of childhood risk factors (high LDL-cholesterol, low HDL-cholesterol, high blood pressure, obesity, diabetes, smoking, low physical activity, infrequent fruit consumption) was associated with a 6-year change in adulthood IMT. In subjects with 0, 1, 2, and > or =3 childhood risk factors, IMT [mean (95% CI)) increased by 35 (28-42), 46 (40-52), 49 (41-57), and 61 (49-73) microm (P = 0.0001). This association remained significant when adjusted for adulthood risk score and genotype score (P = 0.007). Of the individual childhood variables, infrequent fruit consumption ((beta (95% CI) for 1-SD change -5(-9 to -1), P = 0.03) and low physical activity (-6(-10 to -2), P = 0.01) were associated with accelerated IMT progression after taking into account these variables assessed in adulthood. CONCLUSION: These findings indicate that children with risk factors have increased atherosclerosis progression rate in adulthood, and support the idea that the prevention of atherosclerosis by means of life style could be effective when initiated in childhood.

Hyperspectral monitor of soil chromium contaminant based on deep learning network model in the Eastern Junggar coalfield.

In China, over 10% of cultivated land is polluted by heavy metals, which can affect crop growth, food safety and human health. Therefore, how to effectively and quickly detect soil heavy metal pollution has become a critical issue. This study provides a novel data preprocessing method that can extract vital information from soil hyperspectra and uses different classification algorithms to detect levels of heavy metal contamination in soil. In this experiment, 160 soil samples from the Eastern Junggar Coalfield in Xinjiang were employed for verification, including 143 noncontaminated samples and 17 contaminated soil samples. Because the concentration of chromium in the soil exists in trace amounts, combined with the fact that spectral characteristics are easily influenced by other types of impurity in the soil, the evaluation of chromium concentrations in the soil through hyperspectral analysis is not satisfactory. To avoid this phenomenon, the pretreatment method of this experiment includes a combination of second derivative and data enhancement (DA) approaches. Then, support vector machine (SVM), k-nearest neighbour (KNN) and deep neural network (DNN) algorithms are used to create the discriminant models. The accuracies of the DA-SVM, DA-KNN and DA-DNN models were 95.61%, 95.62% and 96.25%, respectively. The results of this experiment demonstrate that soil hyperspectral technology combined with deep learning can be used to instantly monitor soil chromium pollution levels on a large scale. This research can be used for the management of polluted areas and agricultural insurance applications.

Conventional cardiovascular risk factors and metabolic syndrome in predicting carotid intima-media thickness progression in young adults: the cardiovascular risk in young Finns study.

BACKGROUND: Conventional risk factors and metabolic syndrome (MetS) are cross-sectionally associated with subclinical atherosclerosis in young adults. We evaluated the relations of conventional risk factors and MetS to the 6-year progression of carotid intima-media thickness (IMT) in a population of young adults. RESULTS AND METHODS: The study included 1809 subjects (aged 32+/-5 years) who had IMT measured in 2001 and 2007. Risk factor measurements included low-density lipoprotein cholesterol, body mass index, C-reactive protein, smoking, and family history of coronary disease in addition to MetS components. We used European Group for the Study of Insulin Resistance, revised National Cholesterol Education Program, and International Diabetes Federation definitions to diagnose MetS in 2001. Waist circumference (P<0.0001), low-density lipoprotein cholesterol (P=0.01), and insulin (P=0.003) were directly associated with IMT progression in a multivariable model adjusted for age, sex, and baseline IMT (model R(2)=24%). When the MetS/European Group for the Study of Insulin Resistance definition was included in the model, it was directly associated with IMT progression (P=0.03), but its inclusion did not improve the model's predictive value. IMT increased 79+/-7 mum (mean+/-SEM) in subjects with MetS according to the MetS/European Group for the Study of Insulin Resistance definition and 42+/-2 mum in subjects without MetS (P<0.0001). In addition, the number of MetS components was linearly associated with IMT progression (P<0.0001). Similar results were seen with MetS/revised National Cholesterol Education Program and MetS/International Diabetes Federation definitions. CONCLUSIONS: Obesity, high low-density lipoprotein cholesterol, and high insulin level predicted IMT progression in young adults. All MetS definitions identified young adults with accelerated IMT progression, but we found no evidence that MetS would predict IMT progression more than expected from the sum of its risk components.

Texture Classification in Extreme Scale Variations Using GANet.

Research in texture recognition often concentrates on recognizing textures with intraclass variations, such as illumination, rotation, viewpoint, and small-scale changes. In contrast, in real-world applications, a change in scale can have a dramatic impact on texture appearance to the point of changing completely from one texture category to another. As a result, texture variations due to changes in scale are among the hardest to handle. In this paper, we conduct the first study of classifying textures with extreme variations in scale. To address this issue, we first propose and then reduce scale proposals on the basis of dominant texture patterns. Motivated by the challenges posed by this problem, we propose a new GANet network where we use a genetic algorithm to change the filters in the hidden layers during network training in order to promote the learning of more informative semantic texture patterns. Finally, we adopt a Fisher vector pooling of a convolutional neural network filter bank feature encoder for global texture representation. Because extreme scale variations are not necessarily present in most standard texture databases, to support the proposed extreme-scale aspects of texture understanding, we are developing a new dataset, the extreme scale variation textures (ESVaT), to test the performance of our framework. It is demonstrated that the proposed framework significantly outperforms the gold-standard texture features by more than 10% on ESVaT. We also test the performance of our proposed approach on the KTHTIPS2b and OS datasets and a further dataset synthetically derived from Forrest, showing the superior performance compared with the state-of-the-art.

Association of Pathobiologic Determinants of Atherosclerosis in Youth risk score and 15-year change in risk score with carotid artery intima-media thickness in young adults (from the Cardiovascular Risk in Young Finns Study).

The Pathobiological Determinants of Atherosclerosis in Youth (PDAY) study of autopsy findings in subjects 15 to 34 years of age developed a risk score using coronary heart disease risk factors (gender, age, serum lipoprotein concentrations, smoking, hypertension, obesity, and hyperglycemia) to estimate the probability of advanced atherosclerotic lesions in the coronary arteries. The Cardiovascular Risk in Young Finns Study measured coronary heart disease risk factors in a population-based cohort in 1986 and 2001 and measured carotid artery intima-media thickness (IMT) with ultrasonography in 2001. We computed the PDAY risk score from risk factors measured in 1,279 subjects who were 12 to 24 years of age in 1986 and 27 to 39 years of age in 2001. The PDAY risk score early in life (i.e., 1986) and the change in risk score in the following 15 years (i.e., 1986 through 2001) were independent predictors of carotid artery intima-media thickness; the multiplicative effect of 1 point in the 1986 risk score was 1.008 (95% confidence interval 1.005 to 1.012) and the multiplicative effect of a 1-point increase between the 1986 and 2001 risk scores was 1.003 (95% confidence interval 1.001 to 1.006; multiplicative effect of 0.997 for a 1-point decrease). In conclusion, the change in risk score over time (decrease or increase) during adolescence and young adulthood, as well as the risk score early in life, are important predictors of atherosclerosis.

Young adults with family history of coronary heart disease have increased arterial vulnerability to metabolic risk factors: the Cardiovascular Risk in Young Finns Study.

OBJECTIVE: Subjects with family history for coronary heart disease (CHD) may be more susceptible to the adverse effects of risk factors than subjects without family history. We investigated the occurrence of subclinical atherosclerosis in young adults with family history of CHD and tested the hypothesis that their arteries are more vulnerable to the proatherogenic effects of metabolic risk factors. METHODS AND RESULTS: Carotid artery intima-media thickness (IMT), carotid artery compliance (CAC), and brachial artery flow-mediated dilation (FMD) were measured in the 21-year follow-up of the Cardiovascular Risk in Young Finns Study in 2265 white adults 24 to 39 years of age. Subjects with positive family history of CHD had greater IMT compared with those with negative history (mean+/-SEM; 0.600+/-0.006 versus 0.578+/-0.002 mm; P=0.003, respectively). No differences were observed in CAC or FMD (both P>0.2). The difference in IMT remained similar after adjustment with current risk factors (P=0.008) or childhood risk factors measured 21 years earlier (P=0.002). The number of metabolic risk factors (components of the NCEP metabolic syndrome) correlated more strongly with IMT in subjects with family history of CHD than those without (P=0.007 for interaction). CONCLUSIONS: Young healthy adults with family history of CHD have increased carotid IMT. This is partly explained by their increased vulnerability to metabolic risk factors.

Dynamic texture recognition using local binary patterns with an application to facial expressions.

Dynamic texture (DT) is an extension of texture to the temporal domain. Description and recognition of DTs have attracted growing attention. In this paper, a novel approach for recognizing DTs is proposed and its simplifications and extensions to facial image analysis are also considered. First, the textures are modeled with volume local binary patterns (VLBP), which are an extension of the LBP operator widely used in ordinary texture analysis, combining motion and appearance. To make the approach computationally simple and easy to extend, only the co-occurrences of the local binary patterns on three orthogonal planes (LBP-TOP) are then considered. A block-based method is also proposed to deal with specific dynamic events such as facial expressions in which local information and its spatial locations should also be taken into account. In experiments with two DT databases, DynTex and Massachusetts Institute of Technology (MIT), both the VLBP and LBP-TOP clearly outperformed the earlier approaches. The proposed block-based method was evaluated with the Cohn-Kanade facial expression database with excellent results. The advantages of our approach include local processing, robustness to monotonic gray-scale changes, and simple computation.

HEp-2 Cell Classification via Combining Multiresolution Co-Occurrence Texture and Large Region Shape Information.

Indirect immunofluorescence imaging of human epithelial type 2 (HEp-2) cell image is an effective evidence to diagnose autoimmune diseases. Recently, computer-aided diagnosis of autoimmune diseases by the HEp-2 cell classification has attracted great attention. However, the HEp-2 cell classification task is quite challenging due to large intraclass and small interclass variations. In this paper, we propose an effective approach for the automatic HEp-2 cell classification by combining multiresolution co-occurrence texture and large regional shape information. To be more specific, we propose to: 1) capture multiresolution co-occurrence texture information by a novel pairwise rotation-invariant co-occurrence of local Gabor binary pattern descriptor; 2) depict large regional shape information by using an improved Fisher vector model with RootSIFT features, which are sampled from large image patches in multiple scales; and 3) combine both features. We evaluate systematically the proposed approach on the IEEE International Conference on Pattern Recognition (ICPR) 2012, the IEEE International Conference on Image Processing (ICIP) 2013, and the ICPR 2014 contest datasets. The proposed method based on the combination of the introduced two features outperforms the winners of the ICPR 2012 contest using the same experimental protocol. Our method also greatly improves the winner of the ICIP 2013 contest under four different experimental setups. Using the leave-one-specimen-out evaluation strategy, our method achieves comparable performance with the winner of the ICPR 2014 contest that combined four features.

A texture-based method for modeling the background and detecting moving objects.

This paper presents a novel and efficient texture-based method for modeling the background and detecting moving objects from a video sequence. Each pixel is modeled as a group of adaptive local binary pattern histograms that are calculated over a circular region around the pixel. The approach provides us with many advantages compared to the state-of-the-art. Experimental results clearly justify our model.

Face description with local binary patterns: application to face recognition.

This paper presents a novel and efficient facial image representation based on local binary pattern (LBP) texture features. The face image is divided into several regions from which the LBP feature distributions are extracted and concatenated into an enhanced feature vector to be used as a face descriptor. The performance of the proposed method is assessed in the face recognition problem under different challenges. Other applications and several extensions are also discussed.

Thorax disease diagnosis using deep convolutional neural network.

Computer aided diagnosis (CAD) is an important issue, which can significantly improve the efficiency of doctors. In this paper, we propose a deep convolutional neural network (CNN) based method for thorax disease diagnosis. We firstly align the images by matching the interest points between the images, and then enlarge the dataset by using Gaussian scale space theory. After that we use the enlarged dataset to train a deep CNN model and apply the obtained model for the diagnosis of new test data. Our experimental results show our method achieves very promising results.

Dynamic Facial Expression Recognition With Atlas Construction and Sparse Representation.

In this paper, a new dynamic facial expression recognition method is proposed. Dynamic facial expression recognition is formulated as a longitudinal groupwise registration problem. The main contributions of this method lie in the following aspects: 1) subject-specific facial feature movements of different expressions are described by a diffeomorphic growth model; 2) salient longitudinal facial expression atlas is built for each expression by a sparse groupwise image registration method, which can describe the overall facial feature changes among the whole population and can suppress the bias due to large intersubject facial variations; and 3) both the image appearance information in spatial domain and topological evolution information in temporal domain are used to guide recognition by a sparse representation method. The proposed framework has been extensively evaluated on five databases for different applications: the extended Cohn-Kanade, MMI, FERA, and AFEW databases for dynamic facial expression recognition, and UNBC-McMaster database for spontaneous pain expression monitoring. This framework is also compared with several state-of-the-art dynamic facial expression recognition methods. The experimental results demonstrate that the recognition rates of the new method are consistently higher than other methods under comparison.

Median Robust Extended Local Binary Pattern for Texture Classification.

Local binary patterns (LBP) are considered among the most computationally efficient high-performance texture features. However, the LBP method is very sensitive to image noise and is unable to capture macrostructure information. To best address these disadvantages, in this paper, we introduce a novel descriptor for texture classification, the median robust extended LBP (MRELBP). Different from the traditional LBP and many LBP variants, MRELBP compares regional image medians rather than raw image intensities. A multiscale LBP type descriptor is computed by efficiently comparing image medians over a novel sampling scheme, which can capture both microstructure and macrostructure texture information. A comprehensive evaluation on benchmark data sets reveals MRELBP's high performance-robust to gray scale variations, rotation changes and noise-but at a low computational cost. MRELBP produces the best classification scores of 99.82%, 99.38%, and 99.77% on three popular Outex test suites. More importantly, MRELBP is shown to be highly robust to image noise, including Gaussian noise, Gaussian blur, salt-and-pepper noise, and random pixel corruption.