Multiprior Learning via Neural Architecture Search for Blind Face Restoration.

Blind face restoration (BFR) aims to recover high-quality (HQ) face images from low-quality (LQ) ones and usually resorts to facial priors for improving restoration performance. However, current methods still suffer from two major difficulties: 1) how to derive a powerful network architecture without extensive hand tuning and 2) how to capture complementary information from multiple facial priors in one network to improve restoration performance. To this end, we propose a face restoration searching network (FRSNet) to adaptively search the suitable feature extraction architecture within our specified search space, which can directly contribute to the restoration quality. On the basis of FRSNet, we further design our multiple facial prior searching network (MFPSNet) with a multiprior learning scheme. MFPSNet optimally extracts information from diverse facial priors and fuses the information into image features, ensuring that both external guidance and internal features are reserved. In this way, MFPSNet takes full advantage of semantic-level (parsing maps), geometric-level (facial heat maps), reference-level (facial dictionaries), and pixel-level (degraded images) information and, thus, generates faithful and realistic images. Quantitative and qualitative experiments show that the MFPSNet performs favorably on both synthetic and real-world datasets against the state-of-the-art (SOTA) BFR methods. The codes are publicly available at: https://github.com/YYJ1anG/MFPSNet.

Taming Self-Supervised Learning for Presentation Attack Detection: De-Folding and De-Mixing.

Biometric systems are vulnerable to presentation attacks (PAs) performed using various PA instruments (PAIs). Even though there are numerous PA detection (PAD) techniques based on both deep learning and hand-crafted features, the generalization of PAD for unknown PAI is still a challenging problem. In this work, we empirically prove that the initialization of the PAD model is a crucial factor for generalization, which is rarely discussed in the community. Based on such observation, we proposed a self-supervised learning-based method, denoted as DF-DM. Specifically, DF-DM is based on a global-local view coupled with de-folding and de-mixing to derive the task-specific representation for PAD. During de-folding, the proposed technique will learn region-specific features to represent samples in a local pattern by explicitly minimizing the generative loss. While de-mixing drives detectors to obtain the instance-specific features with global information for more comprehensive representation by minimizing the interpolation-based consistency. Extensive experimental results show that the proposed method can achieve significant improvements in terms of both face and fingerprint PAD in more complicated and hybrid datasets when compared with the state-of-the-art methods. When training in CASIA-FASD and Idiap Replay-Attack, the proposed method can achieve an 18.60% equal error rate (EER) in OULU-NPU and MSU-MFSD, exceeding the baseline performance by 9.54%. The source code of the proposed technique is available at https://github.com/kongzhecn/dfdm.

EDFace-Celeb-1 M: Benchmarking Face Hallucination With a Million-Scale Dataset.

Recent deep face hallucination methods show stunning performance in super-resolving severely degraded facial images, even surpassing human ability. However, these algorithms are mainly evaluated on non-public synthetic datasets. It is thus unclear how these algorithms perform on public face hallucination datasets. Meanwhile, most of the existing datasets do not well consider the distribution of races, which makes face hallucination methods trained on these datasets biased toward some specific races. To address the above two problems, in this paper, we build a public Ethnically Diverse Face dataset, EDFace-Celeb-1 M, and design a benchmark task for face hallucination. Our dataset includes 1.7 million photos that cover different countries, with relatively balanced race composition. To the best of our knowledge, it is the largest-scale and publicly available face hallucination dataset in the wild. Associated with this dataset, this paper also contributes various evaluation protocols and provides comprehensive analysis to benchmark the existing state-of-the-art methods. The benchmark evaluations demonstrate the performance and limitations of state-of-the-art algorithms. https://github.com/HDCVLab/EDFace-Celeb-1M.

Enhanced Spatio-Temporal Interaction Learning for Video Deraining: Faster and Better.

Video deraining is an important task in computer vision as the unwanted rain hampers the visibility of videos and deteriorates the robustness of most outdoor vision systems. Despite the significant success which has been achieved for video deraining recently, two major challenges remain: 1) how to exploit the vast information among successive frames to extract powerful spatio-temporal features across both the spatial and temporal domains, and 2) how to restore high-quality derained videos with a high-speed approach. In this paper, we present a new end-to-end video deraining framework, dubbed Enhanced Spatio-Temporal Interaction Network (ESTINet), which considerably boosts current state-of-the-art video deraining quality and speed. The ESTINet takes the advantage of deep residual networks and convolutional long short-term memory, which can capture the spatial features and temporal correlations among successive frames at the cost of very little computational resource. Extensive experiments on three public datasets show that the proposed ESTINet can achieve faster speed than the competitors, while maintaining superior performance over the state-of-the-art methods. https://github.com/HDCVLab/Enhanced-Spatio-Temporal-Interaction-Learning-for-Video-Deraining.

Hemicelluloses-based sprayable and biodegradable pesticide mulch films for Chinese cabbage growth.

In this study, one high-performance hemicelluloses (HC)-based sprayable and biodegradable pesticide mulch film was developed. Firstly, HC was transesterified with vinyl acetate (VA) to improve its solubility and film-forming ability. Then abamectin (ABA) was encapsulated by ß-cyclodextrin (ß-CD) to endow mulch film persistent anti-pesticide activity. After that, sodium alginate (SA) and gelatin were added to develop the mechanical performances of the mulch film. As a result, the obtained mulch film showed good characteristics, with optimum mechanical strength, elongation at break, water vapor permeability (WVP), swelling ratio (SR), and weight loss (biodegradability) of 7.9 ± 0.3 MPa, 43.6 ± 2.0 %, 2.1 ± 0.1 × 10-11 g mm m-2 s-1 kPa-1, 73.8 ± 2.0 %, and 69.3 %, respectively. After covering with mulch film, the soil moisture and temperature were developed to 90.8 % and 19.3 ± 0.2 °C, respectively, which could facilitate Chinese cabbage growth, with optimum germination rate of 98.6 ± 6.4 %.

Development of functional hydroxyethyl cellulose-based composite films for food packaging applications.

Cellulose-based functional composite films can be a good substitute for conventional plastic packaging to ensure food safety. In this study, the semi-transparent, mechanical strengthened, UV-shielding, antibacterial and biocompatible films were developed from hydroxyethyl cellulose Polyvinyl alcohol (PVA) and ε-polylysine (ε-PL) were respectively used as reinforcing agent and antibacterial agent, and chemical cross-linking among these three components were constructed using epichlorohydrin The maximum tensile strength and elongation at break were 95.9 ± 4.1 MPa and 148.8 ± 2.6%, respectively. TG-FTIR and XRD analyses indicated that chemical structure of the composite films could be well controlled by varying component proportion. From UV-Vis analysis, the optimum values of the percentage of blocking from UV-A and UV-B and ultraviolet protection factor values were 98.35%, 99.99% and 60.25, respectively. Additionally, the composite films exhibited good water vapor permeability, swelling behavior, antibacterial activity and biocompatibility. In terms of these properties, the shelf life of grapes could be extended to 6 days after packing with the composite film.

Acquiring new memories in neocortex of hippocampal-lesioned mice.

The hippocampus interacts with the neocortical network for memory retrieval and consolidation. Here, we found the lateral entorhinal cortex (LEC) modulates learning-induced cortical long-range gamma synchrony (20-40 Hz) in a hippocampal-dependent manner. The long-range gamma synchrony, which was coupled to the theta (7-10 Hz) rhythm and enhanced upon learning and recall, was mediated by inter-cortical projections from layer 5 neurons of the LEC to layer 2 neurons of the sensory and association cortices. Artificially induced cortical gamma synchrony across cortical areas improved memory encoding in hippocampal lesioned mice for originally hippocampal-dependent tasks. Mechanistically, we found that activities of cortical c-Fos labeled neurons, which showed egocentric map properties, were modulated by LEC-mediated gamma synchrony during memory recall, implicating a role of cortical synchrony to generate an integrative memory representation from disperse features. Our findings reveal the hippocampal mediated organization of cortical memories and suggest brain-machine interface approaches to improve cognitive function.

ASH1L haploinsufficiency results in autistic-like phenotypes in mice and links Eph receptor gene to autism spectrum disorder.

ASD-associated genes are enriched for synaptic proteins and epigenetic regulators. How those chromatin modulators establish ASD traits have remained unknown. We find haploinsufficiency of Ash1l causally induces anxiety and autistic-like behavior, including repetitive behavior, and alters social behavior. Specific depletion of Ash1l in forebrain induces similar ASD-associated behavioral defects. While the learning ability remains intact, the discrimination ability of Ash1l mutant mice is reduced. Mechanistically, deletion of Ash1l in neurons induces excessive synapses due to the synapse pruning deficits, especially during the post-learning period. Dysregulation of synaptic genes is detected in Ash1l mutant brain. Specifically, Eph receptor A7 is downregulated in Ash1l+/- mice through accumulating EZH2-mediated H3K27me3 in its gene body. Importantly, increasing activation of EphA7 in Ash1l+/- mice by supplying its ligand, ephrin-A5, strongly promotes synapse pruning and rescues discrimination deficits. Our results suggest that Ash1l haploinsufficiency is a highly penetrant risk factor for ASD, resulting from synapse pruning deficits.

Liquid Warping GAN With Attention: A Unified Framework for Human Image Synthesis.

We tackle human image synthesis, including human motion imitation, appearance transfer, and novel view synthesis, within a unified framework. It means that the model, once being trained, can be used to handle all these tasks. The existing task-specific methods mainly use 2D keypoints (pose) to estimate the human body structure. However, they only express the position information with no ability to characterize the personalized shape of the person and model the limb rotations. In this paper, we propose to use a 3D body mesh recovery module to disentangle the pose and shape. It can not only model the joint location and rotation but also characterize the personalized body shape. To preserve the source information, such as texture, style, color, and face identity, we propose an Attentional Liquid Warping GAN with Attentional Liquid Warping Block (AttLWB) that propagates the source information in both image and feature spaces to the synthesized reference. Specifically, the source features are extracted by a denoising convolutional auto-encoder for characterizing the source identity well. Furthermore, our proposed method can support a more flexible warping from multiple sources. To further improve the generalization ability of the unseen source images, a one/few-shot adversarial learning is applied. In detail, it first trains a model in an extensive training set. Then, it finetunes the model by one/few-shot unseen image(s) in a self-supervised way to generate high-resolution ( 512 ×512 and 1024 ×1024) results. Also, we build a new dataset, namely Impersonator (iPER) dataset, for the evaluation of human motion imitation, appearance transfer, and novel view synthesis. Extensive experiments demonstrate the effectiveness of our methods in terms of preserving face identity, shape consistency, and clothes details. All codes and dataset are available on https://impersonator.org/work/impersonator-plus-plus.html.

Preparation and Characterization of Chinese Leek Extract Incorporated Cellulose Composite Films.

This study aimed to prepare microcrystalline cellulose (MCC) films with good mechanical properties via plasticization using a Chinese leek (CL, Allium tuberosum) extract. The microstructure, crystal structure, mechanical properties, barrier ability, and thermal properties of the films were investigated. The chemical structure analysis of CL extract showed the existence of cellulose, lignin, and low-molecular-weight substances, such as polysaccharides, pectins, and waxes, which could act as plasticizers to enhance the properties of MCC:CL biocomposite films. The results of scanning electron microscopy and atomic force microscopy analyses indicated the good compatibility between MCC and CL extract. When the volume ratio of MCC:CL was 7:3, the MCC:CL biocomposite film exhibited the best comprehensive performance in terms of water vapor permeability (2.11 × 10-10 g/m·s·Pa), elongation at break (13.2 ± 1.8%), and tensile strength (24.7 ± 2.5 MPa). The results of a UV absorption analysis demonstrated that the addition of CL extract improved the UV-shielding performance of the films. Therefore, this work not only proposes a facile method to prepare MCC films with excellent mechanical properties via plasticization using CL extract but also broadens the potential applications of MCC films in the packaging area.

Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal.

Rain streaks and raindrops are two natural phenomena, which degrade image capture in different ways. Currently, most existing deep deraining networks take them as two distinct problems and individually address one, and thus cannot deal adequately with both simultaneously. To address this, we propose a Dual Attention-in-Attention Model (DAiAM) which includes two DAMs for removing both rain streaks and raindrops. Inside the DAM, there are two attentive maps - each of which attends to the heavy and light rainy regions, respectively, to guide the deraining process differently for applicable regions. In addition, to further refine the result, a Differential-driven Dual Attention-in-Attention Model (D-DAiAM) is proposed with a "heavy-to-light" scheme to remove rain via addressing the unsatisfying deraining regions. Extensive experiments on one public raindrop dataset, one public rain streak and our synthesized joint rain streak and raindrop (JRSRD) dataset have demonstrated that the proposed method not only is capable of removing rain streaks and raindrops simultaneously, but also achieves the state-of-the-art performance on both tasks.

Blind Motion Deblurring Super-Resolution: When Dynamic Spatio-Temporal Learning Meets Static Image Understanding.

Single-image super-resolution (SR) and multi-frame SR are two ways to super resolve low-resolution images. Single-Image SR generally handles each image independently, but ignores the temporal information implied in continuing frames. Multi-frame SR is able to model the temporal dependency via capturing motion information. However, it relies on neighbouring frames which are not always available in the real world. Meanwhile, slight camera shake easily causes heavy motion blur on long-distance-shot low-resolution images. To address these problems, a Blind Motion Deblurring Super-Reslution Networks, BMDSRNet, is proposed to learn dynamic spatio-temporal information from single static motion-blurred images. Motion-blurred images are the accumulation over time during the exposure of cameras, while the proposed BMDSRNet learns the reverse process and uses three-streams to learn Bidirectional spatio-temporal information based on well designed reconstruction loss functions to recover clean high-resolution images. Extensive experiments demonstrate that the proposed BMDSRNet outperforms recent state-of-the-art methods, and has the ability to simultaneously deal with image deblurring and SR.

Deep Dense Multi-Scale Network for Snow Removal Using Semantic and Depth Priors.

Images captured in snowy days suffer from noticeable degradation of scene visibility, which degenerates the performance of current vision-based intelligent systems. Removing snow from images thus is an important topic in computer vision. In this paper, we propose a Deep Dense Multi-Scale Network (DDMSNet) for snow removal by exploiting semantic and depth priors. As images captured in outdoor often share similar scenes and their visibility varies with depth from camera, such semantic and depth information provides a strong prior for snowy image restoration. We incorporate the semantic and depth maps as input and learn the semantic-aware and geometry-aware representation to remove snow. In particular, we first create a coarse network to remove snow from the input images. Then, the coarsely desnowed images are fed into another network to obtain the semantic and depth labels. Finally, we design a DDMSNet to learn semantic-aware and geometry-aware representation via a self-attention mechanism to produce the final clean images. Experiments evaluated on public synthetic and real-world snowy images verify the superiority of the proposed method, offering better results both quantitatively and qualitatively. https://github.com/HDCVLab/Deep-Dense-Multi-scale-Network https://github.com/HDCVLab/Deep-Dense-Multi-scale-Network.

Single Image Dehazing via Dual-Path Recurrent Network.

An image can be decomposed into two parts: the basic content and details, which usually correspond to the low-frequency and high-frequency information of the image. For a hazy image, these two parts are often affected by haze in different levels, e.g., high-frequency parts are often affected more serious than low-frequency parts. In this paper, we approach the single image dehazing problem as two restoration problems of recovering basic content and image details, and propose a Dual-Path Recurrent Network (DPRN) to simultaneously tackle these two problems. Specifically, the core structure of DPRN is a dual-path block, which uses two parallel branches to learn the characteristics of the basic content and details of hazy images. Each branch consists of several Convolutional LSTM blocks and convolution layers. Moreover, a parallel interaction function is incorporated into the dual-path block, thus enables each branch to dynamically fuse the intermediate features of both the basic content and image details. In this way, both branches can benefit from each other, and recover the basic content and image details alternately, therefore alleviating the color distortion problem in the dehazing process. Experimental results show that the proposed DPRN outperforms state-of-the-art image dehazing methods in terms of both quantitative accuracy and qualitative visual effect.

AD-VAT+: An Asymmetric Dueling Mechanism for Learning and Understanding Visual Active Tracking.

Visual Active Tracking (VAT) aims at following a target object by autonomously controlling the motion system of a tracker given visual observations. To learn a robust tracker for VAT, in this article, we propose a novel adversarial reinforcement learning (RL) method which adopts an Asymmetric Dueling mechanism, referred to as AD-VAT. In the mechanism, the tracker and target, viewed as two learnable agents, are opponents and can mutually enhance each other during the dueling/competition: i.e., the tracker intends to lockup the target, while the target tries to escape from the tracker. The dueling is asymmetric in that the target is additionally fed with the tracker's observation and action, and learns to predict the tracker's reward as an auxiliary task. Such an asymmetric dueling mechanism produces a stronger target, which in turn induces a more robust tracker. To improve the performance of the tracker in the case of challenging scenarios such as obstacles, we employ more advanced environment augmentation technique and two-stage training strategies, termed as AD-VAT+. For a better understanding of the asymmetric dueling mechanism, we also analyze the target's behaviors as the training proceeds and visualize the latent space of the tracker. The experimental results, in both 2D and 3D environments, demonstrate that the proposed method leads to a faster convergence in training and yields more robust tracking behaviors in different testing scenarios. The potential of the active tracker is also shown in real-world videos.

Coupled Network for Robust Pedestrian Detection With Gated Multi-Layer Feature Extraction and Deformable Occlusion Handling.

Pedestrian detection methods have been significantly improved with the development of deep convolutional neural networks. Nevertheless, detecting ismall-scaled pedestrians and occluded pedestrians remains a challenging problem. In this paper, we propose a pedestrian detection method with a couple-network to simultaneously address these two issues. One of the sub-networks, the gated multi-layer feature extraction sub-network, aims to adaptively generate discriminative features for pedestrian candidates in order to robustly detect pedestrians with large variations on scale. The second sub-network targets on handling the occlusion problem of pedestrian detection by using deformable regional region of interest (RoI)-pooling. We investigate two different gate units for the gated sub-network, namely, the channel-wise gate unit and the spatio-wise gate unit, which can enhance the representation ability of the regional convolutional features among the channel dimensions or across the spatial domain, repetitively. Ablation studies have validated the effectiveness of both the proposed gated multi-layer feature extraction sub-network and the deformable occlusion handling sub-network. With the coupled framework, our proposed pedestrian detector achieves promising results on both two pedestrian datasets, especially on detecting small or occluded pedestrians. On the CityPersons dataset, the proposed detector achieves the lowest missing rates (i.e. 40.78% and 34.60%) on detecting small and occluded pedestrians, surpassing the second best comparison method by 6.0% and 5.87%, respectively.

STFlow: Self-Taught Optical Flow Estimation Using Pseudo Labels.

The Deep learning of optical flow has been an active area for its empirical success. For the difficulty of obtaining accurate dense correspondence labels, unsupervised learning of optical flow has drawn more and more attention, while the accuracy is still far from satisfaction. By holding the philosophy that better estimation models can be trained with betterapproximated labels, which in turn can be obtained from better estimation models, we propose a self-taught learning framework to continually improve the accuracy using self-generated pseudo labels. The estimated optical flow is first filtered by bidirectional flow consistency validation and occlusion-aware dense labels are then generated by edge-aware interpolation from selected sparse matches. Moreover, by combining reconstruction loss with regression loss on the generated pseudo labels, the performance is further improved. The experimental results demonstrate that our models achieve state-of-the-art results among unsupervised methods on the public KITTI, MPI-Sintel and Flying Chairs datasets.

Construction of EVA/chitosan based PEG-PCL micelles nanocomposite films with controlled release of iprodione and its application in pre-harvest treatment of grapes.

A novel nanocomposite poly(ethylene-co-vinyl acetate) (EVA) film with controlled in vitro release of iprodione (ID) was prepared. Chitosan (CS) was used as the reinforcement which enhances the water and oxygen permeability of films. ID loaded poly(ethylene glycol)-poly(ε-caprolactone) (PEG-PCL) (IPP) micelles were used as the drug carrier which endows the films with antifungal and controlled release ability. IPP micelles with spherical shape and uniform size were obtained, and the maximum encapsulation efficacy (EE) was 91.17 ± 5.03% by well controlling the feeding amount of ID. Incorporation CS could improve the oxygen and moisture permeability of films, and the maximum oxygen permeability (OP) and water vapor transmission rate (WVTR) were 477.84 ± 13.03 cc/(m2·d·0.1 MPa) and 8.60 ± 0.25 g m-2 d-1, respectively. After loading IPP micelles, the films showed an improved antifungal ability and temperature-sensitive drug release behavior, and were found to enhance the quality of grapes by pre-harvest spraying.

End-to-End Active Object Tracking and Its Real-World Deployment via Reinforcement Learning.

We study active object tracking, where a tracker takes visual observations (i.e., frame sequences) as input and produces the corresponding camera control signals as output (e.g., move forward, turn left, etc.). Conventional methods tackle tracking and camera control tasks separately, and the resulting system is difficult to tune jointly. These methods also require significant human efforts for image labeling and expensive trial-and-error system tuning in the real world. To address these issues, we propose, in this paper, an end-to-end solution via deep reinforcement learning. A ConvNet-LSTM function approximator is adopted for the direct frame-to-action prediction. We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training. The tracker trained in simulators (ViZDoom and Unreal Engine) demonstrates good generalization behaviors in the case of unseen object moving paths, unseen object appearances, unseen backgrounds, and distracting objects. The system is robust and can restore tracking after occasional lost of the target being tracked. We also find that the tracking ability, obtained solely from simulators, can potentially transfer to real-world scenarios. We demonstrate successful examples of such transfer, via experiments over the VOT dataset and the deployment of a real-world robot using the proposed active tracker trained in simulation.

Mutations in ASH1L confer susceptibility to Tourette syndrome.

Tourette syndrome (TS) is a childhood-onset neuropsychiatric disorder characterized by repetitive motor movements and vocal tics. The clinical manifestations of TS are complex and often overlap with other neuropsychiatric disorders. TS is highly heritable; however, the underlying genetic basis and molecular and neuronal mechanisms of TS remain largely unknown. We performed whole-exome sequencing of a hundred trios (probands and their parents) with detailed records of their clinical presentations and identified a risk gene, ASH1L, that was both de novo mutated and associated with TS based on a transmission disequilibrium test. As a replication, we performed follow-up targeted sequencing of ASH1L in additional 524 unrelated TS samples and replicated the association (P value = 0.001). The point mutations in ASH1L cause defects in its enzymatic activity. Therefore, we established a transgenic mouse line and performed an array of anatomical, behavioral, and functional assays to investigate ASH1L function. The Ash1l+/- mice manifested tic-like behaviors and compulsive behaviors that could be rescued by the tic-relieving drug haloperidol. We also found that Ash1l disruption leads to hyper-activation and elevated dopamine-releasing events in the dorsal striatum, all of which could explain the neural mechanisms for the behavioral abnormalities in mice. Taken together, our results provide compelling evidence that ASH1L is a TS risk gene.