Line-scanning technique using a PDMS grating in a microscope configuration.

We report the concept of a 1550 nm laser line scanning microscope based on a polydimethylsiloxane (PDMS) grating with scanning by stretching the PDMS grating to improve the scanning speed and enable low-cost scanning. Zemax is used to verify the possibility of realizing the system by simulating the illumination light path and the emission light path. The scanning field of view is 0.11m m×0.11m m, and the modulation transfer function (MTF) data of the 0th, ±1st, and ±2 nd diffraction orders in the illumination light path and the emission light path, respectively, meet the requirements of the diffraction limit resolution at the cutoff frequencies.

An integrated strategy for rapid on-site screening and determination of prometryn residues in herbs.

We produced a prometryn-specific monoclonal antibody and propose a strategy for convenient on-site detection of prometryn residues in herbs for the first time. This strategy has perfect applicability in a complex herbal medicine matrix. The strategy combines a semiquantitative immunochromatographic strip assay with a heterologous indirect competitive ELISA. When there was no matrix interference, the ELISA had a half-maximal inhibitory concentration of 2.6 ng·mL-1 and a limit of detection of 0.2 ng·mL-1. The immunochromatographic strip assay can be completed within 5 min with a visual limit of detection of 1 ng·mL-1. Although the sample matrix had different effects on the sensitivity of the antibody, excellent repeatability and accuracy were achieved. The method was successfully applied for the screening and determination of prometryn residue in multiple complex herb samples for the first time, and the results were in good agreement with those obtained by liquid chromatography-tandem mass spectrometry. The proposed strategy is rapid, of high-throughput, and of low cost, and may be a promising choice for on-site detection of prometryn in different kinds of herbs. Graphical abstract.

[Preparation of highly sensitive monoclonal antibody against aflatoxin B_1 and its application in rapid detection of contamination in Ziziphi Spinosae Semen].

A highly sensitive monoclonal antibody against aflatoxin B_1(AFB_1) was prepared and an indirect competition enzyme-linked immunosorbent assay(ic-ELISA) was established based on the antibody which was used for high-throughput and rapid screening of AFB_1 contamination in Chinese herbal medicines to ensure the safety of medication. In this study, the structure of AFB_1 was modified by improved oxime method, and the carrier protein was coupled by EDC-NHS method to obtain the complete antigen of AFB_1, which was more convenient and environmental friendly. The Balb/c female mice were immunized using increasing the immunization dose and various ways of injection, and finally the AFB_1 monoclonal antibody was prepared. The AFB_1 monoclonal antibody belongs to IgG_(2 b) immunoglobulin by identifying its immunological characteristics, and its sensitivity(IC_(50)) can reach 0.15 µg·L~(-1), and the affi-nity is 2.81×10~8 L·mol~(-1). The cross-reaction rates of AFB_2, AFG_1, and AFG_2 were 35.07%, 8.75%, and 1.15%, respectively, and there was almost no cross-reactivity with other mycotoxins. Based on the high sensitivity and specificity of the antibody, an ic-ELISA method was established and applied to the determination of AFB_1 contamination in Ziziphi Spinosae Semen. According to the matrix matching standard curve, the linear concentration range for AFB_1 was 0.05-0.58 µg·L~(-1)(R~2=0.992), the recoveries were 88.00%-119.0%, and the detection limit was 1.69 µg·kg~(-1). The AFB_1 in 33 batches of Ziziphi Spinosae Semen samples was determined by ic-ELISA, and the contamination level was 3.62-206.58 µg·kg~(-1). The linear correlation coefficient between the detection results of ic-ELISA and UHPLC-MS/MS was 0.996, and there were no false positive and false negative cases. It indicates that the established ic-ELISA is accurate and reliable, and could provide a simple and effective technique for fast screening of AFB_1 contamination in Ziziphi Spinosae Semen, and also could be considered as the reference for the detection and monitoring of AFB_1 contamination in other Chinese herbal medicines.

[QuEchERS pretreatment coupled to gas chromatography and tandem mass spectrometry to fast determination of 34 pesticide residues in Glycyrrhizae Radix et Rhizoma].

This study proposed a quantitative method for 34 pesticides including organochlorine,organophosphorus and pyrethroids in Glycyrrhizae Radix et Rhizoma herbs and medicinal slices,and analyzed the pesticide residues of collected Glycyrrhizae Radix et Rhizoma samples from different regions. With acetonitrile extraction and optimized Qu Ech ERS purification,the 32 batches of Glycyrrhizae Radix et Rhizoma herbs and medicinal slices were analyzed by matrix matching standard curve quantitative analysis under GC-MS/MS multi-response monitoring( MRM) mode. This study investigated the pretreatment of Glycyrrhizae Radix et Rhizoma samples based on the Qu Ech ERS method of Chinese Pharmacopoeia( 2015 edition,4),and the result showed that the recoveries of some pesticide was low and pigment has a strong interference in analysis,which result in worse purification effect. Therefore,this paper further optimized the Qu Ech ERS method and corrected the matrix matching standard curve method,and compensated the qualitative and quantitative effects of matrix effects on the detected target compounds in Glycyrrhizae Radix et Rhizoma. The results showed that 34 kinds of pesticide had good linear( R~2 of 0. 996 4 or higher) within a covering 0. 01-0. 2 mg·kg~(-1) concentration range. The limits of quantitation are less than 0. 01 mg·kg~(-1). This method was further applied to the simultaneous determination of 34 pesticide residues of typical organochlorine,organophosphorus and pyrethroids in 32 batches of Glycyrrhizae Radix et Rhizoma herbs and medicinal slices. Six batches containing beta-endosulfan,thiosulphate,o,p'-DDD and thrta-cypermethrin were detected,but none of them exceeded the limit of pesticide residues stipulated in the Chinese Pharmacopoeia and the EU Pharmacopoeia. This study indicates that the established method is rapid,convenient,accurate,and sensitive,which provides a rapid and efficient method for the simultaneous determination of typical organochlorine,organophosphorus and pyrethroids in Glycyrrhizae Radix et Rhizoma.

Knockoffs-SPR: Clean Sample Selection in Learning With Noisy Labels.

A noisy training set usually leads to the degradation of the generalization and robustness of neural networks. In this article, we propose a novel theoretically guaranteed clean sample selection framework for learning with noisy labels. Specifically, we first present a Scalable Penalized Regression (SPR) method, to model the linear relation between network features and one-hot labels. In SPR, the clean data are identified by the zero mean-shift parameters solved in the regression model. We theoretically show that SPR can recover clean data under some conditions. Under general scenarios, the conditions may be no longer satisfied; and some noisy data are falsely selected as clean data. To solve this problem, we propose a data-adaptive method for Scalable Penalized Regression with Knockoff filters (Knockoffs-SPR), which is provable to control the False-Selection-Rate (FSR) in the selected clean data. To improve the efficiency, we further present a split algorithm that divides the whole training set into small pieces that can be solved in parallel to make the framework scalable to large datasets. While Knockoffs-SPR can be regarded as a sample selection module for a standard supervised training pipeline, we further combine it with a semi-supervised algorithm to exploit the support of noisy data as unlabeled data. Experimental results on several benchmark datasets and real-world noisy datasets show the effectiveness of our framework and validate the theoretical results of Knockoffs-SPR.

Brain-machine interactive neuromodulation research tool with edge AI computing.

Closed-loop neuromodulation with intelligence methods has shown great potentials in providing novel neuro-technology for treating neurological and psychiatric diseases. Development of brain-machine interactive neuromodulation strategies could lead to breakthroughs in precision and personalized electronic medicine. The neuromodulation research tool integrating artificial intelligent computing and performing neural sensing and stimulation in real-time could accelerate the development of closed-loop neuromodulation strategies and translational research into clinical application. In this study, we developed a brain-machine interactive neuromodulation research tool (BMINT), which has capabilities of neurophysiological signals sensing, computing with mainstream machine learning algorithms and delivering electrical stimulation pulse by pulse in real-time. The BMINT research tool achieved system time delay under 3 ms, and computing capabilities in feasible computation cost, efficient deployment of machine learning algorithms and acceleration process. Intelligent computing framework embedded in the BMINT enable real-time closed-loop neuromodulation developed with mainstream AI ecosystem resources. The BMINT could provide timely contribution to accelerate the translational research of intelligent neuromodulation by integrating neural sensing, edge AI computing and stimulation with AI ecosystems.

ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priors.

Image inpainting involves filling missing areas of a corrupted image. Despite impressive results have been achieved recently, restoring images with both vivid textures and reasonable structures remains a significant challenge. Previous methods have primarily addressed regular textures while disregarding holistic structures due to the limited receptive fields of Convolutional Neural Networks (CNNs). To this end, we study learning a Zero-initialized residual addition based Incremental Transformer on Structural priors (ZITS++), an improved model upon our conference work, ZITS (Dong et al. 2022). Specifically, given one corrupt image, we present the Transformer Structure Restorer (TSR) module to restore holistic structural priors at low image resolution, which are further upsampled by Simple Structure Upsampler (SSU) module to higher image resolution. To recover image texture details, we use the Fourier CNN Texture Restoration (FTR) module, which is strengthened by Fourier and large-kernel attention convolutions. Furthermore, to enhance the FTR, the upsampled structural priors from TSR are further processed by Structure Feature Encoder (SFE) and optimized with the Zero-initialized Residual Addition (ZeroRA) incrementally. Besides, a new masking positional encoding is proposed to encode the large irregular masks. Compared with ZITS, ZITS++ improves the FTR's stability and inpainting ability with several techniques. More importantly, we comprehensively explore the effects of various image priors for inpainting and investigate how to utilize them to address high-resolution image inpainting with extensive experiments. This investigation is orthogonal to most inpainting approaches and can thus significantly benefit the community.

Pixel2Mesh++: 3D Mesh Generation and Refinement From Multi-View Images.

We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. While many previous works learn to hallucinate the shape directly from priors, we adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Instead of building a direct mapping function from images to 3D shape, our model learns to predict series of deformations to improve a coarse shape iteratively. Inspired by traditional multiple view geometry methods, our network samples nearby area around the initial mesh's vertex locations and reasons an optimal deformation using perceptual feature statistics built from multiple input images. Extensive experiments show that our model produces accurate 3D shapes that are not only visually plausible from the input perspectives, but also well aligned to arbitrary viewpoints. With the help of physically driven architecture, our model also exhibits generalization capability across different semantic categories, and the number of input images. Model analysis experiments show that our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable renderer for test-time optimization.

DeepSFM: Robust Deep Iterative Refinement for Structure From Motion.

Structure from Motion (SfM) is a fundamental computer vision problem which has not been well handled by deep learning. One of the promising solutions is to apply explicit structural constraint, e.g. 3D cost volume, into the neural network. Obtaining accurate camera pose from images alone can be challenging, especially with complicate environmental factors. Existing methods usually assume accurate camera poses from GT or other methods, which is unrealistic in practice and additional sensors are needed. In this work, we design a physical driven architecture, namely DeepSFM, inspired by traditional Bundle Adjustment, which consists of two cost volume based architectures to iteratively refine depth and pose. The explicit constraints on both depth and pose, when combined with the learning components, bring the merit from both traditional BA and emerging deep learning technology. To speed up the learning and inference efficiency, we apply the Gated Recurrent Units (GRUs)-based depth and pose update modules with coarse to fine cost volumes on the iterative refinements. In addition, with the extended residual depth prediction module, our model can be adapted to dynamic scenes effectively. Extensive experiments on various datasets show that our model achieves the state-of-the-art performance with superior robustness against challenging inputs.

H4MER: Human 4D Modeling by Learning Neural Compositional Representation With Transformer.

Despite the impressive results achieved by deep learning based 3D reconstruction, the techniques of directly learning to model 4D human captures with detailed geometry have been less studied. This work presents a novel neural compositional representation for Human 4D Modeling with transformER (H4MER). Specifically, our H4MER is a compact and compositional representation for dynamic human by exploiting the human body prior from the widely used SMPL parametric model. Thus, H4MER can represent a dynamic 3D human over a temporal span with the codes of shape, initial pose, motion and auxiliaries. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation, followed by per-frame compensation for pose and geometry details with the residual encoded in the auxiliary codes. We present a novel Transformer-based feature extractor and conditional GRU decoder to facilitate learning and improve the representation capability. Extensive experiments demonstrate our method is not only effective in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks, including monocular video fitting, motion retargeting, 4D completion, and future prediction.

Exploring Structural Sparsity of Deep Networks Via Inverse Scale Spaces.

The great success of deep neural networks is built upon their over-parameterization, which smooths the optimization landscape without degrading the generalization ability. Despite the benefits of over-parameterization, a huge amount of parameters makes deep networks cumbersome in daily life applications. On the other hand, training neural networks without over-parameterization faces many practical problems, e.g., being trapped in the local optimal. Though techniques such as pruning and distillation are developed, they are expensive in fully training a dense network as backward selection methods; and there is still a void on systematically exploring forward selection methods for learning structural sparsity in deep networks. To fill in this gap, this paper proposes a new approach based on differential inclusions of inverse scale spaces. Specifically, our method can generate a family of models from simple to complex ones along the dynamics via coupling a pair of parameters, such that over-parameterized deep models and their structural sparsity can be explored simultaneously. This kind of differential inclusion scheme has a simple discretization, dubbed Deep structure splitting Linearized Bregman Iteration (DessiLBI), whose global convergence in learning deep networks could be established under the Kurdyka-Lojasiewicz framework. Particularly, we explore several applications of DessiLBI, including finding sparse structures of networks directly via the coupled structure parameter and growing networks from simple to complex ones progressively. Experimental evidence shows that our method achieves comparable and even better performance than the competitive optimizers in exploring the sparse structure of several widely used backbones on the benchmark datasets. Remarkably, with early stopping, our method unveils "winning tickets" in early epochs: the effective sparse network structures with comparable test accuracy to fully trained over-parameterized models, that are further transferable to similar alternative tasks. Furthermore, our method is able to grow networks efficiently with adaptive filter configurations, demonstrating the good performance with much less computational cost. Codes and models can be downloaded at https://github.com/DessiLBI2020/DessiLBI.

A simple and rapid strategy for monitoring vancomycin concentration in serum using a multicolor immunosensor based on the ratio of gold nanobipyramids and a product of cetyltrimethylammonium bromide-blue oxide of 3,3',5,5'-tetramethylbenzidine interaction.

Rapid and simple monitoring of vancomycin (VAN) concentration in blood is a vital strategy for maximizing therapeutic efficacy, minimizing toxicity and developing a personalized treatment plan. In this work, a simple multicolor immunosensor is proposed to enable rapid monitoring of VAN concentration in serum, without using any expensive and bulky instrument. The multicolor immunosensor platform is a system that works based on the principle that the product of cetyltrimethylammonium bromide-blue oxide of 3,3',5,5'-tetramethylbenzidine interaction (CTAB/TMB+) and TMB+ increases simultaneously with the decrease in VAN concentration, whereas AuNBPs are insensitive to VAN. The result indicates that the reaction system has multiple distinct color variants. These distinct vivid color changes can be easily distinguished with the naked eye, and smartphone-relied red-green-blue (RGB) analysis can be used for quantitative detection, without the need for any sophisticated apparatus. The construction of this multicolor system omitted the hydrochloric acid (HCl) addition step, growth or etch procedure of noble metal nanoparticles in traditional multicolor immunosensors, which can improve the time-cost and tedious operation. The proposed method achieves a good linear relationship (r2 = 0.9679), accuracy (recoveries, 99.25-126.96%) and repeatability (n = 3, RSD, 1.27-2.17%). Moreover, a good correlation was observed between the results obtained from the new method and liquid chromatography-tandem mass spectrometry (r2 = 0.8993, n = 8). In summary, this work provides a new low-cost, facile and user-friendly immunosensor platform with high potential for rapid detection of VAN and various other drugs at home, hospital rooms and rural areas.

PatchMix Augmentation to Identify Causal Features in Few-Shot Learning.

The task of Few-shot learning (FSL) aims to transfer the knowledge learned from base categories with sufficient labelled data to novel categories with scarce known information. It is currently an important research question and has great practical values in the real-world applications. Despite extensive previous efforts are made on few-shot learning tasks, we emphasize that most existing methods did not take into account the distributional shift caused by sample selection bias in the FSL scenario. Such a selection bias can induce spurious correlation between the semantic causal features, that are causally and semantically related to the class label, and the other non-causal features. Critically, the former ones should be invariant across changes in distributions, highly related to the classes of interest, and thus well generalizable to novel classes, while the latter ones are not stable to changes in the distribution. To resolve this problem, we propose a novel data augmentation strategy dubbed as PatchMix that can break this spurious dependency by replacing the patch-level information and supervision of the query images with random gallery images from different classes from the query ones. We theoretically show that such an augmentation mechanism, different from existing ones, is able to identify the causal features. To further make these features to be discriminative enough for classification, we propose Correlation-guided Reconstruction (CGR) and Hardness-Aware module for instance discrimination and easier discrimination between similar classes. Moreover, such a framework can be adapted to the unsupervised FSL scenario. The utility of our method is demonstrated on the state-of-the-art results consistently achieved on several benchmarks including miniImageNet, tieredImageNet, CIFAR-FS, CUB, Cars, Places and Plantae, in all settings of single-domain, cross-domain and unsupervised FSL. By studying the intra-variance property of learned features and visualizing the learned features, we further quantitatively and qualitatively show that such a promising result is due to the effectiveness in learning causal features.

Reinforcing Generated Images via Meta-learning for One-Shot Fine-Grained Visual Recognition.

One-shot fine-grained visual recognition often suffers from the problem of training data scarcity for new fine-grained classes. To alleviate this problem, off-the-shelf image generation techniques based on Generative Adversarial Networks (GANs) can potentially create additional training images. However, these GAN-generated images are often not helpful for actually improving the accuracy of one-shot fine-grained recognition. In this paper, we proposes a meta-learning framework to combine generated images with original images, so that the resulting hybrid training images can improve one-shot learning. Specifically, the generic image generator is updated by a few training instances of novel classes, and a Meta Image Reinforcing Network (MetaIRNet) is proposed to conduct one-shot fine-grained recognition as well as image reinforcement. Our experiments demonstrate consistent improvement over baselines on one-shot fine-grained image classification benchmarks. Furthermore, our analysis shows that the reinforced images have more diversity compared to the original and GAN-generated images.

Generalized Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data.

The vanilla Few-shot Learning (FSL) learns to build a classifier for a new concept from one or very few target examples, with the general assumption that source and target classes are sampled from the same domain. Recently, the task of Cross-Domain Few-Shot Learning (CD-FSL) aims at tackling the FSL where there is a huge domain shift between the source and target datasets. Extensive efforts on CD-FSL have been made via either directly extending the meta-learning paradigm of vanilla FSL methods, or employing massive unlabeled target data to help learn models. In this paper, we notice that in the CD-FSL task, the few labeled target images have never been explicitly leveraged to inform the model in the training stage. However, such a labeled target example set is very important to bridge the huge domain gap. Critically, this paper advocates a more practical training scenario for CD-FSL. And our key insight is to utilize a few labeled target data to guide the learning of the CD-FSL model. Technically, we propose a novel Generalized Meta-learning based Feature-Disentangled Mixup network, namely GMeta-FDMixup. We make three key contributions of utilizing GMeta-FDMixup to address CD-FSL. Firstly, we present two mixup modules - mixup-P and mixup-M that help facilitate utilizing the unbalanced and disjoint source and target datasets. These two novel modules enable diverse image generation for training the model on the source domain. Secondly, to narrow the domain gap explicitly, we contribute a novel feature disentanglement module that learns to decouple the domain-irrelevant and domain-specific features. By stripping the domain-specific features, we alleviate the negative effects caused by the domain inductive bias. Finally, we repurpose a new contrastive learning module, dubbed ConL. ConL prevents the model from only capturing category-related features via introducing contrastive loss. Thus, the generalization ability on novel categories is improved. Extensive experimental results on two benchmarks show the superiority of our setting and the effectiveness of our method. Code and models will be released.

How to Trust Unlabeled Data? Instance Credibility Inference for Few-Shot Learning.

Deep learning based models have excelled in many computer vision tasks and appear to surpass humans' performance. However, these models require an avalanche of expensive human labeled training data and many iterations to train their large number of parameters. This severely limits their scalability to the real-world long-tail distributed categories, some of which are with a large number of instances, but with only a few manually annotated. Learning from such extremely limited labeled examples is known as Few-Shot Learning (FSL). Different to prior arts that leverage meta-learning or data augmentation strategies to alleviate this extremely data-scarce problem, this paper presents a statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the support of unlabeled instances for few-shot visual recognition. Typically, we repurpose the self-taught learning paradigm to predict pseudo-labels of unlabeled instances with an initial classifier trained from the few shot and then select the most confident ones to augment the training set to re-train the classifier. This is achieved by constructing a (Generalized) Linear Model (LM/GLM) with incidental parameters to model the mapping from (un-)labeled features to their (pseudo-)labels, in which the sparsity of the incidental parameters indicates the credibility of the corresponding pseudo-labeled instance. We rank the credibility of pseudo-labeled instances along the regularization path of their corresponding incidental parameters, and the most trustworthy pseudo-labeled examples are preserved as the augmented labeled instances. This process is repeated until all the unlabeled samples are included in the expanded training set. Theoretically, under the conditions of restricted eigenvalue, irrepresentability, and large error, our approach is guaranteed to collect all the correctly-predicted pseudo-labeled instances from the noisy pseudo-labeled set. Extensive experiments under two few-shot settings show the effectiveness of our approach on four widely used few-shot visual recognition benchmark datasets including miniImageNet, tieredImageNet, CIFAR-FS, and CUB. Code and models are released at https://github.com/Yikai-Wang/ICI-FSL.

A comprehensive review on the pretreatment and detection methods of neonicotinoid insecticides in food and environmental samples.

In recent years, the residues of neonicotinoid insecticide in food and environmental samples have attracted extensive attention. Neonicotinoids have many adverse effects on human health, such as cancer, chronic disease, birth defects, and infertility. They have substantial toxicity to some non-target organisms (especially bees). Hence, monitoring the residues of neonicotinoid insecticides in foodstuffs is necessary to guarantee public health and ecological stability. This review aims to summarize and assess the metabolic features, residue status, sample pretreatment methods (solid-phase extraction (SPE), Quick, Easy, Cheap, Effective, Rugged, and Safe (QuEChERS), and some novel pretreatment methods), and detection methods (instrument detection, immunoassay, and some innovative detection methods) for neonicotinoid insecticide residues in food and environmental samples. This review provides detailed references and discussion for the analysis of neonicotinoid insecticide residues, which can effectively promote the establishment of innovative detection methods for neonicotinoid insecticide residues.

Pixel2Mesh: 3D Mesh Model Generation via Image Guided Deformation.

In this paper, we propose an end-to-end deep learning architecture that generates 3D triangular meshes from single color images. Restricted by the nature of prevalent deep learning techniques, the majority of previous works represent 3D shapes in volumes or point clouds. However, it is non-trivial to convert these representations to compact and ready-to-use mesh models. Unlike the existing methods, our network represents 3D shapes in meshes, which are essentially graphs and well suited for graph-based convolutional neural networks. Leveraging perceptual features extracted from an input image, our network produces the correct geometry by progressively deforming an ellipsoid. To make the whole deformation procedure stable, we adopt a coarse-to-fine strategy, and define various mesh/surface related losses to capture properties of various aspects, which benefits producing the visually appealing and physically accurate 3D geometry. In addition, our model by nature can be adapted to objects in specific domains, e.g., human faces, and be easily extended to learn per-vertex properties, e.g., color. Extensive experiments show that our method not only qualitatively produces the mesh model with better details, but also achieves the higher 3D shape estimation accuracy compared against the state-of-the-arts.

Learning Layer-Skippable Inference Network.

The process of learning good representations for machine learning tasks can be very computationally expensive. Typically, we facilitate the same backbones learned on the training set to infer the labels of testing data. Interestingly, This learning and inference paradigm, however, is quite different from the typical inference scheme of human biological visual systems. Essentially, neuroscience studies have shown that the right hemisphere of the human brain predominantly makes a fast processing of low-frequency spatial signals, while the left hemisphere more focuses on analyzing high-frequency information in a slower way. And the low-pass analysis helps facilitate the high-pass analysis via a feedback form. Inspired by this biological vision mechanism, this paper explores the possibility of learning a layer-skippable inference network. Specifically, we propose a layer-skippable network that dynamically carries out coarse-tofine object categorization. Such a network has two branches to jointly deal with both coarse and fine-grained classification tasks. The layer-skipping mechanism is proposed to learn a gating network by generating dynamic inference graphs, and reducing the computational cost by detouring the inference path from some layers. This adaptive path inference strategy endows the network with better flexibility and larger capacity and makes the high-performance deep networks with dynamic structures. To efficiently train the gating network, a novel ranking-based loss function is presented. Furthermore, the learned representations are enhanced by the proposed top-down feedback facilitation and feature-wise affine transformation, individually. The former one employs features of a coarse branch to help the finegrained object recognition task, while the latter one encodes the selected path to enhance the final feature representations. Extensive experiments are conducted on several widely used coarse-to-fine object categorization benchmarks, and promising results are achieved by our proposed model. Quite surprisingly, our layer-skipping mechanism improves the network robustness to adversarial attacks. The codes and models are released on https://github.com/avalonstrel/DSN.

Pose-Guided Person Image Synthesis in the Non-iconic Views.

Generating realistic images with the guidance of reference images and human poses is challenging. Despite the success of previous works on synthesizing person images in the iconic views, no efforts are made towards the task of poseguided image synthesis in the non-iconic views. Particularly, we find that previous models cannot handle such a complex task, where the person images are captured in the non-iconic views by commercially-available digital cameras. To this end, we propose a new framework - Multi-branch Refinement Network (MR-Net), which utilizes several visual cues, including target person poses, foreground person body and scene images parsed. Furthermore, a novel Region of Interest (RoI) perceptual loss is proposed to optimize the MR-Net. Extensive experiments on two non-iconic datasets, Penn Action and BBC-Pose, as well as an iconic dataset - Market-1501, show the efficacy of the proposed model that can tackle the problem of pose-guided person image generation from the non-iconic views. The data, models, and codes are downloadable from https://github.com/loadder/MR-Net.