Transferring Annotator- and Instance-dependent Transition Matrix for Learning from Crowds.

Learning from crowds describes that the annotations of training data are obtained with crowd-sourcing services. Multiple annotators each complete their own small part of the annotations, where labeling mistakes that depend on annotators occur frequently. Modeling the label-noise generation process by the noise transition matrix is a powerful tool to tackle the label noise. In real-world crowd-sourcing scenarios, noise transition matrices are both annotator- and instance-dependent. However, due to the high complexity of annotator- and instance-dependent transition matrices (AIDTM), annotation sparsity, which means each annotator only labels a tiny part of instances, makes modeling AIDTM very challenging. Without prior knowledge, existing works simplify the problem by assuming the transition matrix is instance-independent or using simple parametric ways, which lose modeling generality. Motivated by this, we target a more realistic problem, estimating general AIDTM in practice. Without losing modeling generality, we parameterize AIDTM with deep neural networks. To alleviate the modeling challenge, we suppose every annotator shares its noise pattern with similar annotators, and estimate AIDTM via knowledge transfer. We hence first model the mixture of noise patterns by all annotators, and then transfer this modeling to individual annotators. Furthermore, considering that the transfer from the mixture of noise patterns to individuals may cause two annotators with highly different noise generations to perturb each other, we employ the knowledge transfer between identified neighboring annotators to calibrate the modeling. Theoretical analyses are derived to demonstrate that both the knowledge transfer from global to individuals and the knowledge transfer between neighboring individuals can effectively help mitigate the challenge of modeling general AIDTM. Experiments confirm the superiority of the proposed approach on synthetic and real-world crowd-sourcing data. The implementation is available at https://github.com/tmllab/TAIDTM.

Inverse Image Frequency for Long-Tailed Image Recognition.

The long-tailed distribution is a common phenomenon in the real world. Extracted large scale image datasets inevitably demonstrate the long-tailed property and models trained with imbalanced data can obtain high performance for the over-represented categories, but struggle for the under-represented categories, leading to biased predictions and performance degradation. To address this challenge, we propose a novel de-biasing method named Inverse Image Frequency (IIF). IIF is a multiplicative margin adjustment transformation of the logits in the classification layer of a convolutional neural network. Our method achieves stronger performance than similar works and it is especially useful for downstream tasks such as long-tailed instance segmentation as it produces fewer false positive detections. Our extensive experiments show that IIF surpasses the state of the art on many long-tailed benchmarks such as ImageNet-LT, CIFAR-LT, Places-LT and LVIS, reaching 55.8% top-1 accuracy with ResNet50 on ImageNet-LT and 26.3% segmentation AP with MaskRCNN ResNet50 on LVIS. Code available at https://github.com/kostas1515/iif.

Redesigning Multi-Scale Neural Network for Crowd Counting.

Perspective distortions and crowd variations make crowd counting a challenging task in computer vision. To tackle it, many previous works have used multi-scale architecture in deep neural networks (DNNs). Multi-scale branches can be either directly merged (e.g. by concatenation) or merged through the guidance of proxies (e.g. attentions) in the DNNs. Despite their prevalence, these combination methods are not sophisticated enough to deal with the per-pixel performance discrepancy over multi-scale density maps. In this work, we redesign the multi-scale neural network by introducing a hierarchical mixture of density experts, which hierarchically merges multi-scale density maps for crowd counting. Within the hierarchical structure, an expert competition and collaboration scheme is presented to encourage contributions from all scales; pixel-wise soft gating nets are introduced to provide pixel-wise soft weights for scale combinations in different hierarchies. The network is optimized using both the crowd density map and the local counting map, where the latter is obtained by local integration on the former. Optimizing both can be problematic because of their potential conflicts. We introduce a new relative local counting loss based on relative count differences among hard-predicted local regions in an image, which proves to be complementary to the conventional absolute error loss on the density map. Experiments show that our method achieves the state-of-the-art performance on five public datasets, i.e. ShanghaiTech, UCF_CC_50, JHU-CROWD++, NWPU-Crowd and Trancos. Our codes will be available at https://github.com/ZPDu/Redesigning-Multi-Scale-Neural-Network-for-Crowd-Counting.

Linear Complexity Self-Attention With 3rd Order Polynomials.

Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the number of spatial positions hence making their use impractical in many real case applications. In this work, we analyze these methods through a polynomial lens, and we show that self-attention can be seen as a special case of a 3 rd order polynomial. Within this polynomial framework, we are able to design polynomial operators capable of accessing the same data pattern of non-local and self-attention blocks while reducing the complexity from quadratic to linear. As a result, we propose two modules (Poly-NL and Poly-SA) that can be used as "drop-in" replacements for more-complex non-local and self-attention layers in state-of-the-art CNNs and ViT architectures. Our modules can achieve comparable, if not better, performance across a wide range of computer vision tasks while keeping a complexity equivalent to a standard linear layer.

Extended T: Learning With Mixed Closed-Set and Open-Set Noisy Labels.

The noise transition matrix T, reflecting the probabilities that true labels flip into noisy ones, is of vital importance to model label noise and build statistically consistent classifiers. The traditional transition matrix is limited to model closed-set label noise, where noisy training data have true class labels within the noisy label set. It is unfitted to employ such a transition matrix to model open-set label noise, where some true class labels are outside the noisy label set. Therefore, when considering a more realistic situation, i.e., both closed-set and open-set label noises occur, prior works will give unbelievable solutions. Besides, the traditional transition matrix is mostly limited to model instance-independent label noise, which may not perform well in practice. In this paper, we focus on learning with the mixed closed-set and open-set noisy labels. We address the aforementioned issues by extending the traditional transition matrix to be able to model mixed label noise, and further to the cluster-dependent transition matrix to better combat the instance-dependent label noise in real-world applications. We term the proposed transition matrix as the cluster-dependent extended transition matrix. An unbiased estimator (i.e., extended T-estimator) has been designed to estimate the cluster-dependent extended transition matrix by only exploiting the noisy data. Comprehensive experiments validate that our method can better cope with realistic label noise, following its more robust performance than the prior state-of-the-art label-noise learning methods.

EDFace-Celeb-1 M: Benchmarking Face Hallucination With a Million-Scale Dataset.

Recent deep face hallucination methods show stunning performance in super-resolving severely degraded facial images, even surpassing human ability. However, these algorithms are mainly evaluated on non-public synthetic datasets. It is thus unclear how these algorithms perform on public face hallucination datasets. Meanwhile, most of the existing datasets do not well consider the distribution of races, which makes face hallucination methods trained on these datasets biased toward some specific races. To address the above two problems, in this paper, we build a public Ethnically Diverse Face dataset, EDFace-Celeb-1 M, and design a benchmark task for face hallucination. Our dataset includes 1.7 million photos that cover different countries, with relatively balanced race composition. To the best of our knowledge, it is the largest-scale and publicly available face hallucination dataset in the wild. Associated with this dataset, this paper also contributes various evaluation protocols and provides comprehensive analysis to benchmark the existing state-of-the-art methods. The benchmark evaluations demonstrate the performance and limitations of state-of-the-art algorithms. https://github.com/HDCVLab/EDFace-Celeb-1M.

WebFace260M: A Benchmark for Million-Scale Deep Face Recognition.

Face benchmarks empower the research community to train and evaluate high-performance face recognition systems. In this paper, we contribute a new million-scale recognition benchmark, containing uncurated 4M identities/260M faces (WebFace260M) and cleaned 2M identities/42M faces (WebFace42M) training data, as well as an elaborately designed time-constrained evaluation protocol. First, we collect 4M name lists and download 260M faces from the Internet. Then, a Cleaning Automatically utilizing Self-Training (CAST) pipeline is devised to purify the tremendous WebFace260M, which is efficient and scalable. To the best of our knowledge, the cleaned WebFace42M is the largest public face recognition training set and we expect to close the data gap between academia and industry. Referring to practical deployments, Face Recognition Under Inference Time conStraint (FRUITS) protocol and a new test set with rich attributes are constructed. Besides, we gather a large-scale masked face sub-set for biometrics assessment under COVID-19. For a comprehensive evaluation of face matchers, three recognition tasks are performed under standard, masked and unbiased settings, respectively. Equipped with this benchmark, we delve into million-scale face recognition problems. A distributed framework is developed to train face recognition models efficiently without tampering with the performance. Enabled by WebFace42M, we reduce 40% failure rate on the challenging IJB-C set and rank 3rd among 430 entries on NIST-FRVT. Even 10% data (WebFace4M) shows superior performance compared with the public training sets. Furthermore, comprehensive baselines are established under the FRUITS-100/500/1000 milliseconds protocols. The proposed benchmark shows enormous potential on standard, masked and unbiased face recognition scenarios. Our WebFace260M website is https://www.face-benchmark.org.

Deep Polynomial Neural Networks.

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose Π-Nets, a new class of function approximators based on polynomial expansions. Π-Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. The unknown parameters, which are naturally represented by high-order tensors, are estimated through a collective tensor factorization with factors sharing. We introduce three tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks. We empirically demonstrate that Π-Nets are very expressive and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, Π-Nets produce state-of-the-art results in three challenging tasks, i.e., image generation, face verification and 3D mesh representation learning. The source code is available at https://github.com/grigorisg9gr/polynomial_nets.

ArcFace: Additive Angular Margin Loss for Deep Face Recognition.

Recently, a popular line of research in face recognition is adopting margins in the well-established softmax loss function to maximize class separability. In this paper, we first introduce an Additive Angular Margin Loss (ArcFace), which not only has a clear geometric interpretation but also significantly enhances the discriminative power. Since ArcFace is susceptible to the massive label noise, we further propose sub-center ArcFace, in which each class contains K sub-centers and training samples only need to be close to any of the K positive sub-centers. Sub-center ArcFace encourages one dominant sub-class that contains the majority of clean faces and non-dominant sub-classes that include hard or noisy faces. Based on this self-propelled isolation, we boost the performance through automatically purifying raw web faces under massive real-world noise. Besides discriminative feature embedding, we also explore the inverse problem, mapping feature vectors to face images. Without training any additional generator or discriminator, the pre-trained ArcFace model can generate identity-preserved face images for both subjects inside and outside the training data only by using the network gradient and Batch Normalization (BN) priors. Extensive experiments demonstrate that ArcFace can enhance the discriminative feature embedding as well as strengthen the generative face synthesis.

A novel fluorescent "turn-on" aptasensor based on nitrogen-doped graphene quantum dots and hexagonal cobalt oxyhydroxide nanoflakes to detect tetracycline.

In this study, a novel fluorescent "turn-on" aptasensor was developed for sensitive and rapid detection of tetracycline (TC) in animal-derived food. It is based on aptamer-functionalized nitrogen-doped graphene quantum dots (N-GQDs-aptamer) coupled with cobalt oxyhydroxide (CoOOH) nanoflakes. The CoOOH nanoflakes are efficient fluorescence quenchers in homogeneous solutions, and this is due to their advantages of excellent optical properties, superior flexibility, and water dispersibility. The proposed method's mechanism is driven by quenching based on the fluorescence resonance energy transfer (FRET) between the donor (N-GQDs) and the acceptor (CoOOH nanoflakes). On the other hand, fluorescence recovery is caused by the structure switching behavior of the aptamer. Compared with previous methods, our developed method exhibits better behavior in terms of being easy to fabricate and being simple in detection procedure and maintains the detection limit low enough in TC determination: a linear range from 1 to 100 ng mL-1 and a detection limit of 0.95 ng mL-1 (S/N = 3). Furthermore, the proposed method was applied to five animal-derived food samples (milk, honey, fish, eggs, and chicken muscle) and demonstrated practical applicability. As well, the method has the advantages of simplicity in pre-treatment and convenience in instruments, saves times, and is cost-effective. Finally, the proposed method demonstrates significant potential for sensitive and rapid detection of specific components in real samples. Graphical abstract.

Side Information for Face Completion: A Robust PCA Approach.

Robust principal component analysis (RPCA) is a powerful method for learning low-rank feature representation of various visual data. However, for certain types as well as significant amount of error corruption, it fails to yield satisfactory results; a drawback that can be alleviated by exploiting domain-dependent prior knowledge or information. In this paper, we propose two models for the RPCA that take into account such side information, even in the presence of missing values. We apply this framework to the task of UV completion which is widely used in pose-invariant face recognition. Moreover, we construct a generative adversarial network (GAN) to extract side information as well as subspaces. These subspaces not only assist in the recovery but also speed up the process in case of large-scale data. We quantitatively and qualitatively evaluate the proposed approaches through both synthetic data and eight real-world datasets to verify their effectiveness.

Joint Multi-view Face Alignment in the Wild.

The de facto algorithm for facial landmark estimation involves running a face detector with a subsequent deformable model fitting on the bounding box. This encompasses two basic problems: i) the detection and deformable fitting steps are performed independently, while the detector might not provide best-suited initialization for the fitting step, ii) the face appearance varies hugely across different poses, which makes the deformable face fitting very challenging and thus distinct models have to be used (e.g., one for profile and one for frontal faces). In this work, we propose the first, to the best of our knowledge, joint multi-view convolutional network to handle large pose variations across faces in-the-wild, and elegantly bridge face detection and facial landmark localization tasks. Existing joint face detection and landmark localization methods focus only on a very small set of landmarks. By contrast, our method can detect and align a large number of landmarks for semi-frontal (68 landmarks) and profile (39 landmarks) faces. We evaluate our model on a plethora of datasets including standard static image datasets such as IBUG, 300W, COFW, and the latest Menpo Benchmark for both semi-frontal and profile faces. Significant improvement over state-of-the-art methods on deformable face tracking is witnessed on 300VW benchmark. We also demonstrate state-ofthe- art results for face detection on FDDB and MALF datasets.

Turn-On Fluorescence Aptasensor on Magnetic Nanobeads for Aflatoxin M1 Detection Based on an Exonuclease III-Assisted Signal Amplification Strategy.

In order to satisfy the need for sensitive detection of Aflatoxin M1 (AFM1), we constructed a simple and signal-on fluorescence aptasensor based on an autocatalytic Exonuclease III (Exo III)-assisted signal amplification strategy. In this sensor, the DNA hybridization on magnetic nanobeads could be triggered by the target AFM1, resulting in the release of a single-stranded DNA to induce an Exo III-assisted signal amplification, in which numerous G-quadruplex structures would be produced and then associated with the fluorescent dye to generate significantly amplified fluorescence signals resulting in the increased sensitivity. Under the optimized conditions, this aptasensor was able to detect AFM1 with a practical detection limit of 9.73 ng kg-1 in milk samples. Furthermore, the prepared sensor was successfully used for detection of AFM1 in the commercially available milk samples with the recovery percentages ranging from 80.13% to 108.67%. Also, the sensor performance was evaluated by the commercial immunoassay kit with satisfactory results.

Multiple advanced logic gates made of DNA-Ag nanocluster and the application for intelligent detection of pathogenic bacterial genes.

The integration of multiple DNA logic gates on a universal platform to implement advance logic functions is a critical challenge for DNA computing. Herein, a straightforward and powerful strategy in which a guanine-rich DNA sequence lighting up a silver nanocluster and fluorophore was developed to construct a library of logic gates on a simple DNA-templated silver nanoclusters (DNA-AgNCs) platform. This library included basic logic gates, YES, AND, OR, INHIBIT, and XOR, which were further integrated into complex logic circuits to implement diverse advanced arithmetic/non-arithmetic functions including half-adder, half-subtractor, multiplexer, and demultiplexer. Under UV irradiation, all the logic functions could be instantly visualized, confirming an excellent repeatability. The logic operations were entirely based on DNA hybridization in an enzyme-free and label-free condition, avoiding waste accumulation and reducing cost consumption. Interestingly, a DNA-AgNCs-based multiplexer was, for the first time, used as an intelligent biosensor to identify pathogenic genes, E. coli and S. aureus genes, with a high sensitivity. The investigation provides a prototype for the wireless integration of multiple devices on even the simplest single-strand DNA platform to perform diverse complex functions in a straightforward and cost-effective way.

A target-induced logically reversible logic gate for intelligent and rapid detection of pathogenic bacterial genes.

For the first time, a target (pathogenic bacterial gene)-induced logically reversible logic gate was constructed as an intelligent biosensor, which has one-to-one mapping functionality and can rapidly distinguish all information of the two targets, the presence of any target and the absence (or presence) of both of the targets, from the unique output signal pattern.

An Ultrasensitive Fluorescence Sensor with Simple Operation for Cu2+ Specific Detection in Drinking Water.

Whether short-term or long-term, overexposure to an abnormal amount of copper ion does significant harm to human health. Considering its nonbiodegradability, it is critical to sensitively detect copper ion. Herein, a novel fluorescent strategy with a "turn-on" signal was developed for highly sensitive and specific detection of copper ion (Cu2+). In the present investigation, we found that Cu2+ exhibits excellent peroxidase-like catalytic activity toward oxidizing the nonfluorescent substrate of Amplex Red into the product of resofurin with outstanding fluorescence emission under the aid of H2O2. Thus, an enzyme-free and label-free sensing system was constructed for copper ion detection with quite simple operation. To ensure the detection sensitivity and reproducibility, the amount of H2O2 and incubation time were optimized. The limit of detection can reach as low as 1.0 nM. In addition, the developed assay demonstrated excellent specificity and could be utilized to detect copper ion in water samples including tap water and bottled purified water without standing recovery.

Implementation of cascade logic gates and majority logic gate on a simple and universal molecular platform.

Wiring a series of simple logic gates to process complex data is significantly important and a large challenge for untraditional molecular computing systems. The programmable property of DNA endows its powerful application in molecular computing. In our investigation, it was found that DNA exhibits excellent peroxidase-like activity in a colorimetric system of TMB/H2O2/Hemin (TMB, 3,3', 5,5'-Tetramethylbenzidine) in the presence of K+ and Cu2+, which is significantly inhibited by the addition of an antioxidant. According to the modulated catalytic activity of this DNA-based catalyst, three cascade logic gates including AND-OR-INH (INHIBIT), AND-INH and OR-INH were successfully constructed. Interestingly, by only modulating the concentration of Cu2+, a majority logic gate with a single-vote veto function was realized following the same threshold value as that of the cascade logic gates. The strategy is quite straightforward and versatile and provides an instructive method for constructing multiple logic gates on a simple platform to implement complex molecular computing.

IL-8 and IP-10 expression from human bronchial epithelial cells BEAS-2B are promoted by Streptococcus pneumoniae endopeptidase O (PepO).

BACKGROUND: The bronchial epithelium serves as the first defendant line of host against respiratory inhaled pathogens, mainly through releasing chemokines (e.g. interleukin-8 (IL-8), interferon-induced protein 10 (IP-10) etc.) responsible for neutrophil or lymphocyte recruitment to promote the clearance of inhaled pathogens including Streptococcus pneumoniae (S. pneumoniae). Previous studies have shown that IL-8 expression is induced by pneumococcal virulence factors (e.g. pneumolysin, peptidoglycan-polysaccharides, pneumococcal surface protein A (PspA) etc.), which contributes to the pathogenesis of pneumonia. Whether other pneumococcal virulence factors are involved in inducing chemokines expression in epithelium is still unknown. RESULTS: We studied the effect of PepO, a widely expressed and newly discovered pneumococcal virulence protein, on the release of proinflammatory cytokines, IL-8 and IP-10, from human bronchial epithelial cell line BEAS-2B and identified the relevant signaling pathways. Incubation of BEAS-2B with PepO resulted in increased synthesis and release of IL-8 and IP-10 in a dose and time independent manner. We also detected the increased and sustained expression of TLR2 and TLR4 transcripts in BEAS-2B stimulated by PepO. PepO activation leaded to the phosphorylation of MAPKs, Akt and p65. Pharmacologic inhibitors of MAPKs, PI3K and IκB-α phosphorylation attenuated IL-8 release, while IP-10 production was just suppressed by inhibitors of IκB-α phosphorylation, PI3K and P38 MAPK. CONCLUSION: These results suggest that PepO enhances IL-8 and IP-10 production in BEAS-2B in a MAPKs-PI3K/Akt-p65 dependent manner, which may play critical roles in the pathogenesis of pneumonia.

Nanozyme-based bio-barcode assay for high sensitive and logic-controlled specific detection of multiple DNAs.

Since HCV and HIV share a common transmission path, high sensitive detection of HIV and HCV gene is of significant importance to improve diagnosis accuracy and cure rate at early stage for HIV virus-infected patients. In our investigation, a novel nanozyme-based bio-barcode fluorescence amplified assay is successfully developed for simultaneous detection of HIV and HCV DNAs with excellent sensitivity in an enzyme-free and label-free condition. Here, bimetallic nanoparticles, PtAuNPs, present outstanding peroxidase-like activity and act as barcode to catalyze oxidation of nonfluorescent substrate of amplex red (AR) into fluorescent resorufin generating stable and sensitive "Turn On" fluorescent output signal, which is for the first time to be integrated with bio-barcode strategy for fluorescence detection DNA. Furthermore, the provided strategy presents excellent specificity and can distinguish single-base mismatched mutant from target DNA. What interesting is that cascaded INHIBIT-OR logic gate is integrated with biosensors for the first time to distinguish individual target DNA from each other under logic function control, which presents great application in development of rapid and intelligent detection.

Efficient $\chi

Explicit feature mapping is an appealing way to linearize additive kernels, such as χ2 kernel for training large-scale support vector machines (SVMs). Although accurate in approximation, feature mapping could pose computational challenges in high-dimensional settings as it expands the original features to a higher dimensional space. To handle this issue in the context of χ2 kernel SVMs learning, we introduce a simple yet efficient method to approximately linearize χ2 kernel through random feature maps. The main idea is to use sparse random projection to reduce the dimensionality of feature maps while preserving their approximation capability to the original kernel. We provide approximation error bound for the proposed method. Furthermore, we extend our method to χ2 multiple kernel SVMs learning. Extensive experiments on large-scale image classification tasks confirm that the proposed approach is able to significantly speed up the training process of the χ2 kernel SVMs at almost no cost of testing accuracy.