Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks.

New classes arise frequently in our ever-changing world, e.g., emerging topics in social media and new types of products in e-commerce. A model should recognize new classes and meanwhile maintain discriminability over old classes. Under severe circumstances, only limited novel instances are available to incrementally update the model. The task of recognizing few-shot new classes without forgetting old classes is called few-shot class-incremental learning (FSCIL). In this work, we propose a new paradigm for FSCIL based on meta-learning by LearnIng Multi-phase Incremental Tasks (Limit), which synthesizes fake FSCIL tasks from the base dataset. The data format of fake tasks is consistent with the 'real' incremental tasks, and we can build a generalizable feature space for the unseen tasks through meta-learning. Besides, Limit also constructs a calibration module based on transformer, which calibrates the old class classifiers and new class prototypes into the same scale and fills in the semantic gap. The calibration module also adaptively contextualizes the instance-specific embedding with a set-to-set function. Limit efficiently adapts to new classes and meanwhile resists forgetting over old classes. Experiments on three benchmark datasets (CIFAR100, miniImageNet, and CUB200) and large-scale dataset, i.e., ImageNet ILSVRC2012 validate that Limit achieves state-of-the-art performance.

3D CNN-based fingerprint anti-spoofing through optical coherence tomography.

Optical coherence tomography (OCT) is a noninvasive high-resolution imaging technology that can accurately acquire the internal characteristics of tissues within a few millimeters. Using OCT technology, the internal fingerprint structure, which is consistent with external fingerprints and sweat glands, can be collected, leading to high anti-spoofing capabilities. In this paper, an OCT fingerprint anti-spoofing method based on a 3D convolutional neural network (CNN) is proposed, considering the spatial continuity of 3D biometrics in fingertips. Experiments were conducted on self-built and public datasets to test the feasibility of the proposed anti-spoofing method. The anti-spoofing strategy using a 3D CNN achieved the best results compared with classic networks.

Randomized Spectrum Transformations for Adapting Object Detector in Unseen Domains.

We propose a Meta Learning on Randomized Transformations (MLRT) to learn domain invariant object detectors. Domain generalization is a problem about learning an invariant model from multiple source domains which can generalize well on unseen target domains. This problem is overlooked in object detection field, which is formally named as domain generalizable object detection (DGOD). Moreover, existing domain generalization methods have the problem of domain bias so that they can easily overfit to some specific domain (e.g., source domain). In order to alleviate the domain bias, in MLRT model, a novel randomized spectrum transformation (RST) module is proposed to increase the diversity of source domains. Specifically, RST randomizes the domain specific information of images in frequency-space, which can transform single or multiple source domains into various new domains. Besides, we observe a prior that the gradient imbalance degree among domains can also reflect the domain bias. Therefore, we further propose to alleviate the domain bias from the perspective of gradient balancing, and a novel gradient weighting (GW) module is proposed to balance the gradients over all domains via a hand-crafted weight. Finally we embed our RST and GW into a general meta learning framework and the proposed MLRT model is formalized for DGOD task. Extensive experiments are conducted on six benchmarks, and our method achieves the SOTA performance.

Single-Atom Cu Stabilized on Ultrathin WO2.72 Nanowire for Highly Selective and Ultrasensitive ppb-Level Toluene Detection.

Various catalysts are developed to improve the performance of metal oxide semiconductor gas sensors, but achieving high selectivity and response intensity in chemiresistive gas sensors (CGSs) remains a significant challenge. In this study, an in situ-annealing approach to synthesize Cu catalytic sites on ultrathin WO2.72 nanowires for detecting toluene at ultralow concentrations (Ra /Rg = 1.9 at 10 ppb) with high selectivity is developed. Experimental and molecular dynamic studies reveal that the Cu single atoms (SAs) act as active sites, promoting the oxidation of toluene and increasing the affinity of Cu single-atom catalysts (SACs)-containing sensing materials for toluene while weakening the association with carbon dioxide or water vapor. Density functional theory studies show that the selective binding of toluene to Cu SAs is due to the favorable binding sites provided by Cu SAs for toluene molecules over other gaseous species, which aids the adsorption of toluene on WO2.72 nanowires. This study demonstrates the successful atomic-level interface regulation engineering of WO2.72 nanowire-supported Cu SAs, providing a potential strategy for the development of highly active and durable CGSs.

Contrastive Learning-Based Embedder for the Representation of Tandem Mass Spectra.

Tandem mass spectrometry (MS/MS) shows great promise in the research of metabolomics, providing an abundance of information on compounds. Due to the rapid development of mass spectrometric techniques, a large number of MS/MS spectral data sets have been produced from different experimental environments. The massive data brings great challenges into the spectral analysis including compound identification and spectra clustering. The core challenge in MS/MS spectral analysis is how to describe a spectrum more quantitatively and effectively. Recently, emerging deep-learning-based technologies have brought new opportunities to handle this challenge in which high-quality descriptions of MS/MS spectra can be obtained. In this study, we propose a novel contrastive learning-based method for the representation of MS/MS spectra, called CLERMS, which is based on transformer architecture. Specifically, an optimized model architecture equipped with a sinusoidal embedder and a novel loss function composed of InfoNCE loss and MSE loss has been proposed for the attainment of good embedding from the peak information and the metadata. We evaluate our method using a GNPS data set, and the results demonstrate that the learned embedding can not only distinguish spectra from different compounds but also reveal the structural similarity between them. Additionally, the comparison between our method and other methods on the performance of compound identification and spectra clustering shows that our method can achieve significantly better results.

Voltage tunable mid-wave infrared reflective varifocal metalens via an optomechanic cavity.

Metalenses enable the multifunctional control of light beams with an optically thin layer of nanoantennas. Efficient on-chip voltage tuning of the focal length is the crucial step toward the integration of metalenses into dynamically tunable optical systems. We propose and numerically investigate the on-chip electrical tuning of a reflective metalens via an optomechanic cavity. Light is focused by an array of silicon nanopillar antennas separated from a deformable metallic reflector by a small air gap. A transparent electrode is inserted into the optomechanic cavity to electrostatically deform the reflector and rearrange the reflection phase profile, resulting in a shift in the focal point. Two modes of voltage tuning via the relative curvature change of the reflector are analyzed. In mode 1, the size of the air gap is modified through the nearly parallel shift of the reflector, whereas in mode 2, the distribution of the air-gap size is tailored by the curvature change of the reflector. With the designed working wavelength of 3.8 µm and the initial focal length of 80.35 µm, the focal length is shifted by 20.3 µm in mode 1 and 7.25 µm in mode 2. Such a device can be used as a free space coupler between quantum cascade lasers and mid-infrared fibers with variable coupling efficiency.

Breast mass detection in digital mammography based on anchor-free architecture.

BACKGROUND AND OBJECTIVE: Accurate detection of breast masses in mammography images is critical to diagnose early breast cancer, which can greatly improve the patients' survival rate. However, it is still a big challenge due to the heterogeneity of breast masses and the complexity of their surrounding environment. Therefore, how to develop a robust breast mass detection framework in clinical practical applications to improve patient survival is a topic that researchers need to continue to explore. METHODS: To address these problems, we propose a one-stage object detection architecture, called Breast Mass Detection Network (BMassDNet), based on anchor-free and feature pyramid which makes the detection of breast masses of different sizes well adapted. We introduce a truncation normalization method and combine it with adaptive histogram equalization to enhance the contrast between the breast mass and the surrounding environment. Meanwhile, to solve the overfitting problem caused by small data size, we propose a natural deformation data augmentation method and mend the train data dynamic updating method based on the data complexity to effectively utilize the limited data. Finally, we use transfer learning to assist the training process and to improve the robustness of the model ulteriorly. RESULTS: On the INbreast dataset, each image has an average of 0.495 false positives whilst the recall rate is 0.930; On the DDSM dataset, when each image has 0.599 false positives, the recall rate reaches 0.943. CONCLUSIONS: The experimental results on datasets INbreast and DDSM show that the proposed BMassDNet can obtain competitive detection performance over the current top ranked methods.

FREE: A Fast and Robust End-to-End Video Text Spotter.

Currently, video text spotting tasks usually fall into the four-staged pipeline: detecting text regions in individual images, recognizing localized text regions frame-wisely, tracking text streams and post-processing to generate final results. However, they may suffer from the huge computational cost as well as sub-optimal results due to the interferences of low-quality text and the none-trainable pipeline strategy. In this article, we propose a fast and robust end-to-end video text spotting framework named FREE by only recognizing the localized text stream one-time instead of frame-wise recognition. Specifically, FREE first employs a well-designed spatial-temporal detector that learns text locations among video frames. Then a novel text recommender is developed to select the highest-quality text from text streams for recognizing. Here, the recommender is implemented by assembling text tracking, quality scoring and recognition into a trainable module. It not only avoids the interferences from the low-quality text but also dramatically speeds up the video text spotting. FREE unites the detector and recommender into a whole framework, and helps achieve global optimization. Besides, we collect a large scale video text dataset for promoting the video text spotting community, containing 100 videos from 21 real-life scenarios. Extensive experiments on public benchmarks show our method greatly speeds up the text spotting process, and also achieves the remarkable state-of-the-art.

Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study.

Existing enhancement methods are empirically expected to help the high-level end computer vision task: however, that is observed to not always be the case in practice. We focus on object or face detection in poor visibility enhancements caused by bad weathers (haze, rain) and low light conditions. To provide a more thorough examination and fair comparison, we introduce three benchmark sets collected in real-world hazy, rainy, and low-light conditions, respectively, with annotated objects/faces. We launched the UG2+ challenge Track 2 competition in IEEE CVPR 2019, aiming to evoke a comprehensive discussion and exploration about whether and how low-level vision techniques can benefit the high-level automatic visual recognition in various scenarios. To our best knowledge, this is the first and currently largest effort of its kind. Baseline results by cascading existing enhancement and detection models are reported, indicating the highly challenging nature of our new data as well as the large room for further technical innovations. Thanks to a large participation from the research community, we are able to analyze representative team solutions, striving to better identify the strengths and limitations of existing mindsets as well as the future directions.

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding.

Recently, convolutional neural network (CNN) has attracted tremendous attention and has achieved great success in many image processing tasks. In this paper, we focus on CNN technology combined with image restoration to facilitate video coding performance and propose the content-aware CNN based in-loop filtering for high-efficiency video coding (HEVC). In particular, we quantitatively analyze the structure of the proposed CNN model from multiple dimensions to make the model interpretable and optimal for CNN-based loop filtering. More specifically, each coding tree unit (CTU) is treated as an independent region for processing, such that the proposed content-aware multimodel filtering mechanism is realized by the restoration of different regions with different CNN models under the guidance of the discriminative network. To adapt the image content, the discriminative neural network is learned to analyze the content characteristics of each region for the adaptive selection of the deep learning model. The CTU level control is also enabled in the sense of rate-distortion optimization. To learn the CNN model, an iterative training method is proposed by simultaneously labeling filter categories at the CTU level and fine-tuning the CNN model parameters. The CNN based in-loop filter is implemented after sample adaptive offset in HEVC, and extensive experiments show that the proposed approach significantly improves the coding performance and achieves up to 10.0% bit-rate reduction. On average, 4.1%, 6.0%, 4.7%, and 6.0% bit-rate reduction can be obtained under all intra, low delay, low delay P, and random access configurations, respectively.