NTCE-KD: Non-Target-Class-Enhanced Knowledge Distillation.

Most logit-based knowledge distillation methods transfer soft labels from the teacher model to the student model via Kullback-Leibler divergence based on softmax, an exponential normalization function. However, this exponential nature of softmax tends to prioritize the largest class (target class) while neglecting smaller ones (non-target classes), leading to an oversight of the non-target classes's significance. To address this issue, we propose Non-Target-Class-Enhanced Knowledge Distillation (NTCE-KD) to amplify the role of non-target classes both in terms of magnitude and diversity. Specifically, we present a magnitude-enhanced Kullback-Leibler (MKL) divergence multi-shrinking the target class to enhance the impact of non-target classes in terms of magnitude. Additionally, to enrich the diversity of non-target classes, we introduce a diversity-based data augmentation strategy (DDA), further enhancing overall performance. Extensive experimental results on the CIFAR-100 and ImageNet-1k datasets demonstrate that non-target classes are of great significance and that our method achieves state-of-the-art performance across a wide range of teacher-student pairs.

Attention-Based Fault-Tolerant Approach for Multi-Agent Reinforcement Learning Systems.

The aim of multi-agent reinforcement learning systems is to provide interacting agents with the ability to collaboratively learn and adapt to the behavior of other agents. Typically, an agent receives its private observations providing a partial view of the true state of the environment. However, in realistic settings, the harsh environment might cause one or more agents to show arbitrarily faulty or malicious behavior, which may suffice to allow the current coordination mechanisms fail. In this paper, we study a practical scenario of multi-agent reinforcement learning systems considering the security issues in the presence of agents with arbitrarily faulty or malicious behavior. The previous state-of-the-art work that coped with extremely noisy environments was designed on the basis that the noise intensity in the environment was known in advance. However, when the noise intensity changes, the existing method has to adjust the configuration of the model to learn in new environments, which limits the practical applications. To overcome these difficulties, we present an Attention-based Fault-Tolerant (FT-Attn) model, which can select not only correct, but also relevant information for each agent at every time step in noisy environments. The multihead attention mechanism enables the agents to learn effective communication policies through experience concurrent with the action policies. Empirical results showed that FT-Attn beats previous state-of-the-art methods in some extremely noisy environments in both cooperative and competitive scenarios, much closer to the upper-bound performance. Furthermore, FT-Attn maintains a more general fault tolerance ability and does not rely on the prior knowledge about the noise intensity of the environment.

Positive therapy of andrographolide in vocal fold leukoplakia.

PURPOSE: Vocal fold leukoplakia is a premalignant precursor of squamous cell carcinoma. Although many efforts have been contributed to therapy of this disease, none exhibits a satisfactory result. The aims of this study were to investigate the effectiveness and feasibility of andrographolide therapy in vocal fold leukoplakia and to explore the preliminary mechanism underlying. MATERIALS AND METHODS: Forty-one eligible patients were enrolled in the study. The patients were treated for 10-minute exposures of 5 ml (25mg/ml) andrographolide injection aerosols twice a day, and 2 weeks was considered as one treatment course. Electronic laryngoscope was used to observe the condition of vocal fold leukoplakia during the treatment. Every patient received one or two treatment courses, and the follow-up was carried out for 12 months. Toxic reactions of treatments were evaluated on the basis of the standards of the United States MD Anderson Cancer Center. Moreover, laryngeal carcinoma cell line Hep2 was applied to explore the mechanism of effect of andrographolide. Anti-proliferative effect on Hep2, cell nuclear morphology, express of mitogen-activated protein kinases (MAPK) and pro-apoptotic protein were detected after andrographolide treatment. RESULTS: We found that andrographolide exhibited significant curative effects on treatments, which were accompanied by thinning of the lesion of leukoplakia, reduction in the whitish surface area, and return of pink or red epithelium. A complete response up to 85% was observed, and no toxic side effect events occurred during the study. No patient with a complete response had a recurrence in the follow-up. Moreover, cellular experiments in Hep2 indicated that andrographolide activated MAPK pathway and caspase cascade, and finally induced apoptosis in laryngeal carcinoma cell. CONCLUSIONS: The advantages of andrographolide are connected with minimally invasive and localized character of the treatment and no damage of collagenous tissue structures, which are more convenient and less painful for patients. These results suggest that andrographolide treatment is a viable strategy for curing vocal fold leukoplakia.

IoUformer: Pseudo-IoU prediction with transformer for visual tracking.

Siamese tracking has witnessed tremendous progress in tracking paradigm. However, its default box estimation pipeline still faces a crucial inconsistency issue, namely, the bounding box decided by its classification score is not always best overlapped with the ground truth, thus harming performance. To this end, we explore a novel simple tracking paradigm based on the intersection over union (IoU) value prediction. To first bypass this inconsistency issue, we propose a concise target state predictor termed IoUformer, which instead of default box estimation pipeline directly predicts the IoU values related to tracking performance metrics. In detail, it extends the long-range dependency modeling ability of transformer to jointly grasp target-aware interactions between target template and search region, and search sub-region interactions, thus neatly unifying global semantic interaction and target state prediction. Thanks to this joint strength, IoUformer can predict reliable IoU values near-linear with the ground truth, which paves a safe way for our new IoU-based siamese tracking paradigm. Since it is non-trivial to explore this paradigm with pleased efficacy and portability, we offer the respective network components and two alternative localization ways. Experimental results show that our IoUformer-based tracker achieves promising results with less training data. For its applicability, it still serves as a refinement module to consistently boost existing advanced trackers.

Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models.

Blind video quality assessment (BVQA) plays an indispensable role in monitoring and improving the end-users' viewing experience in various real-world video-enabled media applications. As an experimental field, the improvements of BVQA models have been measured primarily on a few human-rated VQA datasets. Thus, it is crucial to gain a better understanding of existing VQA datasets in order to properly evaluate the current progress in BVQA. Towards this goal, we conduct a first-of-its-kind computational analysis of VQA datasets via designing minimalistic BVQA models. By minimalistic, we restrict our family of BVQA models to build only upon basic blocks: a video preprocessor (for aggressive spatiotemporal downsampling), a spatial quality analyzer, an optional temporal quality analyzer, and a quality regressor, all with the simplest possible instantiations. By comparing the quality prediction performance of different model variants on eight VQA datasets with realistic distortions, we find that nearly all datasets suffer from the easy dataset problem of varying severity, some of which even admit blind image quality assessment (BIQA) solutions. We additionally justify our claims by comparing our model generalization capabilities on these VQA datasets, and by ablating a dizzying set of BVQA design choices related to the basic building blocks. Our results cast doubt on the current progress in BVQA, and meanwhile shed light on good practices of constructing next-generation VQA datasets and models.

MotionTrack: Learning motion predictor for multiple object tracking.

Significant progress has been achieved in multi-object tracking (MOT) through the evolution of detection and re-identification (ReID) techniques. Despite these advancements, accurately tracking objects in scenarios with homogeneous appearance and heterogeneous motion remains a challenge. This challenge arises from two main factors: the insufficient discriminability of ReID features and the predominant utilization of linear motion models in MOT. In this context, we introduce a novel motion-based tracker, MotionTrack, centered around a learnable motion predictor that relies solely on object trajectory information. This predictor comprehensively integrates two levels of granularity in motion features to enhance the modeling of temporal dynamics and facilitate precise future motion prediction for individual objects. Specifically, the proposed approach adopts a self-attention mechanism to capture token-level information and a Dynamic MLP layer to model channel-level features. MotionTrack is a simple, online tracking approach. Our experimental results demonstrate that MotionTrack yields state-of-the-art performance on datasets such as Dancetrack and SportsMOT, characterized by highly complex object motion.

Smooth-Guided Implicit Data Augmentation for Domain Generalization.

The training process of a domain generalization (DG) model involves utilizing one or more interrelated source domains to attain optimal performance on an unseen target domain. Existing DG methods often use auxiliary networks or require high computational costs to improve the model's generalization ability by incorporating a diverse set of source domains. In contrast, this work proposes a method called Smooth-Guided Implicit Data Augmentation (SGIDA) that operates in the feature space to capture the diversity of source domains. To amplify the model's generalization capacity, a distance metric learning (DML) loss function is incorporated. Additionally, rather than depending on deep features, the suggested approach employs logits produced from cross entropy (CE) losses with infinite augmentations. A theoretical analysis shows that logits are effective in estimating distances defined on original features, and the proposed approach is thoroughly analyzed to provide a better understanding of why logits are beneficial for DG. Moreover, to increase the diversity of the source domain, a sampling-based method called smooth is introduced to obtain semantic directions from interclass relations. The effectiveness of the proposed approach is demonstrated through extensive experiments on widely used DG, object detection, and remote sensing datasets, where it achieves significant improvements over existing state-of-the-art methods across various backbone networks.

Tackling Noisy Labels With Network Parameter Additive Decomposition.

Given data with noisy labels, over-parameterized deep networks suffer overfitting mislabeled data, resulting in poor generalization. The memorization effect of deep networks shows that although the networks have the ability to memorize all noisy data, they would first memorize clean training data, and then gradually memorize mislabeled training data. A simple and effective method that exploits the memorization effect to combat noisy labels is early stopping. However, early stopping cannot distinguish the memorization of clean data and mislabeled data, resulting in the network still inevitably overfitting mislabeled data in the early training stage. In this paper, to decouple the memorization of clean data and mislabeled data, and further reduce the side effect of mislabeled data, we perform additive decomposition on network parameters. Namely, all parameters are additively decomposed into two groups, i.e., parameters w are decomposed as w=σ+γ. Afterward, the parameters σ are considered to memorize clean data, while the parameters γ are considered to memorize mislabeled data. Benefiting from the memorization effect, the updates of the parameters σ are encouraged to fully memorize clean data in early training, and then discouraged with the increase of training epochs to reduce interference of mislabeled data. The updates of the parameters γ are the opposite. In testing, only the parameters σ are employed to enhance generalization. Extensive experiments on both simulated and real-world benchmarks confirm the superior performance of our method.

Meta attention for Off-Policy Actor-Critic.

Off-Policy Actor-Critic methods can effectively exploit past experiences and thus they have achieved great success in various reinforcement learning tasks. In many image-based and multi-agent tasks, attention mechanism has been employed in Actor-Critic methods to improve their sampling efficiency. In this paper, we propose a meta attention method for state-based reinforcement learning tasks, which combines attention mechanism and meta-learning based on the Off-Policy Actor-Critic framework. Unlike previous attention-based work, our meta attention method introduces attention in the Actor and the Critic of the typical Actor-Critic framework, rather than in multiple pixels of an image or multiple information sources in specific image-based control tasks or multi-agent systems. In contrast to existing meta-learning methods, the proposed meta-attention approach is able to function in both the gradient-based training phase and the agent's decision-making process. The experimental results demonstrate the superiority of our meta-attention method in various continuous control tasks, which are based on the Off-Policy Actor-Critic methods including DDPG and TD3.

Discriminative Geometric-Structure-Based Deep Hashing for Large-Scale Image Retrieval.

Deep hashing reaps the benefits of deep learning and hashing technology, and has become the mainstream of large-scale image retrieval. It generally encodes image into hash code with feature similarity preserving, that is, geometric-structure preservation, and achieves promising retrieval results. In this article, we find that existing geometric-structure preservation manner inadequately ensures feature discrimination, while improving feature discrimination of hash code essentially determines hash learning retrieval performance. This fact principally spurs us to propose a discriminative geometric-structure-based deep hashing method (DGDH), which investigates three novel loss terms based on class centers to induce the so-called discriminative geometrical structure. In detail, the margin-aware center loss assembles samples in the same class to the corresponding class centers for intraclass compactness, then a linear classifier based on class center serves to boost interclass separability, and the radius loss further puts different class centers on a hypersphere to tentatively reduce quantization errors. An efficient alternate optimization algorithm with guaranteed desirable convergence is proposed to optimize DGDH. We theoretically analyze the robustness and generalization of the proposed method. The experiments on five popular benchmark datasets demonstrate superior image retrieval performance of the proposed DGDH over several state of the arts.

SiamDF: Tracking training data-free siamese tracker.

Much progress has been made in siamese tracking, primarily benefiting from increasing huge training data. However, very little attention has been really paid to the role of huge training data in learning an effective siamese tracker. In this study, we undertake an in-depth analysis of this issue from a novel optimization perspective, and observe that training data is particularly adept at background suppression, thereby refining target representation. Inspired by this insight, we present a data-free siamese tracking algorithm named SiamDF, which requires only a pre-trained backbone and no further fine-tuning on additional training data. Particularly, to suppress background distractors, we separately improve two branches of siamese tracking by retaining the pure target region as target input with the removal of template background, and by exploring an efficient inverse transformation to maintain the constant aspect ratio of target state in search region. Besides, we further promote the center displacement prediction of the entire backbone by eliminating its spatial stride deviations caused by convolution-like quantification operations. Our experimental results on several popular benchmarks demonstrate that SiamDF, free from both offline fine-tuning and online update, achieves impressive performance compared to well-established unsupervised and supervised tracking methods.

Highly Efficient Active Learning With Tracklet-Aware Co-Cooperative Annotators for Person Re-Identification.

Supervised person re-identification (ReID) has attracted widespread attentions in the computer vision community due to its great potential in real-world applications. However, the demand of human annotation heavily limits the application as it is costly to annotate identical pedestrians appearing from different cameras. Thus, how to reduce the annotation cost while preserving the performance remains challenging and has been studied extensively. In this article, we propose a tracklet-aware co-cooperative annotators' framework to reduce the demand of human annotation. Specifically, we partition the training samples into different clusters and associate adjacent images in each cluster to produce the robust tracklet which decreases the annotation requirements significantly. Besides, to further reduce the cost, we introduce a powerful teacher model in our framework to implement the active learning strategy and select the most informative tracklets for human annotator, the teacher model itself, in our setting, also acts as an annotator to label the relatively certain tracklets. Thus, our final model could be well-trained with both confident pseudo-labels and human-given annotations. Extensive experiments on three popular person ReID datasets demonstrate that our approach could achieve competitive performance compared with state-of-the-art methods in both active learning and unsupervised learning (USL) settings.

Learning to Purification for Unsupervised Person Re-Identification.

Unsupervised person re-identification is a challenging and promising task in computer vision. Nowadays unsupervised person re-identification methods have achieved great progress by training with pseudo labels. However, how to purify feature and label noise is less explicitly studied in the unsupervised manner. To purify the feature, we take into account two types of additional features from different local views to enrich the feature representation. The proposed multi-view features are carefully integrated into our cluster contrast learning to leverage more discriminative cues that the global feature easily ignored and biased. To purify the label noise, we propose to take advantage of the knowledge of teacher model in an offline scheme. Specifically, we first train a teacher model from noisy pseudo labels, and then use the teacher model to guide the learning of our student model. In our setting, the student model could converge fast with the supervision of the teacher model thus reduce the interference of noisy labels as the teacher model greatly suffered. After carefully handling the noise and bias in the feature learning, our purification modules are proven to be very effective for unsupervised person re-identification. Extensive experiments on two popular person re-identification datasets demonstrate the superiority of our method. Especially, our approach achieves a state-of-the-art accuracy 85.8% @mAP and 94.5% @Rank-1 on the challenging Market-1501 benchmark with ResNet-50 under the fully unsupervised setting. Code has been available at: https://github.com/tengxiao14/Purification_ReID.

Contrastive Transformer Hashing for Compact Video Representation.

Video hashing learns compact representation by mapping video into low-dimensional Hamming space and has achieved promising performance in large-scale video retrieval. It is challenging to effectively exploit temporal and spatial structure in an unsupervised setting. To fulfill this gap, this paper proposes Contrastive Transformer Hashing (CTH) for effective video retrieval. Specifically, CTH develops a bidirectional transformer autoencoder, based on which visual reconstruction loss is proposed. CTH is more powerful to capture bidirectional correlations among frames than conventional unidirectional models. In addition, CTH devises multi-modality contrastive loss to reveal intrinsic structure among videos. CTH constructs inter-modality and intra-modality triplet sets and proposes multi-modality contrastive loss to exploit inter-modality and intra-modality similarities simultaneously. We perform video retrieval tasks on four benchmark datasets, i.e., UCF101, HMDB51, SVW30, FCVID using the learned compact hash representation, and extensive empirical results demonstrate the proposed CTH outperforms several state-of-the-art video hashing methods.

An Input-Current Shaping and Soft-Switching Drive Circuit Applied to a Piezoelectric Ceramic Actuator.

Piezoelectric ceramic actuators utilize an inverse piezoelectric effect to generate high-frequency vibration energy and are widely used in ultrasonic energy conversion circuits. This paper presents a novel drive circuit with input-current shaping (ICS) and soft-switching features which consists of a front AC-DC full-wave bridge rectifier and a rear DC-AC circuit combining a stacked boost converter and a half-bridge resonant inverter for driving a piezoelectric ceramic actuator. To enable ICS functionality in the proposed drive circuit, the inductor of the stacked boost converter sub-circuit is designed to operate in boundary-conduction mode (BCM). In order to allow the two power switches in the proposed drive circuit to achieve zero-voltage switching (ZVS) characteristics, the resonant circuit of the half-bridge resonant inverter sub-circuit is designed as an inductive load. In this paper, a prototype drive circuit for providing piezoelectric ceramic actuators was successfully implemented. Experimental results tested at 110 V input utility voltage show that high power factor (PF > 0.97), low input current total harmonic distortion (THD < 16%), and ZVS characteristics of the power switch were achieved in the prototype drive circuit.

Heterogeneous Pseudo-Supervised Learning for Few-shot Person Re-Identification.

How to obtain good retrieval performance in the case of few-shot labeled samples is the current research focus of Person Re-Identification. To facilitate formal analysis, we formally put forward the concept of Pseudo-Supervised Learning (PSL) to represent a series of research works based on label generation under few-shot condition. Through extensive investigations, we find that the main problem that needs to be solved of PSL is how we can improve the quality of pseudo-label. To solve this problem, in this work, we proposed a simple yet effective Heterogeneous Pseudo-Supervised Learning (H-PSL) framework based on classical PSL to implement asynchronous match, which boosts the feature expression and then a better label prediction in the following. Specifically, a novel isomer is constructed as the feature extractor and is trained with a much larger amount of pseudo-supervised data, i.e., samples with pseudo-labels. In this way, the isomer obtains advanced feature expression. We then deliberately implement a cross-level asynchronous match mechanism between model and pseudo-supervised data. As a result, the quality of pseudo-label is greatly improved and the feature expression performance also be optimized accordingly. In addition, to make better use of pseudo-supervised data, we also designed a knowledge fusion strategy to integrate the pseudo labels and their confidence which are easily obtained by the base model and isomer. Encouragingly, knowledge fusion strategy further removes the noise-labeled samples from candidate data. We conduct experiments on four popular datasets to fully verify the universality of the proposed method. The experimental results show that the proposed method improves the performance of all compared baseline works.

Nocal-Siam: Refining Visual Features and Response With Advanced Non-Local Blocks for Real-Time Siamese Tracking.

Siamese trackers contain two core stages, i.e., learning the features of both target and search inputs at first and then calculating response maps via the cross-correlation operation, which can also be used for regression and classification to construct typical one-shot detection tracking framework. Although they have drawn continuous interest from the visual tracking community due to the proper trade-off between accuracy and speed, both stages are easily sensitive to the distracters in search branch, thereby inducing unreliable response positions. To fill this gap, we advance Siamese trackers with two novel non-local blocks named Nocal-Siam, which leverages the long-range dependency property of the non-local attention in a supervised fashion from two aspects. First, a target-aware non-local block (T-Nocal) is proposed for learning the target-guided feature weights, which serve to refine visual features of both target and search branches, and thus effectively suppress noisy distracters. This block reinforces the interplay between both target and search branches in the first stage. Second, we further develop a location-aware non-local block (L-Nocal) to associate multiple response maps, which prevents them inducing diverse candidate target positions in the future coming frame. Experiments on five popular benchmarks show that Nocal-Siam performs favorably against well-behaved counterparts both in quantity and quality.

A Novel and Cost-Effective Drive Circuit for Supplying a Piezoelectric Ceramic Actuator with Power-Factor-Correction and Soft-Switching Features.

This paper proposes a novel and cost-effective drive circuit for supplying a piezoelectric ceramic actuator, which combines a dual boost AC-DC converter with a coupled inductor and a half-bridge resonant DC-AC inverter into a single-stage architecture with power-factor-correction (PFC) and soft-switching characteristics. The coupled inductor of the dual boost AC-DC converter sub-circuit is designed to work in discontinuous conduction mode (DCM), so the PFC function can be realized in the proposed drive circuit. The resonant tank of the half-bridge resonant inverter sub-circuit is designed as an inductive load, so that the two power switches in the presented drive circuit can achieve zero-voltage switching (ZVS) characteristics. A 50 W-rated prototype drive circuit providing a piezoelectric ceramic actuator has been successfully implemented in this paper. From the experimental results at 110 V input utility-line voltage, the drive circuit has the characteristics of high power factor and low input current total-harmonic-distortion factor, and two power switches have ZVS characteristics. Therefore, satisfactory outcomes from measured results prove the function of the proposed drive circuit.

Interacting Tracklets for Multi-object Tracking.

In this paper, we propose to exploit the interactions between non-associable tracklets to facilitate multi-object tracking. We introduce two types of tracklet interactions, close interaction and distant interaction. The close interaction imposes physical constraints between two temporally overlapping tracklets and more importantly, allows us to learn local classifiers to distinguish targets that are close to each other in the spatiotemporal domain. The distant interaction, on the other hand, accounts for the higher-order motion and appearance consistency between two temporally isolated tracklets. Our approach is modeled as a binary labeling problem and solved using the efficient Quadratic Pseudo-Boolean Optimization (QPBO). It yields promising tracking performance on the challenging PETS09 and MOT16 dataset. Our code will be made publicly available upon the acceptance of the manuscript.

DEHP exposure destroys blood-testis barrier (BTB) integrity of immature testes through excessive ROS-mediated autophagy.

Di-(2-ethylhexyl) phthalate (DEHP), is known to impair testicular functions and reproduction. However, its effects on immature testis Blood-testis barrier (BTB) and the underlying mechanisms remain obscure. We constructed a rat model to investigate the roles of autophagy in BTB toxicity induced by DEHP. Sprague-Dawley rats were developmentally exposed to 0, 250 and 500 mg/kg DEHP via intragastric administration from postnatal day (PND) 1 to PND 35. Testicular morphology, expressions of BTB junction proteins and autophagy related proteins were detected. In addition, expressions of oxidative stress markers were also analyzed. Our results demonstrated that developmental DEHP exposure induced decreasing organ coefficients of immature testes and severe testicular damage in histomorphology. The expressions of junctional proteins were down-regulated significantly after DEHP treatment. Intriguingly, DEHP simultaneously increased the number of autophagosomes and the levels of autophagy marker LC3-II and p62, suggesting that the accumulated autophagosomes resulted from impaired autophagy degradation. Moreover, the expressions of HO-1 and SOD levels remarkably decreased after DEHP exposure. Vitamins E and C could alleviate the DEHP-induced oxidative stress, reverse the autophagy defect and restore the BTB impairment. Taken together, DEHP exposure destroys immature testis blood-testis barrier (BTB) integrity through excessive ROS-mediated autophagy.