Enhancing Few-Shot Learning in Lightweight Models via Dual-Faceted Knowledge Distillation.

In recent computer vision research, the pursuit of improved classification performance often leads to the adoption of complex, large-scale models. However, the actual deployment of such extensive models poses significant challenges in environments constrained by limited computing power and storage capacity. Consequently, this study is dedicated to addressing these challenges by focusing on innovative methods that enhance the classification performance of lightweight models. We propose a novel method to compress the knowledge learned by a large model into a lightweight one so that the latter can also achieve good performance in few-shot classification tasks. Specifically, we propose a dual-faceted knowledge distillation strategy that combines output-based and intermediate feature-based methods. The output-based method concentrates on distilling knowledge related to base class labels, while the intermediate feature-based approach, augmented by feature error distribution calibration, tackles the potential non-Gaussian nature of feature deviations, thereby boosting the effectiveness of knowledge transfer. Experiments conducted on MiniImageNet, CIFAR-FS, and CUB datasets demonstrate the superior performance of our method over state-of-the-art lightweight models, particularly in five-way one-shot and five-way five-shot tasks.

Multi-Type Missing Imputation of Time-Series Power Equipment Monitoring Data Based on Moving Average Filter-Asymmetric Denoising Autoencoder.

Supervisory control and data acquisition (SCADA) systems are widely utilized in power equipment for condition monitoring. For the collected data, there generally exists a problem-missing data of different types and patterns. This leads to the poor quality and utilization difficulties of the collected data. To address this problem, this paper customizes methodology that combines an asymmetric denoising autoencoder (ADAE) and moving average filter (MAF) to perform accurate missing data imputation. First, convolution and gated recurrent unit (GRU) are applied to the encoder of the ADAE, while the decoder still utilizes the fully connected layers to form an asymmetric network structure. The ADAE extracts the local periodic and temporal features from monitoring data and then decodes the features to realize the imputation of the multi-type missing. On this basis, according to the continuity of power data in the time domain, the MAF is utilized to fuse the prior knowledge of the neighborhood of missing data to secondarily optimize the imputed data. Case studies reveal that the developed method achieves greater accuracy compared to existing models. This paper adopts experiments under different scenarios to justify that the MAF-ADAE method applies to actual power equipment monitoring data imputation.

Distributed Optimization of Multiagent Systems Subject to Inequality Constraints.

In this paper, we study a distributed convex optimization problem with inequality constraints. Each agent is associated with its cost function, and can only exchange information with its neighbors. It is assumed that each cost function is convex and the optimization variable is subject to an inequality constraint. The objective is to make all the agents reach consensus, and meanwhile converge to the minimum point of the sum of local cost functions. A distributed protocol is proposed to guarantee that all agents can reach consensus in finite time and converge to the optimal point within the inequality constraints. Based on the ideas of parameter projection, the protocol includes two decent directions. One makes the cost function decrease, and the other makes agents step forward to the constraint set. It is shown that the proposed protocol solves the problem under connected undirected graphs without using a Lagrange multiplier technique. Especially, all of the agents could reach the constraint sets in finite time and stay in there after. The method could also be used in the centralized optimization problems.