Compressing Deep Networks by Neuron Agglomerative Clustering.
Sensors (Basel)
; 20(21)2020 Oct 23.
Article
em En
| MEDLINE
| ID: mdl-33114078
ABSTRACT
In recent years, deep learning models have achieved remarkable successes in various applications, such as pattern recognition, computer vision, and signal processing. However, high-performance deep architectures are often accompanied by a large storage space and long computational time, which make it difficult to fully exploit many deep neural networks (DNNs), especially in scenarios in which computing resources are limited. In this paper, to tackle this problem, we introduce a method for compressing the structure and parameters of DNNs based on neuron agglomerative clustering (NAC). Specifically, we utilize the agglomerative clustering algorithm to find similar neurons, while these similar neurons and the connections linked to them are then agglomerated together. Using NAC, the number of parameters and the storage space of DNNs are greatly reduced, without the support of an extra library or hardware. Extensive experiments demonstrate that NAC is very effective for the neuron agglomeration of both the fully connected and convolutional layers, which are common building blocks of DNNs, delivering similar or even higher network accuracy. Specifically, on the benchmark CIFAR-10 and CIFAR-100 datasets, using NAC to compress the parameters of the original VGGNet by 92.96% and 81.10%, respectively, the compact network obtained still outperforms the original networks.
Palavras-chave
Texto completo:
1
Base de dados:
MEDLINE
Assunto principal:
Análise por Conglomerados
/
Redes Neurais de Computação
/
Compressão de Dados
/
Neurônios
Idioma:
En
Ano de publicação:
2020
Tipo de documento:
Article