Efficient design of complex-valued neural networks with application to the classification of transient acoustic signals.

A paper by the current authors Paul and Nelson [JASA Express Lett. 3(9), 094802 (2023)] showed how the singular value decomposition (SVD) of the matrix of real weights in a neural network could be used to prune the network during training. The paper presented here shows that a similar approach can be used to reduce the training time and increase the implementation efficiency of complex-valued neural networks. Such networks have potential advantages compared to their real-valued counterparts, especially when the complex representation of the data is important, which is the often case in acoustic signal processing. In comparing the performance of networks having both real and complex elements, it is demonstrated that there are some advantages to the use of complex networks in the cases considered. The paper includes a derivation of the backpropagation algorithm, in matrix form, for training a complex-valued multilayer perceptron with an arbitrary number of layers. The matrix-based analysis enables the application of the SVD to the complex weight matrices in the network. The SVD-based pruning technique is applied to the problem of the classification of transient acoustic signals. It is shown how training times can be reduced, and implementation efficiency increased, while ensuring that such signals can be classified with remarkable accuracy.

Matrix analysis for fast learning of neural networks with application to the classification of acoustic spectra.

Neural networks are increasingly being applied to problems in acoustics and audio signal processing. Large audio datasets are being generated for use in training machine learning algorithms, and the reduction of training times is of increasing relevance. The work presented here begins by reformulating the analysis of the classical multilayer perceptron to show the explicit dependence of network parameters on the properties of the weight matrices in the network. This analysis then allows the application of the singular value decomposition (SVD) to the weight matrices. An algorithm is presented that makes use of regular applications of the SVD to progressively reduce the dimensionality of the network. This results in significant reductions in network training times of up to 50% with very little or no loss in accuracy. The use of the algorithm is demonstrated by applying it to a number of acoustical classification problems that help quantify the extent to which closely related spectra can be distinguished by machine learning.

Efficient design of neural networks for the classification of acoustic spectra.

A previous paper by Paul and Nelson [(2021). J. Acoust. Soc. Am. 149(6), 4119-4133] presented the application of the singular value decomposition (SVD) to the weight matrices of multilayer perceptron (MLP) networks as a pruning strategy to remove weight parameters. This work builds on the previous technique and presents a method of reducing the size of a hidden layer by applying a similar SVD algorithm. Results show that by reducing the neurons in the hidden layer, a significant amount of training time is saved compared to the algorithm presented in the previous paper while no or little accuracy is being lost compared to the original MLP model.