Experimental implementation of a neural network optical channel equalizer in restricted hardware using pruning and quantization.

The deployment of artificial neural networks-based optical channel equalizers on edge-computing devices is critically important for the next generation of optical communication systems. However, this is still a highly challenging problem, mainly due to the computational complexity of the artificial neural networks (NNs) required for the efficient equalization of nonlinear optical channels with large dispersion-induced memory. To implement the NN-based optical channel equalizer in hardware, a substantial complexity reduction is needed, while we have to keep an acceptable performance level of the simplified NN model. In this work, we address the complexity reduction problem by applying pruning and quantization techniques to an NN-based optical channel equalizer. We use an exemplary NN architecture, the multi-layer perceptron (MLP), to mitigate the impairments for 30 GBd 1000 km transmission over a standard single-mode fiber, and demonstrate that it is feasible to reduce the equalizer's memory by up to 87.12%, and its complexity by up to 78.34%, without noticeable performance degradation. In addition to this, we accurately define the computational complexity of a compressed NN-based equalizer in the digital signal processing (DSP) sense. Further, we examine the impact of using hardware with different CPU and GPU features on the power consumption and latency for the compressed equalizer. We also verify the developed technique experimentally, by implementing the reduced NN equalizer on two standard edge-computing hardware units: Raspberry Pi 4 and Nvidia Jetson Nano, which are used to process the data generated via simulating the signal's propagation down the optical-fiber system.

Neural networks for computing and denoising the continuous nonlinear Fourier spectrum in focusing nonlinear Schrödinger equation.

We combine the nonlinear Fourier transform (NFT) signal processing with machine learning methods for solving the direct spectral problem associated with the nonlinear Schrödinger equation. The latter is one of the core nonlinear science models emerging in a range of applications. Our focus is on the unexplored problem of computing the continuous nonlinear Fourier spectrum associated with decaying profiles, using a specially-structured deep neural network which we coined NFT-Net. The Bayesian optimisation is utilised to find the optimal neural network architecture. The benefits of using the NFT-Net as compared to the conventional numerical NFT methods becomes evident when we deal with noise-corrupted signals, where the neural networks-based processing results in effective noise suppression. This advantage becomes more pronounced when the noise level is sufficiently high, and we train the neural network on the noise-corrupted field profiles. The maximum restoration quality corresponds to the case where the signal-to-noise ratio of the training data coincides with that of the validation signals. Finally, we also demonstrate that the NFT b-coefficient important for optical communication applications can be recovered with high accuracy and denoised by the neural network with the same architecture.

Convolutional long short-term memory neural network equalizer for nonlinear Fourier transform-based optical transmission systems.

We evaluate improvement in the performance of the optical transmission systems operating with the continuous nonlinear Fourier spectrum by the artificial neural network equalisers installed at the receiver end. We propose here a novel equaliser designs based on bidirectional long short-term memory (BLSTM) gated recurrent neural network and compare their performance with the equaliser based on several fully connected layers. The proposed approach accounts for the correlations between different nonlinear spectral components. The application of BLSTM equaliser leads to a 16x improvement in terms of bit-error rate (BER) compared to the non-equalised case. The proposed equaliser makes it possible to reach the data rate of 170 Gbit/s for one polarisation conventional nonlinear Fourier transform (NFT) based system at 1000 km distance. We show that our new BLSTM equalisers significantly outperform the previously proposed scheme based on a feed-forward fully connected neural network. Moreover, we demonstrate that by adding a 1D convolutional layer for the data pre-processing before BLSTM recurrent layers, we can further enhance the performance of the BLSTM equaliser, reaching 23x BER improvement for the 170 Gbit/s system over 1000 km, staying below the 7% forward error correction hard decision threshold (HD-FEC).

Combining nonlinear Fourier transform and neural network-based processing in optical communications.

We propose a method to improve the performance of the nonlinear Fourier transform (NFT)-based optical transmission system by applying the neural network post-processing of the nonlinear spectrum at the receiver. We demonstrate through numerical modeling about one order of magnitude bit error rate improvement and compare this method with machine learning processing based on the classification of the received symbols. The proposed approach also offers a way to improve numerical accuracy of the inverse NFT; therefore, it can find a range of applications beyond optical communications.