Generalized Rayleigh quotient optimization method for inter-channel nonlinearity compensation in coherent optical communication systems.

ABSTRACT

Inter-channel nonlinearity compensation plays a crucial role in wavelength division multiplexing (WDM) systems for improving transmission capacity and distance. In this work, we propose a novel, to the best of our knowledge, inter-channel nonlinearity compensation method called generalized Rayleigh quotient optimization (GRQO) method with two different working modes. In an 8 × 64 GBaud 16-ary quadrature amplitude modulation (16-QAM) experimental system over 1600 km standard single-mode fiber (SSMF), the proposed method shows a 0.40 dB Q2 factor improvement over nonlinear polarization cross talk canceller (NPCC) with a moderately low computational complexity of about 2000 real multiplications per bit (RMb).