An End-to-End Autoencoder-based Probabilistic Shaping Method for Optical Communication

The advent of services such as big data and cloud computing has driven a continuous increase in the required transmission rate and capacity of optical fiber communication systems, which now support over 90% of global data traffic. Probability shaping (PS), which adjusts the probability distribution of constellation points, enhances both data transmission rates and the resilience of signals to channel nonlinearity. However, traditional PS methods, such as those using constant composition distribution matching (CCDM), cannot dynamically adjust the probability distribution of constellation points based on channel conditions. To address this limitation, we propose an end-to-end Autoencoder (AE)-based PS technique for optical communication. Simulation results show that compared to traditional 16/64/256QAM, this scheme achieves a maximum improvement of 0.356 bit/symbol in mutual information over the AWGN channel. In the optical fiber channel, under varying signal power and fiber length conditions, the mutual information is improved in different degrees and the influence of channel nonlinearity on the signal is reduced. Innovatively, residual connections are added to the decoder to enhance the model's generalization performance, enabling a deeper network structure and improving its ability to handle complex optical fiber channels.