This paper presents a Bit Error Rate (BER) performance analysis of a Public Key Cryptosystem based on Systematic Polar Encoding (PKC-SPE) for Additive White Gaussian Noise (AWGN) and Rayleigh Fading channels. The PKC-SPE uses a combination of public key cryptography and systematic polar encoding to protect against eavesdropping and message interception. The system’s BER performance has been evaluated using numerical simulation by processing 10,000 random messages and averaging the obtained results. The study demonstrates that PKC-SPE exhibits strong error resilience under AWGN channel conditions. However, the analysis also reveals an increase in BER under Rayleigh fading channels due to multipath effects, which highlights the challenges associated with wireless communication in real-world environments. The analysis also highlights the impact of different design parameters, such as code rate, channel noise and blocklength, on the BER performance of cryptosystem. Our results underscore PKC-SPE’s robustness in maintaining reliable performance despite channel distortions, making it a viable candidate for secure communication applications. This research provides valuable insights into PKC-SPE capabilities and lays the groundwork for future optimization and practical implementation.