Based on neural activities, deep learning based neural signal decoding has become a classical problem in computational neuroscience. Due to their excellent capability and performance, deep learning approaches have played a critical role in neural decoding of audio and visual data. Deep learning approaches have enabled high-accuracy reconstruction of audio data based on cochlear traveling basilar membrane (BM) motion waves. During pattern recognition for audio data, existing deep neural network (DNN)-based processing methods tend to ignore associations between center frequency (CF) and have exhibited limitations in multi-scale feature extraction and fusion. In order to address these issues, this study proposes ACReNet, a knowledge-enhanced deep learning model designed for high-precision neural signal decoding, using convolution optimized through CF association modeling. Experiments on diverse regional audio datasets demonstrate that ACReNet significantly outperforms Connear, achieving a lower MSE ( \(\downarrow \) 58%), higher PPMCC ( \(\uparrow \) 11%), and improved SI-SNR ( \(\uparrow \) 3.7), thus validating its effectiveness in neural audio decoding. The proposed ACReNet method provides novel insights for high-precision neural signal processing and advances DNN-based pathological system treatments.