Single Image Super-Resolution (SISR) techniques reconstruct high-resolution (HR) image from given low-resolution (LR) ones. Traditional frequency-guided SISR techniques adopt pure high-frequency band as the guidance for feature extraction. However, the high-frequency information of LR images are usually seriously affected by the disturbance like noise, blurry, leading to the rollback of SISR performance. To address this issue, we propose a novel frequency based SISR network guided by bilateral frequency adaption via Discrete Cosine Transform (DCT). On the basis of current methods that remove low-frequency information to construct frequency guidance, we propose to further remove the high-frequency information that is sensitive to degradation disturbance adaptively to further refine the frequency guidance, which we named the bilateral frequency learning strategy. Our method consists of two key modules: Firstly, we propose an Bilateral Adaptive Frequency Selection Module (BAFS), which adaptively learn frequency bands as guidance based on the Discrete Cosine Transform (DCT); secondly, we present a Frequency Attention Module (FAM) that further refines the learned frequency bands to guide image reconstruction. Our method conforms to human subjective visual characteristics and achieves better subjective visual results. It also achieves an increase of 0.36dB in PSNR and 0.0061 in SSIM when compared with the SISR SOTA method in \(\times 4\) task on the Urban100 dataset.