Image deblurring has received significant progress due to the use of deep residual learning as it can help network training and model high-frequency information that is vital for image restoration. However, the blur does not only smooth the details (i.e., high-frequency information) but also distort the main structures (i.e., low-frequency information). Although several existing approaches utilize the frequency domain with residual learning to model both low and high frequency information for image deblurring, these methods cannot adaptively adjust their attention to pixels at different positions. To overcome this problem, we introduce an attention mechanism into residual learning and present the attention-guided residual fourier transformation block, capable of treating information contained in different pixels adaptively. The proposed module can be integrated into existing deblurring networks for performance improvement. Experimental results show that the proposed network performs more favorably against other state-of-the-art methods on several datasets such as GoPro, HIDE and RealBlur.