Most existing single image super-resolution (SR) methods employ the same fixed convolution kernels for extensively varied image contents, which are often suboptimal for the varied content in different image regions. To address this problem, we propose a novel building block called Subimage-Adaptive Convolution Block (SACB), which generates spatial-variant convolution kernels for HR image reconstruction, leveraging prior knowledge extracted from diverse subimages. In the SACB, multiple base convolution kernels are employed to progressively capture distinct hints from different subimages by parallel optimization during training, which can be re-parameterized into a single spatial-variant kernel in the inference stage to maintain decent efficiency. In essence, it utilizes predicted, adaptive linear combination coefficients of the base kernels for SR reconstruction. Based on the SACB, a unified solution pipeline named SACBSR is advanced. It consists of a shallow prediction module to dynamically generate different fusing coefficients for varying subimages, and an SR module made of a series of SACBs for adaptive SR reconstruction on the input image. Furthermore, our proposed SACB exhibits broad applicability across various CNN-based SR methods with enhanced image quality and promising computational efficiency. Extensive experiments over six benchmark datasets demonstrate the effectiveness and efficiency of SACB and SACBSR.