Affine registration is a key component in a wide range of medical image processing frameworks and is extensively applied in clinical settings. Nevertheless, recent works often emphasize the integration of affine and deformable registration, resulting in the standalone performance of affine registration being underexplored. Despite its clinical significance, affine registration faces two major challenges: it involves larger transformations compared to deformable registration, and it requires fast runtime for high-resolution 3D medical images to meet real-time clinical demands. To address these challenges, we introduce a Transformer block into the CNN-based affine registration network to effectively model the long-range dependencies in high-level features. This integration also preserves the CNN’s ability to capture local features while avoiding the high computational cost of a full Transformer architecture. Moreover, for the CNN stage, we use a Siamese network to extract low-level features for fixed and moving images. We also add parallel convolution branches to enhance the receptive field of convolution operations. Experiments on common and practical affine registration tasks, i.e., template-matching normalization and atlas-based registration, demonstrate that our method achieves higher registration accuracy than the state-of-the-art methods while being 10 times faster and having only half of the model parameters.