Recent studies have started incorporating Transformers into 3D medical image segmentation tasks. Transformer-based models, known for their remarkable success in 2D medical image segmentation, differ significantly from CNNs in that they use self-attention mechanisms, facilitating their ability to effectively encompass distant dependencies within voxels. We introduce an efficient combined CNN-Transformer architecture for brain tumor segmentation, called PA-Swin UNETR (Paired Attention Swin UNETR). First, our Paired Attention blocks efficiently downsample spatial dimensions while concurrently learning both channel and spatial data derived from 3D feature maps, enhancing segmentation performance. Within PA-Swin UNETR, the challenge of segmentation is reconceptualized as a sequence-to-sequence prediction task. In this approach, the input three-dimensional data is converted to one-dimensional sequence of embeddings, which are then processed by a hierarchical Swin Transformer encoder. The encoder captures features across five distinct resolutions through the application of shifted windows to perform self-attention and connects to a CNN decoder at every resolution level via skip connections. PA-Awin UNETR is validated on the BraTS-2021 dataset and on a private post-operative glioblastoma dataset obtained from Uppsala University Hospital. The average dice score obtained is 0.7035 for the residual tumors of the post-operative brain tumor dataset.