With the ability to capture long-range dependencies through self-attention mechanisms, Vision Transformer has demonstrated outstanding performance for many vision tasks. However, global interactions between tokens result in high computational complexity, especially for high-resolution images. Thus, it is a challenging task to efficiently capture global context features, especially at shallow layers. To address this problem, we propose Context Aggregation and Transmission Transformer (CATFormer), a novel hybrid architecture that unifies the strengths of Convolutions and Transformers. Specifically, we add sparse query tokens to the self-attention module to gather context, which helps in efficiently capturing important global information. These aggregated global representations are then transmitted into convolutional pathways, allowing the network to propagate global context into local regions. This transmission mechanism empowers the model to effectively unify short-range local patterns and long-range global dependencies within a cohesive framework. Extensive experiments demonstrate that CATFormer achieves superior performance and efficiency over existing lightweight networks in various vision tasks.