We propose CANpose, a lightweight model for human pose recognition that integrates convolutional networks with a bi-modal cross-attention mechanism to effectively capture both local and global features. In contrast to conventional CNN-based and hybrid attention approaches, CANpose leverages the complementary nature of pose depth maps and binary silhouettes through cross-attention, enhancing its ability to distinguish between visually similar poses. By employing depth-wise separable convolutions for efficient local feature extraction, CANpose significantly reduces computational complexity–achieving up to 17 \(\times \) fewer parameters and 11 \(\times \) lower FLOPs–while maintaining high recognition accuracy across multiple benchmark datasets. These results position CANpose as a scalable and efficient solution for real-world pose recognition applications.