Image description generation is an intricate process incorporating computer vision and natural language processing to produce descriptive captions for photographs. This paper presents a novel architecture combining XceptionNet as the image encoder with Nyströmformer, a linear-complexity transformer variant, as the caption decoder. By leveraging linear attention mechanisms, the proposed model mitigates the quadratic complexity of traditional transformers, ensuring efficient and scalable caption creation. Evaluated on the Microsoft Common Objects in Context dataset using BLEU, ROUGE, and CIDEr, the model demonstrates competitive performance while significantly reducing computational overhead. Additionally, a mathematical discussion of the Nyströmformer’s attention mechanism highlights its advantages over conventional transformer-based approaches.