Image captioning models are typically limited to generating captions based on patterns learned from the training data. EVCap (External Visual-Name Memory for Retrieval-Augmented Image Captioning) addresses this limitation by using a visual-object name memory to augment LLMs with retrieved object names. However, this approach focuses primarily on objects and overlooks other important factors such as context and actions. In this paper, we present an enhancement of EVCap by introducing a new memory component that stores visual-caption pairs, which can be used to extract both actions and objects to further improve model performance. The incorporation of visual-caption pairs enables better integration of multi-modal information, enhancing the model’s ability to generate more accurate, contextually rich captions. Experimental results demonstrate that our model consistently outperforms the original EVCap, notably achieving a score of up to 119.5 in CIDEr and 15.4 in SPICE on the NoCaps validation set.