<p>Visual language pre-training empowers image captioning models with the capacity to describe open-world scenarios. However, this capacity is often limited, as manifested in the models’ poor performance on out-domain datasets. Since advanced models typically need to store knowledge within their own parameters, augmenting their ability to describe the open world requires additional training, which is both expensive and labor-intensive. In this study, we propose an external knowledge augmentation approach for visual language pre-trained image captioning models. We leverage visual-textual concepts to construct an expandable knowledge repository, where open-world knowledge is stored externally rather than within the model itself. Specifically, we design a knowledge retrieval module to extract highly relevant text concepts from the knowledge repository that are related to the given image. During the inference process, the extracted text concepts are embedded into a multimodal space using an external knowledge extraction structure, serving as supplementary knowledge to assist the model in generating more precise captions. Consequently, we enhance the model’s capability to describe the open world in a training-free manner. We construct the EKA-BLIP model based on the BLIP Caption as the foundational model and evaluate its performance on out-domain benchmarks, such as Flickr30K and NoCaps, without utilizing additional data for training. The results demonstrate a significant improvement in performance on these datasets.</p>