Act-retrieval: a framework for action-based image retrieval
摘要
Semantic image retrieval remains a significant challenge in computer vision, and this work addresses the problem of retrieving images based on human actions. The Act-Retrieval framework is presented as a solution for action-based image retrieval, which relies solely on still images for both training and testing. The framework consists of modules for action detection, hint-learning, and an attention mechanism. The authors conducted an extensive analysis of their results, which shows that the Act-Retrieval framework outperforms relying only on the entire image by more than 25% and outperforms state-of-the-art works on Content-based image retrieval (CBIR) by more than 10% in mean Average Precision (mAP). The framework reduces the semantic gap between still images and actions and can be extended to various image retrieval applications based on static actions. The contributions of this work include an efficient framework that addresses a semantic retrieval problem with many possible combinations of strategies, the investigation of the effect of multiple information as input, and the exploration of a hint-learning approach combining the attention module with more global features to increase context information. The authors also show that the additional input of human-made bounding boxes at test time is not mandatory, and propose to use their action detection instead of multiple object detectors or manually made bounding boxes.