I2V-CMGAN: Generative Adversarial Cross-Modal Network-Based Image-to-Video Person Re-identification
摘要
Information asymmetry situation amongst image and video features in image-to-video (I2V) person re-identification (Re-ID) refers to the difficulty in extracting consistent and dependable features from both image and video data in order to accurately match and identify a person in both modalities. The problem arises because images and videos have different characteristics and represent different aspects of a person. This difference in representation can result in inconsistent and unreliable features being extracted from image and video data, leading to difficulty in accurately matching and re-identifying a person between modalities. The temporal information provided by videos can also boost the accuracy of person re-identification, especially in crowded and cluttered environments. To address the information asymmetry problem, a generative adversarial cross-modal network–based I2V Person Re-ID (I2V-CMGAN) is proposed, which works by using a generator to transform the features learned from the video network into an image network with an additional loss function to improve the consistency and reliability of features extracted from both image and video data and also preserve identity information. Extensive studies show the efficacy of the proposed approach, and the aggregate results on the MARS dataset outperform the state-of-the-art methods by a substantial margin and achieved rank-1 accuracy of 88.9% (+ 2.9), rank-5 accuracy of 95.5% (+ 2.3), rank-10 accuracy of 97.1% (+ 2.9), and mean average precision of 81.2% (+ 1.1) for I2V Re-ID. On iLIDS-VID and PRID2011 datasets, the proposed method attains outstanding margins with rank-1, rank-5, rank-10, mAP of 64.7%, 89.3%, 92.7%, 74.2%, and 80.9%, 93.3%, 98.9%, and 86.8% respectively.