Evaluating fashion compatibility based on mutual reference momentum contrast with weak-positive samples
摘要
Fashion compatibility refers to the matching degree of clothing items in an outfit. Most DNN-based methods for evaluating the fashion compatibility use a triplet loss for training. However, a triplet only encodes a single contrastive relation, resulting in a weak supervisory signal. Moreover, unreliable negative samples may inappropriately participate in the training process due to lack of a screening mechanism. To alleviate these problems, this paper proposes to evaluate the fashion compatibility based on mutual reference momentum contrast with weak-positive samples. Specifically, we design a screening mechanism to mine weak-positive samples based on style consistency and visual similarity. The weak-positive samples, which serve as a supplement to the positive ones, are contrasted with the negative ones. This enriches the contrastive relations and thus enhances the supervisory signals for training. On this basis, we establish two encoders and a memory bank to extract and store features of the clothing items, respectively. In this way, multiple positive and negative samples can be introduced into a contrastive loss without compromising computational efficiency. Further, we extend a momentum update to a mutual reference mode, which enables bidirectional learning between the two encoders. Extensive experiments, including performance comparisons, ablation studies, hyperparameter settings and visualizations, are conducted on a benchmark dataset called FashionVC. Experimental results show that the proposed method, which achieves high AUC and MRR scores of 0.7709 and 0.5472, outperforms the existing baseline models.