Multi-view Multi-label Canonical Correlation Analysis with Dual Correlations for Cross-modal Multimedia Retrieval
摘要
We present Multi-view Multi-label Canonical Correlation Analysis with Dual Correlations (MVMLCCA-DC), a novel generalization of the Canonical Correlation Analysis (CCA), for performing cross-modal multimedia retrieval on multi-view data in presence of multi-labelled meta-data. Unlike existing CCA-based methods that take only either feature correlations or semantic correlations into account, MVMLCCA-DC simultaneously integrates both of them. Specifically, it collates semantic correlations in the form of similarity in multi-labelled meta-data, and feature correlations in the form of correlations between features from diverse modalities. Further, MVMLCCA-DC can be applied to multi-view data with two or more views/modalities unlike some of the earlier CCA-based methods. We conduct extensive experimental comparisons on three popular datasets (IAPRTC-12, MS-COCO and MirFlickr-25K) to demonstrate the flexibility and effectiveness of MVMLCCA-DC.