In today’s data-rich world, we often deal with multiple types of information such as images, text, and audio. Traditional deep learning models usually focus on a single type of data, but real-world applications need systems that can understand and connect across these different formats—a concept known as multi-modal learning. This paper explores cross-modal retrieval, where a user can input one type of data (like an image) and retrieve another (like related text). To make this possible, we map different data types into a common space using deep learning methods like CNN for images and LSTM for text. One of the key challenges in this area is comparing vectors of different lengths, which affects similarity estimation. Most traditional methods use inner product similarity, which is not ideal for vectors with varying magnitudes. To overcome this, we normalize the vectors using cosine similarity, which focuses only on the angle between vectors, not their length. This improves retrieval accuracy by reducing noise caused by vector size differences. We also discuss the benefits of using deep learning to jointly learn features and generate hash codes for faster and more accurate retrieval. Experiments on datasets like Google News show that cosine similarity outperforms Euclidean distance in terms of retrieval performance, especially when combined with models like CBOW.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Joint Feature Learning and Hashing for Multi-modal Data Via Cosine Normalization

  • Nikita Bhatt,
  • Nirav Bhatt,
  • Purvi Prajapati

摘要

In today’s data-rich world, we often deal with multiple types of information such as images, text, and audio. Traditional deep learning models usually focus on a single type of data, but real-world applications need systems that can understand and connect across these different formats—a concept known as multi-modal learning. This paper explores cross-modal retrieval, where a user can input one type of data (like an image) and retrieve another (like related text). To make this possible, we map different data types into a common space using deep learning methods like CNN for images and LSTM for text. One of the key challenges in this area is comparing vectors of different lengths, which affects similarity estimation. Most traditional methods use inner product similarity, which is not ideal for vectors with varying magnitudes. To overcome this, we normalize the vectors using cosine similarity, which focuses only on the angle between vectors, not their length. This improves retrieval accuracy by reducing noise caused by vector size differences. We also discuss the benefits of using deep learning to jointly learn features and generate hash codes for faster and more accurate retrieval. Experiments on datasets like Google News show that cosine similarity outperforms Euclidean distance in terms of retrieval performance, especially when combined with models like CBOW.