Short-text similarity is a vital research area in NLP with significant implications for use cases like search recommendations and question-answer systems. Traditional models often focus solely on semantic similarity, overlooking syntactic factors. Our approach uses the Angle Embedding (AnglE) method for semantic similarity, which transforms text into high-dimensional vectors to capture the nuanced meanings and relationships between words and phrases. The cosine similarity measure is then employed to calculate the similarity score from these vectors. We apply the weighted tree edit distance (TED) method for syntactic similarity, which measures structural differences between parse trees by calculating the minimum cost required to convert one tree into another through a series of edit operations. By integrating these two complementary similarity measures, our approach aims to deliver a more thorough and accurate evaluation of text similarity. This methodology introduces an advanced technique that combines semantic and structural information to enhance the assessment of short-text similarity. The integrated methodology introduces a sophisticated framework that not only enhances the precision of similarity evaluations but also bridges the gap between semantic and syntactic analyses, thereby offering a more comprehensive evaluation of text similarity.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Hybrid Framework for Short-Text Similarity Detection by Integrating Semantic and Syntactic Measures

  • M. Mumthaz Beegum,
  • Raseena Beevi,
  • S. Aji

摘要

Short-text similarity is a vital research area in NLP with significant implications for use cases like search recommendations and question-answer systems. Traditional models often focus solely on semantic similarity, overlooking syntactic factors. Our approach uses the Angle Embedding (AnglE) method for semantic similarity, which transforms text into high-dimensional vectors to capture the nuanced meanings and relationships between words and phrases. The cosine similarity measure is then employed to calculate the similarity score from these vectors. We apply the weighted tree edit distance (TED) method for syntactic similarity, which measures structural differences between parse trees by calculating the minimum cost required to convert one tree into another through a series of edit operations. By integrating these two complementary similarity measures, our approach aims to deliver a more thorough and accurate evaluation of text similarity. This methodology introduces an advanced technique that combines semantic and structural information to enhance the assessment of short-text similarity. The integrated methodology introduces a sophisticated framework that not only enhances the precision of similarity evaluations but also bridges the gap between semantic and syntactic analyses, thereby offering a more comprehensive evaluation of text similarity.