RASR: A Multi-perspective Semantic Text Similarity Computation Method Integrating RAG
摘要
Semantic Text Similarity (STS) is a fundamental task in Natural Language Processing (NLP), aimed at measuring the similarity between sentence pairs, with wide applications in information retrieval and language understanding. Traditional STS models rely on manually designed features and shallow representations, struggling to capture complex semantic relationships and lacking scalability and flexibility. In recent years, large language models (LLMs), with their superior contextual understanding, have provided new approaches to address this issue. However, these models still face challenges in STS tasks, such as inconsistent prompt example quality, single-perspective scoring, and difficulties in fully capturing complex semantic relationships. To address these challenges, this paper proposes RASR (Refined Augmented Scoring with Reflection), a method, by integrating the Retrieval-Augmented Generation (RAG) approach, RASR enhances few-shot prompting by incorporating high-quality relevant examples, improving prompt consistency and scoring accuracy. RASR enables LLMs to score sentence pairs from multiple perspectives, enhancing scoring diversity. This approach more effectively captures complex semantic relationships, significantly improving the accuracy and robustness of semantic text similarity computation. We applied the RASR method to both open-source and closed-source models and evaluated it on four datasets using six LLMs. Compared to traditional methods and chain-of-thought approaches, RASR achieved significant improvements across nearly all metrics.