Community-based question answering (CQA) platforms are websites that let users ask questions, get answers, and exchange information. One of these services’ biggest challenges is identifying duplicate questions and identifying high-quality answers from previous responses. Many efforts have been made to represent words as numerical vectors, including wordtovec. We converted preprocessed text input into a numerical vector by combining these techniques with OpenAI (artificial intelligence) embedding. Bidirectional encoder representations from transformers (BERT) is a model build by using encoder and transformer to analysis textual data which applied on QA system, sentiment analysis applications. Sentence-BERT and Distil-BERT are next version of BERT both used in same task. There are different generative pre-trained transformers (GPT) architecture like GPT-1, GPT-2, and GPT-3. We examine Sentence-BERT, Distil-BERT, and GPT-3 model on CQA system. Result demonstrate that any algorithm trained on OpenAI embedding yields excellent performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Need of OpenAI GPT-3 Embedding for Duplicate Text Detection to Improve Performance of CQA Website

  • Gayatri G. Asalkar,
  • Bechoo Lal,
  • Nilesh B. Korade

摘要

Community-based question answering (CQA) platforms are websites that let users ask questions, get answers, and exchange information. One of these services’ biggest challenges is identifying duplicate questions and identifying high-quality answers from previous responses. Many efforts have been made to represent words as numerical vectors, including wordtovec. We converted preprocessed text input into a numerical vector by combining these techniques with OpenAI (artificial intelligence) embedding. Bidirectional encoder representations from transformers (BERT) is a model build by using encoder and transformer to analysis textual data which applied on QA system, sentiment analysis applications. Sentence-BERT and Distil-BERT are next version of BERT both used in same task. There are different generative pre-trained transformers (GPT) architecture like GPT-1, GPT-2, and GPT-3. We examine Sentence-BERT, Distil-BERT, and GPT-3 model on CQA system. Result demonstrate that any algorithm trained on OpenAI embedding yields excellent performance.