Research question is crucial to scientific development, serving as both the logical starting point of research and the core that guides and directs scientific inquiry. In this study, we construct a feature words set about research question sentences by analyzing the syntactic structure and quantitatively expresses the influence of each feature word on the semantics of the sentence. Base on the RoBERTa (Robustly Optimized BERT Pretraining Approach) model, we introduce the weight vectors, absolute and relative position information to establish a universal text classification model. The classification results show that compared to BERT (Bidirectional Encoder Representation from Transformers), BERT-CNN (Convolutional Neural Networks), RoBERTa, and feature word-regularized RoBERTa, our proposed model achieves a 97.8% F1 score in identifying research question sentences, with improvements of 9.2, 3.0, 2.2, and 1.3%, respectively. In addition, compare with LLMs identification results, we demonstrate that the F1 score of our approach is 11.7% higher in identifying research question sentence. Overall, our approach can be applied to the classification of research question sentences in different domains. Additionally, by updating or replacing the feature words, the model can also serve as a theoretical foundation and model basis for the classification of different types of sentences. More details and model weights are public at https://huggingface.co/wmsr22/Research_question_classification/tree/main .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Modeling Sentence Classification: Considering Syntactic Analysis and Position Embedding

  • Meng Wang,
  • Zhixiong Zhang,
  • Hanyu Li,
  • Boran Li

摘要

Research question is crucial to scientific development, serving as both the logical starting point of research and the core that guides and directs scientific inquiry. In this study, we construct a feature words set about research question sentences by analyzing the syntactic structure and quantitatively expresses the influence of each feature word on the semantics of the sentence. Base on the RoBERTa (Robustly Optimized BERT Pretraining Approach) model, we introduce the weight vectors, absolute and relative position information to establish a universal text classification model. The classification results show that compared to BERT (Bidirectional Encoder Representation from Transformers), BERT-CNN (Convolutional Neural Networks), RoBERTa, and feature word-regularized RoBERTa, our proposed model achieves a 97.8% F1 score in identifying research question sentences, with improvements of 9.2, 3.0, 2.2, and 1.3%, respectively. In addition, compare with LLMs identification results, we demonstrate that the F1 score of our approach is 11.7% higher in identifying research question sentence. Overall, our approach can be applied to the classification of research question sentences in different domains. Additionally, by updating or replacing the feature words, the model can also serve as a theoretical foundation and model basis for the classification of different types of sentences. More details and model weights are public at https://huggingface.co/wmsr22/Research_question_classification/tree/main .