The development of language models capable of generating text with a high degree of similarity to human-written content presents significant challenges regarding the authenticity and reliability of information, both online and in academic and professional contexts. These models have raised concerns due to the difficulty in distinguishing between human and machine-generated content, which impacts the evaluation and consumption of information. This study proposes a method based on machine learning and Natural Language Processing (NLP) techniques to differentiate texts written by humans from those generated by machines. The approach focuses on analyzing textual features such as phrasal patterns, the use of discourse markers, and punctuation signs. The multilingual dataset from the IberLEF 2024 task was used, which includes texts in Spanish, English, Catalan, Basque, Galician, and Portuguese, covering various domains such as news and essays, generated by models like GPT and LLaMA. Classification algorithms such as Random Forest, Multilayer Perceptron, and XGBoost were employed, along with an ensemble model combining these methods. The ensemble model outperformed the others, followed by Random Forest. The results are promising and highlight the effectiveness of the proposed method, as well as the relevance of textual features in model learning.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Detection of Automatically Generated Texts: A Classification-Based Approach and Text Features

  • César Espin-Riofrio,
  • Joel Alejandro Barba-Salazar,
  • Edison Cruz-Navarrete,
  • Mariuxi Del Carmen Toapanta Bernabé,
  • Oswaldo Vergara-Bello,
  • Pamela Carriel-Castillo,
  • Irving Garzón-Soledispa,
  • Arturo Montejo-Ráez

摘要

The development of language models capable of generating text with a high degree of similarity to human-written content presents significant challenges regarding the authenticity and reliability of information, both online and in academic and professional contexts. These models have raised concerns due to the difficulty in distinguishing between human and machine-generated content, which impacts the evaluation and consumption of information. This study proposes a method based on machine learning and Natural Language Processing (NLP) techniques to differentiate texts written by humans from those generated by machines. The approach focuses on analyzing textual features such as phrasal patterns, the use of discourse markers, and punctuation signs. The multilingual dataset from the IberLEF 2024 task was used, which includes texts in Spanish, English, Catalan, Basque, Galician, and Portuguese, covering various domains such as news and essays, generated by models like GPT and LLaMA. Classification algorithms such as Random Forest, Multilayer Perceptron, and XGBoost were employed, along with an ensemble model combining these methods. The ensemble model outperformed the others, followed by Random Forest. The results are promising and highlight the effectiveness of the proposed method, as well as the relevance of textual features in model learning.