Keyword extraction from text is an important task in the field of natural language processing (NLP). This process aims to identify meaningful and summative words or phrases from large bodies of text. Keywords are important elements that summarize the content of a text and enable users to quickly understand the main theme of the text. This process is a critical component for many applications such as information retrieval text mining summarization and search engine optimization. TF-IDF (Term Frequency-Inverse Document Frequency) one of the statistical methods for keyword extraction measures how important a word is in a particular document. TF-IDF multiplies how often the word occurs in the document (TF) by how rare that word is among all documents (IDF). A high TF-IDF value indicates that the word is important for that document. Another statistical method Word Frequency: calculates the number of words in the text and selects the most frequently occurring words as keywords. The third statistical method the Chi-Square Test is used to determine whether words belong to certain classes. It can be especially effective in classification problems. In this study 3 statistical NLP steps namely TF-IDF Word Frequency and Chi-Square Test were applied on 100 texts collected on the internet and at least 200 words long and keywords were generated according to the number of keywords preferred by the user. Three methods were compared and it was determined that the most successful method among them was Word Frequency.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Keyword Extraction From Turkish Text Using Natural Language Processing Techniques

  • Tuncay Yiğit,
  • Yeşim Aktaş

摘要

Keyword extraction from text is an important task in the field of natural language processing (NLP). This process aims to identify meaningful and summative words or phrases from large bodies of text. Keywords are important elements that summarize the content of a text and enable users to quickly understand the main theme of the text. This process is a critical component for many applications such as information retrieval text mining summarization and search engine optimization. TF-IDF (Term Frequency-Inverse Document Frequency) one of the statistical methods for keyword extraction measures how important a word is in a particular document. TF-IDF multiplies how often the word occurs in the document (TF) by how rare that word is among all documents (IDF). A high TF-IDF value indicates that the word is important for that document. Another statistical method Word Frequency: calculates the number of words in the text and selects the most frequently occurring words as keywords. The third statistical method the Chi-Square Test is used to determine whether words belong to certain classes. It can be especially effective in classification problems. In this study 3 statistical NLP steps namely TF-IDF Word Frequency and Chi-Square Test were applied on 100 texts collected on the internet and at least 200 words long and keywords were generated according to the number of keywords preferred by the user. Three methods were compared and it was determined that the most successful method among them was Word Frequency.