All components following the Preprocessing (Text Handling, Language and Topic Identification) are language specific, and need resources in the supported languages. The first group of analysis tools is intended to create word candidates (‘tokens’) from texts (paragraphs) as produced by the text handling: Sentence splitting, tokenisation, and normalisation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Text Segmentation

  • Gregor Thurmair

摘要

All components following the Preprocessing (Text Handling, Language and Topic Identification) are language specific, and need resources in the supported languages. The first group of analysis tools is intended to create word candidates (‘tokens’) from texts (paragraphs) as produced by the text handling: Sentence splitting, tokenisation, and normalisation.