One method to aid the understanding of a document corpus is to try constructing automatically a word/term/knowledge map for that corpus by analyzing the contents of the documents. Several methods have been proposed in the literature for this task. In this paper we investigate a novel method that is based on Association Rule Mining (ARM). ARM was proposed for databases, for structured data in general, as a method for data mining, e.g. for market basket analysis. Here we investigate its application over documents. In particular, we leverage association rule mining, through the Apriori algorithm, to find pairs of terms that co-occur in documents and their association. Each rule is characterized by its confidence and support. Then we map these rules to graphical elements. A key merit of the approach is that the user can interactively change the confidence and support threshold and obtain a different visualization.  The evaluation over small datasets, up to datasets with 125.654 distinct words, showed that this approach is feasible and can produce maps that show the dominating words and connections. Source code: https://github.com/EfthimisM/AssociationMaps Video: https://youtu.be/eN9VrmmS6Ls .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Interactive Association Map Creation from Documents Using Association Rule Mining

  • Efthimios Mitkousis,
  • Yannis Tzitzikas

摘要

One method to aid the understanding of a document corpus is to try constructing automatically a word/term/knowledge map for that corpus by analyzing the contents of the documents. Several methods have been proposed in the literature for this task. In this paper we investigate a novel method that is based on Association Rule Mining (ARM). ARM was proposed for databases, for structured data in general, as a method for data mining, e.g. for market basket analysis. Here we investigate its application over documents. In particular, we leverage association rule mining, through the Apriori algorithm, to find pairs of terms that co-occur in documents and their association. Each rule is characterized by its confidence and support. Then we map these rules to graphical elements. A key merit of the approach is that the user can interactively change the confidence and support threshold and obtain a different visualization.  The evaluation over small datasets, up to datasets with 125.654 distinct words, showed that this approach is feasible and can produce maps that show the dominating words and connections. Source code: https://github.com/EfthimisM/AssociationMaps Video: https://youtu.be/eN9VrmmS6Ls .