Scene graphs are structured and powerful representations of images that extract semantic information from images to enhance visual understanding and image reasoning. To improve the utilization of predicate prior knowledge in scene graphs, we use a probability mass function (PMF) for the predicate, given a subject-object pair during the initialization of the graph neural network. The PMFs are updated through a gating function during message passing of the graph neural network. To mitigate the effects of long-tailed distributions in datasets, we employ a data resampling strategy. During each training iteration, we try to balance the head, body and tail predicates as much as possible. Experiments show that the proposed methods achieve competitive performance compared to previous methods on dataset we created for home environment, demonstrating their effectiveness and generality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PR-GB-NET: Probabilistic Graph Bridging Network with Data Resampling for Scene Graph Generation

  • Hongyu Lu,
  • Guoliang Liu,
  • Guohui Tian,
  • Jian Jiang,
  • Shanmei Wang

摘要

Scene graphs are structured and powerful representations of images that extract semantic information from images to enhance visual understanding and image reasoning. To improve the utilization of predicate prior knowledge in scene graphs, we use a probability mass function (PMF) for the predicate, given a subject-object pair during the initialization of the graph neural network. The PMFs are updated through a gating function during message passing of the graph neural network. To mitigate the effects of long-tailed distributions in datasets, we employ a data resampling strategy. During each training iteration, we try to balance the head, body and tail predicates as much as possible. Experiments show that the proposed methods achieve competitive performance compared to previous methods on dataset we created for home environment, demonstrating their effectiveness and generality.