VQA combines computer vision and natural language processing to deal with multimodal challenges. The conventional models cannot cope with deep thinking, semantic understanding, and learning and thus require advanced methodologies for application. Developing a better Visual-NLP system by using the mechanism of adaptive attention with meta-learning, as well as knowledge graphs for better cross-modal reasoning and response to queries. CNNs extract visual features, while knowledge graphs use transformer-based embeddings to improve semantic enrichment. Relevance is given priority via adaptive attention, while meta-learning enhances flexibility across various datasets and related visual domains. It performed better than the multimodal models existing currently. Its precision, recall, and F1-score were 93%. High attention, very effective reasoning, and reduced computing complexity to (O(n3)) were reported.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhanced Visual-NLP Systems Using Knowledge Graphs, Meta-Learning, and Adaptive Attention Networks

  • Mohan Reddy Sareddy,
  • R. Veerandra Kumar,
  • M. Thanjaivadivel

摘要

VQA combines computer vision and natural language processing to deal with multimodal challenges. The conventional models cannot cope with deep thinking, semantic understanding, and learning and thus require advanced methodologies for application. Developing a better Visual-NLP system by using the mechanism of adaptive attention with meta-learning, as well as knowledge graphs for better cross-modal reasoning and response to queries. CNNs extract visual features, while knowledge graphs use transformer-based embeddings to improve semantic enrichment. Relevance is given priority via adaptive attention, while meta-learning enhances flexibility across various datasets and related visual domains. It performed better than the multimodal models existing currently. Its precision, recall, and F1-score were 93%. High attention, very effective reasoning, and reduced computing complexity to (O(n3)) were reported.