In this chapter, a new methodology is introduced to improve the performance of Knowledge Graph Question Answering (KGQA) systems. The Text-Graphs 17 shared task dataset was utilized to study text-graph representations for KGQA. Our experiment use four different methods: Term Frequency-Inverse Document Frequency (TF-IDF), Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM), and Multi-head Attention (Multi-head). Our goal is to handle the inherent complexities in KGQA tasks by incorporating textual questions, candidate answer entities, and subgraphs extracted from Wikidata. In this article, a new methodology is introduced to improve the performance of Knowledge Graph Question Answering (KGQA) systems. The Text-Graphs 17 shared task dataset was utilized to study text-graph representations for KGQA. Our experiment use four different methods: Term Frequency-Inverse Document Frequency (TF-IDF), Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM), and Multi-head Attention (Multi-head). Our goal is to handle the inherent complexities in KGQA tasks by incorporating textual questions, candidate answer entities, and subgraphs extracted from Wikidata. Our goal is to identify the right answer entity from a group of candidates. This turns it into a binary classification issue. When we added graph-based data to LLM-based question answering systems, we saw significant improvements in answer selection accuracy. Our findings highlight the potential of merging textual and graph-based data to enhance KGQA systems. Our experiments were thoroughly tested on given datasets, with the F1 score as the main measure. The results showed significant performance improvements compared to standard models. This highlights the effectiveness of our text-graph fusion approaches. The highest score of 0.7679 F1 Score for the shared task dataset Text-Graphs 17 was accomplished by our multi-head attenuation method.