Document-level relation extraction is an essential task in natural language processing. Achieving high performance on this task is challenging due to the complexity of reasoning over long texts. While large language models have shown promising results in natural language processing, their effectiveness in document-level relation extraction remains limited. Techniques such as chain-of-thought and few-shot prompting offer insufficient improvements for real-world applications. Reinforcement learning with human feedback (RLHF) has proven beneficial in enhancing large language model capabilities; however, its reliance on manual annotation makes it costly and impractical at scale. To address these challenges, we introduce RLKGF, an approach that incorporates knowledge graph feedback into RLHF and replacing human feedback with knowledge graph feedback. Rather than integrating knowledge graphs into the learning process, RLKGF utilizes them solely as a source of feedback for the document-level relation extraction, reducing reliance on manual supervision while lowering annotation costs and improving feasibility. The proposed method complements existing RLHF techniques and can be easily incorporated with them. In our evaluation on two benchmark datasets, our method outperforms both traditional document-level relation extraction and large language model-based approaches, achieving F1-score improvements of 10% and 3%, respectively. These results highlight RLKGF’s potential to advance on this task while minimizing human effort and annotation costs.