Document Information Extraction (DIE) plays a crucial role in document understanding systems by identifying structured information from various documents with applications spanning academic research to industry. While recent end-to-end generative models have demonstrated promising results, their over-reliance on language information often leads to imbalanced multimodal integration and utilization, compromising the trustworthiness of extracted results. To address this issue, we present a novel trustworthy framework that integrates a self-evaluation mechanism to enhance the reliability of document information extraction. This mechanism allows the model to evaluate and identify its generated results, facilitating our model to enhance the utilization of visual information without reducing the ability to use language information, thereby utilizing the visual and language information in a balanced way. Besides, an algorithm is introduced to automatically generate negative samples for the self-evaluation mechanism during training. Moreover, to ensure the negative samples do not adversely affect the model’s learning process, we elaborately design a simple joint training optimization strategy. Extensive evaluations across three benchmark datasets (Ticket, CORD, and SROIE) demonstrate our method’s superiority, achieving F1-score improvements of 2.1% over OmniParser, 2.9% over Donut in CORD dataset.