Government auditing is a pillar of democratic governance, enabling transparency, strengthening public trust, and ensuring responsibility in the use of taxpayer resources. However, government auditing faces significant challenges, including the growing volume and complexity of financial data, limited resources for oversight bodies, and the need for specialized technical expertise. Furthermore, the manual nature of many document classification and verification tasks introduces a high risk of human error, potentially leading to rejected financial statements. This work investigates Transfer Learning (TL) to improve automation and accuracy in classifying financial commitment notes. The focus of this research is the Brazilian public expenditure cycle. We fine-tuned a pre-trained Brazilian Portuguese language model called BERTimbau to help government accountants reduce classification errors and ensure regulatory compliance. Following the CRISP-DM methodology, it was analyzed and preprocessed commitment note classifications from 2023. Two samples with similar data volumes, but varying classification counts were selected due to class imbalance. Even in a multiclass context with reduced class numbers, BERTimbau achieved accuracy around 98%, with an error rate of 0.10 in the test set, demonstrating its effectiveness for public financial auditing applications. The results indicate that the TL models have significant potential to optimize financial auditing processes, with positive implications for wider adoption throughout Brazil’s public administration system.