Retrieval-Augmented Generation (RAG) shows strong potential for knowledge-intensive reasoning. However, it performs unreliably in government budget auditing, often producing responses that are linguistically sound yet institutionally invalid. We identify two domain-specific failure modes: Caliber Shift, where fiscal indicators lose comparability due to boundary changes across fiscal years, and Causal Mismatch, where generations violate the policy-expenditure-performance auditing logic. To address these issues, we propose Causality-Aware Retrieval-Augmented Generation (CRAG). This framework enforces causal completeness and caliber consistency during retrieval through a closed-loop mechanism, driven by Causal Adaptive Retrieval Control (CARC) and Consistency-based Evidence Scrutiny (CES). Using a high-quality government budget auditing QA dataset we constructed and domain-specific metrics we designed, CRAG substantially outperforms state-of-the-art RAG baselines. The CRAG framework has been deployed as a prototype within real audit data workflows, showing promising potential for intelligent budget auditing. Our code is available at https://github.com/budget-auditing/CRAG .