Detecting abnormal subgraphs is crucial for structural-level anomaly detection, offering insights into atypical interactions overlooked by traditional single-node anomaly detection methods, particularly crucial in financial networks for spotting potential money laundering activities. Current challenges arise from the diverse and complex transaction distributions and the vast scale of real-world financial networks. Addressing these, we propose a novel Reinforcement-based Anomalous subgraph DIscOvery algorithm (RADIO). RADIO incorporates an innovative subgraph encoder along with a coarse prototype discovery module, enabling efficient and accurate identification of anomalous subgraphs amidst intricate transaction distributions. It further enhances subgraph detection through strategic reward design, directing optimization towards the most significant abnormalities. Our comprehensive evaluation, using four real financial transaction datasets and comparing with twelve existing methods, confirms its exceptional performance. It outperforms the current state-of-the-art approach by an average of 7 \(\times \) in abnormal degrees of detected subgraphs and demonstrates high efficiency in handling networks with millions of nodes.