<p>Fraudulent activities within banking transactions pose a significant challenge for the banking sector, occurring either individually or as part of an organized scheme. It is always difficult to identify such illegal activities. Despite the development of various models and algorithms to tackle this issue, the intricate and diverse nature of fraud patterns presents difficulties in detecting all suspicious transactions. Researchers have suggested using graph theory to consider the interactions between transactions in order to overcome this challenge. Another challenge is the increased false positive error when investigating individual transactions that exhibit behavior similar to high-risk behavior. To improve the understanding of the transaction process, the use of sequence-based approaches has been proposed. In this article, a model that combines graph and sequence theory was developed to detect organized fraud. The first phase of the model involved extracting network features from the transaction graph and applying a hidden Markov chain to capture the sequential nature of the transactions. In the second phase, fraud detection was performed using a combination of the support vector machine and the improved honey badger metaheuristic algorithm. This algorithm aims to enhance fraud detection efficiency by adjusting the parameters of the support vector machine. The proposed model was evaluated on three datasets—two real-world datasets and one benchmark dataset. Performance was assessed using precision, accuracy, recall, F1-score, ROC-AUC, and PR-AUC metrics. On average, the method achieved an F1-score of 92% in fraud detection.</p>