Fraud detection in financial networks requires models that are both accurate and efficient for real-time inference. In this work, we propose a knowledge distillation framework where a GraphTransformer based teacher model distills its key and query components to a lightweight Simple attention model (SAM) student model. Our approach enables the student model to learn meaningful attention representations from the teacher without direct access to graph edges, significantly reducing computational overhead. We evaluate our method on a financial transaction dataset from the Unified Payments Interface (UPI) for fraud detection. Experimental results demonstrate that our distilled G-TED SAM model achieves improvement over baseline models while significantly reducing inference time and model size. This method offers a promising direction for deploying efficient fraud detection systems in UPI.