The detection and prevention of criminal and violent activities is critical for enhancing public safety. This research proposes a novel approach to using surveillance video data for deep learning-based crime detection in a federated and distributed setting. The proposed methodology leverages the SlowFast network architecture for video’s spatial and temporal feature extraction relevant for efficient violence identification and classification. By utilising federated learning, the model ensures that privacy and data security are maintained, as the surveillance videos need not leave the local devices. Models are trained collaboratively across distributed nodes, reducing the risk of data breaches in high-priority security tapes. This makes our methodology potentially scalable to real-time security systems. The results achieved through this framework outperform previous methods, in both accuracy of classification and computational efficiency. This research demonstrates the efficacy of proposed methodology by evaluating it on the UCF-Crime Dataset, where it achieves high accuracy in detecting spatio-temporal anomalies associated with crime events, while offering scalability and robustness in a decentralized environment. Extensive empirical results have shown that the proposed framework significantly outperforms traditional classification models, achieving accuracies upwards of 99% while also significantly reducing training time and computational resource inefficiency by over 240% in comparison to previously proposed spatio-temporal architectures.