<p>Among the various sports activities that are carried out all over the world, football is undoubtedly the most popular, most participated in and most watched activity and sport. The increasing spread of sports has caused it to break down geographical, racial, ethnic, political and ideological boundaries. Social and cultural analysts have conducted research on various customs, rituals, values and social patterns prevalent in football in recent decades. Among the important issues that have attracted the attention of scholars, especially in the last two decades, is the study of violent events and incidents related to football. This paper introduces a real-time violence detection system that uses deep learning techniques, including convolutional neural networks and long short-term memory (LSTM) for extracting spatial features and understanding temporal relationships. The proposed model was then optimized by a modified version of shuffled shepherd optimizer (MSSO) to provide better efficiency by optimal selection of network hyperparameters. The model is then compared with earlier studies to show its superiority to be the most efficient in violence detection.</p>