Motion detection methods are essential in many fields. This paper introduces a new approach to Energy-based Motion Pattern Detection (EMPD). The research aims to provide a theoretical basis for concepts of motion patterns, energy distance, and EMPD. This method was tested on the FBMS-59 dataset with 3,105 frames, including various objects such as bears, cats, lions, people, marple1, marple3, rabbits, and horses, showing that the motion pattern detection method with the energy model achieves good efficiency. The study uses clustering through two methods: Hierarchical clustering and K-means clustering, with Silhouette Score values reaching 0.64 and above. In particular, the horses06 data has a hierarchical clustering index of 0.84 and K-means reaches 0.85, the datasets have Silhouette Score results differing from 0 to 5%. The results show that both clustering methods are good. However, the K-means clustering method achieves better efficiency. The research results show that the EMPD method outperforms other methods and has significant importance in the field of motion pattern detection.