Human action recognition is a critical field in artificial intelligence and computer vision, with wide-ranging applications such as healthcare, surveillance, and virtual reality. This study enhances action recognition performance by integrating transfer learning with the Quaternion Discrete Fourier Transform (QDFT), a novel approach that leverages the mathematical properties of quaternions for advanced signal processing. Using a subset of the UCF50 dataset, the study evaluates the effectiveness of this method across various actions, including BaseballPitch, Basketball, and Biking. The approach involves extracting features from pre-trained convolutional neural networks (CNNs) and applying the Fourier transform on quaternions to these features, which are then combined with those processed through fully connected layers. The experimental results demonstrate that incorporating QDFTs significantly improves the accuracy, precision, recall, and F1-scores of transfer learning models compared to conventional methods. Furthermore, comparative analysis with baseline models, such as MobileNet and EfficientNet, highlights the superiority of this hybrid approach. This research contributes to the field of action video recognition by providing a robust framework that can be adapted to various applications, with potential for further optimization and exploration across different datasets and recognition tasks.