<p>Real-time Video Anomaly Detection (VAD) in surveillance video has recently become one of the most developed areas of research in Computer Vision. In this paper, we propose a novel model for VAD called Tensor Deep-Anomaly (TD-A), which is structured into two main blocks: the first one uses Transfer Learning (TL) to extract feature maps from videos frames using pre-trained deep learning models. The second block employs a robust tensor classifier called Tensor Reduced Fully Connected Network (TR-FCN), which directly processes the multidimensional output feature maps of the TL block, bypassing the need of the Flatten operation typically used in classical Fully Connected Network (FCN). This approach allows us to generate a multidimensional weight and bias tensors instead of a weight matrix and bias vector. These weight and bias tensors are adjusted via the Tensor Gradient Descent (TGD) algorithm to minimize the error between network predictions and actual values. Additionally, we apply Higher Order Singular Value Decomposition (HOSVD) to the weight and bias tensors to enhance the model’s performance. Numerical tests have been showcased to validate the effectiveness of this approach.</p>