To alleviate workers’ labor intensity, enhance inspection efficiency, and improve worker safety, a method for automated detection of railway container loading and unloading status is proposed, leveraging the deep learning convolutional neural network YOLOv5s. Firstly, to enhance the model's capability in learning and expressing essential features, this study incorporates the Convolutional Block Attention Module (CBAM) into the feature extraction framework of YOLOv5s. This augmentation enhances the capacity for extracting features from diminutive targets, leading to increased accuracy in small target detection. Furthermore, the trained YOLOv5s model undergoes quantization, a process that converts some of the high-precision floating-point weights and activations within the neural network to lower-precision integer values. This accelerates the model's inference speed, thereby meeting the real-time detection requirements for container loading and unloading conditions in railway freight scenarios. The amalgamation of these improvements markedly enhances the model's performance. Utilizing the collected dataset for testing, the experimental results demonstrate that the confidence score of the modified model is improved by 1% compared to the original model, and the detection speed of the quantized model is increased by 74%. It will greatly improve the efficiency of freight railway transportation and promote the application of freight loading and unloading with automated gantry cranes.