The substation plays an essential role in the security and reliability of the power supply of the grid as it serves as the energy conversion hub of the entire power system. To address the challenges posed by complex on-site environments, low detection accuracy, and excessive resource consumption in existing deep learning models, we propose a lightweight substation safety inspection system designed for front-end devices. The system consists of software and hardware modules. The software module utilizes weighted bidirectional feature pyramid network, attention mechanism, and pruning-quantization-distillation operations to improve and lightweight the YOLOXs (you only look once version-xs) model, effectively compressing the model size while maintaining accuracy. The hardware module mainly achieves quantization compilation and hardware acceleration of the lightweight YOLOXs detection model on the FPGA frontend device, enabling low-latency, high-precision real-time detection for on-site operations at substations. In Experiment, the improved YOLOXs model shows an average detection accuracy increase of 2.71% compared to the original model, with a reduction in model size of 86.9% after light weighting. The FPGA front-end device achieves a single-image detection time of 87.33 ms, which satisfies the practical engineering requirements for substation safety inspection.