The You Only Look Once (YOLO) algorithm series is acclaimed for its efficient and real-time object detection capabilities, which exhibit outstanding performance in various data processing tasks, especially in satellite remote sensing. However, in practice, it is a formidable challenge to directly deploy the YOLO model with massive parameters on resource-constrained edge devices. The principal challenge lies in the fact that the training and inference phases of the YOLO model, due to its extensive parameterization, demand considerable computational power and substantial hardware storage. To overcome this difficulty, we propose a new lightweight YOLO model via random matrix factorization, named CUR-YOLO, capable of significantly reducing the computational complexity and the number of parameters in the YOLO model The key innovation of CUR-YOLO involves breaking down each large weight matrix of the YOLO model into three smaller sub-matrices, effectively replacing the computation and storage demands of the original matrix with those of the more efficient sub-matrices. Furthermore, we present a novel rank-constrained back-propagation algorithm, which enables direct parallel training of these small-scale sub-matrices on edge devices. In contrast to existing lightweight methods, the proposed CUR-YOLO effectively reduces the original YOLO’s computational complexity and parameter count, under the same recognition accuracy conditions. These findings highlight the enormous potential of deploying our CUR-YOLO on resource-constrained edge devices to enable real-time analytics and decision-making for satellite remote sensing tasks.