Lane detection is an important task for lane keeping in automated driving. However, due to the complexity of road conditions, maintaining real-time lane detection with limited GPU resources and occupied storage on mobile devices remains a challenge. As a result, we propose a lightweight segmentation network for lane detection (ULNet). Firstly, we incorporate lane line coordinates to enable the model to perceive the absolute location of lane lines, and design a fusion module (MHLM) that incorporates multiple masks of different scales, and high and low dimensional features through a lightweight approach to improve the model’s ability to perceive lane lines at different scales. Then, a GhostConv group multi-direction point-by-point product attention (GMPPA) is designed to extract the features in different directions of the lane line by grouping the features and multiplying them element-by-element in different directions respectively. Finally, a two-branch Decoder module (BI-Trans) combining inverse convolution and bilinear interpolation is designed so that the fine state can be maintained while the image is subjected to an up-sampling operation. Experiments of the final model on the lane detection datasets TuSimple and CuLane datasets show that the model achieves good segmentation results and is capable of lightweight lane detection with only 0.26 MB of parameters and 0.56 GFLOPs of computational cost in the segmentation phase. In addition, the trained model occupies about 0.4 MB of memory, which meets the requirements for deployment on resource-limited mobile devices.