<p>In the development of fishery intelligence, accurate and efficient fish recognition is of vital importance. However, traditional recognition algorithms suffer from insufficient accuracy and real-time performance in complex scenarios. This study proposes an improved YOLOv8 algorithm that integrates SEAttention and LDConv. The SEAttention mechanism is introduced to enhance the network’s focus on important features. Through channel weighting, it highlights the key features of fish and suppresses redundant information. Meanwhile, LDConv is adopted to replace some traditional convolutional layers, expanding the receptive field while reducing computational load, thereby improving the network’s ability to detect multi-scale fish targets and effectively balancing model accuracy and computational efficiency. The experimental results show that the improved algorithm achieved an average accuracy rate (mAP@0.5) of 68.10% on the enhanced fish dataset, which was 2.01% points higher than the original YOLOv8 algorithm’s 66.09%. The recall rate also increased from 93.43 to 96.60%. In terms of processing speed, the improved algorithm reached 29.88 FPS in the specified experimental environment, meeting the real-time requirements. Moreover, this algorithm has better adaptability to fish images of different scales, postures, and complex backgrounds, effectively solving the problems of easy missed detection of small target fish and loss of details of large target fish.</p>