Addressing the challenge of effectively tracking floating litter in water bodies, which undergoes significant changes in appearance due to buoyancy and mobility, compounded by its small size, this paper proposes the D-StrongSORT algorithm. This algorithm designs a cost association model comprising motion and appearance models. The motion model incorporates ECC camera compensation and NSA Kalman filter, while the appearance model employs the ResNeSt50 feature extractor and EMA feature update strategy. The motion model compensates for camera noise through ECC compensation, predicts target positions using the Kalman filter, and calculates motion association costs by correlating with detection results. Optimizing the association matching with the DIOU algorithm accurately captures the motion trajectories and appearance features of floating litter. Experimental results demonstrate that compared to StrongSORT, D-StrongSORT reduces the number of identity switches from 19 to 15 and increases the number of multiple tracked targets from 74 to 79, with other metrics showing varying degrees of improvement, reflecting the algorithm’s enhanced ability to sustain tracking of the same target over time.