<p>Obtaining the trajectory data of traffic objects from rainy videos can be of great significance for traffic investigations. Since there are usually some interferences in these videos recorded by traffic surveillance, this makes it difficult to obtain the trajectory information of traffic objects in rainy conditions. In this paper, we present spatial-frequency domain guided residual network with deep association tracking (SFD-ResTrack), a rain-adaptive trajectory extraction framework that comprises a spatial-frequency domain algorithm and a density prediction branch. Performance evaluations are conducted on the synthetic multiple-object-tracking 2016 (MOT16) data with rain ratios of 5%, 10%, 20% and 40%, as well as on natural rain data including light rain (LR), moderate rain (MR), heavy rain (HR), and torrential rain (TR). For the performance evaluations on the synthetic data, the presented framework can achieve the 61.5% accuracy on the MOT16 data with rain ratios of 5%; the 52.2% accuracy on the MOT16 data with rain ratios of 10%; the 46.1% accuracy on the MOT16 data with rain ratios of 20%; the 32.7% accuracy on the MOT16 data with rain ratios of 40%. For the performance evaluations on the natural data, our framework can achieve the 67.9% accuracy on the LR data; the 61.2% accuracy on the MR data; the 52.7% accuracy on the HR data; the 42.1% accuracy on the TR data. And our presented framework can achieve the MOTA value of 60.9% (± 0.5%) for diverse complex conditions across multiple runs. Lastly, our presented framework is verified to be applicable for assisting traffic investigations in rainy conditions.</p>