Wood defect detection plays a critical role in ensuring wood quality and minimizing resource waste in industrial production. Advanced object detectors show excellent performance in wood defect detection. However, the texture of wood defects is complex and categories are diverse, and existing methods still face challenges in the detection of defects with small size and unclear boundaries. To address the issue, we propose a novel wavelet detector based on DETR for real-time wood defect detection, which is called WT-RT-DETR. Firstly, we optimize the network structure by incorporating Partial Self Attention (PSA) as the object detection head, which enhances the multi-scale shallow and deep features. Secondly, we propose an Inverse Haar Wavelet Transform Upsampling module (UP-IHWT), which preserves details and multi-scale information based on the wavelet inverse transform. Finally, we propose the Reparameterized Wavelet Transform Convolution 3 module (RepWTC3) by employing wavelet convolution into RepC3, which strengthens the boundary features extraction capabilities of the neck network. The experimental results show that the mAP50 of WT-RT-DETR is 69.1%, which is higher than the state-of-the-art methods, and the model parameters are slightly reduced. It can better balance efficiency and accuracy.