The presence of specular reflection on endoscopic images can hide important underlying features and limit extraction of clinically relevant information. To address this challenge, software-based solutions to remove specular reflection can be applied, which requires the specular reflection detection and removal. In this paper, we proposed an automated detection of specular reflection using ResUNet++, which was trained using weakly annotated data to overcome the lack of annotated data and the requirement of repetitive parameter adjustment. Analytical specular reflection detection methods: K-means clustering and histogram thresholding were evaluated to provide weak annotations to train the ResUNet++. The performance of ResUNet++ when trained using the weak annotations was compared with convolutional-based model, the simple Unet, and the state-of-the-art transformer-based model. Experimental results showed that fully unsupervised K-means clustering and histogram thresholding were sufficient to provide weak annotation for training the deep learning models. This shows that the fully analytical method can minimize human supervision on the creation of training data and on the training of the deep learning models. Moreover, we showed that ResUNet++ had better performance than the state-of-the-art when trained solely using weakly annotated data and faster inference time, making it suitable for future near-real-time applications. Our code is available at https://github.com/ajsugiarti/Weak-Annotation-Specs.