Images obtained by the visual sensors of automated vehicles often suffer from diminished quality due to a multitude of atmospheric phenomena such as haze, moisture, rain, and storms. These phenomena usually disrupt the proper functioning of visual sensors of vision-assisted transportation systems or Advanced Driver Assistance System (ADAS) as well as several other surveillance based systems. Removal of weather effects (deweathering) from these images has been crucial and has gotten a lot of attention in order to address these issues. Therefore, in order to accurately trigger the necessary deweathering (removal of haze, moisture, etc.) operation, a solution is needed to automatically ascertain the weather conditions depicted in an input image. This paper presents a novel deep learning-based framework for weather image recognition that takes into account 11 typical weather phenomena, including dew, rainbows, lightning, sandstorms, snow, rain, fog smog, rime, glaze, hail, and frost. For an input image, the proposed solution approach automatically classifies the image into one of the eleven categories (e.g., sunny or others). In the proposed approach, transfer learning on five state-of-the-art models, namely EfficientNetB7, ResNet50, MobileNet, VGG19, and DenseNet201, has been implemented which are pretrained on the ImageNet dataset. Using this approach, an accuracy of up to 92.5% has been obtained.