We address the problem of automating the previously manual inspection of asphalt distress and damage, which is time-consuming and subjective. With the help of a semantic segmentation neural network, we automate this process and make a pixel perfect detection. Our method is based on a previously published approach, validated for detecting distress symptoms in asphalt surfaces, for highway road data. The main challenge with the previous approach was, that the original training material did not yet contain significant samples of classes like foliage, drainage and additional types of road surface damage relevant to smaller secondary roads. Upon trying to retrain the underlying neural network it was discovered that methodological improvements were necessary to accurately detect these new classes. For now, we tried a new combination of a momentum-based Adam-Optimizer and a Cosine based learning Rate scheduler with restarting epoch also called Cosine Annealing. In theory, this should evade local minima of the error function, compared to the previously used polynomial decay without restarting epoch. The combination of both should yield a better minimum of the error function and higher accuracy rate.