Recent advancements in airborne platforms equipped with ultra-high-resolution imaging sensors have significantly improved our capability to acquire detailed urban imagery. These systems offer exceptional clarity and precision in capturing urban data, paving the way for innovative approaches to urban analysis. However, manually extracting information from this data can be a slow and labour-intensive process. Thus, employing deep learning algorithms for data extraction appears to be a promising solution. However, deep learning methods require significant amounts of training data, processing power, and memory to achieve satisfactory results. To tackle these challenges, we developed a new approach called the Color Range Mask (CRM) layer, which facilitates an in-depth understanding and assessment of input images. This method enhances the effectiveness of CNN-based deep learning algorithms for image processing while maintaining computational and memory efficiency. We evaluated our approach using both U-Net and pre-trained ResNetUNet deep learning architectures. The CRM layer proved to be effective, slightly improving segmentation accuracy, particularly for complex classes such as Crosswalks and Water bodies, which have a limited amount of training data. Comparative assessments revealed that pre-trained ResNetUNet models incorporating the CRM layer outperformed baseline U-Net models across various metrics, including Intersection over Union (IoU) and Accuracy.