Semantic segmentation models are widely used in various computer vision tasks. However, these models often suffer from biases and inefficiencies, which can limit their performance. In this paper, we propose a novel approach of model pruning using explainable AI (XAI) techniques. The proposed approach aims at identifying and eliminating non-pertinent channels in convolutional layers. It is applied for segmentation-based models, where we use XAI as the criterion for pruning techniques specifically tailored for segmentation tasks. By guiding the pruning process by means of explainability metrics, thorough experiments were conducted on the DIVA-HisDB benchmark dataset to correct model biases and enhance overall efficiency and performance.