With the widespread adoption of semantic segmentation models across numerous applications, safeguarding the intellectual property (IP) of these models has become a pressing concern. The predominant landscape of IP protection strategies primarily revolves around passive methods, such as fingerprints or watermarking, which are used for ownership verification after a potential IP infringement. However, these approaches are all designed for classification tasks and come with challenges and delays in practical applications. To address these issues, we are the first to introduce an innovative active IP protection method tailored specifically for semantic segmentation models. Several mutual information-based strategies are designed for training semantic segmentation models to deter illicit and covert model usage in our proactive protection scheme. By training these models to yield visually appealing yet unrecognizable segmentation predictions in response to unauthorized inputs, we effectively mitigate the risk of unauthorized deployment or misuse of the protected model. Empirical experimental results have demonstrated the effectiveness of our approach where the protected models can produce distorted output masks in response to unauthorized inputs while maintaining high mIOU scores for users with authorization tokens. Furthermore, our method incorporates a designed encoder network to enhance the invisibility of tokens against potential attacker inspection aimed at bypassing the protection measures.