Multi-task learning has gained significant industry attention due to its efficient data use and robust generalization, making it ideal for delivering high-quality intelligent services. Edge devices, serving as primary user platforms, are essential for multi-task service delivery. However, existing multi-task models are often large, and user demands are increasingly diverse. Deploying these models directly on edge devices strains resources and leads to task redundancy. To tackle this challenge, this paper introduces an innovative pre-trained multi-task model pruning method tailored for edge computing. The objective is to leverage existing pre-trained models to create a compact multi-task model suitable for edge devices. The implementation involves three key steps: First, decompose tasks within the pre-trained model and select those aligned with actual user needs. Second, while preserving the original model’s knowledge, assess parameter importance and employ a parameter fusion technique to integrate shared parameters across tasks effectively. Finally, derive a compact multi-task model optimized for edge devices. Experiments conducted on three public image datasets validate the method’s effectiveness and efficiency, offering a novel solution for multi-task learning on edge devices. Our code and related baseline methods are available at: https://github.com/zxccvdql/CUT .