Privacy-Preserving Federated Learning (PPFL), a new paradigm for secure and efficient Federated Learning, has the advantage of security aggregation without losing accuracy. However, PPFL focuses on protecting the privacy of private data, but the privacy of models is equally important. Model sharing among institutions increases the risk of model privacy leakage and economic loss when federated learning is applied to real-world scenarios. To address the above issues, we design a model privacy-preserving scheme based on fully homomorphic encryption, PriM. Specifically, we adopt the CKKS fully homomorphic encryption scheme to guarantee the security of the model, which supports both floating-point and vector encryption and avoids the significant overhead of single-value encryption in the homomorphic encryption-based PPFL scheme. Then, we maintain the confidentiality of the model computation process without revealing the model information to the participating entities. Finally, experiments show that our scheme achieves equal efficiency with the baseline algorithm FedAvg but is more realistic.