With the data security law granting users the right to be forgotten, it has become essential to tackle the challenge of unlearning specific training data from the global model in federated learning (FL). Most existing federated unlearning researches employ model retraining methods to forget clients’ data, which brings high computational costs and low training efficiency. Furthermore, the issue of fine-grained deletion and forgetting of part of the data within clients has yet to be addressed. To achieve efficient unlearning of part of the client’s data in FL, this paper proposes a novel approximate federated unlearning scheme based on gradient ascent. Specifically, this scheme first adopts a constrained gradient ascent method for local unlearning of the deleted data of the target client, using a dynamic penalty mechanism to reduce catastrophic forgetting in the local model. Secondly, the scheme optimizes the local unlearning model through projected gradient ascent, improving the accuracy of the global unlearning model on normal data. Additionally, extensive experiments have been conducted to verify the performance of the federated unlearning, and comparing our scheme with the model retraining. The experimental results demonstrate the effectiveness and efficiency of the proposed scheme.