Federated learning (FL) enables clients to train global models under server coordination without sharing data, thereby addressing privacy concerns. However, FL faces multiple challenges in noisy environments, including vulnerability to client data poisoning attacks and inefficiency when handling dynamic class distribution changes. To address these issues, we propose an adaptive robust federated learning framework named RAFL-DSV. We introduce a dynamic class contribution assessment module that quantifies the Shapley value based on gradient alignment. To enhance robustness in real-world scenarios, we develop an adaptive aggregation module that adjusts client weights with dynamic Shapley values. Additionally, we propose a dual-weight balanced sampling module, which achieves a trade-off between convergence and exploration through the balance of convergence weights and exploration weights and client importance sampling optimisation. We conducted performance, efficiency, and fairness experiments on the CIFAR-10, Fashion-MNIST, and MNIST datasets. The experiments demonstrate that under noisy perturbation scenarios, our method achieves a 7% to 16% performance improvement over baseline methods while ensuring client selection fairness and a 2- to 3-fold improvement in time efficiency.