Privacy preservation and fairness constraints for few-shot learning based on Lagrange duality
摘要
Few-shot learning focuses on training models that generalize to new classes or tasks with limited training samples. However, the datasets utilized in few-shot learning often contain sensitive information, raising concerns about potential privacy breaches and fairness issues. Rényi differential privacy (RDP) has gained recognition as a promising alternative to standard differential privacy, offering enhanced flexibility in composition rules and providing more precise analytical guarantees. This approach enforces privacy by introducing noise into the data or model. However, the noise introduced to ensure privacy often adversely affects both model utility and fairness. To address these challenges, this paper proposes a novel sample-level adaptive privacy filtering algorithm, individual Rényi differential privacy stochastic gradient descent, which leverages RDP as a privacy filter to achieve more accurate privacy loss estimation. Additionally, the paper proposes a privacy and fairness constraint algorithm based on Lagrange duality. This approach reformulates fairness metrics as optimization constraints, incorporates the RDP filter for precise privacy measurement, and dynamically adjusts clipping bounds by leveraging the gradients of the fairness constraints. Model parameters are optimized via gradient descent to achieve a balanced trade-off among privacy, fairness, and utility. The experimental results demonstrate that the proposed approach enhances model performance while effectively preserving privacy and fairness in few-shot learning tasks, highlighting its practical applicability in real-world scenarios.