<p>Few-shot learning focuses on training models that generalize to new classes or tasks with limited training samples. However, the datasets utilized in few-shot learning often contain sensitive information, raising concerns about potential privacy breaches and fairness issues. Rényi differential privacy (RDP) has gained recognition as a promising alternative to standard differential privacy, offering enhanced flexibility in composition rules and providing more precise analytical guarantees. This approach enforces privacy by introducing noise into the data or model. However, the noise introduced to ensure privacy often adversely affects both model utility and fairness. To address these challenges, this paper proposes a novel sample-level adaptive privacy filtering algorithm, individual Rényi differential privacy stochastic gradient descent, which leverages RDP as a privacy filter to achieve more accurate privacy loss estimation. Additionally, the paper proposes a privacy and fairness constraint algorithm based on Lagrange duality. This approach reformulates fairness metrics as optimization constraints, incorporates the RDP filter for precise privacy measurement, and dynamically adjusts clipping bounds by leveraging the gradients of the fairness constraints. Model parameters are optimized via gradient descent to achieve a balanced trade-off among privacy, fairness, and utility. The experimental results demonstrate that the proposed approach enhances model performance while effectively preserving privacy and fairness in few-shot learning tasks, highlighting its practical applicability in real-world scenarios.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Privacy preservation and fairness constraints for few-shot learning based on Lagrange duality

  • Jinghong Wang,
  • Hongbo Yang,
  • Wenxin Li,
  • Jusheng Mi,
  • Haokang Li

摘要

Few-shot learning focuses on training models that generalize to new classes or tasks with limited training samples. However, the datasets utilized in few-shot learning often contain sensitive information, raising concerns about potential privacy breaches and fairness issues. Rényi differential privacy (RDP) has gained recognition as a promising alternative to standard differential privacy, offering enhanced flexibility in composition rules and providing more precise analytical guarantees. This approach enforces privacy by introducing noise into the data or model. However, the noise introduced to ensure privacy often adversely affects both model utility and fairness. To address these challenges, this paper proposes a novel sample-level adaptive privacy filtering algorithm, individual Rényi differential privacy stochastic gradient descent, which leverages RDP as a privacy filter to achieve more accurate privacy loss estimation. Additionally, the paper proposes a privacy and fairness constraint algorithm based on Lagrange duality. This approach reformulates fairness metrics as optimization constraints, incorporates the RDP filter for precise privacy measurement, and dynamically adjusts clipping bounds by leveraging the gradients of the fairness constraints. Model parameters are optimized via gradient descent to achieve a balanced trade-off among privacy, fairness, and utility. The experimental results demonstrate that the proposed approach enhances model performance while effectively preserving privacy and fairness in few-shot learning tasks, highlighting its practical applicability in real-world scenarios.