This chapter addresses the necessity of impact mitigation on alignment. Building on prior discussions about the challenges of aggregating human preferences into coherent learning signals, it foregrounds a fundamental limitation—the inherent imperfections and misalignments that may emerge during the value learning stage when our target is only tied to the behavior and preferences of subjects. Given that human behavior and societal values are complex, context-dependent, and sometimes contradictory, AI systems optimized solely based on subjective preferences risk perpetuating harmful, unintended, or socially unacceptable outcomes. Hence, developing methods that confine systems within boundaries of socially acceptable behavior must not be forgotten during alignment. On this line, the chapter presents a series of impact mitigation strategies and techniques that can serve as alignment interventions during training and inference time. Conceptually, these will be framed by ideas tied to environmental values and impact mitigation. The chapter concludes by discussing current challenges and open questions in impact mitigation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dynamic Normativity: Impact Mitigation

  • Nicholas Kluge Corrêa

摘要

This chapter addresses the necessity of impact mitigation on alignment. Building on prior discussions about the challenges of aggregating human preferences into coherent learning signals, it foregrounds a fundamental limitation—the inherent imperfections and misalignments that may emerge during the value learning stage when our target is only tied to the behavior and preferences of subjects. Given that human behavior and societal values are complex, context-dependent, and sometimes contradictory, AI systems optimized solely based on subjective preferences risk perpetuating harmful, unintended, or socially unacceptable outcomes. Hence, developing methods that confine systems within boundaries of socially acceptable behavior must not be forgotten during alignment. On this line, the chapter presents a series of impact mitigation strategies and techniques that can serve as alignment interventions during training and inference time. Conceptually, these will be framed by ideas tied to environmental values and impact mitigation. The chapter concludes by discussing current challenges and open questions in impact mitigation.