We outline four problem settings that illustrate the type of distinctive conceptual, methodological and empirical challenges involved in the design of AI systems that are aligned with human values. The four settings complement each other but each one is meant to reveal some specific aspects of the overall problem. The settings are: alignment on collective coordination, alignment of individual actions, an experimental approach to the engineering of value alignment and the empirical problems of making value alignment work in practice. This discussion is meant to provide substance for the exploration of the interplay of the notions of autonomy, governance and values that is motivated by the existence of artificial autonomous entities that can interact with humans. \(^{*}\)