This chapter explores the challenge of aggregating human preferences, examining how one could combine diverse and sometimes conflicting preference orderings into a coherent signal for learning. In it, we will review standard methods based on majority voting, exposing their limitations. We will also present alternative consensus-based methods, like Borda Count, that avoid some of the pitfalls of majority and Condorcet approaches. By analyzing the inherent trade-offs and conceptual challenges of different preference aggregation methods, we will see that no method can satisfy all the desirable criteria we wish to have. Ultimately, this chapter emphasizes that preference aggregation is a crucial, yet often underappreciated, step in AI alignment, requiring careful theoretical and practical consideration to ensure that AI systems reflect a balanced and consistent set of human values.