Flight and migration often trigger strong, emotional, and very diverse reactions in the societies where people arrive, ranging from welcoming the incoming people to their massive rejection. Why is this so? Is it solely the environment, our upbringing, that makes us react so differently to the “others”, or is there also an evolutionary background? New archaeological, archaeogenetic, and behavioral genetic data, as well as evolutionary assumptions, and also our own investigations suggest that the individual differences in our “in-group” vs. “out-group” behavior, i.e., our attitude towards the “own group” vs. “the others”, also have an evolutionary/genetic basis. As is the case with many other attitudes and behaviors, we are therefore not as free as we believe. The origin of these individual differences is highly likely to lie in our millennia-long life in small groups, where other groups could always be both: dangerous enemies and competitors for resources on the one hand, but also vital partners for genetic and cultural exchange on the other hand. Looking at our xenophile vs. xenophobic attitude from this evolutionary perspective, both attitudes were therefore important and sensible for our survival in small groups during our evolution: a rather xenophobic attitude that warned of the dangers of encounters between groups, as well as a xenophile attitude that ensured genetic and cultural exchange across group boundaries. This chapter sheds light on the evolutionary and behavioral genetic foundations of this topic, which so strongly divides our modern societies.