AI progress has skyrocketed in the last decade, fueled by the deep learning and generative AI revolutions. Yet these unprecedented accomplishments bring about unprecedented risks: Experts warn that the explicit goal of leading AI companies, superintelligence or smarter-than-human AIs, poses a risk of human extinction. Despite the straightforwardness of the arguments behind these risks, no single academic survey brings them all together coherently, explaining where the risks come from, why current AI safety and AI governance are not up to the task of solving them, and how AI companies keep it that way. This chapter fills this gap. The main difference between AI and other technologies is that AIs are grown, not designed by humans. This means that AI progress does not rely on human insight and can be expected to scale until it passes human-level intelligence. The end result of this trend is a world where autonomous, powerful, incomprehensible AIs run the show. These AIs will disempower humanity, not out of spite, but out of indifference. Neither ad hoc safety tricks nor weak advisory governance will be enough to avoid this fate. And AI companies will not stop by themselves; they are already undermining any effort to stop this race to the bottom. Yet all is not lost. It is still possible to regulate and throttle the dangerous progress toward superintelligence without forbidding most specialized and actually valuable AI innovations. But this requires implementing real regulations, with teeth, that take this problem seriously and refuse to fall for the racing dynamics peddled by the AI companies.