Dependability requirements for some systems are so stringent that sufficient assurance of their satisfaction cannot be achieved by evidence of successful operation before deployment. The dominant concern is often that critical design faults may still be present when the system is deployed. To gain regulatory approval to operate such system, a convincing demonstration must be produced that accidents will be as unlikely as required. Yet experience shows that every now and then such a claim, despite the complex process in place to ensure it is correct, is proved wrong in operation. The Boeing 737 MAX is just one recent, striking example. We contend that the practice of risk assessment needs to take into account the inevitable doubt that affects any claim of extreme safety. We first outline how this doubt affects the bounds one can reasonably claim for probability of accidents. During early operation, this “reasonable” bound is much higher than the formal claim accepted by regulators, and depends heavily on the probability of that accepted claim being wrong. But this reasonable estimate then improves over time, if the system does operate without accidents or other surprises. We thus outline an argument that gives a more solid basis to current practices for authorising early operation of critical systems. We then show how evidence supporting “fall-back” arguments for even modest levels of safety can improve the bounds that can be claimed during early operation. Last, we discuss possible improvements to the risk assessment processes, and research directions to address and mitigate the impact of doubt on a system safety justification.