Neural Networks are known to be opaque in their decision-making process. In particular, it is known that, when encountering out-of-distribution (OOD) data, they can confidently provide an erroneous output without warning the user. It is well known that the “class probabilities” output by the softmax layer of a neural network are only weakly correlated with how confident the model should be about the prediction. Therefore, identifying out-of-distribution input data at inference time is critical for many applications of machine learning. We present XOOD: a self-supervised extreme value-based OOD detection framework for image classification. The algorithm relies on the signals captured by the extreme values of the data in the activation layers of the neural network in order to distinguish between in-distribution and OOD instances. We show experimentally that XOOD outperforms state-of-the-art OOD detection methods on many benchmark data sets in both efficiency and accuracy, reducing false-positive rate (FPR95) by 50%, while improving the inferencing time by an order of magnitude. The source code is available at https://github.com/MdSaifulIslamSajol/xood-icann/ .