The ability to detect abnormal examples is a critical function in building reliable AI systems. Various methods have been developed to detect adversarial examples (AE) and out-of-distribution (OOD) cases. However, existing methods suffer from limitations such as high execution time and limited accuracy. In addition, most existing approaches aim to solve only one of the AE or OOD detection tasks. We propose Kernel Path Distribution (KPD), a novel abnormal sample detector that is accurate, fast, and seamlessly applicable to both AE and OOD detection tasks. Our key idea is to selectively utilize a small number of crucial kernels for each layer, which are highly confident to in-distribution samples. A probability density function drawn from the paths of the selected kernels is then used to filter out abnormal samples. Empirically, we show that KPD achieves the best performance on both the AE and OOD detection problems while being computationally efficient. We also confirm the robustness of KPD against the adaptive attack specifically designed to defeat it. Our code is available at: https://github.com/gyeomo/KPD .