Despite remarkable practical success of Deep Learning (DL) networks, their theoretical understanding is still limited. Recently, in DL community there has been many new theories that explain generalization properties of heuristic learning methods. These new theories appear to be unaware of scientific understanding of inductive inference, dating back to Fisher’s (1935) paper. Notably, phenomenon called ‘double descent’, discovered by DL practitioners, seems to contradict known statistical theories. However, we show that double descent phenomenon is in perfect agreement with classical VC-theory. In particular, VC-theory explains both first and second descent regime, corresponding to under and over-parameterized estimators. Proposed theoretical explanation is supported by empirical modeling of double descent curves using analytic VC-bounds, under standard classification setting. Finally, we discuss methodological reasons for misunderstanding of VC-theoretical concepts, such as VC-dimension and Structural Risk Minimization, in machine learning research community.