Traditional data-driven methods for structural damage identification rely on high-dimensional features extracted from measurement data. However, these methods predominantly operate as black-box models, lacking the interpretability and explainability. In this study, Neural Ordinary Differential Equations (NODEs) comprising two distinct terms are employed to approximate governing dynamic equations: one term represents the state-space-based equation of the dynamic structural system, serving as the physical constraints based on prior knowledge, while the other term, a feed-forward neural network, captures discrepancies between the feed data and the prior structural knowledge. A novel approach is proposed to identify structural damage using the discrepancy term, and the closed-form expression from the neural network for reconstructing structural system parameters is explored to enhance the interpretability and transparency of the trained network model. The efficacy of the proposed NODEs strategy is validated through numerical simulations and the results show its accuracy and efficiency in identifying structural damage. The proposed NODEs model could potentially serve as a cornerstone for the digital twin technology in the lifecycle management of infrastructure assets.