Algorithm-Based Fault Tolerance (ABFT), which verifies computing correctness by checking the sum of large-scale tensor operations with a single comparison, has proven effective in high-performance computing systems, including ML systems. However, its substantial performance overhead often limits practical adoption. In this work, we revisit classical ABFT design by leveraging the characteristics of deep learning to enable more lightweight implementations. We observe that many computational errors have negligible impact on deep learning model accuracy, prompting the proposal of Approximate ABFT (ApproxABFT), which relaxes error detection and recovery thresholds to filter out non-significant errors. Additionally, we introduce a lightweight detector to characterize the fine-grained error sensitivity across different model components and inputs, enabling adaptive fault tolerance at runtime. The adaptive ABFT (AdaptABFT) approach achieves lightweight error protection without compromising system resilience.