Deep Neural Networks (DNNs) have found great success in a wide range of applications, including image classification, speech recognition, and natural language modeling. In general, the performance of DNNs is determined by two factors: their architectures and the weights associated with them. Weights are usually optimized through a learning process, which involves using a continuous loss function to measure the discrepancies between the prediction and the real one, and then using gradient-based algorithms to minimize the discrepancies.