Addendum: A Brief Mathematical Digression on Complexity, Reproducibility, Interpretability, and Explainability of AI
摘要
Before concluding, I would like to offer some quick reflections, inspired by the work (Quarteroni et al. 2025), on certain mathematical aspects of AI. As discussed in in the previous chapters, a fundamental component of the learning process is the choice of the model, that is, the function f that maps inputs (training data) to outputs (the response to our question). Whether it involves a least squares process or an artificial neural network, the model f is defined by a set of parameters and, possibly, hyperparameters. In both cases, the parameters are determined through a process of minimizing an appropriate function J (the cost function, also called the objective function or loss function). From a strictly mathematical perspective, even a least squares method (Quarteroni et al. 2014) could be considered a machine learning process. However, there are some key differences between commonly used least squares approaches and supervised training of artificial neural networks that deserve to be highlighted.