The intelligent monitoring of the structural health of buildings through agnostic methods is a difficult research area. Despite of recent advancements in multi-sensor systems, to date a limited amount of historical data are still available. As a result, data-driven techniques are often not practical for long-term evaluation. However, certain well-known historical buildings have been under monitoring for many years, prior to the emergence of smart sensors and Deep Learning technology. This paper presents a deep learning (DL) method for evaluating structural changes in an agnostic manner. The proposed approach has been tested on the stabilization intervention that took place on the Leaning Tower of Pisa in Italy from 2000–2002. The data set includes both operational and environmental measurements collected from 1993 to 2006. The approach is compared to both traditional and more recent methods, including Multiple Linear Regression, Long Short-Term Memory (LSTM) and Transformer. The results are encouraging and demonstrate that the LSTM method is more sensitive to changes, and that the Transformer method has a higher modeling accuracy.