Infrared thermography (IRT) is increasingly being used to detect anomalies in buildings. In recent years, IRT results are being integrated with computer vision techniques to automate the detection and classification of defects in buildings. To deepen knowledge in this area, an experimental facility was built, consisting of small-scale walls with different defects, fully characterised and located, so that thermal images could be obtained in different conditions and over time and an image database could be created, allowing machine learning models to be trained and validated in the future. Preliminary results have shown that environmental conditions have an impact on surface temperatures, but do not influence defect detection. The type of coating and the test approach can have an impact on the measurement. Heating period and temperature differences between environments are also key elements.