Recent advancements in machine learning and neural network algorithms have introduced new approaches for reconstructing full-field data from sparse sets of measurements data. However, these approaches have shown limited accuracy when applied to optical data due to the complex physical modeling requirements. For example, traditional autoencoders (AEs), a subset of neural network algorithms, lack the ability to capture complex phenomena and their underlying physics in the latent space. To overcome these limitations, this study proposes a novel framework called time-inferred sparse autoencoder (TIS-AE). The TIS-AE aims to learn the underlying physics of a system of interest by using a physics-based regularizer (i.e., the governing equation of the targeted system) to enhance the accuracy of data reconstruction. To validate the proposed approach, data were generated using a finite element model collected during the cooling process of a metallic plate from two different states A and B. The data from system A were then used to train the TIS-AE model, along with the physics-based regularizer. Then, the trained TIS-AE model was used to reconstruct the full-field data from system B using sparse measurements. The robustness of the proposed TIS-AE model was evaluated by using metrics such as average reconstruction error, average peak signal to noise ratio, and Kullback–Leibler divergence. If further developed, the TIS-AE holds potential for applications in structural health monitoring and nondestructive evaluation.