Estimating accurate nitrogen oxides (NOx) emissions with advanced analysis tools is essential to monitor the pollution and the health condition of a turbomachine. Furthermore, providing advanced analysis tools that are compliant with environmental safety laws is fundamental. While machine learning models show promising solutions in estimating particle pollution levels through virtual sensor modeling of turbomachines, the scarcity and quality of real-world data pose significant challenges. To address these challenges, our study focuses on evaluating the performance of time series generative models on the turbomachinery sensor data generation quality using a comprehensive set of quantitative and qualitative metrics. To this end, we propose a task-specific metric to assess the effectiveness of the generated sensor data in supporting our industrial application. Moreover, we conduct a critical analysis regarding the evaluation metrics of the commonly proposed methods in the literature, for validating and assessing the effectiveness of the generative models. The analysis highlights the limitations of these metrics, as they do not consider the temporal correlations present in time series data, heavily rely on the specific implementation of the evaluation model, and do not provide a robust description of the overall model performance. By delving into those concerns, we believe that this work contributes to the advancement of knowledge on temporal synthetic data generation, stimulating discussion in how it can be reliably evaluated in real-world settings.