From Positive to Negative: On the Role of Negative Data in Enhancing Generative Models for Engineering Constraint Satisfaction
摘要
Generative Artificial Intelligence has the potential to transform engineering sectors by enhancing design innovation and automating processes. However, despite advances in their data, training, and architectures, generative models still struggle to effectively and reliably satisfy constraints. This shortcoming presents a significant challenge with their adoption in engineering design tasks, where design constraints are ubiquitous. This difficulty is rooted in the similarity-based training objective of generative AI models, in which they learn to mimic the statistical distribution of a dataset of constraint-satisfying examples (positive data). We assert that generative models can be more effectively trained by examining constraint-violating examples (negative data) in addition to positive data. These “Negative Data Generative Models” (NDGMs) can thereby learn to avoid sampling from constraint-violating regions of the sample space. To demonstrate this principle, we propose a type of NDGM, then benchmark this formulation against vanilla models on two 2D test problems and two engineering design problems related to gearbox and concrete beam design. We showcase that NDGMs achieve significantly (2-30x) better constraint satisfaction compared to vanilla generative models. Moreover, they learn these constraints with only a fraction of the training data compared to vanilla generative models. Since NDGMs require only a handful of example to adjust their learned densities, they are significantly more agile and adaptable than vanilla generative models and may be much more effective in continuous data streams as seen in Dynamic Data-Driven Application Systems (DDDAS). Our findings suggest that NDGMs could play a crucial role in overcoming the constraints satisfaction challenges in current generative models, thereby broadening the scope and applicability of generative AI in critical engineering domains.