The diffusion model has shown success in generating high-quality and diverse solutions to trajectory optimization problems. However, it inevitably violates the constraint that leads to unmet goals or collisions. This paper presents a novel constraint-aware diffusion model for trajectory optimization, utilizing principles from the Dynamic Data-driven Application Systems (DDDAS) framework. We improve on the original diffusion model by introducing a novel hybrid loss function in training that takes into account noisy data in the diffusion process. Demonstrated on tabletop manipulation and two-car reach-avoid problems, we outperform traditional diffusion models in minimizing constraint violations while generating samples close to locally optimal solutions. This method can be further incorporated into the DDDAS framework to dynamically update the model in real-time for efficient online trajectory adaptation.