Cardiac image analysis often involves assessing the heart’s anatomy and motion from images and understanding their association with clinical factors like gender, age, and diseases. While image segmentation and motion tracking algorithms address the first issue, modeling the second remains challenging. In this work, we propose a novel conditional generative model to describe the 4D spatio-temporal anatomy of the heart and its interaction with non-imaging clinical factors. By integrating these clinical factors as conditions, our model can investigate their influence on cardiac anatomy. We evaluate the model’s performance on two main tasks: anatomical sequence completion and sequence generation. It achieves high performance in anatomical sequence completion, comparable to or surpassing state-of-the-art generative models. For sequence generation, the model generates realistic synthetic 4D sequential anatomies that align with real data distributions given clinical conditions. The code and trained generative model are available at https://github.com/MengyunQ/CHeart .