Direct Preference Optimization (DPO) has expanded its application beyond aligning large language models (LLMs) to also include the adaptation of text-to-image (T2I) models based on human preferences, exemplified by approaches such as Diffusion-DPO and SPO. In this study, we identify a limitation in the preference optimization process, namely the insufficient disparity in the image entropy between preferred and dispreferred generations. To tackle this issue, we introduce the Entropy-Aware Regularization paradigm, which explicitly enhances the entropy disparity. Meanwhile, in order to achieve dynamic entropy sensitivity adjustment, we further introduce the Entropy-Aware Normalization paradigm. Furthermore, by integrating them with the step-aware preference alignment paradigm, we propose the Entropy-Aware Preference Alignment (EAPA) method. Through a comprehensive evaluation and comparison of alignment performance, it is demonstrated that EAPA significantly outperforms current state-of-the-art methods. We also conduct ablation studies on the two aforementioned paradigms, demonstrating that both significantly improve the alignment performance. Code, dataset and model are available at https://github.com/dqtcyh/EAPA