Fine-tuning Alignment plays a key role in large language model training. However, the ORPO method suffers from overfitting and unstable training due to standard cross-entropy loss, as well as semantic degradation in intermediate layers caused by excessive focus on output optimization. To address these issues, this paper proposes a Hierarchical Contrastive Alignment (HCA) framework that combines dynamic label smoothing and intermediate-layer contrastive learning. The dynamic smoothing module adaptively adjusts the supervision based on the training stage and input structure, reducing overconfidence, and improving generalization. Meanwhile, the contrastive module introduces structured supervision into intermediate representations to improve semantic discrimination and prevent representational collapse. Experiments on benchmarks validate the effectiveness of HCA, showing improved performance over existing baselines and highlighting its potential for stable, semantically aligned preference learning.