<p>Predicting the future motions of surrounding traffic agents is critical for the safety of autonomous vehicles. Most previous methods largely rely on long-term historical observations, which may fail to deal with rapid response. Rare methods focus on short-term observations, which often lead to prediction instability and challenges in interaction modeling due to limited information. In this study, we propose a novel trajectory prediction network for short-term observations, named CCS-Net. By incorporating Cross-Branch Feature Learning and Discriminative Enhancement Contrastive Learning, the model effectively improves prediction accuracy under limited short-term observation data. Firstly, a novel Cross-Branch Feature Learning Module (CBFL-Module) is proposed based on a hierarchical feature extraction mechanism, this module aligns short-term and long-term features through contrastive branch and focuses on the trajectory regression task through the prediction branch, making it possible to resolve the target conflict between short-term and long-term networks and mitigate prediction instability. Secondly, we propose a Discriminative Enhanced Contrastive Learning Module (DECL-Module) based on the adaptive sample selection mechanism to enhance the feature consistency between short-term and long-term networks through cross-sample contrastive learning, while filtering out pseudo-negative samples to reduce false negative interference and improve model performance, alleviate modeling difficulties caused by insufficient short-term information. Finally, we propose a Cross-Temporal Attention Distillation Module (CTAD-Module), which optimizes the similarity measure of the attention matrix, enhancing the short-term observation network’s ability to model global dependencies and solving the problem of misaligned attention distribution. Experiments demonstrate CCS-Net’s competitive performance in short-term trajectory prediction.</p>