Multimodal Sentiment Analysis (MSA) has received increasing attention in recent years but faces some challenges: Information conflicts between unimodal tokens and multimodal sentiment labels lead to errors in sentiment analysis, while inter-modal correlation learning is equally difficult when multimodal fusion is performed. In this paper, we propose a new framework, PamSA, which incorporates parallel attention and correlation fusion. Specifically, we first propose parallel attention, which utilises parallel cross-fusion modules to implicitly guide cross-modal learning, correcting the sentiment information for each modality. After that, proceed to correlation fusion. We use distance-aware contrastive learning to further exploit the obtained intermodal features to learn the mixed-modal correlations. Finally, we perform a weighted fusion of mixed modalities to obtain multimodal fusion features to identify sentiment information. Experimental results show that our proposed PamSA achieves state-of-the-art performance on three datasets, including CMU-MOSI, CMU-MOSEI, and CH-SIMS. The code link is https://github.com/lxq666666/PamSA