Feature engineering and deep learning methods have allowed to decoding cognitive workload with neural data. However, the low efficiency and low cross-subject performance remain challenges. In this study, we propose a domain-guided cross-subject contrastive adversarial learning framework for cognitive workload decoding based on electroencephalogram (EEG). This framework employs contrastive adversarial learning to train an EEG encoder that projects individual EEG data from diverse sources into a low-dimensional invariant subspace, where subjects from both source and target domains share a common representation. The robustness of the model is enhanced by confusing the domain-specific representations of new subjects in the target domain and effectively interfering with the domain discriminator. These shared representations are subsequently used to classify cognitive workload. We conduct extensive experiments to investigate the influence of noise intensity from different sources and the contribution of different brain regions to decoding. Our method achieves state-of-the-art (SOTA) performance in EEG-based workload decoding, addressing the challenges of cross-subject variability and accuracy, and paving the way for more reliable cognitive workload assessment in diverse populations.