In recent years, the widespread dissemination of fake news across various domains has drawn significant academic attention to multi-domain fake news detection. Among different approaches, single-modal text modification-based detection remains the predominant method in this field. However, existing methods still face two fundamental challenges: (1) the varying semantic meanings of the same word across different domains, known as the domain knowledge shift problem, and (2) the model bias caused by imbalanced data distribution across domains, referred to as the domain distribution imbalance problem. To address these challenges, we proposes the Domain Adaptation Network with Dual-Encoder for Fake News Detection framework(DADE). The framework incorporates a domain adaptation expert network to effectively extract and process domain-specific information, thereby mitigating the domain knowledge shift issue. Additionally, it employs expert-level contrastive learning to explicitly optimize the weighting of features from different domains, resolving the domain distribution imbalance problem. Furthermore, DADE utilizes a dual-encoder architecture to jointly extract textual features, enhancing the richness of input representations. Experimental results on multiple real-world datasets demonstrate that DADE significantly outperforms state-of-the-art methods in detection performance.