Network traffic analysis serves as a fundamental pillar in network management. The sophisticated analysis of encrypted network traffic leverages side-channel characteristics to enhance network monitoring, privacy detection, and security-crucial for inferring insights. Despite this, the limitations inherent in network probes often preclude the capture of complete sessions within the high-speed context of the real-world internet, leading to out-of-distribution (OOD) challenges in encrypted network traffic analysis. Current methodologies are impeded by feature drift, which significantly diminishes their efficacy. Addressing this gap, this paper proposes NetTrans, a Transformer-based classifier designed for the analysis of encrypted network traffic, ensuring robust performance. Utilizing a pre-training paradigm, NetTrans initially absorbs knowledge from a vast array of unlabeled data, enriched with multi-modal side-channel features. Subsequently, it undergoes fine-tuning with task-specific traffic datasets to accommodate diverse analytical objectives. Empirical evaluations across three real-world scenarios corroborate NetTrans’s superior efficiency and efficacy, markedly surpassing state-of-the-art (SOTA) models in encrypted network traffic classification.