With the development of smart manufacturing, data-driven intelligent fault diagnosis has emerged as a prominent topic recently. Despite the potential for fault diagnosis in rotating machines, traditional deep learning methods face challenges, including manual feature extraction, the requirement of massive, labeled data, and the assumption of the same distributions of training and testing data. To overcome these limitations, this paper proposes an end-to-end semi-supervised domain adaptation method to learn knowledge from the labeled source domain to the limited labeled target domain under time-varying variable working conditions to improve generalization to target domain. The proposed strategy aligns not only the marginal distribution across the domains but also the conditional distribution between source and target domains. To achieve this, multi-kernel maximum mean discrepancy and Wasserstein distance losses are utilized between different layers of 1D-CNN to learn domain-invariant and class-discriminative features and reduce the discrepancies across the domains explicitly. To determine the robustness and effectiveness of the proposed method, we conduct different experiments, including the scenarios involving both acceleration and deceleration of the motor speed during operation. The proposed method is more efficient than existing transfer learning approaches, yielding more accurate results and illustrating the need for marginal and conditional distribution alignment.