Transfer learning aims to improve the performance of a target model by leveraging data from related source populations. In this paper, we consider the data are inherently distributed and focus on distributed transfer learning in the presence of heavy-tailed and/or asymmetric errors. When the transferable sources are known, based on adaptive Huber regression (AHR) to achieve the bias-robustness tradeoff, a two-step communication-efficient transfer learning for AHR (CTrans-AHR) estimator is proposed to deal with the distributed data. To correct the biases caused by the Lasso penalty, a debiased Lasso CTrans-AHR estimator is constructed and its asymptotic distribution is also studied. When the transferable sources are unknown, a data-driven transferable source detection algorithm is proposed with the theoretical guarantee. Simulation results show that our detection algorithm can consistently select all transferable sources with probability approaching one. Compared to the transfer learning estimator using one site alone or the distributed estimator based solely on target data, our CTrans-AHR estimator achieves the lowest \(\ell _\infty \) and \(\ell _2\) estimation errors under various error distributions. The accurate coverage probabilities demonstrate the effectiveness of our proposed debiased Lasso CTrans-AHR estimator. An application to the DVL1 gene expression of brain tissues in the Genotype-Tissue Expression dataset is also presented.
