Decentralized distributed learning has recently attracted considerable interest due to its advantages in system stability, data privacy, and efficient communication and computation. Despite these benefits, the approach encounters three significant challenges: arbitrary noise, high dimensionality, and data heterogeneity. To tackle the first two challenges, we propose integrating composite quantile regression with an \(\ell _1\) penalty. This method introduces a doubly nonsmooth objective function, presenting new difficulties for both algorithmic and theoretical development. Traditional optimization algorithms typically demonstrate a slow sublinear convergence rate in such scenarios. To speed the convergence rate, we introduce an innovative local smoothing technique that effectively overcomes nonsmoothness, allowing our algorithm to achieve a rapid linear convergence rate with simple implementation. Additionally, this technique addresses the third challenge by managing heterogeneous covariates and noise across various nodes. From a theoretical perspective, we provide statistical guarantees for estimation accuracy and support recovery. Specifically, the proposed estimator achieves a near-oracle rate without imposing stringent requirements on the number of nodes. Extensive experiments and a real data application confirm the efficacy of our method.
