<p>We revisit median-of-means (MoM) estimation from a deterministic optimisation viewpoint and develop a family of block-<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(L_p\)</EquationSource> </InlineEquation> estimators tailored to robust learning with heavy-tailed and adversarially corrupted data. In a block contamination model with at least <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\((1-\varepsilon )\)</EquationSource> </InlineEquation> good blocks, we first show that every convex block <i>M</i>-estimator has worst-case robustness constant at least <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(1/(1-2\varepsilon )\)</EquationSource> </InlineEquation>, matching the classical MoM bound and proving that the trimmed-block oracle constant <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(1/(1-\varepsilon )\)</EquationSource> </InlineEquation> is unattainable within the convex class. We then introduce a nonconvex block-<InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(L_p\)</EquationSource> </InlineEquation> family, <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(p\in (0,1]\)</EquationSource> </InlineEquation>, and derive finite-sample deterministic robustness bounds for all global minimisers. As <i>p</i> decreases from 1 to 0, these bounds interpolate continuously between <InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(1/(1-2\varepsilon )\)</EquationSource> </InlineEquation> and the block-<InlineEquation ID="IEq8"> <EquationSource Format="TEX">\(L_0\)</EquationSource> </InlineEquation> oracle <InlineEquation ID="IEq9"> <EquationSource Format="TEX">\(1/(1-\varepsilon )\)</EquationSource> </InlineEquation>; for small <i>p</i> the global minimisers coincide with those of the oracle under a mild separation condition. We further show that the energy landscape of the block-<InlineEquation ID="IEq10"> <EquationSource Format="TEX">\(L_p\)</EquationSource> </InlineEquation> objectives is benign: all local minima lie near the truth and there are no bad basins. Combining these results with block-level concentration yields sub-Gaussian deviation bounds under finite <InlineEquation ID="IEq11"> <EquationSource Format="TEX">\((2+\delta )\)</EquationSource> </InlineEquation> moments and high-dimensional extensions for robust mean estimation and sparse regression with optimal rates. The analysis places MoM estimators on a continuous 1–<i>p</i>–0 path that approaches trimmed-block performance while remaining computationally tractable and directly applicable to modern robust learning problems.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Median-of-Means as an Extremal Convex Estimator and a Nonconvex Route to the Trimmed Oracle

  • Angshul Majumdar

摘要

We revisit median-of-means (MoM) estimation from a deterministic optimisation viewpoint and develop a family of block- \(L_p\) estimators tailored to robust learning with heavy-tailed and adversarially corrupted data. In a block contamination model with at least \((1-\varepsilon )\) good blocks, we first show that every convex block M-estimator has worst-case robustness constant at least \(1/(1-2\varepsilon )\) , matching the classical MoM bound and proving that the trimmed-block oracle constant \(1/(1-\varepsilon )\) is unattainable within the convex class. We then introduce a nonconvex block- \(L_p\) family, \(p\in (0,1]\) , and derive finite-sample deterministic robustness bounds for all global minimisers. As p decreases from 1 to 0, these bounds interpolate continuously between \(1/(1-2\varepsilon )\) and the block- \(L_0\) oracle \(1/(1-\varepsilon )\) ; for small p the global minimisers coincide with those of the oracle under a mild separation condition. We further show that the energy landscape of the block- \(L_p\) objectives is benign: all local minima lie near the truth and there are no bad basins. Combining these results with block-level concentration yields sub-Gaussian deviation bounds under finite \((2+\delta )\) moments and high-dimensional extensions for robust mean estimation and sparse regression with optimal rates. The analysis places MoM estimators on a continuous 1–p–0 path that approaches trimmed-block performance while remaining computationally tractable and directly applicable to modern robust learning problems.