<p>The authors study the efficient subsampling estimating equation for massive data sets. A two-step procedure is proposed: (i) Design a subsampling probability (SP) to draw a subsample from the full data set, and (ii) construct an estimating function based on the subsample. To improve estimation efficiency, the SP in Step (i) is allowed to be informative, i.e., to depend on the response. However, informative subsampling typically induces bias in the estimating function used in Step (ii). To correct this bias, three approaches are developed to modify the estimating function: Inverse probability weighting (IPW), generalized IPW (GIPW), and projection (PJT), yielding three subsampling estimators. IPW is widely applicable but may inflate variance. GIPW adds a non-informative weight to mitigate this issue. PJT relies on likelihood information and is often the most efficient. Asymptotic properties are then established, based on which the authors propose data-driven methods for selecting the SP and weights by minimizing the estimated asymptotic variances. Numerical results support our theoretical findings.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Efficient Subsampling Estimating Equations from Massive Data Sets

  • Yan Chen,
  • Lu Lin,
  • Yongfeng Wu

摘要

The authors study the efficient subsampling estimating equation for massive data sets. A two-step procedure is proposed: (i) Design a subsampling probability (SP) to draw a subsample from the full data set, and (ii) construct an estimating function based on the subsample. To improve estimation efficiency, the SP in Step (i) is allowed to be informative, i.e., to depend on the response. However, informative subsampling typically induces bias in the estimating function used in Step (ii). To correct this bias, three approaches are developed to modify the estimating function: Inverse probability weighting (IPW), generalized IPW (GIPW), and projection (PJT), yielding three subsampling estimators. IPW is widely applicable but may inflate variance. GIPW adds a non-informative weight to mitigate this issue. PJT relies on likelihood information and is often the most efficient. Asymptotic properties are then established, based on which the authors propose data-driven methods for selecting the SP and weights by minimizing the estimated asymptotic variances. Numerical results support our theoretical findings.