Scholarlink

<p>The study of human behaviour has shifted in the past 15 years. It increasingly relies on opt-in non-probability online data sources. Here we offer an analysis of nine such sources (<i>N</i> = 13,053), aiming to inform researchers conducting experiments or correlational studies. We assess response validity (attentiveness, effort, honesty, speeding and attrition), the extent to which samples represent the underlying population (demographics, attitude representativeness and experimental treatment effects) and professionalism (number and frequency of studies taken and modality of device on which studies are taken). We document substantial variation across samples on each dimension. Samples using demographic quotas display relatively higher amounts of representativeness across multiple indicators (beyond demographics) but often exhibit less response validity. However, inclusion of two attention checks early in a survey can enhance response validity without negatively impacting representativeness. We offer guidance for choosing opt-in samples, depending on the purpose of the research and resource constraints.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Representativeness and response validity across nine opt-in online samples

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月22日

查看全文
  • Michael N. Stagnaro,
  • James N. Druckman,
  • Adam J. Berinsky,
  • Antonio A. Arechar,
  • Robb Willer,
  • David G. Rand

摘要

The study of human behaviour has shifted in the past 15 years. It increasingly relies on opt-in non-probability online data sources. Here we offer an analysis of nine such sources (N = 13,053), aiming to inform researchers conducting experiments or correlational studies. We assess response validity (attentiveness, effort, honesty, speeding and attrition), the extent to which samples represent the underlying population (demographics, attitude representativeness and experimental treatment effects) and professionalism (number and frequency of studies taken and modality of device on which studies are taken). We document substantial variation across samples on each dimension. Samples using demographic quotas display relatively higher amounts of representativeness across multiple indicators (beyond demographics) but often exhibit less response validity. However, inclusion of two attention checks early in a survey can enhance response validity without negatively impacting representativeness. We offer guidance for choosing opt-in samples, depending on the purpose of the research and resource constraints.