Design Sensitivity and the Choice of Statistical Methods
摘要
Design sensitivity is a number, \(\widetilde {\varGamma }\) ; it is the limiting sensitivity to unmeasured bias as the sample size increases. It contrasts two situations: (i) a favorable situation with a treatment effect and no unmeasured bias in treatment assignment and (ii) an unfavorable situation with no treatment effect and a bias in treatment assignment. Can these two situations be distinguished in a large observational study? Consider the upper bound on the P-value testing the null hypothesis of no treatment effect in the presence of a bias of at most Γ. That bound is tending to 0 as \(I\rightarrow \infty \) if the sensitivity analysis is performed with \(\varGamma < \widetilde {\varGamma }\) , but it is tending to 1 with \(\varGamma > \widetilde {\varGamma }\) . In a given favorable situation, a wise choice of test statistic can increase \(\widetilde {\varGamma }\) . An unwise choice of test statistic may lead to a claim that an observational study is sensitive to small unmeasured biases when that claim is untrue.