The t-Value Mirage: Why Parametric Tests on Bootstrap Resamples Mislead in Network Science
摘要
Bootstrapping is increasingly popular in cognitive network science as a way to estimate variability in graph-theoretic measures and compare groups of participants. A critical misuse nevertheless persists: applying classical t-tests to bootstrap resamples. This practice treats resampled statistics as if they were independent data, artificially inflating t-values as the number of iterations increases. The result is a “t-value mirage” – apparent significance arising from inappropriate computation rather than true group differences. We demonstrate the problem using semantic fluency data for lists of animal names from English monolingual participants (N = 94) vs. bilingual English-dominant participants (N = 93). Networks were constructed using correlation-based and naïve random walk methods, and three standard metrics were analyzed: average shortest path length (ASPL), clustering coefficient (CC), and modularity (Q). Across increasing bootstrap iteration counts (500, 1,000, 2,000), classical t-tests produced progressively inflated t-values, suggesting false differences between monolingual and bilingual participants in their semantic networks. In contrast, percentile confidence intervals from the bootstrapping distribution remained stable and conservative, indicating no reliable differences. We also conducted permutation testing, which yielded the same inferential outcomes as bootstrap confidence intervals, further confirming that no robust differences exist between the two groups’ network structures. These findings highlight a methodological artifact created by misapplied t-tests and provide a cautionary case study: the t-value mirage is not unique to cognitive data but threatens inference validity across network science. We conclude that bootstrap confidence intervals and permutation tests should replace parametric t-tests for robust inference. Such correction is essential for improving reproducibility and reliability in complex network research.