Assessing Power Analysis Practices in Eyewitness Research: A Review and Simulation Study

Abstract

To conduct meaningful power analyses and estimate the required sample size, researchers need to indicate which effect sizes are of practical or theoretical interest. This is especially relevant for research fields in eyewitness research (e.g., eyewitness memory, eyewitness identification, investigative interviewing) wherein findings can inform legal practice. In the current study, I examined how power analyses were conducted in 155 eyewitness articles including 353 studies with a focus on ANOVA family based designs and how the effect size was taken into account. None of the power analyses used meaningful effect sizes, most were irreproducible, and many did not reflect the hypothesis of interest. A simulation study showed the potential statistical power issues when the hypothesis is not accurately reflected in the power analysis. To conduct meaningful and reproducible power analyses, I recommend to set a smallest effect size of interest, conduct simulations, and provide Rcode to guide researchers.