Perspectives on Psychological Science· 2014Q1
Beyond Power Calculations
- 1,369citations
- Q1SCImago
- 2014year
Short summary
This paper proposes a new approach to research design that moves beyond traditional statistical power calculations to address misleading results in small-sample studies.
AI-generated from the title and abstract; the full text is not read.
Key points
- Traditional power analysis can lead to misleading results in small-sample studies.
- Proposes 'design calculations' to estimate Type S (sign) error and Type M (magnitude) error.
- Type S error quantifies the chance of an estimate being in the wrong direction.
- Type M error quantifies the potential overestimation of an effect's magnitude.
- Estimating plausible effect sizes from external information is a key challenge.
AI-generated from the title and abstract; the full text is not read.
Abstract
Statistical power analysis provides the conventional approach to assess error rates when designing a research study. However, power analysis is flawed in that a narrow emphasis on statistical significance is placed as the primary focus of study design. In noisy, small-sample settings, statistically significant results can often be misleading. To help researchers address this problem in the context of their own studies, we recommend design calculations in which (a) the probability of an estimate being in the wrong direction (Type S [sign] error) and (b) the factor by which the magnitude of an effect might be overestimated (Type M [magnitude] error or exaggeration ratio) are estimated. We illustrate with examples from recent published research and discuss the largest challenge in a design calculation: coming up with reasonable estimates of plausible effect sizes based on external information.
The authors' abstract, as published at the source. Perspectives on Psychological Science, 2014 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: General Psychology
General PsychologyPsychology