Published online Dec 26, 2017. doi: 10.5662/wjm.v7.i4.112
Peer-review started: October 29, 2017
First decision: November 20, 2017
Revised: November 23, 2017
Accepted: December 3, 2017
Article in press: December 3, 2017
Published online: December 26, 2017
Processing time: 58 Days and 10.4 Hours
A statistically significant research finding should not be defined as a P-value of 0.05 or less, because this definition does not take into account study power. Statistical significance was originally defined by Fisher RA as a P-value of 0.05 or less. According to Fisher, any finding that is likely to occur by random variation no more than 1 in 20 times is considered significant. Neyman J and Pearson ES subsequently argued that Fisher’s definition was incomplete. They proposed that statistical significance could only be determined by analyzing the chance of incorrectly considering a study finding was significant (a Type I error) or incorrectly considering a study finding was insignificant (a Type II error). Their definition of statistical significance is also incomplete because the error rates are considered separately, not together. A better definition of statistical significance is the positive predictive value of a P-value, which is equal to the power divided by the sum of power and the P-value. This definition is more complete and relevant than Fisher’s or Neyman-Peason’s definitions, because it takes into account both concepts of statistical significance. Using this definition, a statistically significant finding requires a P-value of 0.05 or less when the power is at least 95%, and a P-value of 0.032 or less when the power is 60%. To achieve statistical significance, P-values must be adjusted downward as the study power decreases.
Core tip: Statistical significance is currently defined as a P-value of 0.05 or less, however, this definition is inadequate because of the effect of study power. A better definition of statistical significance is based upon the P-value’s positive predictive value. To achieve statistical significance using this definition, the power divided by the sum of power plus the P-value must be 95% or greater.