Predictive power of statistical significance

Table 1 Statistically significant research findings can represent a true positive or false positive

	Reality
Study findings		Alternative hypothesis true	Null hypothesis true
	Significant P-value ≤ 0.05	True positive	False positive
	Insignificant P-value > 0.05	False negative	True negative

Similarly, statistically insignificant findings may represent a true or false negative.

Table 2 When the P-value is utilized to determine whether or not a finding is statistically significant, 1-beta represents the sensitivity for identifying the alternative hypothesis, and 1-alpha represents the specificity

	Reality
Study findings		Alternative hypothesis true	Null hypothesis true
	Significant P-value ≤ 0.05	1 - beta (power)	Alpha (exact P-value)
	Insignificant P-value > 0.05	Beta	1 - alpha

Table 3 A Type I error corresponds to 1-specificity and a Type II error corresponds to 1-sensitivity when study findings are determined to be significant or insignificant based upon the P-value

	Reality
Study findings		Alternative hypothesis true	Null hypothesis true
	Significant P-value ≤ 0.05	Correct	Type I error
	Insignificant P-value > 0.05	Type II error	Correct

Table 4 This 2 × 2 contingency table shows the corresponding values for a research study where a study finding is determined to be significant based upon a P-value of 0.05 and when the study’s power is 80%

	Reality
Study findings		Alternative hypothesis true	Null hypothesis true
	Significant P-value ≤ 0.05	0.8	0.05
	Insignificant P-value > 0.05	0.2	0.95

Table 5 P-values corrected for study power

Study power	P-value
0.95	0.05
0.9	0.047
0.85	0.045
0.8	0.042
0.75	0.039
0.7	0.037
0.65	0.034
0.6	0.032