A common misconception among nonstatisticians is that p-values can tell you the probability that a result occurred by chance. This interpretation is dead wrong, but you see it again and again and again and again. The p-value only tells you something about the probability of seeing your results given a particular hypothetical explanation — it cannot tell you the probability that the results are true or whether they’re due to random chance…

Nor can a p-value tell you the size of an effect, the strength of the evidence or the importance of a result. Yet despite all these limitations, p-values are often used as a way to separate true findings from spurious ones, and that creates perverse incentives…

If there’s one takeaway from the ASA statement, it’s that p-values are not badges of truth and is not a line that separates real results from false ones. They’re simply one piece of a puzzle that should be considered in the context of other evidence.

I'm a Professor of Mathematics and a University Distinguished Teaching Professor at the University of North Texas. For eight years, I was co-director of Teach North Texas, UNT's program for preparing secondary teachers of mathematics and science.
View all posts by John Quintanilla