The No Bullshit Guide to P-value: Introduction + Frequentist Statistics
If you read articles in the scientific literature, you'll often see people report P-values when they report statistical tests. P-values are widely used, and it's important to understand what they mean. They're also widely criticized, because people often misinterpret p-values. So in this lecture, the goal is to understand what they mean and how to correctly interpret them. When we talk about p-values, the first question we should ask ourselves is why are they so popular in scientific articles? Well, there's a reason for this, and Benjamini expresses it quite nicely. He says in some sense it offers a first line of defense against being fooled by randomness, separating the signal from the noise. So, this is what the p-values allow you to do. When you interpret your data, you might be very likely to interpret data in favor of the hypothesis that you have, even when the effect might be only slightly in the right direction. The risk is that you're fooling yourself. You might be too likely to declare that something is going on, when you're actually looking at random variation in data. So, p-values are one way to prevent you from fooling yourself. P-values tell you how surprising the data is, assuming that there is no effect. And we'll look at all these aspects in more detail. What surprising means, why they're statements about the data, and why they're built on the idea that there is no effect. Now, some people say that p-values are more accurately explained as what you use if you don't know Bayesian statistics yet. In Bayesian statistics, people don't use p-values.