Comparing P-values to different significance levels

- [Instructor] What we're going to do in this video is talk about significance levels which are denoted by the Greek letter alpha and we're gonna talk about two things, the different conclusions you might make based on the different significance levels that you might set and also why it's important to set your significance levels ahead of time, before you conduct an experiment and calculate the p-values, for, frankly, ethical purposes. So to help us get this, let's look at a scenario right over here which tells us Rahim heard that spinning, rather than flipping, a penny raises the probability above 50% that the penny lands showing heads. That's actually quite fascinating if that's true. He tested this by spinning 10 different pennies 10 times each, so that would be a total of a hundred spins. His hypotheses were, his null hypothesis is that by spinning, your proportion doesn't change rather versus flipping, it's still 50% and his alternative hypothesis is that by spinning, your proportion of heads is greater than 50%, where p is the true proportion of spins that a penny would land showing heads. In his 100 spins, the penny landed showing heads in 59 spins. Rahim calculated that the statistic, so this is the sample proportion here, it's 59 out of a hundred were heads so that's 0.59 or 59 hundredths, and he calculated had an associated p-value of approximately 0.036. So based on this scenario, if ahead of time, Rahim had set his significance level at 0.05, what conclusions would he now make? And while you're pausing it, think about how that may or may not have been different if he set his significance levels ahead of time at 0.01. Pause the video and try to figure that out. So let's first of all remind ourselves what a p-value even is. You could view it as the probability of getting a sample proportion at least this large if you assume that the null hypothesis is true. And if that is low enough, if it's below some threshold, which is our significance level, then we will reject the null hypothesis. And so in this scenario, we do see that 0.036, our p-value is indeed less than alpha. It is indeed less than 0.05 and because of that, we would reject the null hypothesis. And in everyday language, rejecting the null hypothesis is rejecting the notion that the true proportion of spins that a penny would land showing heads is 50%. And if you reject your null hypothesis, you could also say that suggests our alternative hypothesis that the true proportion of spins that a penny would land showing heads is greater than 50%. Now what about the situation where our significance level was lower? Well in this situation, our p-value, our probability of getting that sample statistic if we assumed our null hypothesis were true, in this situation, it's greater than or equal to, and it's greater than in this particular situation, than our threshold, than our significance level. And so here, we would say that we fail to reject our null hypothesis so we're failing to reject this right over here and it will not help us suggest our alternative hypothesis. And so because of the difference between what you would conclude given this change in significance levels, that's why it's really important to set these levels ahead of time because you could imagine it's human nature, if you're a researcher of some kind, you want to have an interesting result. You want to discover something, you want to be able to tell your friends, hey, my alternative hypothesis it actually is suggested, we can reject the assumption, the status quo. I found something that actually makes a difference and so it's very tempting for a researcher to calculate your p-values and then say oh, well maybe no one will notice if I then set my significance values so that it's just high enough so that I can reject my null hypothesis. If you did that, that would be very unethical. In future videos, we'll start thinking about the question of okay, if I'm doing it ahead of time, if I'm setting my significance level ahead of time, how do I decide to set the threshold? When should it be one-hundredths? When should it be five-hundredths? When should it be 10-hundredths? Or when should it be something else?