Introduction to experiment design

- [Instructor] Let's say that we've come up with a new pill that we think has a good chance of helping people with diabetes control their blood sugar. When someone has diabetes, their blood sugars unusually high, that damages their body in a bunch of different ways. So we want to conduct an experiment to test if this pill really can help people lower their blood sugar. So the first thing we need to think about is how do we even measure or test whether peoples' blood sugar is getting lower. Well, for our experiment, what's typically done is we measure folks hemoglobin A1C. You don't have to worry too much about this in the context of statistics. But hemoglobin A1C test is a way that's typically used to measure your average blood sugar over the last three months, and we have whole videos on Khan Academy explaining how that works. So our hope would be that our pill lowers peoples' blood sugar which shows up as a lowered A1C. Now we have terms for this. The thing that is causing something else to change, we call this the explanatory variable. Explanatory variable, and the thing that might get changed by that explanatory variable, depending on whether you take the pill or not, we call that our response. Response variable. So now let's actually conduct the experiment. So what we would do is we would go to the population. Population of diabetics, and we would want to take a random sample from that population of diabetics. A reasonably large one, and later in statistics we talk about what a good size sample might be. But let's say that we randomly sample. Randomly sample 100 folks. So we randomly sample 100 folks from that population of diabetics, and then you would want to assign these folks randomly to two different groups. One would be your control group, and this would be the group of people who won't take the new medicine, and then you would have your treatment group. These are the groups of folks who will be given the new medicine. The treatment group. Now in some cases, you can just randomly assign these 100 folks between these two groups, and one way to do it is you could give all of them a random number between one and 100 and then the top 50 go into treatment and the bottom 50 go into the control, or there's, you can use a computer to randomly assign folks. Now sometimes you might want to be a little bit more sophisticated than that. For example, there might be evidence that someone's sex might somehow influence how they respond to a drug. So what you could do is something called block design where, let's say, this group just happens to have 60 females and 40 males. One block design. You can randomly assign, but you can do it in a way that you can ensure that both of these groups have the same proportions of male and females. So for example, if you have 60 females here, you can ensure that 30 of them end up in the control, and 30 end up in the treatment. But you would assign those, those 60 females randomly between these two groups, and similarly, you can do block design of these 40 males, 20 end up in the control, and 20 end up in the treatment. So once you have folks in both of these groups, what you would probably want to do is measure their A1C at the beginning. You can view that as a baseline, and then, over the course of the experiment, you would give the pill to the treatment group, and in the control group, you might be saying, well, we would just wouldn't do anything. But the best practice is actually to give a pill that looks just like the real thing to the control group. This is known as a placebo, and the reason why we do that is there's definitely evidence that when people think they are taking a pill that might help them, that even psychologically it can have an effect on them, and sometimes it helps them. This is known as the placebo effect, and not only would you give both groups a pill that looks the same, even though this one in the treatment group actually has the medicine in it, you also would not want to tell folks which group they are in. When you don't tell them which group they're in, that's known as a blind experiment, and you probably also don't want to tell the people who are administering the experiment which group they are administering, and that's called a double blind. So even the doctors or the nurses that are administering the experiment, when they're giving a pill to the control group, they don't know that that pill is the placebo, and you might say, well, why is it important for an experiment to be blind, or especially double blind. Well, that avoids, one, any type of psychological effect on the, from the point of the patient. Or from the, say the caregivers in this situation, so that they don't kind of give it away. They don't tell these folks, hey, you're actually just pretending to take a pill, and so that ensures that we minimize the amount of influence or bias that might happen. You might even have a triple blind experiment where even the folks who are analyzing the eventual data from this experiment don't know whether they're analyzing the data from the control or the treatment. They just compare the two different groups. But anyway, you do, you people take the medicine and the placebo over the course of the experiment. Maybe this lasts for three months, and then you would want to measure their A1C later, and then you would see their change in the A1C. Now if you saw that there wasn't really a difference in the change in A1C between the control and the treatment group, then you'd say, well, that probably means that my pill didn't work. Now if you do get a greater reduction in the treatment group, and you do the statistical analysis which we will learn in statistics and you show that, hey, there's a very low probability that's happened purely due to chance. Well then, you've got something. You could probably conclude that there is a causal connection between taking the pill and lowering your A1C level. But once again, you cannot be 100% sure, and so, this is why it's very important for people to be able to replicate your experiment. Because what you'd want to do either yourself or other researchers might want to conduct the experiment with different sample sizes and different countries and different populations, maybe with different ages at different times of year to ensure that they continue to see this result.