If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

Main content

Matched pairs experiment design

The video presents an in-depth exploration of experimental design in statistics, focusing on the use of control and treatment groups, block design, and matched pairs design. It emphasizes the importance of random assignment to mitigate lurking variables and bias, and the value of double-blind experiments. The video also discusses the potential for imbalance in experimental groups and how matched pairs design can help address this issue.

Want to join the conversation?

  • purple pi purple style avatar for user Trurl
    At , Sal defines matched pairs as putting everyone in one round in either the experimental or control group and then switching them. However, in the following practice, I get this question:

    "An online retailer wants to study whether more lenient return policies increase purchasing behaviors. They select a random sample of 200020002000 customers who have made purchases in the past year.
    The company ranks the customers according to total cost of purchases made the previous year. For every 222 customers, in order, from the list, the online retailer randomly assigns one customer to the treatment group and the other to the control group."

    To my thinking, this is simply random, because according to Sal's definition above, you need to follow the 2-step process of assigning two groups and then swapping them. Yet the answer to the above example is also a matched pair. The reasoning given:

    "Customers are split into pairs before assignment, matched as nearly as possible on a common trait. Then they are randomly assigned from those pairs into the treatment or control group."

    There is no indication that two sets of experiments were run. What am I missing here? If anything, that seems if not random, more like a block. Or is this matched pair definition more expansive than what is mentioned in the video?

    I'm not really struggling with the concepts. The video seemed pretty clear, but it also seems that matched pair as a term is being used for what seem to me two different concepts. I am missing something somewhere.
    (47 votes)
    Default Khan Academy avatar avatar for user
    • mr pink red style avatar for user Chris
      It looks to me like there may be a mistake in what came through when you typed, correct me if I'm wrong, but should that say "for every 2 customers,"?

      If so I think I can help. matched pairs means you are dividing your sample into blocks of size 2. So in your example they are taking that list and blocking by the top 2 purchasers, then the next 2, and so on. That helps mitigate the random chance of more of the top purchasers in one treatment group.

      The example Sal uses is matched pairs because each individual is paired up, it's just that they are paired up with themselves. Because if you think about trying to get 2 treatment groups as similar as possible, using all the same individuals is as similar as it gets. You do have to randomize the order of the treatments, however, so that the factor order is not confounding.
      (17 votes)
  • duskpin ultimate style avatar for user Anwar
    It looks like Sal used a crossover study example for matched pairs experiment. At he says "and then you do another round where you switch, where the people who are in the treatment go to the control, and the people who are in the control go into the treatment". This sounds like a crossover design.

    "A crossover design is a repeated measurements design such that each experimental unit (patient) receives different treatments during the different time periods, i.e., the patients cross over from one treatment to another during the course of the trial."

    In a matched pair design patients would be divided into pairs based on age and gender for example and then assigned different treatment within pairs.

    "A matched pairs design is a special case of a randomized block design. It can be used when the experiment has only two treatment conditions; and subjects can be grouped into pairs, based on some blocking variable. Then, within each pair, subjects are randomly assigned to different treatments."
    (36 votes)
    Default Khan Academy avatar avatar for user
  • blobby green style avatar for user Idan Harat
    I believe this is called a cross-over design and not a matched pair design. Am I wrong?
    (13 votes)
    Default Khan Academy avatar avatar for user
  • mr pink red style avatar for user andrei.borodaenko
    Can we consider this particular example be incorrect because of the different starting conditions in the 1st and 2nd stages of the experiment? For instance, the 1st stage treatment group might have post-effects caused by the drug.
    (6 votes)
    Default Khan Academy avatar avatar for user
    • starky seedling style avatar for user deka
      nice point

      and that might be a whole new direction of research for the drug

      i mean you may better design an experiment mainly focusing on that post drug effects, while doing the original experiment under the assumption of no post effects in parallel
      (2 votes)
  • duskpin ultimate style avatar for user Khoa Nguyen
    But for the placebo effect to actually work, don't both groups need to think that they are taking the real pill, or am I wrong?
    (3 votes)
    Default Khan Academy avatar avatar for user
  • blobby green style avatar for user eric.stefko
    Is the explanation of matched pair correct? I don't think that running the experiment twice with each group getting the other treatment is required as far as I know A matched pairs design is an experimental design where researchers match pairs of participants by relevant characteristics. Then the researchers randomly assign one person from each pair to the treatment group and the other to the control group
    (3 votes)
    Default Khan Academy avatar avatar for user
    • blobby green style avatar for user daniella
      The explanation of matched pairs design is partially correct. Matched pairs design is indeed an experimental design where researchers match pairs of participants based on relevant characteristics before assigning them to groups. However, the statement about running the experiment twice with each group receiving the other treatment is not accurate. In a matched pairs design, participants are typically matched based on specific characteristics (e.g., age, gender, baseline measurements) that may influence the outcome variable. Once matched, one individual from each pair is randomly assigned to the treatment group, while the other is assigned to the control group. This approach helps control for potential confounding variables and increases the precision of the comparison between treatment and control groups. There is no need to repeat the experiment with each group receiving the opposite treatment, as the matching process ensures balance between groups at the outset.
      (2 votes)
  • aqualine ultimate style avatar for user Daniel Jing
    I am pretty sure that a matched pairs design is one where an experimental unit receives both treatments
    (2 votes)
    Default Khan Academy avatar avatar for user
    • orange juice squid orange style avatar for user Evan
      Yes, that's what happens in this video. Each person got both the treatment and the placebo. Although, you can also have a matched pairs design where you pair different experimental units together that have similar qualities. Like if you paired up people with similar ages, and gave one person in each pair the placebo, and the other the treatment. Hope this helps! (:
      (3 votes)
  • starky sapling style avatar for user ForgottenUser
    Is there any reason to do block design with matched pairs? It seems to me like since each individual is going through both treatment groups, it wouldn't matter.
    (3 votes)
    Default Khan Academy avatar avatar for user
  • blobby green style avatar for user 10055628
    What is the control group again?
    (2 votes)
    Default Khan Academy avatar avatar for user
  • starky tree style avatar for user bella m
    differences between an experiment and an observational study?
    (1 vote)
    Default Khan Academy avatar avatar for user
    • primosaur seed style avatar for user Ian Pulizzotto
      An experiment, unlike an observational study, often involves randomly assigning people (or other living things or other objects) to a control group and to one or more treatment groups. A researcher controls the conditions in an experiment, but does not control the conditions in an observational study.
      (4 votes)

Video transcript

- [Narrator] The last video we constructed an experiment where we had a drug that we thought might help control people's blood sugar. We looked for something that we could measure as an indicator for their blood sugar's being controlled, and hemoglobin A1c is actually what people measure in a blood test. It is, we have a whole video on it on Khan Academy. But it is an average measure of your blood sugar over roughly a three month period. So that's the explanatory variable. Whether or not you're taking the pill. And the response variable is well, what does it do to your hemoglobin A1c. And we constructed a somewhat classic experiment where we had a control group and a treatment group, and we randomly assigned folks to either the control or the treatment group. And to ensure that one group of the other, or I guess both of them, don't end up with an imbalance of, in the case, in the case of the last video, an imbalance of men or women, we did what we call a block design, where we took our 100 people, and we just happened to have 60 women and 40 men. We said, okay, let's split the 60 women randomly between the two groups, and let's split the 40 men between these two groups, so that we have at least an even distribution with respect to sex. And we would measure folks' A1c's before they get the treatment or the placebo. Then we would wait three months of getting either the treatment or the placebo. And then we'll see if there is a statistically significant improvement. Now this was a pretty good, and it's a bit of a classic experimental design. We would also do it so that the patients don't know which one they're getting, placebo or the actual treatment, so it's a blind experiment. And it'll probably be good if even the nurses or the doctors who are administering the pills or giving the pills also don't know which one they're getting. So it would be a doubles blind experiment. But this doesn't mean that it's a perfect experiment. There seldom is a perfect experiment, and that's why it should be able to be replicated, that other people should try to prove the same thing. It may be in different ways, But even the way that we designed it, There's still a possibility that there's some lurking variables in here. Maybe we took care to make sure our distribution of men and women was roughly even across both of these groups, but maybe my throwing at random sampling, we got a disproportionate number of young people and the treatment group. And maybe young people responded better to taking a pill. Maybe that changes their behaviors in other ways. But maybe older people, when they take a pill, they decide to eat worse because they say the pills are gonna solve all my problems. And so you can have these other lurking variables like age, or where in the country the live, or other types of things, that just by the random process you might have things get uneven one way or another. Now one technique To help control for this a little bit. And I shouldn't use controlled too much. Another technique to help mitigate this, is something called matched pairs design. Matched, matched pairs. Pairs design of an experiment, and it's essentially instead of going through all this trouble saying, oh boy maybe we do block design or all this random sampling. Instead, you randomly put people first into either the control or the treatment group, and then we do another round you measure, and then you do another round where you switch, where the people who are in the treatment go to the control, and the people who are in the control go into the treatment. So we can even extend from what we have here. We can imagine a a world where the first three months we have the 50 people in this treatment group, we have another 50 people in this control group, they are taking the placebo. We see what happens to the A1Cs, and then we switch. Where this group over here. Then, they don't know, they don't know. First of all, ideally it's a blind experiment so they don't even know they're on the treatment group, so hopefully the pills look identical. So now that same group for the next three months is now going to be the control group. And so they got the medicine for the first there months and we saw what happens to their A1c, and now we're gonna pet the placebo, then we're going to get the placebo for the second three months, And then we're going to see what happens to their A1c and likewise the other group is going to be switched around. The thing that the folks that used to be getting the placebo could now get the treatment. They are now going to get the treatment. And the value here is, is that because everyone is going through, is it for one period in the control group, and once period is in the treatment group. But they don't know when, which one happening you are less likely to have a lurking variable like age or geographic region or behavior, cause an imbalance or in somehow skew the results of give you a biased result. So this is an interesting thing, even when I've talked about it in this video in the last one, these aren't just different ways to approach it, and as you construct experiments, this is in medicine. You'll obviously construct experiments in other fields. It's important to think about what types of things are practical to do. And also (stammering) have the best chance at giving you real, I guess you could say, an unbiased and real information as to in the case of an experiment to the efficacy of something, or whether a certain variable, an explanatory variable really does drive, have a causal effect on a response variable.