Question 1

Just to clarify it in my own mind, as the scale of n grows large, the width of any spike grows small, so its impact on the average approaches zero - while in general, the 'height' will always hover around the expected value - so the sample mean will approach E(x) as n-> infinity. Is that a good way to visualize it?

Accepted Answer

Yes, exactly. Early spikes in values that move the average away from the true mean will be drowned out over time by many, many more samples, and the average will converge to the true mean.  The best example of this is the Galton Board - it's a physical example of a normal distribution appearing over time.  You can see that any early spikes in the distribution dissappear as more sand is poured through.  http://www.youtube.com/watch?v=5_HVBhwhwV8

Question 2

How can you prove the law of large numbers?

Accepted Answer

I am pretty certain that this law was empirically derived. A lot of the early mathematicians who worked on probability would roll die or flip coins for hours.... and then painstakingly work through the data. I actually think that the Law of Large Numbers grew out of Measurement Theory, where scientists were struggling trying to find accurate numbers for physical constants.

Like the calculus -- which was also historically an intuitive subject rooted in observation -- Probability and Statistics were "formalized" in the late 19th through the early 20th Century.

Like the calculus -- which was also historically an intuitive subject rooted in observation --  Probability and Statistics were "formalized" in the late 19th through the early 20th Century.

Question 3

Why is the n-th sample n at 3:29? Is this a mistake?

Accepted Answer

That is defined like ( X1 + X2 + X3 + ... + Xn ) / n
So yes that was an mistake i thik ;) but dont want to be a smarty panst :D

Question 4

At 3:35 when Sal says that the sample mean converges on the population mean when the sample size (n) approaches infinity, would it not also be true for when the sample size (n) approaches the population size (N) for a finite population?

Accepted Answer

That's a great question!  It seems to me that our statistical practices make the hidden assumption that we do our sampling "with replacement" as we said back in the probability days.  If you look up the difference between the binomial distribution and the hypergeometric distribution, you might guess why mathematicians wanted to avoid that headache. :)

That assumption an insignificant source of error as long as n is small relative to N, but you're right that you'd notice the difference as the size of the sample grows.  That difference becomes very stark when the sample size is equal to the population size -- without replacement you know you've tested the entire population but with replacement you are all but certain that you've double-checked some people and ignored others, so it doesn't precisely reflect the population for any finite sample size.

Question 5

Please tell me if my understanding is correct or incorrect. It's really hard to explain, but I'll do my best. I apologize in advance if I'll confuse you.
Reading from wikipedia, I understood, that gambler's fallacy states that:
Let's say after 99 flips of fair coins, that turned out to be heads (very small probability, but still, let's assume that's the case), the probability of 100th coin to be tails is still 50%. But, what I am trying to wrap my mind around is this:

Let's say you are going to flip a coin INDEFINITELY. So, E (X) would be 50. After FIRST 50 SEQUENTIAL flips you have, let's say, 45 heads and 5 tails. According to this Law of Large Numbers, you have infinity. That means, that at some region on that infinite graph, you'll get to the point where you'll be having 45 tails and 5 heads (not necessarily sequential draws) - to even out the average value, is that correct? Please remember, that I am not talking about finite number of draws. I'm talking about infinity. As I understand, that even if you don't know what the outcome would be at particular draw, there always be opposite outcome to even that one out. Am I right?

Accepted Answer

It is a strange idea, but *luminoustedium* is right, there does not need to be a particular 'balancing' effect. Think about it like this:

You first 50 flips, you get 45 heads and 5 tails. Then for the rest of the flips (up ti infinity), you get _exactly_ half heads and the other half tails. So the number of heads is:

(n-45)/2 + 45 = n/2 + 22.5

That is, for n flips total, we have the 45 heads, and for the rest of them (n-45), we have exactly half of them as heads. We can see that this is an _increasing function of n_ (as n increases, so does the number of heads). Then for the proportion of heads, we divide by n:

(n/2 + 22.5)/n = 1/2 + 22.5/n

Now if we take n out to infinity (or take the "limit" over n, if you are familiar with calculus), the 1/2 stays as it is, but the 22.5/n slowly gets smaller and smaller as n goes to infinity. The limit of this term (when n is "equal" to infinity_ is zero, meaning that the proportion of heads becomes just 1/2. So even though we had 45 more heads than tails, the expected value would still be 50%, because the slight bias gets washed out by the sample size.

Now, in general it won't happen that the first 50 will be 45 heads and 5 tails, and the rest will be evenly split, but the general idea remains the same: if we are generating from a known distribution (i.e. flipping a fair coin), then while there may be a bias in small samples, that bias will get washed out by the sample size when we take it up to infinity.

Question 6

At 2:41 it said that expected value is number of trial multiply by probability of success. So by this we arrived at E(X) = 100 * (0.5) = 50.

But in Expected value lesson it was never discussed that expected value can be estimated in such a way. The only thing mentioned around calculating expected value was that it is sum of the product of probability of an outcome and actual outcome. The value of outcomes and their probabilities is something which is random and can be derived either by experiment or doing a sampling.

So how can we say that expected value will be 50.

Accepted Answer

I have a feeling that the videos used to be in a different order. He covers it in this video:
https://www.khanacademy.org/math/probability/random-variables-topic/binomial_distribution/v/expected-value-of-binomial-distribution

The basic idea is this:
For just 1 coin flip, it's easy to get E[X], right? Then, when we're adding up a lot of things (tossing a coin 100 times and counting the number of heads is the same as adding up the number of heads from 100 coin tosses).

Secondly, the expected value can be distributed over addition/subtraction. So if we have two random variables X and Y, E{ X+Y ] = E[X] + E[Y]. So, if we have 100 tosses we're adding up, we can just multiply 100 by the expected value.

Question 7

At numerous points in this video (i.e., 3:47), Sal explains that knowing the past results of a series of experiments will not help with predicting future results.  Does this also hold true for a situation where the results have already been performed but there are so many of them that the count is effectively infinite?  For example, a random number generator (like a coin-flipping machine or a thermal sensor that outputs the least significant digit of its measurement) fills a terabyte hard drive with 0s and 1s, which is about 8 million million bits.  We then choose an address each minute (a different address each time) that points us to one of these bits.  With this setup, it would take more than 15 years to reveal even a millionth of the total bits stored.  Is there a way then to make a guess if the bit is 0 or 1 noticably better than 50/50 before the end of the world?

Accepted Answer

Nope.  Not if the number generator really was random.

Question 8

Sal: I am confused between "the number of draws" which is not the same as the "sample size of a draw". 
 If there are x_1, x_2, ... x_n  and you draw x_1,x_2,x_3 and create a mean (divide by 3).  Then you say draw x_1 ... x_4 sum them and divide by 4 etc.. Then you say as n --> infinity.  How can n go to infinity?  Are you saying n is the entire population? Let's take a different case here:  Now, say you draw 3 X's from a pool and create a mean; then put that back in, and draw another set of 3 X

Accepted Answer

I think your problem is that a single x_k is an entire draw.  So the x's don't represent things that you are drawing; each single x is a result of a full-size draw.  In Sal's example, there are 100 coins (x's are NOT coins, so n is NOT 100).  He 'draws' them all and gets 55 heads, so x_1=55.  Then he has to start over and 'draw' all 100 coins AGAIN just to get x_2.  So n is the number of times he 'draws' all 100 coins, and this is also the "Sample Size", but not the "sample size of a draw".

Statistics and probability

Course: Statistics and probability > Unit 9

Law of large numbers

Want to join the conversation?

Video transcript