Main content
Course: Linear algebra > Unit 3
Lesson 2: Orthogonal projections- Projections onto subspaces
- Visualizing a projection onto a plane
- A projection onto a subspace is a linear transformation
- Subspace projection matrix example
- Another example of a projection matrix
- Projection is closest vector in subspace
- Least squares approximation
- Least squares examples
- Another least squares example
© 2024 Khan AcademyTerms of usePrivacy PolicyCookie Notice
Projection is closest vector in subspace
Showing that the projection of x onto a subspace is the closest vector in the subspace to x. Created by Sal Khan.
Want to join the conversation?
- It seems that the prof works in IR3, x - v = a + b, triangularly.
Should I accept this in IRn?(1 vote)- The proof demonstrated in the video makes no assumption about what the vector space is. It is applicable in any Rn.
x-v = a+b simply carries from the definitions of vector addition and how we have constructed our vectors a and b.(6 votes)
- How come the vector b is equal to projection of xvector on v minus the vector v can some one explain this(2 votes)
- It looks like ||x-v||^2 is equal to ||a-b||^2 and not ||a+b||^2, which still gives the same result.(1 vote)
- As far as proofs go, I don't really like this one for projection being the closest vector in a subspace... Aren't you assuming some b exists s.t b+a = x-v? Or is this simply a trick to find a relationship between the two? (There is another proof that we did in a problem set I'm just wondering about this one)(0 votes)
- b is just a vector between two points in subspace, therefore it always exists. So Sal doesn't assume it exists - he knows :) Other thing is that it allows to write this relationship and complete the proof.(2 votes)
Video transcript
Let's say I've got
some subspace, v, that's a plane in R3. So let me see if I can draw
it respectably well. Let me draw some plane in R3. Maybe it looks something
like that. That is my subspace. I think that's good enough. Let me see if I can draw it a
little bit better than that. There you go. So that is my plane in R3. This is v. This is a subspace. And let's say that I have
some other vector, x, any vector in R3. So my vector x looks
like this. That is my vector x. Now what I want to show you
in this video is that the projection of x onto our
subspace-- and let's say that this is our 0 vector
right there. I want to show you that the
projection of x onto our subspace is the closest vector
in our subspace to x. So let me draw that out,
and maybe it'll make a little more sense. So the projection of x on to
the subspace will look something like this. It will look something
like that. That right there, that green
vector right there, is the projection of the vector
x onto our subspace v. That's our vector x. Now, let's take some arbitrary
other vector in our subspace. Let's just take this one. This is just some
other arbitrary vector in our subspace. Let me draw a little
bit differently. We draw it like that. Let's call that vector v. That's clearly another vector
on our subspace. It lies on that plane. What I want to show you is that
the distance from x to our projection of x on to v is
shorter than the distance from x to any other vector. From the distance from x
to any other vector. And obviously, the way I've
drawn it, it looks pretty clear that this line is shorter
than that line. But that was just a particular
choice that I picked. Let's prove it that's general. So what I want to prove is that
the distance between x and its projection onto the
subspace-- and the way we can get that is essentially just
take the length of the vector of x minus the projection
of x onto my subspace. This length right here is this
length right here, is this length right here. So x minus the projection of x
onto v, that's going to be this vector right there. Let me do that in a
different color. Don't want to reuse
colors too often. That's going to be that
vector right there. We could call that vector a. It's clearly in the orthogonal
complement of v, because it's orthogonal to this guy. And that's the definition of a
projection, actually, so this is equal to a. My claim, what I want to show
you, is that this distance a is shorter than any distance
here, is less than or equal to the distance between x and
v, where v is any member. So that's this distance,
right here. This vector right here, that
distance right-- let me draw this vector. The vector x minus v
looks like this. It looks like that. That is the vector
x minus v, right? If you take v plus x minus v,
you're going to go to x. So what I want to show is that
this distance, the length of a, of the difference between x
and its projection, is always going to be less than the
distance between x and any other vector in the subspace. So that is x minus v. So let's see if we
can prove that. So let's take the square
of this distance. So the square of x-- actually,
let me do that. Yeah, let me write that way. So we want to concern ourselves
with the square of the distance of x minus v, where
x is some vector in R3, and v is some vector
in R3 that's also a member of our subspace. It sits on this plane. So what's the square of
this going to be? Well, x minus v is equal
to this vector. Let me draw a new vector here. It is equal to this--
wait, let me draw it in in this yellow. It's equal to this vector. It's equal to this yellow
vector plus a. Right? x minus v is-- this
magenta vector that starts here and goes there-- clearly
equal to this yellow vector plus this orange vector. So let me call that
yellow vector b. Now, what is b equal to? Well, b is going to be equal
to this vector, this green vector, which is the projection
of x onto v, minus this purple vector. Minus this mauve vector,
I guess. Minus v. That's what b is. So we could write x minus v as
being equal to the sum of the vector b plus the vector a. So x minus v is equal
to b plus a. And if we're taking the length
of x minus v squared, that's the same thing as the length
of b plus a squared. And that's just equal to b plus
a dotted with b plus a, which is the same thing
as b dot b. Let me write that a
little bit neater. Let me write it down here. This is going to be equal to,
I'll switch colors, b dot b. Right, b dot b plus b
dot a plus a dot b. So plus 2 times a dot
b, plus a dot a. Now, a and b are clearly
orthogonal. b is the difference between two
vectors in our subspace. The subspace is closed under
addition and subraction. So b is a member of
our subspace. a is orthogonal to everything
in our subspace, by definition. So since a is clearly orthogonal
to b, a is-- by definition-- going to be in the
orthogonal compliment of the subspace. This is going to be 0. And then this right here
will simplify to the length of b squared. And then this right here
is going to be plus the length of a squared. So we get the distance between x
and some arbitrary member of our subspace squared is equal
to the length of b, right here, plus the length
of a squared. Now, a was a distance between
our vector x and our projection, right? That's what the definition of
a was. a was the distance between our vector x
and our projection. Now, this number right
here is going to be at least 0 or positive. So this right here is definitely
going to be greater than or equal to a squared. Or another way to say it is that
the distance between x minus v squared is definitely
going to be greater than or equal to the distance
of a squared. Or the distance between x minus
v-- this is still going to be a positive quantity,
length is always going to be positive-- is greater
than or equal to the length of vector a. Or what's that length of vector
a? a is just this thing right here. So let's write our result. The length of the vector x
minus v, or the distance between x and some arbitrary
member of our subspace, is always going to be greater than
or equal to the length of a, which is just the distance
between x and the projection of x onto our subspace. So there you have it. We've shown, and the original
graph kind of hinted at it, that the projection of x onto v
is the closest vector in our subspace to x. It's closer than any other
vector in v to our arbitrary vector in R3, x. And we've proven
it right there.