Main content

## Linear algebra

### Unit 2: Lesson 2

Linear transformation examples# Introduction to projections

Determining the projection of a vector on s line. Created by Sal Khan.

## Want to join the conversation?

- I don't see how you're generalizing from lines that pass thru the origin to the set of all lines. Wouldn't it be more elegant to start with a general-purpose representation for any line L, then go fwd from there? Seems like this special case is missing information....positional info in particular.(3 votes)
- You can get any other line in R2 (or RN) by adding a constant vector to shift the line. Transformations that include a constant shift applied to a linear operator are called affine. Note, affine transformations don't satisfy the linearity property.(18 votes)

- Explain projection of a vector(1 vote)
- Imagine you are standing outside on a bright sunny day with the sun high in the sky. You point at an object in the distance then notice the shadow of your arm on the ground. If your arm is pointing at an object on the horizon and the rays of the sun are perpendicular to your arm then the shadow of your arm is roughly the same size as your real arm... but if you raise your arm to point at an airplane then the shadow of your arm shortens... if you point directly at the sun the shadow of your arm is lost in the shadow of your shoulder.

The shadow is the projection of your arm (one vector) relative to the rays of the sun (a second vector).(21 votes)

- Just a quick question, at9:38you cannot cancel the top vector v and the bottom vector v right? Is this because they are dot products and not multiplication signs?(10 votes)
- You get a different answer (a vector divided by a vector, not a scalar), and the answer you get isn't defined.(2 votes)

- why are you saying a projection has to be orthogonal?(3 votes)
- to use Sal's method, then "x - cv" must be orthogonal to v (or cv) to get the projection. Why? because if x and v are at angle t, then to get ||x||cost you need a right triangle(1 vote)

- Please remind me why we CAN'T reduce the term (x*v / v*v) to (x / v), like we could if these were just scalars in numerator and denominator... but we CAN distribute ((x - c*v) * v) to get (x*v - c*v*v) ?

Where do I find these "properties" (is that the correct word? i.e. what I can and can't transform in a formula), preferably all conveniently** listed?

Presumably, coming to each area of maths (vectors, trig functions) and not being a mathematician, I should acquaint myself with some "rules of engagement" board (because if math is like programming, as Stephen Wolfram said, then to me it's like each area of maths has its own "overloaded" -, +, * operators. But where is the doc file where I can look up the "definitions"??).

--------------

**i.e. without diving into Ancient Greek or Renaissance history ;)_(5 votes)- For example, does:

(u dot v)/(v dot v) = ((1, 2)dot(2, 3))/((2, 3)dot(2, 3)) = (1, 2)/(2, 3)?

No. The quotient of the vectors u and v is undefined, but (u dot v)/(v dot v) is.(1 vote)

- At12:56, how can you multiply vectors such a way? They are (2x1) and (2x1). Can they multiplied to each other in a first place?(3 votes)
- That is Sal taking the dot product. Sal explains the dot product at https://www.khanacademy.org/math/linear-algebra/vectors_and_spaces/dot_cross_products/v/vector-dot-product-and-vector-length.(4 votes)

- What does orthogonal mean?(2 votes)
- In an inner product space, two elements are said to be
*orthogonal*if and only if their inner product is zero. In Euclidean n-space,**R**ⁿ, this means that if**x**and**y**are two n-dimensional vectors, then**x**and**y**are orthogonal if and only if**x**·**y**= 0, where · denotes the dot product.(4 votes)

- hi there, how does unit vector differ from complex unit vector?(3 votes)
- Many vector spaces have a norm which we can use to tell how large vectors are. R^2 has a norm found by ||(a,b)||=a^2+b^2. The complex vectors space C also has a norm given by ||a+bi||=a^2+b^2. The look similar and they are similar. Unit vectors are those vectors that have a norm of 1. There is a pretty natural transformation from C to R^2 and vice versa so you might think of them as the same vector space. But they are technically different and if you get more advanced with what you are doing with them (like defining a multiplication operation between vectors) that you want to keep them distinguished.(3 votes)

- Why not mention the unit vector in this explanation?(2 votes)
- v actually is not the unit vector. The unit vector for L would be (2/sqrt(5), 1/sqrt(5)) .

If you want to solve for this using unit vectors here's an alternative method that relates the problem to the dot product of x and v in a slightly different way:

First, the magnitude of the projection will just be ||x||cos(theta), the dot product gives us x dot v = ||x||*||v||*cos(theta) , therefore ||x||*cos(theta) = (x dot v) / ||v|| . This gives us the magnitude so if we now just multiply it by the unit vector of L this gives our projection (x dot v) / ||v|| * (2/sqrt(5), 1/sqrt(5)) . Which is equivalent to Sal's answer.(3 votes)

- Since dot products "means" the "same-direction-ness" of two vectors (ie. if the two vectors are perpendicular, the dot product is 0; as the angle between them get smaller and smaller, the dot product gets bigger). According to the equation Sal derived, the scaling factor is ("same-direction-ness" of vector x and vector v) / (square of the magnitude of vector v). Does it have any geometrical meaning? How does it geometrically relate to the idea of projection?

Thank you in advance! I hope I could express my idea more clearly...(2 votes)- The factor 1/||v||^2 isn't thrown in just for good luck; it's based on the fact that unit vectors are very nice to deal with. If you're in a nice scalar field (such as the reals or complexes) then you can always find a way to "normalize" (i.e. make the length 1) of any vector. The dot product is exactly what you said, it is the projection of one vector onto the other. You can draw a nice picture for yourself in R^2 - however sometimes things get more complicated. Later on, the dot product gets generalized to the "inner product" and there geometric meaning can be hard to come by, such as in Quantum Mechanics where up can be orthogonal to down.(2 votes)

## Video transcript

Let's say I have a line that
goes through the origin. I'll draw it in R2, but this
can be extended to an arbitrary Rn. Let me draw my axes. Those are my axes right there,
not perfectly drawn, but you get the idea. Let me draw a line that goes
through the origin here. So that is my line there. And we know that a line in any
Rn-- we're doing it in R2-- can be defined as just all of
the possible scalar multiples of some vector. So let's say that this is
some vector right here that's on the line. We can define our line. We could say l is equal to
the set of all the scalar multiples-- let's say that
that is v, right there. So it's all the possible scalar
multiples of our vector v where the scalar multiples,
by definition, are just any real number. So obviously, if you take all
of the possible multiples of v, both positive multiples and
negative multiples, and less than 1 multiples, fraction
multiples, you'll have a set of vectors that will essentially
define or specify every point on that line that
goes through the origin. And we know, of course, if this
wasn't a line that went through the origin,
you would have to shift it by some vector. It would have to be some
other vector plus cv. But anyway, we're starting off
with this line definition that goes through the origin. What I want to do in this video
is to define the idea of a projection onto l of
some other vector x. So let me draw my
other vector x. Let's say that this right here
is my other vector x. Now, a projection, I'm going
to give you just a sense of it, and then we'll define it a
little bit more precisely. A projection, I always imagine,
is if you had some light source that were
perpendicular somehow or orthogonal to our line-- so
let's say our light source was shining down like this, and
I'm doing that direction because that is perpendicular
to my line, I imagine the projection of x onto this line
as kind of the shadow of x. So if this light was coming
down, I would just draw a perpendicular like that, and the
shadow of x onto l would be that vector right there. So we can view it as the shadow
of x on our line l. That's one way to think of it. Another way to think of it,
and you can think of it however you like, is how much of
x goes in the l direction? So the technique would
be the same. You would draw a perpendicular
from x to l, and you say, OK then how much of l would have to
go in that direction to get to my perpendicular? Either of those are how I
think of the idea of a projection. I think the shadow is part of
the motivation for why it's even called a projection,
right? When you project something,
you're beaming light and seeing where the light
hits on a wall, and you're doing that here. You're beaming light and you're
seeing where that light hits on a line in this case. But you can't do anything
with this definition. This is just kind of an
intuitive sense of what a projection is. So we need to figure out some
way to calculate this, or a more mathematically precise
definition. And one thing we can do is,
when I created this projection-- let me actually
draw another projection of another line or another vector
just so you get the idea. If I had some other vector over
here that looked like that, the projection of this
onto the line would look something like this. You would just draw a
perpendicular and its projection would be like that. But I don't want to talk
about just this case. I want to give you the sense
that it's the shadow of any vector onto this line. So how can we think about it
with our original example? In every case, no matter how
I perceive it, I dropped a perpendicular down here. And so if we construct a vector
right here, we could say, hey, that vector is
always going to be perpendicular to the line. And we can do that. I wouldn't have been talking
about it if we couldn't. So let me define this vector,
which I've not even defined it. What is this vector
going to be? If this vector-- let me
not use all these. We know we want to somehow
get to this blue vector. Let me keep it in blue. That blue vector is the
projection of x onto l. That's what we want to get to. Now, one thing we can
look at is this pink vector right there. What is that pink vector? That pink vector that I just
drew, that's the vector x minus the projection, minus this
blue vector over here, minus the projection
of x onto l, right? If you add the projection to
the pink vector, you get x. So if you add this blue
projection of x to x minus the projection of x, you're, of
course, you going to get x. We also know that this pink
vector is orthogonal to the line itself, which means it's
orthogonal to every vector on the line, which also means that
its dot product is going to be zero. So let me define the projection
this way. The projection, this is going
to be my slightly more mathematical definition. The projection onto l of some
vector x is going to be some vector that's in l, right? I drew it right here,
this blue vector. I'll trace it with
white right here. Some vector in l where, and
this might be a little bit unintuitive, where x minus the
projection vector onto l of x is orthogonal to my line. So I'm saying the projection--
this is my definition. I'm defining the projection of x
onto l with some vector in l where x minus that projection
is orthogonal to l. This is my definition. That is a little bit more
precise and I think it makes a bit of sense why it connects to
the idea of the shadow or projection. But how can we deal with this? I mean, this is still
just in words. How can I actually calculate
the projection of x onto l? Well, the key clue here is this
notion that x minus the projection of x is
orthogonal to l. So let's see if we can
use that somehow. So the first thing we need to
realize is, by definition, because the projection of x onto
l is some vector in l, that means it's some scalar
multiple of v, some scalar multiple of our defining vector,
of our v right there. So we could also say, look, we
could rewrite our projection of x onto l. We could write it as some scalar
multiple times our vector v, right? We can say that. This is equivalent to
our projection. Now, we also know that x minus
our projection is orthogonal to l, so we also know that x
minus our projection-- and I just said that I could rewrite
my projection as some multiple of this vector right there. You could see it the
way I drew it here. It almost looks like it's
2 times its vector. So we know that x minus our
projection, this is our projection right here,
is orthogonal to l. Orthogonality, by definition,
means its dot product with any vector in l is 0. So let's dot it with
some vector in l. Or we could dot it with
this vector v. That's what we use
to define l. So let's dot it with v,
and we know that that must be equal to 0. We're taking this vector right
here, dotting it with v, and we know that this has
to be equal to 0. That has to be equal to 0. So let's use our properties of
dot products to see if we can calculate a particular value of
c, because once we know a particular value of c, then we
can just always multiply that times the vector v, which we
are given, and we will have our projection. And then I'll show it to you
with some actual numbers. So let's see if we can
calculate a c. So if we distribute this c--
oh, sorry, if we distribute the v, we know the dot
product exhibits the distributive property. This expression can be rewritten
as x dot v, right? x dot v minus c times v dot v. I rearranged things. We know that c minus cv dot
v is the same thing. We could write it as minus cv. This is minus c times v dot v,
and all of this, of course, is equal to 0. And if we want to solve for c,
let's add cv dot v to both sides of the equation. And you get x dot v is equal
to c times v dot v. Solving for c, let's divide
both sides of this equation by v dot v. You get-- I'll do it in
a different color. c is equal to this: x dot
v divided by v dot v. Now, what was c? We are saying the projection of
x-- let me write it here. The projection of x onto
l is equal to some scalar multiple, right? We know it's in the line, so
it's some scalar multiple of this defining vector,
the vector v. And we just figured out
what that scalar multiple is going to be. It's going to be x dot v over v
dot v, and this, of course, is just going to be
a number, right? This is a scalar still. Even though we have all these
vectors here, when you take their dot products, you just end
up with a number, and you multiply that number times v. You just kind of scale v and
you get your projection. So in this case, the way I
drew it up here, my dot product should end up with some
scaling factor that's close to 2, so that if I start
with a v and I scale it up by 2, this value would be 2, and
I'd get a projection that looks something like that. Now, this looks a little
abstract to you, so let's do it with some real vectors, and
I think it'll make a little bit more sense. And nothing I did here
only applies to R2. Everything I did here can be
extended to an arbitrarily high dimension, so even though
we're doing it in R2, and R2 and R3 is where we tend to
deal with projections the most, this could apply to Rn. Let me do this particular
case. Let me define my line l to
be the set of all scalar multiples of the vector-- I
don't know, let's say the vector 2, 1, such that
c is any real number. Let me draw my axes here. That's my vertical axis. This is my horizontal
axis right there. And so my line is all the
scalar multiples of the vector 2 dot 1. And actually, let me just call
my vector 2 dot 1, let me call that right there the vector v. Let me draw that. So I go 1, 2, go up 1. That right there
is my vector v. And the line is all of
the possible scalar multiples of that. So let me draw that. So all the possible scalar
multiples of that and you just keep going in that direction, or
you keep going backwards in that direction or anything
in between. That's what my line is, all
of the scalar multiples of my vector v. Now, let's say I have another
vector x, and let's say that x is equal to 2, 3. Let me draw x. x is 2, and
then you go, 1, 2, 3. So x will look like this. Vector x will look like that. Well, let me draw it a little
bit better than that. Vector x will look like that. That is vector x. But what we want to do
is figure out the projection of x onto l. We can use this definition
right here. So let me write it down. The projection of x onto
l is equal to what? It's equal to x dot v, right? Where v is the defining
vector for our line. So it's equal to x, which is
2, 3, dot v, which is 2, 1, all of that over v dot v. So all of that over 2, 1, dot
2, 1 times our original defining vector v. So what's our original
defining vector? It's this one right
here, 2, 1. So times the vector, 2, 1. And what does this equal? When you take these two dot of
each other, you have 2 times 2 plus 3 times 1, so 4 plus
3, so you get 7. This all simplified to 7. And then this, you get 2
times 2 plus 1 times 1, so 4 plus 1 is 5. So you get 7/5. That will all simplified to 5. That was a very fast
simplification. You might have been daunted
by this strange-looking expression, but when you take
dot products, they actually tend to simplify very quickly. And then you just multiply
that times your defining vector for the line. So we're scaling it up
by a factor of 7/5. So multiply it times
the vector 2, 1, and what do you get? You get the vector-- let me
do it in a new color. You get the vector, 14/5
and the vector 7/5. And just so we can visualize
this or plot it a little better, let me write
it as decimals. 14/5 is 2 and 4/5,
which is 2.8. And this is 1 and 2/5,
which is 1.4. And so the projection of x
onto l is 2.8 and 1.4. So 2.8 is right about there,
and I go 1.4 is right about there, so the vector is going
to be right about there. I haven't even drawn
this too precisely, but you get the idea. This is the projection. Our computation shows
us that this is the projection of x onto l. If we draw a perpendicular right
there, we see that it's consistent with our idea of
this being the shadow of x onto our line now. Well, now we actually can
calculate projections. In the next video, I'll actually
show you how to figure out a matrix
representation for this, which is essentially a
transformation.