Orthogonal complements

Say I've got a subspace V. So V is some subspace, maybe of Rn. I'm going to define the orthogonal complement of V, let me write that down, orthogonal complement of V is the set. And, this is shorthand notation right here, would be the orthogonal complement of V. So we write this little orthogonal notation as a superscript on V. And you can pronounce this as 'V perp', not for 'perpetrator' but for 'perpendicular.' So V perp is equal to the set of all x's, all the vectors x that are a member of our Rn, such that x dot V is equal to 0 for every vector V that is a member of our subspace. So we're essentially saying, look, you have some subspace, it's got a bunch of vectors in it. Now if I can find some other set of vectors where every member of that set is orthogonal to every member of the subspace in question, then the set of those vectors is called the orthogonal complement of V. And you write it this way, V perp, right there. So the first thing that we just tend to do when we are defining a space or defining some set is to see, hey, is this a subspace? Is V perp, or the orthogonal complement of V, is this a subspace? Well, you might remember from many, many videos ago, that we had just a couple of conditions for a subspace. That if-- let's say that a and b are both a member of V perp, then we have to wonder whether a plus b is a member of V perp. That's our first condition. It needs to be closed under addition in order for this to be a subspace. And the next condition as well, if a is a member of V perp, is some scalar multiple of a also a member of V perp? And the last one, it has to contain the zero vector. Which is a little bit redundant with this, because if any scalar multiple of a is a member of our orthogonal complement of V, you could just multiply it by 0. So it would imply that the zero vector is a member of V. So what does this imply? What is the fact that a and b are members of V perp? That means that a dot V, where this V is any member of our original subspace V, is equal to 0 for any V that is a member of our subspace V. And it also means that b, since b is also a member of V perp, that V dot any member of our subspace is also going to be 0, or any b that is a member of V. So what happens if we take a plus b dot V? Let's do that. So if I do a plus b dot V, what is this going to be equal to? This is going to be equal to a dot V plus b dot V. And we just said, the fact that both a and b are members of our orthogonal complement means that both of these quantities are going to be equal to 0. So this is going to be equal to 0 plus 0 which is equal to 0. So a plus b is definitely a member of our orthogonal complement. So we got our check box right there I'll do it in a different color than the question mark. Check, for the first condition, for being a subspace. Now is ca a member of V perp? Well let's just take c. If we take ca and dot it with any member of our original subspace this is the same thing as c times a dot V. And what is this equal to? By definition a was a member of our orthogonal complement, so this is going to be equal to 0. So this is going to be c times 0, which is equal to 0. So this is also a member of our orthogonal complement to V. And of course, I can multiply c times 0 and I would get to 0. Or you could just say, look, 0 is orthogonal to everything. You take the zero vector, dot it with anything, you're going to get 0. So the zero vector is always going to be a member of any orthogonal complement, because it obviously is always going to be true for this condition right here. So we know that V perp, or the orthogonal complement of V, is a subspace. Which is nice because now we can apply to it all of the properties that we know of subspaces. Now the next question, and I touched on this in the last video, I said that if I have some matrix A, and lets just say it's an m by n matrix. In the last video I said that the row space of A is -- well, let me write this way. I wrote that the null space of A is equal to the orthogonal complement of the row space of A. And the way that we can write the row space of A, this thing right here, the row space of A, is the same thing as the column space of A transpose. So one way you can rewrite this sentence right here, is that the null space of A is the orthogonal complement of the row space. The row space is the column space of the transpose matrix. And the claim, which I have not proven to you, is that this is the orthogonal complement of this. This is equal to that, the little perpendicular superscript. That's the claim, and at least in the particular example that I did in the last two videos this was the case, where I actually showed you that 2 by 3 matrix. But let's see if this applies generally. So let me write my matrix A like this. So my matrix A, I can write it as just a bunch of row vectors. But just to be consistent with our notation, with vectors we tend to associate as column vectors, so to represent the row vectors here I'm just going to write them as transpose vectors. Because in our reality, vectors will always be column vectors, and row vectors are just transposes of those. r1 transpose, r2 transpose and you go all the way down. We have m rows. So you're going to get rm transpose. Don't let the transpose part confuse you. I'm just saying that these are row vectors. I'm writing transposes there just to say that, look these are the transposes of column vectors that represent these rows. But if it's helpful for you to imagine them, just imagine this is the first row of the matrix, this is the second row of that matrix, so on and so forth. Now, what is the null space of A? Well that's all of the vectors here. Let me do it like this. The null space of A is all of the vectors x that satisfy the equation that this is going to be equal to the zero vector. Now to solve this equation, what can we do? We've seen this multiple times. This matrix-vector product is essentially the same thing as saying-- let me write it like this-- it's going to be equal to the zero vector in rm. You're going to have m 0's all the way down to the m'th 0. So another way to write this equation, you've seen it before, is when you take the matrix-vector product, you essentially are taking the dot product. So to get to this entry right here, this entry right here is going to be this row dotted with my vector x. So this is r1, we're calling this row vector r1 transpose. This is the transpose of some column vector that can represent that row. But that dot, dot my vector x, this vector x is going to be equal to that 0. Now, if I take this guy-- let me do it in a different color-- if I take this guy and I dot him with vector x, it's going to be equal to that 0. So r2 transpose dot x is going to be equal to that 0 right there. So, another way to write this equation is that r1 transpose dot x is equal to 0, r2 transpose dot x is equal to 0, all the way down to rn transpose dot x is equal to 0. And by definition the null space of A is equal to all of the x's that are members of-- well in this case it's an m by n matrix, you're going to have n columns-- so it's all the x's that are members of rn, such that Ax is equal to 0. Or, you could alternately write it this way: that if you were to dot each of the rows with x, you're going to be equal to 0. So you could write it this way, such that Ax is equal to 0. That's an easier way to write it. So let's think about it. If someone is a member, if by definition I give you some vector V. If I were to tell you that V is a member of the null space of A. Let's call it V1. V1 is a member of null space of A. That means it satisfies this equation right here. That means A times V is equal to 0. A times V is equal to 0 means that when you dot each of these rows with V, you get equal to 0. Or another way of saying that is that V1 is orthogonal to all of these rows, to r1 transpose-- that's just the first row-- r2 transpose, all the way to rm transpose. So this is orthogonal to all of these guys, by definition, any member of the null space. Well, if you're orthogonal to all of these members, all of these rows in your matrix, you're also orthogonal to any linear combination of them. You can imagine, let's say that we have some vector that is a linear combination of these guys right here. So let's say vector w is equal to some linear combination of these vectors right here. I wrote them as transposes, just because they're row vectors. But I can just write them as regular column vectors, just to show that w could be just a regular column vector. So let's say w is equal to c1 times r1, plus c2 times r2, all the way to cm times rm. That's what w is equal to. So what happens when you take V, which is a member of our null space, and you dot it with w? So if you take V, and dot it with w, it's going to be V dotted with each of these guys, because our dot product has the distributive property. So if you dot V with each of these guys, it's going to be equal to c1-- I'm just going to take the scalar out-- c1 times V dot r1, plus c2 times V dot r2-- this is an r right here, not a V-- plus, all the way to, plus cm times V dot rm. And we know, we already just said, that V dot each of these r's are going to be equal to 0. So all of these are going to be equal to 0. So this whole expression is going to be equal to 0. So if you have any vector that's a linear combination of these row vectors, if you dot it with any member of your null space, you're going to get 0. So let me write this way, what is any vector that's any linear combination of these guys? Well, that's the span of these guys. Or you could say that the row space, that's the row space. So if w is a member of the row space, which you can just represent as a column space of A transpose, then we know that V is a member of our null space. We know that V dot w is going to be equal to 0, I just showed that to you right there. So every member of our null space is definitely orthogonal to every member of our row space. So that's what we know so far. Every member of null space of A is orthogonal to every member of the row space of A. Now, that only gets us halfway. That still doesn't tell us that this is equivalent to the orthogonal complement of the null space. For example, there might be members of our orthogonal complement of the row space that aren't a member of our null space. So let's say that I have some other vector u. Let's say that u is a member of the orthogonal complement of our row space. I know the notation is a little convoluted, maybe I should write an r there. But I want to really get set into your mind that the row space is just the column space of the transpose. Let's say that u is some member of our orthogonal complement. What I want to do is show you that u has to be in your null space. And when I show you that, then we know. So far we just said that, OK then, everything in the null space is orthogonal to the row space, but we don't know that everything that's orthogonal to the row space, which is represented by this set, is also going to be in your null space. That's what we have to show, in order for those two sets to be equivalent, in order for the null space to be equal to this. So if we know this is true, then this means that u dot w, where w is a member of our row space, is going to be equal to 0. Let me write this down right here, that is going to be equal to 0. And what does that mean? That means that u is also orthogonal. So this implies that u dot-- well, r, j, any of the row vectors-- is also equal to 0, where j is equal to 1, through all the way through m. How do I know that? Well, I'm saying that look, you take u as a member of the orthogonal complement of the row space, so that means u is orthogonal to any member of your row space. So in particular the basis vectors of your row space-- we don't know whether all of these guys are basis vectors-- these guys are definitely all members of the row space. Some of them are actually the basis for the row space. So that means if you take u dot any of these guys, it's going to be equal to 0. So if u dot any of these guys is equal to 0, that means that u dot r1 is 0, u dot r2 is equal to 0, all the way to u dot rm is equal to 0. Well, if all of this is true, that means that A times the vector u is equal to 0. That implies this, right? You stick u there, you take all the dot products, it's going to satisfy this equation. Which implies that u is a member of our null space. So we've just shown you that every member of your null space is definitely a member of the orthogonal complement. And now we've said that every member of our orthogonal complement is a member of our null space. And actually I just noticed that I made a slight error here. This dot product, I don't have to write the transpose here, because we've defined our dot product as the dot product of column vectors. So this is the transpose of some column vectors. So you can un-transpose it here and just take the dot product. Anyway, minor error there. But that diverts me from my main takeaway, my punch line, the big picture. We now showed you, any member of our null space is a member of the orthogonal complement. So we just showed you, this first statement here is another way of saying, any member of the null space-- or that the null space is a subset of the orthogonal complement of the row space. So that's our row space, and that's the orthogonal complement of our row space. And here we just showed that any member of the orthogonal complement of our row space is also a member of your null space. Well, if these two guys are subsets of each other, they must be equal to each other. So we now know that the null space of A is equal to the orthogonal complement of the row space of A or the column space of A transpose. Now, I related the null space with the row space. I could just as easily make a bit of a substitution here. Let's say that A is equal to some other matrix, B transpose. Right? It's going to be the transpose of some matrix, you could transpose either way. So if I just make that substitution here, what do we get? We get, the null space of B transpose is equal to the column space of B transpose, right? A transpose is B transpose transposed. Let me get my parentheses right. And then that thing's orthogonal complement. So what is this equal to? The transpose of the transpose is just equal to B. So I can write it as, the null space of B transpose is equal to the orthogonal complement of the column space of B. So just like this, we just show that the left-- B and A are just arbitrary matrices. So this showed us that the null space, sometimes it's nice to write in words, is the orthogonal complement of row space. And this right here is showing us, that the left null space which is just the same thing as a null space of a transpose matrix, is equal to, orthogonal-- I'll just shorthand it-- complement of the column space. Which are two pretty neat takeaways. We saw a particular example of it a couple of videos ago, and now you see that it's true for all matrices.