Null space and column space basis

What I want to do in this video -- and it'll probably occur over several videos -- is really integrate everything we know about matrices, and null spaces, and column spaces, and linear independence. So I have this matrix here, this matrix A. And I guess a good place to start is, let's figure out its column space and its null space. The column space is actually super easy to figure out. It's just the span of the column vectors of A. So we can right from the get-go write that the column space of our matrix A-- Let me do it over here. I can write the column space of my matrix A is equal to the span of the vectors 1, 2, 3. 1, 1, 4. 1, 4, 1. And 1, 3, 2. I'm done. That was pretty straightforward, a lot easier than finding null spaces. Now this may or may not be satisfying to you. And there's a lot of open questions. Is this a basis for the space, for example? Is this a linear independent set of vectors? How can we visualize this space? And I haven't answered any of those yet. But if someone just says, hey what's the column space of A? This is the column space of A. And then we can answer some of those other questions. If this is a linearly independent set of vectors, then these vectors would be a basis for the column space of A. We don't know that yet. We don't know whether these are linearly independent. But we can figure out if they're linearly independent by looking at the null space of A. Remember these are linearly independent if the null space of A only contains the 0 vector. So let's figure out what the null space of A is. And remember, we can do a little shortcut here. The null space of A is equal to the null space of the row, the reduced row echelon form of A. And I showed you that when we first calculated the null space of a vector, because when you performed these -- essentially if you want to solve for the null space of A, you create an augmented matrix. And you put the augmented matrix in reduced row echelon form, but the 0's never change. So essentially you're just taking A and putting it in reduced row echelon form. Let's do that. So I'll keep row one the same, 1, 1, 1, 1. And then let me replace row two with, row two minus row one. So what do I get? No, actually I want to zero this out here. So row two minus, 2 times row one. Actually even better because I eventually want to get a 1 here. So let me do 2 times row one, minus row two. So let me say 2 times row one, and I'm going to minus row two. So 2 times 1 minus 2 is 0, which is exactly what I wanted there. 2 times 1 minus 1 is 1. That's nice to have right there. 2 times 1 minus 4 is minus 2. 2 times 1 minus 3 is minus 1. All right, now let me see if I can zero out this guy here. So what can I do? I could do any combination, anything that essentially zeroes this guy out. But I want to minimize my number of negative numbers. So let me take this third row, minus 3 times this first row. So I'm going take minus 3 times that first row and add it to this third row. So 3 minus 3 times 1 is 0. These are just going to be a bunch of 3's. 4 minus 3 times 1 is 1. 1 minus 3 times 1 is minus 2. And 2 minus 3 times 1 is minus 1. Now if we want to get this into reduced row echelon form we need to target that one there and that one there. And what can we do? So let's keep my middle row the same. My middle row is not going to change. 1, 1, minus 2, minus 1. And to get rid of this one up here I can just replace my first row with my first row minus my second row. Because then this won't change. I'll have 1 minus 0 is 1. 1 minus 1 is 0. That's what we wanted. 1 minus minus 2 is 3. That's 1 plus 2. 1 minus minus 1. That's 1 plus 1. That is 2. Fair enough? Now let me do my third row. Let me replace my third row with my third row subtracted from my first row. They are obviously the same thing. So if I subtract the third row from the second row I'm just going to get a bunch of 0's. 0 minus 0 is 0. 1 minus 1 is 0. Minus 2 minus minus 2 is 0. And minus 1 minus minus 1. That's minus 1 plus 1. That's equal to 0. And just like that we have it now in reduced row echelon form. So this right here is the reduced row echelon form of A. That straightforward. Now the whole the reason why we even went through this exercise is we wanted to figure out the null space of A. And we already know that the null space of A is equal to the null space of the reduced row echelon form of A. So if this is the reduce row echelon form of A, let's figure out its null space. So the null space is the set of all of vectors in R4, because we have 4 columns here. 1, 2, 3, 4. The null space is the set of all of vectors that satisfy this equation, where we're going to have three 0's right here. That's the 0 vector in R3, because we have three rows right there, and you can figure it out. This times this has to equal that 0. That dotted with that essentially is going to equal that 0. That dotted with that is equal to that 0. I say essentially because I didn't define a row vector dot a column vector. I've only defined column vectors dotted with other column vectors. But we've been over that in a previous video, where you can say this is a transpose of a column vector. So let's just take this, and write a system of equations with this. So we get 1 times x1. So this times this is going to be equal to that 0. So one times x1, that is x1. Plus 0 times x2. Let me just write that out. Plus 3 times x3. Plus 2 times x4 is equal to that 0. And then -- I'll do it in yellow right here -- I have 0 times x1. Plus 1 times x2. Minus 2 times x3. Minus x4 is equal to 0. And then this gives me no information. 0 times all this is equal to 0. So it just turns into 0 equals 0. So let's see if we can solve for our pivot entries, or our pivot variables. What are our pivot entries? This is a pivot entry. That's a pivot entry. That's what reduced row echelon form is all about, getting these entries that are 1 and they're the only non-zero term in their respective columns. And that every pivot entry is to the right of a pivot entry above it. And then the columns that don't have pivot entries? These columns represent the free variables. So this column has no pivot entry. And so when you take the dot product, this column turned into this column in our system of equations. So we know that x3 is a free variable. x3 is free. We can set it equal to anything. Likewise x4 is a free variable. X1 and x2 are pivot variables, because their corresponding columns in our reduced row echelon form have pivot entries in them. Fair enough. So let's see if we can simplify this into a form we know. And we've seen this before. So if I solve for x1 -- this 0 I can ignore. That 0 I can ignore -- I could say that x1 is equal to minus 3x3 minus 2x4. I just subtracted these two from both sides of the equation and I can say that x2 is equal to 2x3 plus x4. And if we want to write our solution set now, so if I wanted to find the null space of A, which is the same thing as the null space of the reduced row echelon form of A, is equal to all of the vectors -- let me do a new color. Maybe I'll do blue -- is equal to all of the vectors x1, x2, x3, x4 that are equal to -- So what are they going to be equal to? X1 has to be equal to minus 3x3 minus 2x4. Just to be clear, these are free variables because I can set these to be anything. And these are pivot variables because I can't just set them to anything. When I determine what my x3's and my x4's are, they determine what my x1's and my x2's have to be. So these are pivoted variables. These are free variables. I can make this guy pi. And I can make this guy minus 2. We can set them to anything. So x1 is equal to -- let's see, let me write it this way -- they're equal to x3 -- let me do it in a different color -- do x3 like this. So it's equal to x3 times some vector plus x4 times some other vector. So any solution set in my null space is going to be a linear combination of these two vectors. We can figure out what these two vectors are just from these two constraints right here. So -- let me do it in a neutral color -- x1 is equal to minus 3 times x3 minus 2 times x4. Straightforward enough. x2 is equal to 2 times x3 plus x4. What's x3 equal to? Well x3 is equal to itself. Whatever we set x3 equal to, that's going to be x3. So x3 is going to be 1 times x3 plus 0 times x4. It is not going to have any x4 in it. X3 is going to be kind of an independent variable. It's going to be free. We can set whatever it is. We set it and then that's going to be x3 in our solution set. x4 is not going to have any x3 in it. It's just going to be 1 times x4. And so our null space is essentially all of the linear combinations of these two vectors. This can be any real number. This is just any real number and x4 is just a member of the real space. So all of these, the set of all of the valid solutions to Ax is equal to 0 -- where did I write that. Did I even write that down? No I haven't even written that anywhere. The set of all Ax is equal to 0, where this is my x, it equals all the linear combinations of this vector and that vector right there. And we know what all of the linear combinations mean. It means my null space is equal to the span of these two guys, the span of minus 3, 2, 1, 0. And minus 2, 1, 0, 1. Now let me ask you a question. Are the columns in A, are they a linearly independent set? Are they a linearly independent set? So if we write these vectors right there, these are the column vectors of A. So let me write that down. So are the column vectors of A -- so what were they? Let's see. 1, 3, 2. No it's 1, 2, 3. 1, 1, 4. 1, 4, 1. And 1, 3, 2. So this is just the column vectors of A. I could just write A is just this much of columns, but my question is, is this a linearly independent set? And here you might immediately start thinking, well when we said that something is linearly independent -- so linearly independence implies that there's only one solution -- we saw this I think two videos ago, that there's only one solution -- one solution to Ax is equal to 0. And that is the 0 solution, that x is equal to the 0 vector. Or another way to say that is that the null space of my matrix A is equal to just the 0 vector. That's what linear independence implies. And it goes both ways. If my null space is just a 0 vector, then I know it's linearly independent. If my null space includes other vectors, then I am not linearly independent. Now my null space of A, what does it include? Is it just the 0 vector? Well, no it includes every linear combination of these guys. It includes actually an infinite number of vectors, not just one solution. Obviously 0 vector is contained here, if you just multiply both of these -- if you pick 0 for that and that. It's contained, but you can get a whole set of vectors. So because the null span of A, the null space, sorry, the null space of A does not just contain the 0 vector. So it has more than just 0. So what does that mean? Well that means that there's more than one solution to this. And that means that this is a linearly dependent set. And what does that mean? At the very beginning of the video I said, what's the column space of A. And we said, the column space of A is just the span of the column vectors. I just wrote it out like that. And I said, well it's not clear whether this is a valid basis for the column space of A. And what's a basis? A basis is a set of vectors that span a subspace, and they are also linearly independent. And we just showed that these guys are not linearly independent. So that means that they are not a basis for the column space of A. They do span the column space of A, by definition really. But they're not a basis. They need to be linearly independent for them to be a basis. So let's see if we can figure out what a basis for this column space would be. And to do that we just have to get rid of some redundant vectors. If I can show you that this guy can be represented by some combination of these two guys, then I can get rid of that guy. He's not adding any new information. Same with that guy. Who knows? So let's see if we can figure this piece of the puzzle out. So we know already that x1, let me write it this way, that x1 times -- Maybe I'll just kind of leave you hanging and continue this in the next video. But we know that x1 times 1, 2, 3. Plus x2 times 1, 1, 4. Plus x3 times 1, 4, 1. Plus x4 times 1, 3, 2. We know that this is equal to 0. Now if we are able to solve for x4 in terms of -- let me just think that I can solve for the vectors that are associated with my free variables using the other vectors. Let me see if I can do that. And you'll see it's actually pretty straightforward. So let's say I want to solve for x4. So if I subtract this from both sides of this equation, I get what? Let me put it this way, let me set x3 equal to 0. It was a free variable. I can do that. So if I set x3 is equal to 0, then what do I get here? If I said x3 equals 0, this guy disappears. And if I subtract this from both sides of this equation, I get x1 times 1, 2, 3. Plus x2 times 1, 1, 4. Is equal to -- I'm just setting x3 equal to 0. That was a free variable. So I'm setting x3 equal to 0. So this whole thing disappears. So that is equal to minus x4 times 1, 3, 2. Now I set x3 equal to 0. Let me set x4 to be equal to minus 1. If x4 is equal to minus 1, what is minus x4? Well then this thing will just be equal to 1. And I'll have x1 times 1, 2, 3. Plus x2 times 1, 1, 4 will equal this fourth vector right here. And can I always find things like this? Well sure I can actually find the particular ones. If x3 is equal to 0, and x4 is minus 1 -- Let me copy and paste this that I have up here -- Let me scroll down a little bit. This is what we got when we figured out our null space, right there. So if I'm setting -- remember these are the free variables -- if I set x3 equal to 0 and x4 is equal to minus 1, what is x1? Then this will imply that x1 is equal to minus 3 times x3, that's just 0, minus 2 times x4. If x4 is minus 1, minus 2 times minus 1, x1 will equal 2. And then what will x2 be equal to? x2 is equal to 2 times x3, which is 0, plus x4. So it's equal to minus 1. So I just showed you that if I set this equal to 2 and this equal to minus 1, I have a linear combination of this vector and this vector that can add up to this fourth vector. And you can even verify it. 2 times 1 minus 1 is equal to 1. 2 times 2 minus 1 is equal to 3. 2 times 3 is 6, minus 4 is equal to 2. So it checks out. So I just showed you using, really, our definitions looking at what were our free variables versus our pivot variables. We were able to show you, kind of just very simply solve for this third, this fourth vector, in terms of these first two. So we know, if we go back to the set that this fourth vector is really unnecessary, really not adding anything to the span of the set of vectors. Because this guy can be written as a combination of this guy and this guy. Now let's see if this guy, this third guy, we can do the same exercise. This is also dictated by a free variable. So let's see if I can write him as a combination of these first two. Well we'll do the exact same thing. Instead of setting x3 equal to 0 and x4 equal to minus 1, let us set x4 is equal to 0 because I want to cross that out. And let me set x3 is equal to minus 1. If x3 is equal to minus 1, what does this equation reduce to? We get x1 times 1, 2, 3. Plus x2 times 1, 1, 4. Is equal to -- if this is minus 1 times 1, 4, 1. And then we add it to both sides of this equation, we get plus 1 times 1, 4, 1. And once again we can just solve for our x1 and x2. If x4 is 0 and x3 is minus 1, then x1 x4 is 0. So x3 is just minus 3, times x3, so x1 would be equal to 3, right? Minus 3 times minus 1. And what would x2 be equal to? x4 is 0, we can ignore that. x2 would be equal to minus 2. So this would be 3, and then this would be minus 2. Let's see if it works out. 3 times 1 minus 2 is 1. 3 times 2 minus 2 is 4. 3 times 3 minus 8 is 1. It checks out. So I'm able to write this vector, that was associated with the free variable, as a linear combination of these two. So we can get rid of him from our set. So now I've shown that this guy can be written as a linear combination of these two. This guy can be written as a linear combination of these two. So the span of all of those guys should be equal to the span -- So let me write it this way. The column space of A, I can now re-write. Before it was the span of all of those vectors. It was the span of all of the column vectors, v1, v2, v3, and v4. Now I just showed you that v3 and v4 can be rewritten in terms of v1 and v2. So they're redundant. So that is equal to the span of v1 and v2 which are just those two vectors. Vector 1, 2, 3, and vector 1, 1, 4. Now are any of these guys redundant? Can I express one of them as a linear combination of the other? Essentially when I'm talking about the linear combination of only one other vector it's just multiplying it by a scalar. Well let's think about that. There are multiple ways you can show this, but the easiest way is well look, to go from this entry to that entry I'm just multiplying by 1. But if I multiply this whole vector times 1, then I'm going to get a 2 here and I'm going to get a 3 here. So it won't work. If I want to represent this guy as a scalar multiple of that guy, so any scalar multiple of 1, 2, 3 is going to be equal to 1c, 2c, 3c. Right? And so we're saying this guy has to be represented somehow like that, if we say that this guy is somehow a scalar, somehow can be represented by that guy. So that would have to be equal to 1, 1, 4. When you look at this top entry it implies that c would have to be equal to 1. But when you look at this second entry you think that c would have to be equal to 1/2. So you get a contradiction. Over here c would have to be equal to 4/3. So there's no c where this will work. There's no multiple of c. And you can work that both ways. So there's no way that you can represent one of these guys as a linear combination of the other. And you can actually prove other ways, maybe more formally, that this is linearly independent. But given that this is linearly independent -- I think you're satisfied with that -- we can then say that the set of vectors 1, 2, 3, and 1, 1, 4, this is a basis for the column span of A. Now I'm going to let you go in this video because I think I've gone well over time. But what I'm going to do in the next few videos is now that I've established that this is a basis for the column span of A, we can attempt to visualize it. Because we can say that the column span of A is equal to the span of these two vectors. And we can think about what the span of those two vectors are. We're going to see that it's a plane in R3. Span of 1, 1, 4. And this is a quick reminder, I've said a couple times. When I say it's a basis all I'm saying is that these guys, they both span the column space of A. When I had four vectors, they also spanned the column space of A. But what makes them a basis is that these guys are linearly independent. There's no extra information, or redundant vectors that can be represented by other vectors within the basis. They are linearly independent. Anyway, I'll let you go for now.