Lecture 06 - Linear Operators

[Music] [Music] an attempt to solve ax equal to B right would be let's say uh this is M cross n so there are n columns right uh a is this B right remember we wanted to solve this problem and from whatever we discussed the first thing should be if we want to solve this first we should find a basis of the space spent by C1 to C right this might be a good idea right why why if CN can be written as a linear combination of C1 then it's kind of useless for us so we can throw it out we can make our problem simpler we can reduce the dimensions right and if we reduce it to two great then we can use the graphical meod right but uh obviously it seems like a good idea to kind of condense this Matrix and that we can study irrespective of this and this right and so let's study matrices the first thing we want to talk about for Matrix is linear operators and can someone tell me what are linear operators okay what is an operator function on vectors so what is a linear operator some constraints okay yes so operator means I'm looking at some function or some operator from a vector space V to Vector space w right operator means for any vector v here there is a image from Vector space w right and then if I talk about linear operator that means uh a linear operator acts linearly right good do we know any example of linear operators good so how are the linear operators what are the which space are they going from and what space are they going to so let's say my Matrix m is some M cross n Matrix right you said it is a linear operator right I agree with you but if it is a linear operator it is going from V to W what is V what is w is also no no no V is RN and W is it takes a vector remember m a equal to B this has to be n CR 1 and what we get correct so m is basically a linear operator you can check you know that agreed and again one of the fundamental theorem is when you when you hear of linear operators when you hear hear of matrices very nice it's not just that matrices are examples of linear operators but every linear operator has a matrix representation right now someone says they have a linear operator and you want to know that linear operator how will they give you the linear operator by set of equations how how okay what what equations will be there no no no the remember what is linear operator it's a function from V to W right the dumb thing would be to to take all possible elements of v and give its give its output but that's going to be a long list exactly so this is another fundamental thing that and this is true because we have a linear operator right so once I give you the action on the basis and I want to know okay what is the action on V I will first write it as yes so let's say my basis is B1 to BK right and I want to understand what FV is then uh F of v v can be written as this because B12 BK is a basis and then this is and uh someone has given you these values on the plate right so you can compute this thing that means as soam said if you know the action of the bases then you know the action on the entire Vector space correct okay great now I have a way to give you the linear operator and every linear operator has a matrix what is the Matrix corresponding to this very good so the question to ask is uh so every laration uh uh every linear operator is a matrix representation and this is something which we forget once we fix the basis okay and then since you answered so nicely I give you the freedom to choose any basis of your choice so I think Dave knows the answer but what about others fix your favorite bases what will be the Matrix for this linear operator you can how how can it be identity that is not correct I'm not saying V1 is equal to W1 V2 is equal to W2 so you have to find a basis such that you can easily write it in The Matrix representation so which basis will you choose and how will you write as a matrix again we are doing these exercises so that you become familiar with linear algebra right that's the aim so think about it okay let me simplify the problem for you you know we want to uh sorry I want to write the the thing in this representation okay so then when I write this as k Co efficients these are the coefficients of V1 to VK okay and I should get the corresponding W what is M right yes what are the entries of M so here here I will give alpha 1 to Alpha K and when I said that my Matrix is written in this basis this is what I mean uh sorry so I choose my favorite bases here on which I'm applying and here also I apply my favorite basis like uh did I do something wrong no yeah sorry Alpha is are scalers yes uh W1 as no W1 you cannot write in terms of V this is your Matrix representation and the idea is this times this is exactly this this is the way to interpret your matrix multiplication right so when you write MV and these are the columns this is uh this is how we multiply Matrix with the vector you can convince yourself that okay so then this is the Matrix which is the which captures the linear operator good we wanted to talk about ax equal to B A was a Matrix or a linear operator right and we want to find out can is there a vector X on which I can apply a and get B this is the question we are asking Okay so and then if you want to study this obvious things to talk about are the image and the kernel of the what is the image image is the set of all y right so then we are asking is B part of the image of a okay and kernel is the uh did I do something wrong right so it's a set of all y's such that there exist an X ax y make sense is this okay is this mathematical notation okay with you right and uh this is the image and the kernel it's the set of all X such that ax equal to Z okay and then if a is a linear operator from V to W then image is in with which Vector space w right it's a Subspace of w it's a subset of w and not just that is a subset of w actually if we have two elements of the image we take the linear combination it still is a element of the same Subspace so image is a Subspace of w kernel is a Subspace of has anyone have this question in mind why image and kernel are vector spaces this is okay sounds good okay so then the next concept is rank of a matrix which is basically the dimension of image okay and uh one reason to ruce Kernel and you will see it behaves very nicely It's Kind it's kind of the orthogonal part of the image I don't want to say orthogonal because they are coming from the different Vector space but they are complementary to each other and uh one of the way to put it is the rank nity theorem right which says dimension of image of M plus let's say uh m m is an M cross n Matrix so V is R to the N how do we prove it what will be your attempt to prove this this uh everyone knows this right ranky theorem uh yes no is there anyone who has was who's seeing it for the first time uh there is no shame in saying yes for this question right uh all of us have some flaws in our background it's okay but I need to know if this is new new to you okay if this is not new to you then you should be able to prove it and let me start with you know let so now this is let's say this is a basis of okay and then I want to create a basis of image of him what size should it be okay I think I have helped you enough now can you tell me how many people know know the answer to this one okay others so if I'm given a basis of V1 to VK how will I create a complete basis of the vector space you remember this diagram right this nice diagram take one if it is not you take another one if you still get didn't get the maximum independent set another one right and we were given a guarantee that every time we will stop after n after we have picked n vectors right so why don't we start with that so we have V1 to VK this is linearly independent because it is a basis let's find something which is not in the span of V12 VK okay now can you tell me what is the basis of image of M you are giving a wrong answer but I know you know the right answer what is a I don't know what a is but okay remember it cannot be VK + 1 to VN why can't it be VK + 1 to VN it's wrong space right I have to go to the W and what is the way to go to w apply M okay so my claim is that this is the basis of the image okay uh what do I need to prove for this you tell me what do you need to prove for this and then I won't ask you to prove it wait wait let anyone else answer what do I need to prove the back row is out the this row is out only either you guys or you what do I need to prove to show that this is a basis of image of M how do I prove that something is a basis linear okay good so I have to show anything here can be generated using these vectors what else wait wait what else do I need to show for it to be a basis yes and both these parts are easy if you can't do it once again look at some linear algebra text okay this is basically saying that some linear combination of VK + 1 to VN is in a kernel of M is a counter example uh which is a uh which is contradiction right and this will use the fact that you can write any Vector as a linear combination of V1 V2 up to okay so both these parts are okay but I leave it as an exercise all of you should make sure that you can do this great let's move on let's see more properties of image of M or the rank so the rank was the dimension of image of right but I also know that image of M is linear combinations of what should come here Columns of M right we just saw what was M * V it was basically taking linear combinations of the column so this is column of right that means uh my rank of M is equal to number of or maximum number of linearly independent colums and if you believe in the sanctity of this world then this should also be maximum number of no there is no good reason to believe that this is equal but we know that this is actually equal right so we know that column rank is equal to row Rank and if you are seeing this thing for the first time this should surprise you there is no good reason to believe it right uh so you have a matrix you can view it as bunch of columns how many l independent columns are there or you can see it as a collection of rows how many linearly independent rows are there it turns out that this is going to be the same number how many people have seen this before how many people have not seen this column rank equal to row rank okay okay so now people who have seen it what is the any answers any two line answer any two words answer yes okay so one of the natural proof which comes uh which proves this is goian elimination but I'm going to give you a two image proof now you can say it is two image or one image that is up to you but I think that will highlight why this is true okay so suppose the column rank is K what does that mean everything here can be represented in terms of K columns here let's say some C12 C there is some linear combination here which will give me C1 there is some linear combination here which will give me C2 so I can have some numbers here a11 up to agreed this is the definition of in some sense is saying that column rank is K right because there are K vectors which I've picked from here and there are some linear combinations which make it m right and now yes now you just view it differently you can write this as Row one write this as row two as RK okay what is the first row of M it's the linear combination of R1 to RK with these coefficients what is the second row of M it is a linear combination of R1 to RK this is just understanding what your matrix multiplication does right so so when you write a equal to C you can view it as you take The Columns of a multiply it with the coefficients in the First Column of B you get the First Column of C or the other way to put it is you take the first row of a mult multiply it with the rows of B you get the first row of C okay so then I just think of them as K rows and I see that every Row in M can be written down in terms of K vectors that proves that if you can show that every element of M you can write it in terms of every row can be represented in terms of K vectors that shows that what does this show about row rank row rank is less than equal to column okay because this need not be linearly inter but then you can run the same argument in the opposite direction Okay so so we had all these uh we have all these nice things and uh this allows us remember our old problem we wanted to find out Solutions of axal to B and B equal to Z in some sense the simplest case this is talking about the kernel of [Music] [Music] [Music] it