Lecture 7 | Applied Linear Algebra | Vector Properties | Prof AK Jagannatham

Hello, welcome to another module in this massive open online course. So, we have seen the concept of a matrix inverse and how to evaluate the matrix inverse. Let us now start looking at applications of matrices. Now, predominantly one of the main applications of a matrix is to formulate a system of linear equations and to solve the system of linear equations. So, let us look at a broad class of applications of matrices, where we use the matrix inverse specifically, where we want to use the concept of the matrix inverse that we have just learnt. This is to solve a system of linear equations. I think that is important, the linear equations, which is essentially you have the system of the form a11 x1 plus a12 x2 plus a1n xn equals b1, a21 x1 plus a22 x2 plus a2n xn equals bn, so on and so forth. And then we have a n1 x1 plus a n2 x2 plus a nn xn equal to bn and so you have the system of n equations and these x1, x2, xn, these are the variables or basically the unknown. So, you have x1, x2, xn, these are your n variables or n unknowns. So, you have a system of n equations and n unknowns and this is a linear system of equations, because it is linear in x1, x2, xn. So, this essentially is your linear system of equations and why is it a linear system, because it is linear in these variables x1, x2, xn. It does not involve the powers x1 square, x1 cube. It is simply depends on x1, x2, xn. So, this is a linear system of equations, because you have n equations, because this is linear. And naturally you can write this as, you can put all these things together as a matrix. matrix a11, a12, a1n, a21, a22, a2n, an1, an2, ann times x1, x2, xn equal to you can say b1, b2, bn. You can call this your n cross n matrix A, this is your n dimensional vector x bar, this is your vector b bar. So, you can write this in the compact form A x bar equal to b bar, that is your compact representation of the linear system of equations. This is your n cross n square matrix, simplest formulation where you have a square matrix. Now, what happens when the matrix is not square, then the problem becomes more interesting. So, in this simple, I mean in this the simplest form, we consider n equations, n unknowns and further more if the matrix A is invertible. If A inverse exists, then the solution is simply given as A inverse x bar, this is the solution to the above linear system of equations, x bar equals A inverse b bar. Of course, as we have said, this is based relies on the assumption that the matrix A is invertible and A inverse exists. Now, of course, what happens when A inverse does not exist or in fact A is not a square matrix, that is you have more equations, less equations, more equations than unknowns or less equations than unknowns. Then it becomes very interesting and through this course, but this is to begin with you can look at the simplest problem, where you have n equations, n unknowns and the matrix A is invertible, the solution is given by A inverse b. And let us take another example, a very simple example to understand this point. For instance, you have let us say the matrix, this is the equation system of equations in three variables. So, this is your 3 cross 3 matrix, these are the variables or the unknowns and this is your vector b bar. So, this is your vector x bar, this is your vector b bar and these represent the equation 7 x 1 plus 2 x 2 plus x 3 equal to 2, 3 x 2 minus x 3 equal to minus 1 and minus 3 x 1 plus 4 x 2 minus 2 x 3 equal to 3. These are the equations and the solution is given as A inverse b, this is equal to minus 2. And we have already calculated A inverse in the previous module, if you remember, we have already calculated A inverse and therefore, the solution is simply given as A inverse, which is essentially your matrix minus 2, 8, minus 5 and look at the previous module to check how to evaluate A inverse 7, 9, minus 34, 21, right, times 2, minus 1, 3. So, this is your matrix A inverse, this is, so I will simply say here, see previous module to evaluate, to see how we evaluated A inverse and then you get x bar equals, you evaluate this and this gives you x bar equals minus 27, 38, I am sorry, minus 27, 38, 1, 1, 5. This is the solution of this problem, of this system, of this 3 cross 3 system of linear equations, which is A x bar equal to b bar, 3 equations, 3 unknowns, matrix A. We have seen it is invertible, we have evaluated its inverse, the solution can be computed using A inverse b bar and that is what we have evaluated. Let us move on to other interesting examples of course, this is an abstract example, let us move on to other interesting practical real life applications of linear algebra, right, the theory of linear algebra and matrices. So, let us start looking at practical applications of this and remember, this course is applied linear algebra. So, we are not only simply going to look at the abstract or the theoretical aspects of linear algebra, but we are going to integrate these concepts, I think that is the most important part of this course, that is not only look at the concepts, but alongside look at the applications, so as to relate this concepts to the applications which will help you understand better and appreciate the concepts and more importantly better understand how and where to apply these concepts that have been learnt, right. So, that is one of the important goals of this course, that is to teach linear algebra, but not purely in a theoretical fashion, but from an application perspective. So, there are several interesting and in fact, one of the interesting things was probably we have discussed at some point that linear algebra is probably one of the most important and one of the most extensively used theoretical tools, because linear algebra, linear modeling arises whenever you have linear systems and linear analysis and a large number of systems in science and engineering can be represented, modeled, analyzed using therefore, the principles of linear systems, which is essentially based on linear algebra, right. So, linear algebra arises where it is used very extensively and applications are numerous and we are only going to scratch the surface of that, I encourage all of you to look at further explore many more such applications, I mean even outside of this course, alright. So, there are several important applications, several I will, we have already noted this several applications of linear algebra and in particular of this concept of matrices and inverse matrices and the inversion that we have studied so far, right. So, I am just going to simply say the concept of matrices. The first example that we are going to look at is basically in wireless communication or modern wireless system or wireless system design, you can also say, where for instance you must, all of you must be familiar that the most, the modern wireless systems are basically 4G, modern wireless networks, you have the 4G and in the future you will have 5G and of course, you have so the Wi-Fi and these are based on a very interesting technology, these modern wireless systems, these are based on what we call as the MIMO technology. Now, what is MIMO? MIMO stands for, so what is MIMO? Multiple input, MIMO stands for multiple input, multiple output and what does this mean? This is a very interesting technology multiple input, multiple output, what it means is in a typical wireless system, you have the transmitter, right? So, for instance let me draw this thing, you have, what is, what is the meaning of this MIMO? You have the transmitter, you have the receiver and then you have these antennas, similar to what we have seen, each of these is an antenna and what MIMO means is that you have multiple antennas, right? Multiple antennas at the transmitter from which you can transmit, multiple transmit antennas and you have, and you have this multiple receive antennas. So, you have the multiple transmit antennas from which you can transmit multiple symbols, right? And you have multiple receive antennas from which you can receive multiple symbols. So, you have multiple inputs, that is the multiple transmitted symbols and you have the multiple outputs, that is the multiple received symbols, that is the reason this is known as, such a system is known as a MIMO system. This is a very advanced wireless technology, which has been, which is used as I have told you in 4G, 5G wireless system and that makes it possible to transmit at very high data rates in such systems, alright? So, MIMO is an important technology. So, MIMO is a very important technology. I would say very important technology for 4G slash 5G, which makes possible to transmit at, MIMO is a very important technology in 4G, 5G, which makes it possible to transmit at very high data rates. Now, just to give you a brief background of this MIMO, so we have for instance H I J. So, each of these is a channel, which we have seen similar to our discussion of the wireless system before and each channel is characterized by that channel, that is H is characterized by the channel coefficient H I J. This is the channel coefficient between T X antenna, between the R X antenna, receive antenna I and transmit antenna J. For instance, H 2 3, this is the channel coefficient between receive antenna 2 and transmit antenna, between receive antenna 2 and transmit antenna 3. So, that is the concept of the channel coefficient H 2 3 and naturally, you can put all these channel coefficients together and you can make them into a matrix. So, once you put all these channel coefficients together, it becomes a matrix, which we call as the, which we call as the, which we call as the, which is basically termed as the MIMO channel matrix. So, therefore, you will have H 1 1, H 1 2, H 1 T, I will talk more about this, H 2 1, H 2 2, H 2 T, H R 1, H R 2, H R T. This is the channel matrix H, as you can see, this is R cross T matrix, R equals number of receive antennas and T, this is equal to the number of transmit antennas, number of transmit, T is the number of transmit antennas. Therefore, this is your R cross T MIMO channel matrix and this is very interesting. You have the channel, which is essentially, when you look at the rank of this MIMO channel. Now, let us look at the rank of this MIMO channel. This has very interesting implication for communication. So, the rank of this, we know is less than or equal to minimum of R comma T, number of receive and transmit antennas. What can be shown is that, in this MIMO system, the maximum number of symbols that can be transmitted at any time instant, which basically has a bearing on the transmitted data rate. That is, the larger the number of symbols that can be transmitted, the higher is the data rate. That depends on the rank of this channel matrix. So, the maximum number of symbols, maximum number of symbols less than equal to rank H, which means, if your rank H is large, you can transmit a large number of symbols at any time instant. So, the data rate is going to be very high and that is how it is possible to transmit at very high data rates in 4G and 5G system. So, rank plays a very important role. Rank of the channel matrix plays a very important role in determining the maximum data rate that is possible in a wireless communication system. So, that is a very important. So, you can see, linear algebra is everywhere in particular modern wireless communication to determine the data rate. For instance, what kind of data rate is it possible to transmit, what kind of data rate is possible to transmit at, for instance, by the base station to your cell phone. So, rank determines the data rate. So, rank, larger rank, rank determines the data rate, larger rank H implies larger data rate. And therefore, once you have the MIMO system, you can transmit a much larger number of symbols. That is basically, in the same time and frequency, you can transmit a much larger number of symbols. That is, you can multiplex a larger number of symbols. This property is termed as spatial multiplexing. So, H because of the MIMO system, MIMO system essentially implies transmit or multiplex a large number of symbols in same time slash frequency. This property is termed as, this property of MIMO is termed as spatial multiplexing. So, this is a very important term. This property is termed as spatial multiplexing. This is a very important property, which makes it possible to transmit at very high data rates in 4G and 5G systems. So, this is what makes possible MIMO technology via this property of spatial multiplexing of MIMO makes it possible to achieve very high data rates in 4G, 5G wireless systems. So, this is the spatial. So, essentially what you are doing is, you are multiplexing multiple streams to through space. So, this is basically also termed as the parallel transmission of multiple data streams in same time frequency. This is the parallel transmission of multiple streams. So, this is basically spatial multiplexing. Now, let us look at how to design the receiver for a MIMO system. How do you design the receiver? Very simple. Let us look at a MIMO receiver. Now, remember we have this multiple antenna system. Let us consider a spatial system with number of receive antennas equal to number of transmit antennas equal to R. Let us call these output symbols as Y1, Y2, YR. So, this is your receiver. This is your transmitter and these are your transmitters. So, this is your transmit. These are your transmitted symbols. So, X1, X2, XR are the transmitters. So, we are considering a spatial system in which R equal to number of transmit antennas equal to number of... So, we are considering a spatial system. We will see what happens if they are not equal later. So, we are considering a system with number of transmit antennas R, number of receive antennas R. So, it becomes a square channel. So, these are basically the transmitter symbols X1, X2, XR. These are basically your transmitter symbols and Y1, Y2, YR. These are the received symbols and these are related by the channel as follows. That is the model for the MIMO system is given as Y1, Y2, YR equals your channel matrix H which is your channel matrix H11 up to H1T, HR1, HRT. This is your R cross. In this case, I am sorry, this is an R cross R channel matrix and this is your X1, X2, XR plus you have your noise vector. This is your N1 into NR. So, this is essentially what we call term as the MIMO system model. So, this is your output vector Y. This is what we are going to use several times in this course. This is your channel matrix. This is your transmit vector X bar. This is your noise vector. So, you have Y bar equals H times X bar plus N bar. This is essentially your MIMO system model and the different components of this model are as follows Y bar equal to H times X bar plus N bar. This is your output vector, output of the received vector. This is your channel matrix. This is your transmit vector and this is your noise vector. Now, the question is how to and remember we are considering a special case where this is an R cross R matrix. So, remember this is only a special case. This is not a general case. This is only a special case. Now, the question is how to determine the transmit vector X? How to determine X bar at receiver given Y bar? This is essentially for this case where R equal to R. Now, you can see H is invertible by R cross R. So, this is then X hat which is basically what we term as the estimate of the transmit vector at the receiver. That is the recovered transmit vector at the receiver based on the output symbols or the observed symbols Y bar. That is given as simply H inverse Y bar. It is like solving a system of linear equations, but mind you this is valid only if only when R equal to T and H is invertible. This is what is termed as the Z F or this is interesting name. This is termed as the zero forcing receiver or essentially it is equalizing. You can also think of it as an equalizer. It is essentially although it is not typically called as sometimes also it is called as an equalizer. Typically, it is known as a zero forcing receiver. That is basically essentially inverting the effect of the channel. However, remember this is only valid when R equal to T and H is invertible. Now, what happens when R is not equal to T? In particular, when the number of receive antennas is greater than the number of transmit antennas, that discussion is going to be very interesting which we are going to see in the course of this MOOC. So, that discussion is going to be there very much going to be there. So, what happens when R is not equal to? So, let me also note that what if R greater than T, then I would say things become interesting. We will see this in the future modules. So, this is the zero forcing receiver also and note that this is I mean not the zero forcing receiver, but this particular application that is your H inverse y. This is valid when R equal to T and H is invertible. So, essentially this is one of the I mean these are very interesting applications of linear algebra. So, essentially the idea is to convey to you the sense that what we are learning these different concepts. These are not just abstract theoretical constructs, but these are really powerful concepts and techniques that have immense applicability in practice. For instance, we have just talked about one such application in the context of MIMO technology, multiple input multiple output technology that employs multiple antennas. For instance, at the base station and at your cell phone and naturally when you have MIMO, the channel becomes a matrix kind of a channel. Therefore, linear algebra and the principles of matrices, concepts related to matrices such as rank, the rows of the matrix, columns of the matrix and all these become very important. As we have already seen, the rank has a very important role to play because that determines the maximum number of symbols that can be transmitted. Larger rank implies larger number of symbols which implies very high data rate. The matrix inverse that basically for an R cross R equal to T system that basically can be used as a receiver to recover your transmitter. So, all these concepts have very powerful applications and we are going to see many such applications of these techniques of these concepts in linear algebra as we go through this module. So, with this motivation let me stop this module. We will keep discovering such applications in the subsequent modules.