noc18-mg33 Lec 12

[Music] [Music] I'm very warm welcome to all my dear students and friends a very good morning good afternoon good evening welcome to this da dm1 course under the NPTEL MOOC series and we are for this course which deals only with statistics multivariate analysis later on obviously these will come and this is a 12 week course for 30 hours each week as you know we have you have 5 lectures each for half an hour and this is the twelfth one that means we are in the third week and I am revin on dancing with the phone I am a department IIT Kanpur so if you remember the last slide on the last concept before we closed the eleventh lecture was related to central limit theorem sort of Senet central limit theorem basically says that considering any distribution with a certain mean certain variance so we are only dealing with the first moment in the second moment it can be done for other cases also and we want to find out what is the sum the expected value of the sum and the average sum average of the sum and in the long-run what is the expected value of that so we found out that for the case if it is normal then the mean value is mu for the average of the sample sample means does number of observations which you are picking up from one to small N and the variance for that sample is Sigma square by n where again and small n is the sample size Sigma square is basically the population variance now considering X Bar n as a random variable obviously it has to be because it has got a mean and a variance then its corresponding distribution as we told that if the population is normal the sample distribution for the mean value is normal mean value I am NOT talking about the distribution of the variance so the mean values of the sample distribution is normal so when you convert that to a standard normal deviate with the mean value of 0 and esther and Varion of one then actually the conversion is like this I'm just saying that listen to it carefully in the numerator you have X bar n minus mu because that mu is the expected value of X bar n divided by generally it's basically the standard deviation so what is the standard deviation in this case e the variance is Sigma square by n so the standard deviation would be Sigma by square root of n and that would be normally distributed with zero mean and one standard deviation now in in in the case of sampling so we are going to the realm of sampling we will consider three different ideas these are very broad ideas but I will just stick to the bare minimum as you quiet one is the point estimation concept of point estimation one in the concept of interval estimation and one is the concept of hypothesis testing so what we mean by point estimation interval estimation and point estimation I'll go through that story consider for the sampling actually what why you do sampling is that you have a population of capital n or infinite and what you have about the information on the population in the population has a distribution like it can be the geometric distribution portion distribution exponential distribution have a geometric distribution negative manual distribution normal distribution uniform discrete distribution uniform general district uniform distribution continuous one now you have the parameters so the parameters for the normal one more new and Sigma square for the exponential one was a or lambda or a or theta whatever you'd apply portion was lambda for the my email it was N and P and all these parameters are there so we'll denote the parameters in general by the theta theta can be there a vector or a scalar so if it is only one parameter so consider like the position distortion theta is lambda and if it is normal distribution theta is a vector consisting of two elements mu and Sigma square now if the parameters are known we know the distribution in totality and we can do our work but the question arrives that two things number one if we basically pick up a set of samples or observations of size small n which means basically we are picking out observations x1 its capital x1 remember that those are random variables so hence as I mentioned time and again in like in the beginning so they would be given by capital X or capital Y or capital Z so they would be random variables capital x1 till capital xn and we want to find out the distribution of the characteristics of the sample so it can be characteristics can be mean assemble characteristics can be mode of the sample characters characteristics can be median of the sample characteristics can be variance of the sample and there are many characteristic I doubt or some functional form also so given the parameters we will try to basically do the study whether the characteristics sampling distribution is available to us point one point number two is that given the parameters are unknown from the population how we can estimate them estimate means fine now them as close as possible using the sample sampling information that means the sampling information means from capital X 1 to X Apple xn such that we are as close as possible to the parameter value provided we meet few characters criterias so obviously what are those criterias I'll come to that also so given a population of of capital N and denoting size population distribution would be given with the parameters known if they are not known you have to find out as I said in the in just few seconds back now the corresponding sample which you pick up as I said capital X 1 to a capital X n is the sample of set of observation until we have a sampling distributions and the corresponding statistic which you see here on the on the slide which is small T suffix and that's the characteristics corresponding to population parameter which you want to find out using the sample size that the sample statistics teen is as close as possible to the population parameter so we want to find out that what is the characteristics TN from the sample size that is close as close as possible to the mean value of the distribution or L so you want to find out that what is the sample characteristics from the samp obviously from the sample which is as close as possible to the median of the population or you want to find out that what is the sample characteristics TN so TN is obviously changing which is as close as possible to the variance of the population so we will try to study not in details just as relevant to this concept or dat so consider an example very simple example there are lot of of Const like rigidity in how you are trying to consider the example but it will give you a feel that what we are actually looking at legitimately by the word rigidity means that we are not assuming all the things to be true but in a very real in an a sense in a very theoretical sense this is the situation and how we can attempt considering what I discussed about the statistic the sampling population considering the parameters being known not known whatever it is consider the population having a following elements 1 3 6 and so obviously rigidly being that capital n is 4 which is not possible but we are considering that so hence capital n is 4 and the MU value would obviously be because if therefore the corresponding probability will consider the corresponding probability to be uniform so the probabilities are 1/4 for 1/3 1/4 for three one four four six one four four seven hence the mean value is 17 by four now what we do is that we take a sample size of n is equal to two so n is equal to 2 so we choose the samples in this way so we could have obviously we will consider that we are choosing in the theoretical sense the sampling are done in such a way that if you pick up number one you still have number one from the population to pick up in the second trial actually that's not true because if we pick up one the chit marked one is not there so we will consider this the sampling being done with replacements so the total number types of Sam samples we can have with two observation the sample can be in one extreme it is one one because you can pick up one and then again you pick up one and another extreme you basically the sampling sample or the observations as seven seven so so this is what it says let me continue reading it and choose the samples then hence the possible values of the samples averages because if it is 1 1 so it will mean 1 plus 1 divided by 2 which is 1 and if it's 7 7 it will be 7 plus 7 divided by 2 which is 7 so whole set of average values for a sample of size 2 would be B starting from 2 to 7 now obviously they are the actual realize value for the sampling distributions average average value which you are talking about and if it is a distribution you would like to find out what is the mean what is the distribution parameter so on and so forth now simple and sampling can be of two types as I sent one is the sampling sorry my my apologies it's just went here and there the slides so sampling can be of two types one is simple random sampling with replacement which is SRS WR and simple random sampling without replacement with these SRS wor that means we do with replacement without replacement so what I mentioned few minutes back when you have that population of four if you picking up observations and if you are doing with replacement hence the corresponding probability distributions or the probabilities for picking up any observations doesn't change as for the case if you pick up one and if you consider that the number one is also present in the population the next time and the next time and so on and so forth hence the probability always remains one-fourth so it is simple random sampling with replacement for without replacement the probability of picking up one twice is one-fourth for the first time and the second time it is zero by three because there is no one and then total number of such observations the population has decreased by one hence three note that if X has a distribution says that the expected value of the distribution is mu suffix X mu is again nothing to do with the normal case variance is Sigma square suffix X again nothing to do with the normal case then as I said in the last class which is the in lecture number 11 if you pick up any observations at random xixi is the ayat place you are going to pick up so the ayat list can be filled up by any one of the observations from the population corresponding the probability T has so if you want to find out the expected value of that I at place which is e X I would exactly it will be equal to MU and the corresponding variance of X I would be Sigma square suffix X now remember one thing if you I want to find out something to do with the covariances which I said so that would basically be be taken into consideration when you find on the covariances so before I go into trying to basically describe very briefly though the probability sampling the simple random sampling on all this thing so let me come to the concept of the covariances which I I didn't go into details but I just mention it here so consider let me use the red color yes so what you have is basically these x1 and x2 are the random variables so the random variables I'll denote by small x1 the first one is basically corresponding to the random variable number the next one is the reading number so this will be 2 comma 1 and it consists quantities 2 1 comma n so this is the number of observations which you are picking up this is the N is the number of observations you have become now if I want to find out n consider this is a large set of values so if I want to find out the expected value so the expected value of e x1 we will consider it's given by so these are just the samples which ever it picked up so let me highlight it with some other colors so these are the off set of samples which are picked up so that we go back to the discussion so this is in win buy one new suffix one this is given by mu suffix two variance of x1 is given by Sigma square one radiance of x2 is given by Sigma square 2 so note this down and then I will proceed so this is the background and I'll it is it but I'll be seeking to now come to the concept of covariances and then again come back to whatever it is typed on the slide so here is it so this part which is highlighted can let it continue remaining because whatever I have written about mu 1 mu 2 Sigma 1 Sigma square or suffix 1 Sigma squares of X 2 are for the population now covariance between X 1 and X 2 would be given the relationship which is between X 1 and X 2 so you want to find out the covariance it will be X 1 minus mu 1 so the e is the expected value so multiplied so there's no space I'll continue in the next 9 X 2 so actually what I am doing when I am trying to find out the corresponding covariances so expected value would mean integration on this or the summation so if there are two variables it will basically double summation so what I am doing is double summation for the discrete case for all X value X 1 values for all X 2 values for the case when I have X 1 minus mu 1 into X 2 minus two x by the case when I have the probabilities so these are the joint probabilities I'm going to come to that details later on don't be too much bother the border and the case when you have to the continuous case it is double integration and in this case probability gets replaced so this will become f this capital F or let me put it as small F source capital e wait for F would be difficult for us to understand immediately I will come to that small F F F F but now it's a joint distribution of x1 and x2 so it will be X 1 comma X 2 DX 1 DX 2 so this is integrated for all X 1 comma minimum X 2 comma minimum so this is the minimum the maximum value X 2 comma maximum X 1 comma maximum so you can find it now if you want to find out the relationship covariance obviously is important so when I basically do the covariance part le raise it again I'll come to those all details later on also so what I am trying to find out is basically the correlation that is the relationship between X 1 and X 2 so that is given by covariance so I am just writing the bracket this covariance between X 1 and X 2 divided by the square root of variance of X 1 which is the standard deviation of X 1 divided by standard we can now do now if I look at the covariances between two random variables so how would they look like so and and obviously remember it can be proved that the covariance I am not putting the suffix it can be between any two random variables is always this inclusive of of minus one and plus one it's always this so let me draw simply the diagram to make you understand so and use the red one I'm so black one to draw the coordinates so the coordinates are I am basically plotting x1 along the x axis x2 along the y axis and now I'll try to denote the correlation coefficient the Rho value corresponding to the case where it is it is negative it is zero and it is positive so I use three different colors the first let be for me for the negative one so if you find out the negative one generally they wouldn't plot it like this so the general trend line would be in this direction if I plot the positive one I only draw the trend line for the positive one so this dot should be there so the positive trend line would be something like this and if I do for the zero case that means correlation naught they let me use the color green so they are everywhere so obviously they are spread over along the four quadrants such that you cannot say what is the correlation value it is almost equal to zero so if I basically consider the correlation coefficient it will give me what is the relationship so now the correlation coefficient is point five it would mean the rate of change of X 1 and X 2 is is of one unit to a half a unit and if it is - it is 1 in it it is minus point five you to 102 minus point it means if it increases for the Navy case it really increases the other decreases now consider that correlation coefficient values and the covariances so first let me consider the correlation coefficient values for n number of random variables so this n has nothing to do with the sample observations small n so again let me it is it so I'm spending a little bit time here so I'm sure you will you'll bear with me so let me draw kin here let 1 1 1 it I'm containing it so now come back to the sampling which i drawn so consider there are random variable steeled X n so these are given by X oh my my mistake let me consider this n as M let me consider is that M so this M 1 till X M n so M number of random variables each B are being set of n number of observations M as in mango n as in nagpur now I want to find out the variance covariance or the covariances between them so if I use that formula which I which I just stated how you find out the covariance between two random variables and when I write them it is basically an M by N by M M by M matrix because there are m observations so I try to find out the covariances between the first the second the first the third the first the fourth the first and the last then I find out between the second and the third second and the fourth second and the fifth and second and the last similarly as I continue doing it I find out the M minus 1 and the M 1 and I basically find out the all the covariances now when I'm doing that covariance is obviously you will ask the question that what if I am trying to find out the covariances between two different random variables would not the covariance also exist between them answer is yes they will exist but we have already considered that the covariances of a random variable with itself is the variance before you put on the formula what you have actually is when you are trying to find out it is basically X minus mu whole square so if in the case for the covariances if it is the formula is X minus mu whole square so in the case when it is a covariance in between two different random variables it is X minus mu 1 into y minus mu 2 hence when when both x and y are same hence mu 1 and mu 2 would be same then we get the the variances corresponding it to the covariances where the covariances are between the same random variable so now when we write the M by M matrix that is mango M is mango not n as in up or the M by M matrix actually this denotes the following let me erase the part else it gets no this should be there so the first variable which is 1 comma 1 cell is Sigma 1 1 which is the covariance of the first with itself the second one along the row is a Sigma 1 2 which is the covariance of the first to the second and I continue doing it till the last one which is the covariance or the first with the M F 1 now I come to the second row the second one is Sigma 2 1 which means they are a mirror image of 1 2 & 2 1 because trying to find out the expected value of X minus mu 1 multiplied by Y minus mu 2 is equal to the same thing X finding the expected value of y minus mu 2 into X minus mu 1 second element on the second row is Sigma 2 2 which is basically the variance of the second or as you can see put in another word is the covariance of the second with itself continue going doing it if I go to the last element it is M comma M 1 and the last one is M so this is the variance covariance matrix for the population you will have a variance covariance matrix for the sample also similarly when I come to the correlation coefficient so variance covariance matrix I will denote buying this for the population and this gets replaced by the the by the variance covariance matrix corresponding to the sample as s capital S these are capital S when I replace and try to find out the correlation so let me remove this whole part everything right again draw it so it will become easy for you so now I am trying to draw the correlation matrix for M cross M so first element always the correlation is basically of the first random variable with itself so that relationships one the second one is basically the correlation of the first true it the second and similarly the last one is this the second rows are accordingly so so correlation one two two one are symmetric so they are same the second element is two because the correlation of the second with itself is is 1 similarly this last element is this so the principal diagonal let me highlight it yes the principal diagonal is 1 and the of the diagonal elements are the correlation coefficient which is symmetric so that is denoted by in the case for propagation which for the population and when it is the sample it is denoted by r so I think I can it is this part III even of the slide is being shown here I'll discuss these details of simple and ins happening and all these things more detail in the next class because I thought it was important so hence I'm going a little bit slow and I hope you are understanding it now coming back to the observations what I've written down so if you remember I I put a yellow highlighted here so there was a reason now when you have a sample and you want to find out the correlation coefficient covariances and all these things what you do is simply you consider the sample consider the averages of the sample as such so they would be the sample averages so the sample averages would be given as X bar 1 similarly till the last one is X bar M so for this for this the variances would be given by s1 square I will come to that later on yes M square so they would be discuss in detail within few class and the correlations would be found on find out the averages here so they would be sample averages given here find out the Co video mean the covariances which the word is not covariance it basically square of standard error or the variance of the sample so they would be given here and you can find out the corresponding correlation coefficient and the covariances also with this I will end the 12th lecture but the slide which is in front of us I have not discussed anything about probably something this I will only discuss in a qualitative framework I'll come to that in the thirteenth lecture and correspondingly other lectures but remember when I have been talking about the correlation coefficient covariances mean value standard deviations for those samples corresponding to the case we will be discussing the sampling distribution I will again come back to that in much further details have a nice day with this I will close this net 12th lecture and thank you very much for attention [Music] [Music] [Music]