noc21-ma36-lec19

[Music] [Music] hello welcome to the course essentials of data science with r software 2 where we are trying to understand the basic concept of sampling theory and linear regression analysis and in this module we are going to continue with the topics of simple random sampling so up to now the last couple of lectures we have considered the basic fundamentals of the simple random sampling and we have learned that how we can draw the samples by srswor and srswr using the r software now today i am going to discuss on another issue my question is why do you do sampling what is your objective you may recall that in the beginning i had explained you that our basic objective is this that we want to know something about the entire population but that is very difficult that is not possible so we try to take the help of sampling theory and we try to draw the sample and then we conduct our entire analysis on the basis of sample we try to apply all the statistical tool only on the sample and whatever are the outcomes they are supposed to be valid for the entire population for example if somebody wants to conduct a clinical trial and suppose there is a new medicine which can control the blood pressure in a much better way than the earlier medicine so how to conduct the clinical trial some number of persons having the problem of blood pressure will be selected in the sample they will be given the medicine and their recordings will be noted after some time and then the two values that what was the blood pressure before giving them medicine and after giving them medicine they will be compared but once the results are analyzed and it is established that the yes the medicine is effective then the medicine is valid for the entire population not only for the selected sample so that is our objective so in simple random sampling also when we try to draw the sample then our objective is to estimate something for the population when i say something for the population from the statistical point of view there can be several statistical quantities they can be population mean population variance skewness vertices and and all sorts of parameters so our basic objective is this how to estimate a population value or a population parameter on the basis of sample which is drawn by simple random sampling technique well now when i come to the aspect of sampling theory or in general people are more interested in finding out the population mean and population variance that means they want to see that where the values are scattered or where the values are more concentrated how the values are scattered how the values are spreaded for example as a student you always try to inquire from your teacher what is the class average based on the class average you always try to determine that whatever are the marks obtained on an average inside the class based on that you try to decide whether what type of grades you are going to get so you are interested in the average value and similarly once you try to see the average value you always try to see how the spread of the values for example if some person has got one mark out of hundred and say another person has got ah say this uh say say hundred marks out of hundred so then the average is going to be close to fifty right 100 plus 1 that is 50.5 and there is another possibility that all the students have got the marks between say 45 and 55 so when you try to take the average of those bunks that will again be close to 50. so in both the samples the average value is coming out to be close to 50 then what is the difference the difference can be ascended can be known can be found using the concept of variability and you would always like that the observation should have as minimum as possible variability whatever tools you are going to use they should give you a result which has got the minimum variability and that we also have discussed in the past right so that is the precise reason that in sampling theory usually you will see people are more concentrated or more interested in estimating the population mean and population variability now the next question is this you know from from your from your statistics knowledge that there are different measures to find out the central tendency of the data that can be arithmetic mean that can be geometric mean that can be harmonic mean that can be median that can be mode and similarly for the variability there is variance there is absolute deviation there are quartile deviation and etcetera there are many many things but now the question is which of the measure you would like to choose well now at this moment i would try to address two things first i will try to concentrate on the mathematical easiness means once you are trying to do do the algebra then algebra should be decent and manageable and second thing is this it becomes very difficult so in the first case when we try to concentrate on the algebra and statistical properties of the estimators then arithmetic mean turns out to be a reasonable choice means you take the observation and try to find out their arithmetic mean and use it to estimate the population mean because from the statistical inference also you have seen that the sample mean has got nice statistical properties and similarly when you want to study the variability in the data one of the most convenient most popular and statistically efficient measure is variance so usually you will see that people are trying to use the sample mean or arithmetic mean of the values to estimate the population mean and they are trying to use the sample variance that means the variance of the sample to measure the population variance so that is the reason that why people are trying to ah trying to consider these two aspects using these two measures but i am not saying at all that these are the only possible measures there are other things also and we will see at a later stage that if you have some complicated types of thing then we have a bootstrap methodology which can help you in computing the different types of statistical function based on only computation so this is the basic idea and this is the basic philosophy behind the computation of sample mean and sample variance or treating sample mean and sample variance as an estimator of population mean and population variance respectively nobody is stopping you in using median mode or say anything else to submit the population mean but these are the reasons in my opinion which motivate us to consider the sample mean as an estimator for population mean and a variant of sample variance as a estimator for the population variance so with this objective let us begin our this lecture in this lecture i am going to consider the estimation of population mean and in the next lecture i will try to address the estimation of population variance so let us begin our lecture now okay so first let me briefly explain you our symbols and notation that we are going to use so as we had discussed earlier also capital y 1 capital y 2 capital y n that will be denoting the sampling units in the population and obviously y is my here random variable okay and then small y 1 is small y 2 small y n they are going to indicate the sampling units in the sample and by looking at this capital n and in this small n you can automatically know that what is my population size and what is the sample size now if i try to [Music] define the population mean one option is this i can use the concept of arithmetic mean and using the concept of arithmetic mean the population mean of this y one y two y n in the population can be defined by here one upon capital n summation i goes from one to capital n y i and this is going to be denoted by capital y bar that will be our standard notation through the entire lecture on sampling theory that capital y bar will be denoting the population mean and whatever sample you have obtained here try to find out the arithmetic mean of these values and this is going to be denoted by here 1 upon small n summation i goes from 1 to n small n y i and this is my here a small y bar this is denoting the sample mean and the same notation will continue throughout the lectures on sampling theory right okay so as we have discussed that one of the main objective after the selection of the random sample is to know about the tendency of the data to cluster around the central value and the scatteredness of the data around the central value but that we want to know in the population what does this mean for example if i say if my data is here like this then we can see here this is the point here where most of the values are concentrated and this distance from the central value to this individual point that is giving us an idea about the spread or the scatteredness of the data so if i try to take here another data set here on the same scale so you can see here this has central tendency somewhere here the data is clustered somewhere here around this point but this variation the difference between the central value and all the individual point this is much larger in comparison to this data set so in this case the variation is going to be higher so we want to know the point where the data is concentrated and what is the scatteredness of data around the central value so the popular choice to know these things for central tendency the popular choice is arithmetic mean and for the scatteredness for the variability the popular choice is variance so that is what we try to denote in the population also so the population mean will be denoted by arithmetic mean and the population variance will be denoted by the concept of variance right yeah sometime we will also try to consider the weighted arithmetic mean depending on the choice for example when we do the stratified sampling there you will see that we will try to consider the weighted arithmetic mean instead of ah simple arithmetic mean okay so as i told you that various estimator for estimating the population mean or the population variance they are they are available and can we try to consider the s the estimation of population mean then sample arithmetic mean is one of the popular choice in comparison to other estimators and the reasons for this are manifold as i told you and one of the important reason you can see that that whenever you are trying to define the sample arithmetic mean it is easy to handle it mathematically and it will also have nice statistical prop properties in most of the cases right ok so now with this objective let us try to move further so now we are going to consider the sample arithmetic mean which is here like this y bar is equal to one upon small n i goes from one to small n y i as in estimator for the population mean which is defined here as a capital y bar as we just defined earlier also so now we will estimate the population mean by sample mean this means what we want to know the value of capital y bar but we do not know because this is in the population due to various region it is not possible for us to consider these things so we are trying to find a sample and then we are trying to find the mean of the values in the sample and then we are estimating small y bar we are using a small y bar to estimate capital y bar in simple words we are saying whatever is the value of the sample mean based on the values of sample of observation this is the same value is going to be the mean of the entire population well there can be many question that how it is possible and whether it will give us a good value or bad value so these those aspect we will now try to address one by one one important property of this sample mean is that that sample mean is an unbiased estimator of the population mean under both type of sampling scheme that is srswr as well as srs wor what does this mean from the statistics point of view in the language of statistics we try to denote this property by saying that expected value of small y bar is equal to capital y bar right what is this expectation expectation those who are from statistics background they know it for those who are not i will request them to please go through with some statistic books and try to see but you can see here these are defined as say some possible values of x i into probability of say here x i i goes from 1 to here capital and if the population is discrete and and capital p is the probability mass function and if a population is continuous then it is a sort of integral f of x x d x right and over the web values of here x you have to integrate at integrate it so essentially this expected value of of a function this indicates the arithmetic mean why because in this case what we are going to say that we have different sample and the probability of drawing any of the sample is the same so now i do try to do one thing first i try to take a very simple example to explain you that what is the concept of unbiasedness and how it is useful in sampling theory or why we want that the estimator has to be an unbiased estimator of the population value suppose i take a population of size 3 and i have a 3 values x 1 x 2 x 3 which takes value 1 3 and 5. so the population mean in this case will become 1 plus 3 plus 5 divided by 3 which is equal to here 3 and number of possible samples which can be drawn by srs w o r here are three choose two which is equal to here three and for a while you assume that the this population mean is unknown to us so we don't know here this year 3 we don't know right now what are we trying to do we don't know this this population mean capital x bar but we want to know its value so what i do i try to take here as a sample and try to compute the arithmetic mean of the sample values okay so here in this case i have three possibilities to draw sample sample number one can be the one and three is unit sample number two can be three and five the unit and sample number three can be one and five unit right so if you try to take the value of here x1 x2 x3 so you can see here x1 is 1 x2 is 3 and x3 is 5. so now if i try to consider the sample mean of the first sample that is 1 plus 3 divided by two this is going to be here two simply the sample mean of uh second sample this is three plus five divided by two which is equal to here four and third sample mean is one plus five divided by two which is equal to here three so now you can see here i have got here 3 sample means 2 4 and 3 okay and now if i try to take the sample mean of these three means so you can see here this i am taking x 1 bar plus x 2 bar plus x 3 by divided by 3 which is equal to 2 plus 4 plus 3 divided by 3 which is here 3 and now you can see here if you observe here the mean the population mean of 1 3 and 5 which was here capital x bar which is here same as here 3. so this is what we mean by the unbiasedness of an estimator we say that sample mean is a is an unbiased emitter of population mean that means if you try to take all possible sample try to find all possible sample mean and then try to find out the arithmetic mean of all the sample mean then it will turn out to be the same as the population mean right so what is the utility of this thing because i will try to ah well i can show you from from here also and this utility is actually more useful from the computation point of view particularly when you are trying to deal with bigger data sets and you know that when you are working in a real life you will have only one sample you will not have sample number one sample number two or sample number three you will have only one sample and you have to work with only one sample so in case if you are trying to see trying to take only one sample mean and you are trying to infer about the entire population you can see here that the sample means are 2 4 and 3 and the population mean here is 3. so that means the difference between this sample mean and population mean this is not actually much in one case this is 2 minus 3 in this case this is sample mean 2 minus population mean this is 3 in the second case the sample mean is 4 minus the population mean is 3 and in the third case the sample mean is 3 and the population mean is 3. actually this is the concept of the bias in the estimator so you can see here if the estimator is unbiased then the chances that the individual values will vary from the unknown population mean large is very very less so usually we expect that the sample mean will be quite close to the population mean so before going going into the further detail let me try to give you here a brief sketch of the proof that how we try to prove it so as we have discussed earlier in the lecture we had computed different types of probability among them one of them was p j i p j i was denoting the probability of selection of the is unit at the gth stage and if you remember we had proved in case of srs w r and w o r that this was 1 upon capital n so now once i try to [Music] try to find out here expected value of y bar so expected value of y bar can be written as 1 upon n summation i goes from 1 to n y i and then means i can take this expected expectation operator inside and i can write down here 1 upon n summation j goes from 1 to n expected value of y j right now the expected value of of any random variable is defined as the sum of the values of random variable multiplied by its corresponding probability so now you can see here y j is my sample value this is one of the value which is obtained from the population so it can take it can be first value it can be second value it can be third value or it can be last nth value and every unit has got a probability 1 upon n pro 1 upon n and so on that is what we have what we already have discussed so now this expectation value of y j will become here i goes from 1 to n y i capital y i because this is on the population p j i so y i so in the next step this p j p j i will be replaced by here 1 upon n and so we have here the quantity inside the bracket as i goes from 1 to capital n y i into 1 upon n and you can see here this quantity is nothing but your population mean so this becomes here 1 upon n summation j goes from 1 to small n times y bar so this becomes here n times y bar upon n which is here same as here capital y bar so this proves that in case of s r s w o r the sample mean y bar is an unbiased estimator of the population mean capital y bar right similarly if you try to take the case of simple random sampling with the replacement in that case also the the similar thing happen the probability of selection of any unit is uh one upon capital n so if you try to consider here respected value of here y bar so that will become expected value of 1 upon n summation i goes from 1 to small n y i and so this can be written here like this and this expectation operator can be taken inside the summation sign so this becomes here 1 upon n summation i goes from 1 to small n expected value of y i small y i now once again this small y i can take what possible value this can be capital y one capital y two or this can be capital y n and all these units can be selected with the probability equal probability one upon capital n so this i can write down expected value of y i here i say y one p 1 plus y 2 p 2 plus y n p n so now this p i are equal to 1 upon n so this becomes here 1 upon n summation i goes from 1 to here n y i or say y 1 into 1 upon n plus y 2 into 1 upon n up to here y n into 1 upon n and this quantity is nothing but your capital y bar so this quantity becomes here 1 upon n summation i goes from 1 to n capital y bar and which is again here and n times y bar upon n which is equal to here capital y bar so once again we can see here that the sample mean is an unbiased emitter of population mean in case of simple random sampling with replacement also now let me try to take the same example which i took in the earlier lectures also and try to give you some more insight that how the things are going to happen in uh in the situation of real data when you have a large number of data under the topic of data science so suppose i am just trying to take a small setup here so i have here 10 observation on the heights of the students in a class so this is my population size so this end now you can imagine that this can be the total number of data set which are available in in the framework of our data science and from there you want to draw a sample of size small n here i am trying to take a small n to be 3. so the name of the students are here a to j and then their heights are denoted by y 1 y 2 y 10 and as i explained you earlier and these values have been taken keeping in mind the convenience so this y 1 is same as here 1 that is 151 second value is 152 third value is 153 and so on right okay now from this suppose i want to draw a sample of size 3 and what is your here the sample value is small y i this is the height of the ith student in the samples so suppose i try to draw here three possible samples so sample one suppose it comes out to be third seventh and ninth students so a small y one is the height of the third student which is 153 small y two second value in the sample is the seventh value in the population which is 157 centimeters and third value in the sample is the ninth value in the population and 59 centimeter and if you try to find out the sample mean of 153 157 and 159 this comes out to be 156.33 centimeters and similarly if you try to take here another example in which i get second fifth or the fourth is 20 in my sample so the sample values are 152 centimeters 155 centimeter 154 centimeter and if you try to find out their arithmetic mean this comes out to be say one fifty three point six six centi centimeter and similarly if you try to take one more sample here in which we select first sixth and tenth student so there the values are hundred fifty one centimeter hundred fifty six centimeter hundred sixty centimeter and you find out its sample mean so this comes out to be hundred fifty and a 55.66 just for the sake of illustration although it is not practically possible means i can here compute my population mean this population mean is denoted by here capital y bar which is based on all the 10 values of capital y is and if you try to compute it this comes out to be 155.5 so now you can see here you have drawn here three possible sample and the sample mean comes out to be six point three three and a fifty three point six six hundred fifty five point six six and if you try to compare these values with the capital y bar is equal to hundred fifty point and fifty five point five centimeters then you can see that there is not much difference between the sample values and this population value and in case if you try to obtain here all possible samples the total number of samples that you can obtain here is 10 choose 3 so that is going to be a big number but if you but as you have computed here y 1 bar y bar 2 up to here you can compute here y bar 10 choose 3 and then if you try to take out the arithmetic mean of all such sample mean this will come out to be capital y bar but in practice this y bar is unknown to us and you have only one of the values among these sample means so you have to work only with the one sample mean so here i have shown you that if you try to choose an unbiased estimator then the difference between the estimated value of the sample mean and the true value to unknown value of the population mean the difference is not much so if you try to hide this information that if you do not know then you can expect that whatever values you are estimating here on the basis of random sample they are not far away from the true population value which is completely unknown to us and this is the basic concept of of unbiasedness from the computational point of view and this is how also we try to test for the for an estimator to be unbiased or not in the simulation also that we try to compute the the means of the values of the estimator and then we try to see whether it is close enough to the population mean or not if it is coming out to be close enough because in the in the setup of a simulation means everything is known to you and so this also gives us an idea that why do we choose sample mean well if you try to choose median or mode harmonic mean or geometric mean they will not come out to be an unbiased estimator of the population mean and one thing what you have to notice here that is very important usually when you try to do any statistical inference you want to do estimation or you want to do something usually in statistics we assume a probability distribution that the sample is coming from a normal population from a binomial population or something else but here in this case we have not assumed any distribution up to now at least we have not assumed that my sample y one y two y n it is coming from a normal population or from a binomial population or a poisson power population so as far as the estimation is concerned estimation of parameter is concerned we are not going to ah make any assumption about the distribution later on when you come to the test of hypothesis and confidence interval estimation then we will require a probability distribution but not now so today i have explained you how to estimate the population mean so the rule is very simple one good distributor is this take a sample and then try to find out its sample mean but now here you can imagine if your sample is bad then it will give you a bad value of the population mean and the value of the population mean is not known to you that is completely unknown that is known only to the god so even you have no choice you have no option to judge whether your estimated where value is good or bad but the only option that is which is in your hand is that you try to select a representative sample a good sample a sample which reflects all the characteristic which are hidden inside the population then you expect that once you try to find out their sample mean those properties of the population will be reflected from the sample mean also i hope i have made it clear think about it try to look into books try to read more and i will see you in the next lecture till then take care and good bye [Music] do [Music] you