Lecture 23 : Univariate Normal Distribution: Theoretical Properties

[Music] Hello friends welcome to the course multivariate procedures with r you can recall that in the last couple of lectures we had talked about random variables and Associated probability density function probability Mass functions in univariate Case bivariate Case multivariate Case Etc now we are going to consider a popular probability density function which is normal distribution but I think I must clarify one thing that there are many many probability density function and probability Mass functions binomial distribution poent distribution geometric distribution beta distri distribution gamma distribution Etc but uh we are going to consider here only the normal distribution and the reason for this is that we are going to use the normal distribution and multivariate normal distribution in the forthcoming lectures when we will be doing the multivariate procedures like classification analysis discriminant analysis Etc and since the objective of this courses not to teach you the probability and probability distribution so I'm not doing it but uh they are uh described in another course which is on the essentials of data science one so if you want to have more details on different types of probability distribution probability Mass functions probability Theory and other aspects you can look into this LP courses or you can consult any other book so now the question comes here what are we going to do so as I discussed in the last lecture about PDF that uh the probability density functions they are going to describe the distribution of the probabilities under different type of situation and this situation depends that how the process is is happening we have no control over the process the process is happening in the natural way we have to Simply observe the phenomena and based on that we have to take some observation and based on that we have to see the probabilities and different type of situations require different types of probability function probability density function or probability Mass functions so among those the normal distribution is uh one distribution which is very popular it is popular because of its nice properties nice statistical properties and most of our statistical inference procedures they are based on the basic Assumption of normality or the normal distribution that our data is is drawn from a normal distribution so that is why it is important for us to understand about the normal distribution its important properties and how are we going to uh implement it in the our software well I'm not saying at all that other probability density functions and probability Mass function are not important or they are not useful they are equally useful but we are going to use the normal distribution so that is why I want to prepare this background in case if I need any other distribution or if I feel the need of any other probability density function or probability Mass function at any point of time I will explain you so with this objective let us begin our lecture okay so we are going to consider here the univariate normal distribution and we are essentially going to consider its theoretical properties basically right I will try to give you example also but mainly we are concentrating on the theoretical property and yeah I will not be giving you many proofs but I will simply be giving you the details of the final result so the normal distribution is one of the most important distribution used in statistics and this distribution is also called as gausian distribution right and uh the most widely used model for the distribution of a random variable is a normal distribution you will see that very often in most of the books they always write let X1 X2 X1 be a random sample from normal distribution let the random errors follow a normal distribution Etc so now the question is what is this normal distribution and how the probability density function of uh normal distribution looks like right so a random variable X is set to follow normal distribution with parameters mu and sigma squ if its probability density function is given by like this say 1 upon Sigma < tk2 Pi exponential of - x - mu s / tce of Sigma s and the range of X here is between minus infinity and plus infinity the range of mu is between minus infinity and plus infinity and the range of Sigma square is between zero and infinity so here you can see here this mu and sigma Square they are the parameter right and these parameter try to explain all the properties of the probability distribution and other properties of the random variable and our objective in real application is to know the values of uh the parameters of probability density functions based on the random R om sample or the data set what we observe during the experimentation so we express this normal distribution as that X follows normal mu Sigma square like this this is a standard notation right and well I'm not going to give you here the mathematical derivatives or mathematical derivation of different properties of normal distribution but I will tell you the important properties of the random variable X so the mean of where X follows the normal distribution it is Mu so this is called as expected value of x Is mu and the variance of X is Sigma Square well you can do some mathematical calculation and you can obtain it very easily and if not you can look into any standard book on statistics and you will find the uh all the details to that how to find out these things now the next question comes what this mu and Sig Square are trying to indicate so this value this mu that is expected value of x equal to Mu expected value of x mean this is actually the mean the mean of X is Mu actually this determines the center of the probability density function and this quantity Sigma Square which is the variance of random variable X this determines the width of the probability density function right now there is another variant of uh normal distribution which is very useful and popular it is standard normal distribution in case if I try to choose mu to be zero and sigma Square to be 1 then X is set to follow a standard normal distribution so if you try to put here this mu here to be zero and the sigma Square here to be one then you will get the probability density function of standard normal distribution right so now you can see here that now both the parameters are known and so the variable random variable lies between minus infinity and plus infinity and we write it as like this X follows normal 01 right so now we try to look at the structure of this probability density function that how it looks like and uh it is important to understand the structure of this PDF and it will help us in Computing different types of quantities and uh it will make us understand that whenever we want to do something something how we have to adjust different types of uh issues for example if I want to compute a particular type of probability by understanding this curve we can compute them easily so that I will show you in the next slides right so if you try to see here this line where I am moving my pen this is the density of normal distribution right so you can see here that this is your here then it curve this density curve is symmetric and bell shaped what do you mean by bell shaped you know the bells are like this and uh yeah it is here like this so this structure is similar to the normal curve right and if you try to look at this middle value where I'm trying to move my pen in red color if you try to look here this is the point here at which this curves attains the maximum value right so this point is here actually the mean me and you can see here that this if you try to divide this curve into this two parts from in on the this side and on this side I will try to remove these lines so that I can make it better and neat presentation right then you can see here that this density is symmetric around mu whatever be the distribution in this area this is the same in this area right so this is what I mean to say and uh now if you try to understand here if you look at this point here mu and then if you try to see this spread that how the values are spread then this spread is trying to give you the value of variable that is Sigma squ and so this point here is Mu minus Sigma and this point here is Mu + Sigma which are called here as a inflection points right and in case if you try to divide the entire density into this two equal parts then this part is 50% and this part on the right hand side it is also 50% of the total area right so now let me try to give you here an idea that by changing the value of Sigma Square how the Curve will look like so as I said that Sigma square is the variability so if I try to make here a line and suppose the mean is here at zero right so then this curve belongs to a value of Sigma square is equal to 0.5 and C me WR here c number one two and here three curve number two here is normal 01 where the value of Sigma square is 1 and in the curve number three the value of Sigma Square here is two so what you can see here this spread from the centrer value this is spread of curve number one the spread of curve number two let me write it 1 one 22 and the spread of here curve number three right you can see here that as the value of Sigma square is increasing the spread of the values around the mean that is becoming more and the shape of the curves changes like this that become more flatter right so a higher value of Sigma squ indicates a flatter density and a lower Val value of Sigma Square indicates a higher concentration around the mean V mu or say zero for example I have taken here but the mean is mean right okay now I try to give you here one more situation which we will encounter when we will try to develop some multivariate procedures in the classification problem you can see here I am drawing here the curves please try to look at the number which I'm trying trying to give here suppose this is your here curve number one and this is same as this curve so this is uh your here normal density curve where Sigma square is equal to 1 right so both are similar but the issue is that their means are different the mean for the first curve is at five and the mean of the second curve is at 15 and even if you try to to see here in the first curve the mean remains here at five but the variance is changing in the first curve it is Sig square is equal to 1 and in this curve which I am trying to highlight this Sigma square is here four right and in case if you try to compare here the curve number one and this here curve number two which I am trying to write down here if you try to compare this curve and this curve you can see that they have got the same mean but different variances and in case if you try to compare curve number one and curve number three which is here like this then you can see here they have got the same variance but different means so sometime many students ask what is the relationship between mean variance if mean happens to be this what will happen to the variant etc etc so this curve is trying to give different types of answer to such questions okay now another important aspect the cumulative distribution function of X following a normal new Sigma Square so by definition you know that CDF is given by this expression Capital FX equal to minus infinity tox 5T DT well I can inform you here that small f is usually try to indicate the normal PDF in many books you will find this symbol so I have just Ed it here like this and uh the CDF of a normal distribution is uh indicated by Capital 5x like this so that's a standard symbols and motation to denote the CDF of normal distribution now in case if you try to write down the expression for this here ft then you will see that there is no explicit formula to solve the integral but it has to be solved numerically using some computational method that is why the values of this quantity for different value of x have been tabulated and in many books the tables for the cdfs are presented right and it is a very important concept because when we will try to deal with test of hypothesis or many other things you will see that this CDF is going to play an important role anyway mean earlier people used to uh do it from the table but now we are going to compute all the things using the r software now an important result that in case if x is following a normal distribution with mean mu and Varian Sigma square and if you try to take any two constant A and B which are not equal to zero and if you try to Define a linear transformation like as y = to a + BX then y will also follow a normal distribution with mean here A + B mu and variance b² Sigma squ right so expected value of y is equal to expected value of a plus BX a is a constant so expected value of a will remain as a and this will become here B into expected value of x which is equal to mu and the variance will become here like this B squ variance of X which is B sare Sigma sare so many times you will see that we would like to make a linear transformation from a given variable and in those case cases and situation this result will help us now another very important result which has several applications if x is following a normal distribution like X normal mu Sigma squ then in case if you make a transformation like x - mu / Sigma that means x - mean divided by standard deviation then the distribution of Zed will be standard normal that is normal 01 and this is a very popular transformation which is called as Zed transformation and the advantage here is that the random variable X is dependent on the choice of parameters mu and sigma Square where Z is independent of the choice of the parameters and its mean is known to be zero and its variance is known to be one so this result helps us in finding different probability statement about X in terms of probabilities of Z that we will try to see with the in many application that it is going to be useful and many times this concept is also used in normalizing the data you can recall that when we were considering the turn off faces in graphics then there was a constraint that the data has to be normalized so this is one way by which we can make the normalization normalization has an advantage that the new variable becomes independent of the unit for example you can see here x may have a unit but when you are trying to subtract it by mu it will have a unit but when you trying to divide it by Sigma then this becomes unit free right there are several advantages but anyway this is one among them now I am would like to show you that uh by looking at the graphic how can you uh understand different results about computation of the CDF of normal mu Sigma square right so you can see here if you try to look at this graphic this is here zero so on the left hand side it will take a negative values and on the right hand side it will try to take positive values so now it will be our convention that with X I will will be denoting the normal mu Sig Square the random variable associated with normal mu Sig square and with Z this will be the normal distribution with me Z and variance one so if you try to see the CDF of Z is given by here this definition but this you cannot solve so we try to compute it by some numerical methods now the first result is 5 - Z is equal to 1 - 5 Z so if you try to see here what is here five if I try to create here this probability density function and if I see here this here is is the point here minus Z and then this area this is actually here the f z and now X follows normal mu Sigma square and Z follows normal 01 both are normal so both are symmetric also symmetric means if I try to take this supposed distance and at the same distance if I try to make here a line on the right hand side then this point will become here plus of Z and then this shaded area will also be equal to F of Z so that is what this result is trying to tell you that in case if you want to find out this 5 minus Z this will be the area over here and that total area of the curve this is actually here one so this is actually here 1 minus this curve and this area here like this so that's a very simple rule of integral but it is very helpful when we try to compute different values of uh in the our software or or from the normal uh tables so 5 - Z + 5 Z is equal to 1 for all Zed and 5 0 is equal to 1 by two that is obvious 5 0 is the value if I try to highlight here you see here this area so the total area under the curve is one so half of the area will be 1 by two straightforward result so now in order to compute this type of probabilities the tables are available for f z and you have to look at the value of Z and the corresponding value of f z will be uh will be available for example if you try to see this I'm just giving you here a one sample suchar computation so here are the values of here Zed and based on different values of here some critical value these values are given here like this but you can see here if the value of Zed here is zero and the probability here is 0.5 so that means the values have been computed from minus infinity yeah and yeah one thing I forgot to tell you on this side this will be here minus infinity and this side it will be your here plus infinity right so these are the table which are available in most of the book but anyway we are not going to use here the table but we will try to see later on that how we can compute them in in the our software but in order to understand that computation we need to understand this rule right so if you try to see here this is here the point 1.5 and if I try to say here integral of minus infinity to 1.5 this here 5 Z DZ so so this will be your here 5 1.5 and that you can obtain from the table without any problem right and similarly as you have seen if you try to take here the center part here as a mu then if you try to go either in the left side or in the right side of the MU this a sigma will be increasing and so if you try to see here this is the area which is here mu minus Sigma and this is the area on the MU plus Sigma right so these limits muus Sigma mu+ Sigma are there and similarly you have here muus 2 Sigma mu + 2 to Sigma in which this area is covered and similarly this is here area between muus 3 Sigma and mu + 3 Sigma so actually the area covered between mu + minus Sigma is 68% of the total area the area covered between mu +us 2 Sigma it is 95% of the area and the area covered by mu +us 3 Sigma is 99.7% of the total area and if you try to see the way I'm telling you mu +us Sigma mu +us 2 Sigma muus 3 Sigma actually these limits are used in different places for example in the quality control we always talk of one Sigma limit two Sigma limit three sigma limit so this uh limits are are actually the 1 Sigma 2 Sigma and three sigma uh limits for example this mu plusus Sigma is the one Sigma limits mu + - 2 Sigma is the two Sigma limits and similarly mu + - 3 Sigma is the three sigma limit and they are very useful you will see later on and now I'm just trying to show it here graphically that okay if you try to see here this is my here mean zero then this area is is here minus infinity so minus infinity to here minus a this green area is given by here Alpha for example right so this area here is Alpha and similarly if you try to see here probability that Z greater than a so Z is greater than a so a is here like this so this area will also be Alpha because probability that Z minus Alpha and probability that Z and this Alpha they are going to be the same and similarly if you want to find out the area between minus a and plus a so this is your here minus a this is your here plus a and this green area is going to show this area so if you try to see that if you try to say that the whole area is one and suppose this shaded area which I'm trying to shade here on the right hand side and left hand side suppose this is Alpha by 2 Alpha by 2 then the area in the middle part will be 1 minus Alpha right so similarly if you try to see into this curve number one so if this area is here Alpha then this area will be 1 minus Alpha and similarly in the c number two here if this area is going to be 1 minus Alpha right so this is for your understanding and as I shown you earlier you can see here probability that Z greater than equal to a which is here this probability is equal to 1 minus 1 means the whole area minus this this area so this is here 1 minus probability of Z less than equal to a right similarly if you try to see what is this area probability Z greater than minus a so this is here your minus a so now this is here Zed so this this is here green area is your probability Z greater than minus a and if you try to see here if this is here a then the probability that Z less than a is here this green area right so you who can see that both have the same value right now uh in many situation what will happen that sometime you are given probability distribution normal mu Sig square and you want to compute different values in order to know those values it is important that you should know the value of mu and sigma square but that is pretty difficult so in order to solve this problem one option is that we try to standardize them or we try to normalize them by x - mu upon Sigma now this becomes your normal 0 1 and somehow if I can convert X into x - mu upon Sigma then we can solve the problem for example if I want to compute here probability X less than equal to B right so you can see here I just simply try to write here like this x less than equal to B I try to subtract mu on both the side and I try to divide on both the side by Sigma so this is what I have done here now this x - mu upon Sigma this is now here Z and this B minus mu upon Sigma that will be some known value so now you can just uh look at this value from the table of CDF so this is how you can compute the probability without knowing anything you simply have to just uh know the value of mu and sigma and then correspondingly you can just this will become a constant and you can find out its value from the table and similarly if I say here probability X greater than equal to a so this I can express a 1 minus probability of X less than a now X less than equal to a it is like I try to subtract on both the side by mu and divide it by here Sigma so now the this will become here say 1us 5 of aus mu upon Sigma so these different results will help you in various types of computation and calculation similarly if you want to know here the the value of probability of x l between A and B the same thing I can do here I can adjust the terms with respect to B and A and B terms with respect to mu and sigma so now the central part just becomes here Z and then you can find it by here F of B minus mu minus 5 of aus mu upon Sigma and similarly if you want to find out this value here probability that X line between minus a and plus a so this will be simply your here 5 a minus 5 of minus a and 5 of minus a is your here 1 - 5 so this will come out to be here tce of 5 minus one right now we come to an end to this uh lecture and uh you can see that we have considered here the univariate normal distribution and uh we have considered its theoretical properties these simple different rules will help you in later on in the competition of different types of probabilities so now it's your turn try to just do this exercise yourself and try to understand these properties the next question for us is that how to compute these quantities in the our software that we are going to handle in the next lecture so you try to practice it and I will see you in the next lecture till then goodbye [Music] [Music] [Music]