Lecture 44

In this lesson, we'll discuss the applications of logit model. To begin with, we'll introduce multinomial dependent variables or what we call as multiple discrete choice variables. Next, we'll discuss Tobit class of models. In the subsequent videos, we'll implement a series of six case studies namely homeownership data case study, group data case study, ATM debit card data case study, grade point average data case study, smokers data case study, and education data case study. These case studies include linear probability models, logit probit models, and multinomial logit model implementations. Till now, our discussion on classification algorithms approaches such as logit probit approaches have focused on situations where dependent variable has two categories and we could classify them into ones and zeros. Let us now focus on multinomial linear dependent variable approaches where the dependent variable can have more than two categories and or we can say more than two choices. Think of a couple of situations. For example, a choice of firm planning to list on NYSE, NASDAQ, or Amex. So they have three choices or as a mutual fund investor choosing between AAA, AA, or A-rated securities. These kind of situations represent themselves as discrete choice multiple discrete multiple choice problems. In such cases, the agent chooses that alternative which maximizes her utility. Now just to give you the reference, when we had two choices we needed one equation where ultimately one became reference choices. Similarly with three choices we'll need two equations where one choice will become reference model. Let us think of each of these three choices as a combination of two choices where one is the reference category and therefore each choice becomes sort of relative choice between these two. Let's take an example. For example, we start with the choice between investing in AAA, AA, or A-rated securities by a set of investors. Now think that there are two particular variables are important in this kind of modeling or this choice is dependent on a leverage variable. So if leverage is high probably you would expect a higher return or this may affect your choice of the security. Also expected return so these two leverage and leverage represents the risk and R is expected return. These two variables represent the choice of going with a particular set of security whether A-rated, AA-rated, AAA-rated. Now let's suppose this AI, AAI, and AAA they represent utility function of investing in securities rated A, AA, and AAA respectively with alpha 1, alpha 2, alpha 3, beta 1, beta 2, beta 3, and gamma 1, gamma 2, gamma 3 parameters. If you think of a dummy variable approach, recall a typical dummy variable representing security A would be where value 1 when investment is in security 1, 0 otherwise. Similarly one can think of it AA as which takes value 1 when investment in AA is 1 and 0 otherwise and similarly for AAA we can define that. Now let's consider category A as a reference choice. What we can do is we subtract AA from A to make a equation that captures the essence of both of these and AAA-A which can capture the essence of AAA-A. So we'll get a question like this. Now instead of modeling the original utility function AI, think of AI as similar to that ZI variable in the logit model where if you recall probability were modeled in the logit approach like this where ZI was the utility function. Think of this AI as utility for investing in AI, AAI as investing in AA and AAA as investing in AAA. Now with the same, in the same beam I can take the difference and the transformed utility. What is this utility? My preference to invest in AA as compared to A. This is the difference of this beta 1 minus alpha 1 plus beta 2 minus alpha 2 LI. Now this is your transformed ZI function or rather one can say some let's call it some kind of utility function Z1. We can also call it in fact AAI minus AI as a notation for this utility. Similarly AAA minus A is the difference this as a ZI function. Now if these are the transformed revised utility functions we know how to model them. Why we do not need the third equation to model separately for AI because we know that probability of AAI plus probability of AI plus probability of AAAI this is has to be one because these are the only three choices. So once two are determined the third is automatically determined. Now let's think of this Z function utility function of this form probability of sort of odds ratio where this ZI represents this function and recall we model this as 1 plus e to the power minus ZI. The probability was modeled like this. In this case if this value is very large that means there is a higher probability more and more probability to invest in AA. If probability is low then that person would invest in A. So this is the typical functional form. Similarly you can think of AAA as well in this form where probability to invest in AAA where event of interest Y equal to 1 would invest would represent investment in AAA while Y equal to 0 would represent investment in A and this would be your typical probability function which you are going to estimate and here if ZI is too large that means this value will approach towards 1 and if ZI is too low this value will approach towards 0 that means investment in AAA and A respectively. Now these equations can be these two equations can be estimated with maximum likelihood estimate and third equation is automatically estimated. So this multinomial system of equation will appear like this. This is the first equation the logit for first so this is sort of logit representation you should call the logit this is the logit representation of equation 1 this is the logit representation of equation 2 they call that odds ratio was sort of logit only so we can represent that in this fashion and the third equation is automatically determined. So we have our system of equation to summarize in this video we saw the multinomial approach to solving this linear dependent variable problem. Now in this case we combine three equations we had three discrete equations but they were driven by this particular constraint that the sum of all the three probabilities were equal to one we could solve two we could formulate a system of two equations we solve these two and with the third this property we could obtain the probability for probability for AI. Similarly one can extend it for the multiple choice case where more than three choices are there. In this video we will discuss a very similar approach similar to logit probability approaches which is Torbett model this is often used to model censored or censored kind of models we will see what are these models. Many times it may happen that some of the values of the dependent variable are not observed. Take for example suppose you are somebody who has a lot of income and would like to pay charity there would be some people whose income is so low that rather than taking charity they would like to rather than giving charity they would like to take charity however taking charity would not be observed and for all such cases you have a y value of 0 if you are trying to model the relationship between charity and income these observations may not be available similarly think of a firm with dividend payment and relationship with market cap for large firms with large market cap large cash flows they may pay dividend however a firm does not have a large market cap or steady state of income probably rather than taking in giving dividend they would like to finance borrow money and in that case dividend would not be paid to be zero but their utility is to take money rather than give money so such observations would not be there now Torbett model provides a very interesting approach and argues that rather than ignoring these observations one should use those observations however the problem is if you have such observations at best what you can do is you can put a value of 0 to the dependent variable if you want to use that the RHS of the model will remain the same you have you observe the attribute Xi all that you do not observe Yi so you constraint it to 0 like if dividends is not paid you take it as 0 but still you use the attribute Xi and model it however modeling such data with regression model creates a problem simple typical OLS approach is not very suitable so what you can do is instead of using OLS you model such relationship with maximum likelihood approach the maximum likelihood approach on such censured data where a particular side is not observable is what the Torbett approach that is why it is also called censored regression now here we are using a cutoff or threshold limit of 0 it may happen that threshold can be on the upper side so Y can be restricted at an upper value maybe a lower value which is not 0 so Y can be observed only when it is greater than a or the threshold or censuring can happen from up and both sides so below B and above A for example in logit probit approaches we had this 0 1 kind of thing so of values could have been only between 0 and 1 if it is probability so in this fashion you can instead of using OLS you can use your MLE approach to model such relationship and not lose the observation this is called Torbett model or censured regression approach let us start with the homeowners chip case study example so this case study example we would like to understand the linear probability modeling first we'll read the homeowner data in R the data set comprises observations with discrete variable denoting homeownership Y equal to 1 if family owns the house and 0 if they do not own the house the data set also has a variable representing income X in rupees lakhs we'll begin by running a simple linear probability model with ownership Y as the discrete variable dependent variable and conduct the following analysis first we'll summarize the linear probability model interpret the coefficients and understand the relationship between income and ownership then we'll find the probability of homeownership when income is rupees 12 lakhs next we'll answer the question what happens to homeownership probability when the income is more than 19 lakh for example 20 lakh or more or what happens to the homeownership probability when the income is less than 9 let's say for example 8 lakh subsequently we'll examine the fitted values of the model these are the probabilities and we'll assess the modest performance we'll visualize the residuals to identify issues related to normality and heteroscedasticity subsequently we'll conduct formal test to assess the presence of heteroscedasticity in the model we'll conduct weighted least square regression analysis using residuals then we'll compare the results with the raw results and those with heteroscedasticity robust standard errors with this we'll conclude the case study from this video we'll start with our logistic regression exercises we'll start with the homeownership case data and case execution so first we'll set the working directory we'll put the working directory where all our data and required information is so here it is will I'll for reference I'll copy paste this code so that in future I do not forget the location so I've just copy pasted this now I'll be needing some libraries for these exercises they will be used in other case studies also so I'll load all the packages so first step we'll load the relevant library so one is tidyverse that we are going to need tidyverse second is margins third is pscl these are some of the relevant libraries have not used all in the first session but still they are useful so we are going to load them rms that is also useful now some of these libraries may not be available as installed packages so I need to install some of them packages now I've already installed them so I can go ahead after rms I'll load lmtext then mozilla another useful library I'll use is unit then car lastly our useful library sandwich so I'm not going to use in the next session itself but still I'll load them all in one go now let's start with in this case first what we'll do is we'll load the data so we'll start the exercise by loading the data we have this data here so I'll load it for reference I'll just copy paste this command and as a preference I do not like this syntax so I need like more simpler this kind of syntax so if you want to have a glimpse of this data what you have so this contains family numbers so there are 40 families and you have ones and zeros indicating homeownership and their income in lakhs so let's start with the LPM exercise this is a typical LPM example let us start by modeling this relationship as a simple linear LPM model linear probability model all I need to do is use this model and take this linear modeling LM Y till X as a function of X and data as homonym so if I run this and I'll summarize somebody model now let us interpret the output so to begin with notice this coefficient of minus 0.9 4 5 6 9 now we are modeling probabilities so effectively this suggests that on average a person with zero income or have probability kind of negative probability that a person will own a house is negative we can take it as 0 so that means if a person has zero income probably obviously probably will not be or her will not be owning a home so at least in a normal case they will not be so this negative probability can be taken as a zero chance of a person owning a home if they do not have income more importantly for us you notice the value of X which is 10.213 which sort of indicates that for each 1 lakh rupee why because X has units in rupees lakhs so for each 1 lakh rupee increase in income the probability that X increases will be 10.21 percent now if you have gone through the lecture discussion you'd realize that this is this probability cannot be fixed it depends on the income for example for a person at a very large income maybe 1 crore or something 1 lakh rupees increase would not marginally contribute to their probability in owning a home similarly person has very low income maybe almost no income and their income increases by some 1 lakh rupees it may not affect their probability of owning a home but still this is our typical linear probability model so it suggests that for each lakh rupees increase whatever the their increases irrespective of the starting point for each 1 lakh rupee increase the probability of home owning increases by 10.21 percent that is how we'll interpret this result in this video we'll try to answer the question at different income levels what is the probability of owning a home so what if income is 12 lakhs so if income let's say income is 12 lakhs we have this income variable now you need model coefficients so the idea is to use the model coefficients to make these predictions so you can see we have these first first variable is constant number and plus the second variable is so you can print it also to check yes it is second variable is the slope variable and this variable we need to multiply with income so we'll get 27.98 so this tells us that if the income level is 12 lakh the probability to own the home is 27.98 percent all result kind of 28 percent similarly what if income is let's say more than 19 lakhs let's say 20 lakh rupees let's say income is 20 lakh rupees so if income is 20 lakh rupees then I can simply change this variable so it is more than one so because this value is more than one we can take it as one or rather hundred percent chance which sort of in practical sense a very high chance of owning the home similarly if income is pretty low let's say less than nine or eight lakhs maybe so if income is eight lakhs it means this value will be eight and if I run it negative which means a very low value of income at this low income homeowning probability is almost zero now in the next part we are supposed to examine the fitted values examine the fitted values the fitted values are probabilities and easily we can extract them with this fitted value so there are many such values 40 values for each value we have a fitted value you can see some are more than one some are less than zero also so I would like to summarize them probably that would give me a good sense of how these values are structured and make a claim so if I do that if I summarize you can see values minimum values minus three point zero point thirty nine and maximize one point three it means quite a lot of values are more than one and quite a lot of them are less than zero one good way would be to just plot them and see how they behave so you can see they are quite structured first and foremost they should give you that intuition of heterodoxy there is the fitted values are changing a lot but more importantly see we look at zero like there are a lot of them below zero and a lot of them above one so that violates the thing but still you can convert them the ones above one as one and lower than zero at zero to get some idea about your data some properties now we would also like to visualize our residuals to see if there is a issue of non-normality and heterodoxy the simplest way to visualize that is so visualize the errors now the simplest way to visualize errors is to just plot raw errors we'll do a lot of plotting of errors but first let's start by just plotting the errors so you can plot the residuals and you can see they are very weirdly behaving a lot of heterodoxy at least visually it appears but a more compact way is to plot this model now this gives us opportunity to visualize four plots summary plot the first plot is residual versus fitted so you can see when fitted values are changing residuals are behaving in a very weird way you can see this very clear that there is a heterodoxy there is a change in variance and also they are very sort of systematically moving with fitted values so that is one second plot that is available to us is QQ residuals so you can see that particularly at extreme levels though in between the residuals are well behaved but on extreme levels there is some odd behavior non-normality this QQ plot helps us identifying deviations from normality so they are residuals are deviating from the straight line straight 45 degree line they can be considered as non-normal so we have some evidence on non-normality non-normality particularly on the positive extremes so that is one now you have more plots again to help you visualize for example the scale location plot again suggests heterodoxy so we have sufficient evidence of heterodoxy in our data although we can conduct some formal test also there is one test which is called BP test now BP test you can put your model and see what does it tell so BP tell BP test a Bruce Pagan test of heterodoxy although the test we are not able to reject the this thing so at least from this test wise we are not able to say but visually we saw there is some issue related to heterodoxy now there are two ways to account for this heterodoxy problem for heterodoxy problem first and foremost I can make use of robust standard errors so that is one way for let's start with this robust standard error approach robust standard error now all you need to do is you need to produce will apply the square term test which is in LM this package provide your model and then write the kind of heterodoxy you want to get so for example there is one VCOV HAC you can read more about these for example you can type question mark and read about this head this approach of correcting standard errors for example this is for heterodoxy and autocorrelation consistent standard error matrix so you can print your coefficient matrix and look at this coefficient matrix t values and also you can produce the original one so original one coefficient matrix can be produced like this you can compare this so here you are getting p value of t value of well and look at the standard error so that is one this is the original value and now when I apply this coef test and look at this let me enlarge this so you can see this t value has increased and standard error has come down so that is one thing to be noted also you can change the form of this variance covariance matrix for example the another another way would be to use pure HC only for heterodoxy and again just compare it this is 12.515 for HAC you have 14 point and this is 0.07 and here also t value is larger the point that we are trying to make that when we account for heterodoxy then standard errors are lower and t values have increased that means our power of test has increased so that there was some problem of heterodoxy which was corrected there's one more approach which is called robot apart from robust hundred is one more approach which is called WLS approach in the WLS approach you look at the magnitude or standard deviation so as a standard deviation measure I am just using their absolute value that is also a good way to account for the scale so all I'm doing I am just taking the magnitude sort of size of my errors and now I'll be correcting my model let's see how so I am running my model but what I am doing is first and foremost I am transforming my values I am adjusting the values each value for example y value here by this y value I can divide it by W what is W this is size of errors absolute magnitude of error so I'm dividing the vice with corresponding error sizes and then I need to remove the constant term I am removing this because I can replace this original raw constant with weighted constant so how to wait I wait this I divide 1 by W so original one constant would be now replaced by a value of 1 by W so that is sorted now the last thing I need to do is I need to also divide my X values with W so with this the complete thing is there now I'll put my data and now I'm good to summarize so now I'll summarize my let's call it okay so rather I should have put it model underscore WLS to differentiate from original model but now I need I will be needed to again create my original model so before I summarize this I go to my original model that is first now yeah so now I have summarized that and I'll summarize this WLS so now let's compare so again look at the original values 12 and point double zero eight now here see the drastic change almost four times my standard errors have decreased four times and as a result my t-value has increased significantly so this way my model has improved quite a bit in fact you'll look at the adjusted R-square which is a measure of fitness has increased like anything so overall the point is that my model fitness has improved and which shows that there was a problem of heterogeneous in the data so such problem and even adjusted R-square is not a good measure of model fitness because this is a typical linear probability model so we saw some of the issues if I model this one zero kind of variable which was my home ownership variable which was sort of in the form of probabilities with the help of LPM which is not a good approach but still we have seen this approach in next set of case studies we will try to look at the logit probit approaches also in this video we'll examine the logit approach to modeling the group data if you recall our discussions we said in group data the log of odds ratio is the linear function and therefore there is an opportunity to still use our linear modeling of WLS approach so in this case we'll consider the group data on income ownership the data is grouped according to the income level X it carries variable income X in rupees lakhs per annum and total families who own home by number of families that own a house we'll read the data in R and conduct the following analysis now construct the logit variable from the given data so we'll construct the logit then model in the linear fashion because the log of odds ratio is logit is linear in nature then we'll summarize the model interpret the coefficients to understand the relationship between these variables we'll also examine the fitted values of the probability to assess the model's performance next we'll visualize the residuals to identify the issues related to normality and heteroscedasticity we'll conduct formal tests to assess the presence of heteroscedasticity in the data we'll also conduct weighted least square regression analysis using residuals we'll compare the results with the raw results and those with heteroscedasticity robust standard errors we'll also examine the ownership probability when income is 20 and 40 lakh rupees and with this we'll conclude the case from this video onwards let us start with the group data exercise so this is our part two let us read the data first so the data is income ownership so we'll read the data for reference as usual we'll copy paste this this will be our data so as a just let me simplify the data in fact this quite a big name for a variable so income so I'll use this data now as a just for a picture I'll just use this glim command to see what is inside you can see first column is income which contains the income level in lakhs then total families and out of it those who own the home so this is the data given to us first and foremost we need to calculate the probability PI that a family owns a home in a given income category this is quite simple we use our income ownership data and in this we will create a probability variable PI which is nothing but it's very simple so it's the firm number of family that own the home so the way to put it is own so see own the well family that are owning the home divided by total family so it's quite simple so this will be my probability and now I want to compute the likelihood the probability that families own home so that is let's call this variable Li likelihood that families the own the home or odds of family owning the home log of odds rather logit function log of odds so log of odds is simply this value which is the odds in favor divided by 1 minus odds again so odds in favor and odds against that is the odds ratio and if I take the log of it it becomes my typical logit function so these are the values that I have now things are simple we know that this log of odds is a linear function is a linear kind of function so how to model it so I can model as we said this Li is that as a linear function of income so notice we said that log of odds is a linear function of predictor variables so we can model it like that now data is equal to income ownership model now just somebody of model let's summarize this and we'll interpret the output now first and foremost notice the cone intercept term and slope term so first we can directly say that for each unit 1 lakh rupee increase in income the odds ratio increases by approximately 7.9 percent why because it is in log of odds so we can directly interpret in the form of percentage so odd log of odds ratio so it's odds ratio increases by 7.9 percent second a person with no income a person with zero income will have their odds as log of odds is minus 1.65 so if you want to know the odds actually you can do the exponential of this which would be a rather low number but still so you can take the exponent of this and say that this is your odds ratio you can get the odds by taking exp exponent of this number to say a person with zero income what would be the odds of them owning the home now that we have interpreted the output let's try to answer the question what is the probability of a what is the probability of owning home if income is 20 lakh rupees the problem here is that we will get the output in the form of odds ratio for example if I apply the typical model dollar coefficients and first is the intercept term then and I have decided income as 20 so if I multiply this as 20 what I get is my odds ratio log of odds ratio other let's call it Li so this is the log of odds ratio that you are getting this is not the probability we have to back calculate probability using this let's suppose we consider our probability as PI we need to use this make fun g equal to make function and this is a very useful function here we will take typical X as problem let's say if X is my probability then we are solving for X we want to solve for X if I subtract from a like this is the function that we want to solve index so this is the function that we want to solve this is the function that you want to solve and here you need to find the value of probability PI for zero so there which is called find zeros and find zeros for DX G as a function of X my is a function of X and we need to provide some guess value so we'll give a ring because we are finding probabilities that has to be 0 and 1 in the range of 0 and 1 and let us solve for so we'll first run this and then this and see if it gives us the solution so it gives us 0.4812 which means that if a person person for with an income of 20 lakh the probability to own the home is 48.12 percent let's see what happens when this income is 40 lakhs so let's increase this to 40 so rather income if I run this I open source yeah 82 percent almost 82 percent so you can see now you need to convert the likelihood number back to the probability if you want that relationship getting likelihood was easier a simple equation would have given us the likelihood like likelihood was 1.50 but he had to back calculate it to the probability recall this is not the marginal impact but if you are in case you are interested in getting the marginal impact you need to further use this probability and if you recall the marginal impact was beta the formula was marginal effect in this formula was beta into probability into 1 minus for logit function this was the typical probability so this is the marginal effect if you are assuming the logit function now therefore you need to first use the coefficient marginal coefficient into 1 2 into P I into 1 minus P I so this will give you a typical value the marginal impact now notice this marginal impact is quite less that is at 20 lakh per unit increase in 1 lakh rupees increases the probability by 1.1 percent what if the income is 40 so I need to okay so I need to do is for 20 lakhs so this is 2 percent actually close to 2 what if the income levels are pretty high so let us do that so here we saw the output is lower so it shows you that this is the marginal impact so as the income increases the marginal effect the marginal utility of income is lesser and lesser in this video we'll start the discussion by analyzing the fitted values of probabilities so we'll examine the fitted values of probabilities now recall in this kind of model we are modeling log of odds ratio or the logit function in the previous video so what this model gives us model dollar fitted values is those log of odds log of logits or rather log of odds ratio which is logit if you recall probability in the model P I is nothing but 1 upon 1 plus e to the power minus those logits so this is the formula which is same as 1 upon 1 plus e to the power minus Zi if you recall the function and this will store our fitted values probability fitted values so if I summarize these probabilities from the model they range from 0.2344 to 0.818 in fact I can visualize all of them I'll just put print all the values and you can see all of them are between 0 and 1 so that means our fitted values nicely aligned to the probabilities in the next step we would like to visualize the error terms visualize the errors to visualize the errors one simple way is to plot the model and so let us view the plots I need to increase the plotting margins let me run the command now so you can see this is residual versus fitted so you can see quite a lot of heteroscedasticity I can see QQ residuals more importantly scale location so there is some evidence of heteroscedasticity in fact I can see here residual versus leverage so while this plot does not show any major concern so leverage is not much so we have visualized the residuals let's let's conduct some formal test for example one way to do that is BP test of the model so we'll conduct the BP test of the model okay so the BP test also is not any major there's also something called non constant variance test that also you can conduct it may give us a similar result similar to BP here also we find that heteroscedasticity is not a major issue so heteroscedasticity does not seem to be a very big issue but still as the convention goes requested that we conduct the WLS kind of test weighted least square as well as heteroscedasticity consistent testing of the model so first we'll what we'll do is we'll use robust standard errors for that we need to do coef test model and we'll use two approaches one is VCOV one is this approach another is VCOVHC approach only heteroscedasticity and the last one is our typical good old model so this is our robust standard error approach standard error approach so and we'll also summarize our typical model here so let's see what happens so in our model T values and P values are given if I compare it okay so they are not very much improving so I can see they are not very improving this can be ascribed to the fact they are very similar in fact so this can be it is 19 here earlier it was 16 so it can be ascribed to the fact that in fact heteroscedasticity was not a major problem but still we tested our results are robust we are getting similar T values so that means our results from that model is robust in fact we can also do the weighted least square WLS procedure now one way to do the WLS thing is to use the magnitude of error term to account for the heteroscedasticity so that for that absolute values of the model residuals now this is the magnitude of the residuals so what I'll do is I'll create another model let's call it model WLS and in this I'll adjust the scale of LIs how by dividing them with W so that is one way to do it now I need to regress this I'll remove the constant term first then I'll artificially add another constant divide by W so essentially I am adjusting by the magnitude of residuals or sort of correcting for heteroscedasticity in the model so dividing by W and now finally data equal to underscore so this is my WLS model now I can summarize this WLS model and compare the results I can summarize the model oh so my because of this correction my T values have skyrocketed the standard errors are substantially down so this model has seems to have boosted the performance of my model and still we can say that overall our results are aligning to the same thing this 0.007 coefficient which is similar here so we can say this result is quite robust how do I interpret this result so right now we are regressing the log of odds ratio logit LI on our income variable so we can say each unit increase in income the odds ratio increases by 7.8 percent or 7.79 percent so with this we conclude this example