Lecture 31

In this lesson, we'll discuss the univariate models of prediction and forecasting with the help of our Maharima class of models. We'll start with the concept of time series stationarity and white noise process. Then we introduce autoregressive AR processes and their structure with the help of a simple example. We also discuss its stationarity property. We'll also discuss the autocorrelation coefficient and autocorrelation function and partial autocorrelation function in the context of AR processes. Next, we introduce the moving average or MA processes and its structure with the help of simple example. We also discuss the ACF and PACF plots for an MA process. In this video, we'll discuss univariate models of prediction and forecasting. Univariate time series models of prediction and forecasting are an extremely important class of models. They are specifications where one attempts to model and predict financial variables in particular, for example, financial market prices using only the information contained in their own past values and possibly current and past values of an error term. We'll have more to say about this error term but generally this is used to mean or denote the information that has just arrived in the market. Unlike the structural models that include some theoretical underpinning and a number of variables, these time series models are a theoretical such as modeling high-frequency trading data. What do we mean by this? In general, any such model would derive or take away from some kind of structural model. However, we do not care what that structural model is as long as we have the variable of interest such as security market returns and its own past values. It is assumed that any such structural model that we do not observe or know about at play, automatically that information that is contained in that structural model is incorporated in the historical time series values of the variable of interest. So, for example, you may have some idea that GDP of a country may affect the returns of nifty-fifty. Let's say Indian GDP affects nifty-fifty returns. Even though I may not have the knowledge about this model but I know as long as this GDP affects nifty-fifty and I know that as per the time series model the historical values of nifty-fifty will help me predict the future values of nifty-fifty, I can be rest assured that this relationship, this underlying structural relationship will be captured while I model this time series model of nifty-fifty with its own previous values in time series model which is rather more atheoretical nature but by this background it automatically incorporates the information content of this structural model which incorporates the relationship between GDP and its impact on nifty-fifty. One of the most important class of such models includes autoregressive integrated moving average, ARIMA class of models. We'll have more to discuss about this model. Why such time series models are important and particularly the univariate models is because often when you are modeling the financial market data with modern financial market instruments that are part of this course, structural variables are not available at high frequency and most of the modern day trading is happening at high frequency domain so whenever you are modeling any such model where high frequency trading or any such relationship on daily or weekly basis where more structural variables are not available such as GDP or central bank policy variables then you have to rely on the historical values of the variable that is available for example, prices PT or returns RT that are available on higher frequencies and you have to rely on such time series model that employs past values of the variable itself. This is one important motivation of using these time series models. To summarize, in this video we discussed the importance of univariate time series models. We noted that in modern financial markets the trading activity and various other dynamics play at extremely high frequencies such as intraday level where various structural model variables such as GDP and monetary policy variables are not available at those frequencies and therefore we have to rely on the historical values of the variable itself such as returns and prices to model the data using models such as ARIMA models and therefore in this condition the univariate models, univariate time series models for prediction and forecasting becomes an extremely important class of variables and even more so it has been found that while examining the prediction accuracy through methods such as out-of-sample prediction these univariate class of models often work even better than the structural models as well and also the data requirements of these models are extremely parsimonious and they are extremely easy to model and that voids us the intuition and motivation to work with these and learn these univariate time series models. Introduction to time series stationarity. In this video we'll discuss weak and strict form of stationarity and also the problems that happen when we employ non-stationary series. Let's start with a strict form of stationary process. In a strictly theoretical sense a time series process XT is said to be strictly stationary if the joint probability distribution of sequence of YT1 to YTn is same as that for YT1 plus M to YTn plus M for all M's. That is the joint distribution of YT1 to YTn is same as YT1 plus M to YTn plus M. That means as long as we have the distribution of observations the different moments starting from first moment that is mean, second moment which is change or second moment which is sort of you can say variance and higher moments are same across as the time progresses across all the observation for example whether you are looking at from YT1 to YTn or M steps ahead or K steps ahead the set of observations that you have its joint distribution for a given set of observations remain same or put it more simply for any observation if I pick for any time TK if I pick observation YK or YTK the probability that would lie in certain interval will remain constant as time progresses. This is called stationarity strictly stationary process which has lot of desirable properties. However in a practical sense we make use of something called weak stationary process. What do we mean by weakly stationary or covariance stationary process? We mean the following this kind of process has the constant mean that means the first moment which is the mean it is constant second moment or variance is also constant sigma square so the variance is also constant which is the second moment expected value of YT minus Mu into YT minus Mu and also the autocorrelation structure that means correlation of YT with its own previous values is constant that means whether you are looking at Y correlation between time T1 to T2 that is YT1 to YT2 or YT1 YT2 to YT3 what is important is what is the difference between those time periods for example T1 to T3 or T3 to T5 the correlation expected to remain same so what it what matters only is the time gap between them for example if the time gap is 5 lakhs or 6 lakhs is what only matters it doesn't mean what time you are standing so you can compute the autocovariance for S lag period like this expected value of YT minus Mu into YT plus S minus Mu this is for S period difference so this is for S lags gamma S and this autocovariance structure remains constant as time progresses while this is an important measure because of scale across different processes is difficult to compare so what we focus on is called autocorrelation that means correlation which is a standardized or normalized version of autocovariance so if you divide the autocovariance at different lags maybe lag S where S can be 0 1 2 with autocovariance at 0 lag that means autocovariance at 0 lag is variance itself that means autocovariance with its own term which essentially nothing but gamma naught which is variance this scaled version this scale value is called autocorrelation so at S equal to 0 this value will be 1 in general this value varies from minus 1 to plus 1 and this autocorrelation value it measures from minus 1 to plus 1 and an extremely important measure to define a weakly stationary process the autocorrelation structure of a weakly stationary process so if you plot this autocorrelation on a graph starting from 0 lag first lag and so on at 0 lag you get a value of 1 and next whatever autocorrelation structure is there you can plot this at different lags 1 2 and 3 this kind of diagram is called autocorrelation function autocorrelation function ACF or in short ACF diagram which will be using very heavily and it is also called corellogram corellogram so this will be called corellogram or autocorrelation plot or ACF plot so let's discuss what are the problems of working with non-stationary series like we have defined the non-stationary series is a series where different moments do not stay constant with time particularly the first moment which is mean the second moment which is variance and the autocorrelation structure tau s does not remain constant with time such non-stationary series has various problems while working with them on econometric estimation for example shocks to such non-stationary series do not die away a very desirable property of any economic series or econometric series time series is that any shock dies away naturally over as time progresses however for non-stationary series because of their trending nature and drastically varying different moments the shocks to the series not die away and they are highly persistent this is very important point the second important point is that estimation with non-stationary data can lead to spurious regressions for example even though the two series may be y1 and x1 they are not very related but still if you run some kind of regression on them they appear to be very spurious resulting in high t statistics and adjusted R square for example if you sample from two non-stationary series and two different period samples and keep on sampling the process over and over and run regressions you'll find a large number of adjusted R square like plotted here a very high values for the sample adjusted R square and also very high t statistic values which should not be the case ideally with two series y1 and x1 are independent any sample and subsequent regression should give you poor adjusted R squares and t stats but because of this trending nature of non-stationary series for example non-stationary series the shocks to the system do not die away and therefore they appear to be moving or trending like this the y1 and x1 so because of this trending nature at whatever point you take a sample y1 dash x1 dash or y2 dash x2 dash they are always trending and this results in some kind of positive or negative spurious relationship when you estimate some kind of correlation or regression measure on them to summarize in this video we discussed a strict stationary process and a weakly stationary process that are that is often employed in a practical sense we also saw some of the shortcomings and problems with estimation of these non-stationary series with econometric methods such as regression use white noise IID or identically independently distributed we'll also understand what is the importance of modeling a process as white noise to begin with white noise means a process which has no discernible structure to put in more detail process with the constant mean generally that mean or expected value is taken as zero but can be a constant number we generally take it to be zero and a constant variance sigma square which is not changing because there is no discernible structure or there is no reason to expect that this sigma square but most importantly there is no autocorrelation or autocorrelation that means for any two values in the process this t1, t2 they need not be immediate preceding terms they can be any two values taken from a time series of value may be t1 to t3 or t1 to t2 or t1 to tk any such two generic values the correlation or covariance between them is zero for different values of t1, t2 and so on where as long as t1 is not equal to t2 if t1 is equal to t2 then this becomes variance of sigma square and in autocorrelation terms this value tau 0 will be equal to 1 as we have already seen so to put or summarize this we are saying a white noise is a process which has a constant or rather zero mean a finite constant variance and it is not correlated sometimes you also put some assumption about the distribution and generally this is a normal distribution so this becomes a white noise normal IID process that means when you say independently and identically distributed that identical distribution is normal in most of the cases but a more generic term is IID which there one I is for identical another I is independent and these for so a normally distributed white noise IID process is a special case of white noise now just to summarize this what is the importance of modeling a process or when I say process it can be financial market returns and so on as a white noise IID process many times you model the prices given a certain model it can be various advanced models that we see like guard kind of model, ARMA, ARIMA model in this case but once you model it you tend to believe now that you have taken out everything every discernible structure in the process and model it so if you have modeled it correctly the residual or error term in the process after you model the series let's say you're modeling the return series you model out with some kind of ARIMA process a generic term and extract the residuals from the model let's call them mutants then the assumption here is that if we have done everything correctly these residuals that are extracted from the ARIMA process should have no discernible structure because if we have modeled it correctly any discernible any kind of pattern that should be modeled will be is modeled to parameters of the ARIMA so in the residuals or the remaining part there should be more discernible structure and therefore it is argued that this muti or error term should be as close as possible to a white noise IID process with no discernible structure and that is why it is very important once you model a given process or return or financial market series time series model then you make an assumption up about residual being white noise and then you test the residual for white noise and that testing of white noise residuals is a sort of verification of your modeling accuracy or how good your model has been so to summarize we are saying that a white noise process in process with a constant generally zero mean constant variance and no autocorrelation structure no discernible structure generally we use such modeling or modeling to white noise we model residuals from an ARIMA model class of model or any other similar model where the assumption is that there is no discernible pattern in the structure because we have modeled out anything possible in our model efficiently so generally there should be no discernible structure in our muti or put it more specifically there is no autocorrelation structure in our muti or error term if in this video we'll introduce the AR or autoregressive process which is a very important component of ARIMA so this AR in ARIMA stands for autoregressive process we'll also discuss the motivation for modeling a process through autoregressive process to begin with autoregressive or AR processes assume that a time series process yt depends on its own previous values the structure of AR model can be given as follows so let's say you assume an AR p process where p indicates the number of lags for example a process yt is modeled with a constant term mu plus its own previous values yt minus 1 yt minus 2 up till yt minus p because this is a p order process and therefore the coefficients of these yt minus 1 that is phi 1 phi 2 run up to phi p so this is a AR p process because it is affected or it is modeled at its own previous p lags this muti is error or innovation term now this can be further simplified by writing yt in a more compact notation by summing up all these p terms into this p notation summation i equal to 1 to p phi i into yt minus i where i runs from 1 to p moreover we can use something called lag operator where lag operator li on a process yt means yt minus i so a lag operator with the subscript i would mean y series with the lag i so this term can be further simplified with the lag operator like this we'll see later what is the importance of denoting a process like this again muti is the error term this muti can be written as a white noise this is assumed to be a white noise iid term ideally if the process is purely an AR process autoregressive process and we have modeled it appropriately then muti should be purely white noise generally we put another assumption of normality so it is considered to be distributed as normally distributed with the mean 0 and constant variance once we have modeled this process in this form we can take out these this phi i this particular part on LHS and therefore the left side becomes some kind of function of yt series with phi l where phi l denotes these set of parameters so the sample space or the sort of parameters that are part of phi l are these phi 1 phi 2 and so on and lag operator indicates that this is applied with the lag so yt minus 1 yt minus 2 and so on or to put it more clearly now phi l here would be 1 minus phi 1 L minus phi 2 L 2 or L square you can also call it L square little we'll see the application of it minus phi P L P so this you can call as LP or L to the power P now why I'm using the word L to the power P because often when you characterize this AR process this is called characteristic of equation of this AR process I repeat this is called characteristic equation of AR process and on LHS we have this phi l yt function where this is the phi l yt function on RHS we have the constant term generally in financial market when you are modeling financial market returns and those kind of financial market process this mu tends to become 0 often sometimes it's not and mu t is this white noise IID error term which is normally distributed now later we will see that in algebra the roots of this characteristic equation are very important to understand this AR process so often these roots are very important in determining properties such as stationary and non-stationary AR processes so this characteristic equation becomes very important and therefore these subscripts become powers for example this L1 become L to the power 1 or lag operator at second level L2 becomes L square and then this will be solved like an any algebraic equation so this is become those similar algebra kind of problems where this equation is called characteristic equation of this ARP process and its root which will be very important to this process you to be able to define the properties of this AR process are employed now in summary and conclusion we will try to understand what is the motivation to model a process like this AR autoregressive process process in financial markets like AR it is often believed and for that you need to read a little bit of financial market literature like sequential information SIH sequential information arrival hypothesis mixture of distribution hypothesis and information asymmetry hypothesis that as information arrives in financial markets in first place complete information never releases in one go so information itself is released in a piecemeal manner in chunks of information pieces like Delta 1 Delta 2 over time and as these information pieces arrive they are not immediately incorporated in prices it takes some time so this arriving information or the information that has arrived incorporates gradually over time in financial market prices and therefore by nature this first in first place this information is autocorrelated because it is being released in the piecemeal manner and also because it is being gradually incorporated in prices by design there is an autoregressive structure in financial market returns that is to suggest that financial market price returns are autocorrelated and that is why theoretically AR process becomes extremely important to model financial market returns the significance of this muted term is also very important in econometrics or in modeling statistics this is often treated as error term with not much value error term or innovation term however in financial economics this mutiny has a very important role generally it is considered that when you model this mutiny this is taken to be as the information shock or innovation or information shock that has just arrived at time T so it is believed that once you model out the previous historical information structure which has been autocorrelated like I said information is arriving in a piecemeal manner so it is bound to be autocorrelated once you model out this autocorrelated information what you are left with an information shock which is completely unexpected so this mutiny is unexpected information shock and this information the information shock has no information content in the previous process so this is unexpected had it been the case that some of this information was already understood or explained by the historical terms it would have been modeled out so whatever left unmodeled in this mutiny is called unexpected information shocks to this YT process and that is why modeling these financial market processes such as returns it is extremely important the AR becomes theoretically extremely important process one last point very important point that often some kind of statistical model may give you a very good fit which is not an AR however that model may not be theoretically justified so always remember in modeling financial market and financial economics or any kind of economic series theory is given preference over statistical and goodness of fit so even if a mathematical model is giving a very good fit but it is not theoretically justified we should not use that we should use a model which is theoretically justified this is important because ultimately you are trying to model this process from some policy perspective maybe you want to do some kind of policy experiment maybe you want to look at the impact of GDP on financial market index returns like 50 50 in that case theoretical relationships are important and even though we are using time series process but is still a theoretically justified structure is very important to have and the second implication of that is often such mathematical models good models which work very well in in sample when you model them out of sample new data they work very poorly in contrast a model which are theoretically justified which have some theoretical underpinning even though they are in sample fit in a given sample in they were modeled may not be as good as some kind of very complex mathematical model when you look at out of sample fit these models generally work these two models which have some theoretical underpinning they work much better than any statistical model which may not have some kind of underpinning and maybe a complex model as well we'll discuss the stationarity of an AR process stationarity for an AR process is very important we have already discussed that what are the problems with a non-stationary process also if a process an AR process is non-stationary that means the impact of some of the previous values on some of the current and future values do not die away what essentially means if there was a shock historically in a system it does not die away and it persists for very long times which is not very desirable property in modeling any such AR or Arima-Arma process let's understand what is stationarity mathematically recall we said an AR process will appear like this where phi L yt is autocorrelation structure represented by this kind of structure where phi L is 1 minus phi 1 L phi 2 L square and so on these phi i's are essentially autocorrelation coefficients and this L is lag operator for example if it is written as Li essentially at yt it is essentially yt minus i and in lag notation conversely this can also be thought of as some kind of polynomial in Z and later as we have already said this is like a characteristic equation a polynomial in so you can think of this Li as L to the power i also as we'll see this needs some understanding of Wohl's theorem the second important point here to understand that this mu is some kind of constant term and mu t is that error or innovation or information shock for modeling financial market data such as financial market returns we take this mu as zero generally it is considered to be zero in financial market process such as return for the following reasons if this mu takes on some kind of costate value that means there is some kind of arbitrage to be made to make absence excess abnormal returns and if it is negative vice versa that is short positions can be taken to make abnormal returns and therefore on average this mu should be insignificant or close to zero and that is why a more practical assumption is to take this as zero which means now phi L y t is equal to mu t that is phi y t of phi operator on y t equal to mu t or with the Wohl's theorem we can write phi t equal to phi L inverse mu t now this kind of transformation taking inverse of this phi L is possible because of something called Wohl's theorem which is out of the scope but the point is for this kind of transformation to exist it is needed that phi L be a stationary function that means this equation must represent a stationary process what do we mean by stationary process finally financially or economically economically if the process stationary that means as you keep on increasing lag length these phi ones the magnitude of these phi ones should become smaller and smaller and converge to zero intuitively it should be the case because as you go further previously or past in time the impact of events or shock that have arrived in historically they should die away gradually over time as time passes and this is what exactly we mean by this stationary condition more importantly that also means that none of these phi should be equal to one or greater than one if they are greater than one then not only shock increases but it explodes if it is equal to one that means even though it does not explode but it does not die away infinitely ahead in the future which is not possible in economics and finance or in general in any case so we say that if this series this process is non-stationary or this sequence is non-stationary then this process is not feasible to model so in general we rely on this characteristic equation and find its roots this characteristic equation we find its roots and we say if the roots lie outside the unit circle in the process stationary that means if the roots of this equation think of this as a polynomial and the root of this equation are greater than zero then this process is stationary what essentially mean think of one of the root as maybe z1 or z2 and so on some of the roots and if I put those roots 1 minus phi1 z equal to 0 then essentially z equal to 1 upon phi1 which means this is just a simple very simplistic example we'll have a numerical example following in the following discussions in next set of videos but just think of it if this z value if this z value is greater than 1 then only the phi1 will be less than 1 in magnitude but if this z value in terms of magnitude is less is inside the unit circle that means either 1 or less than 8 then phi1 will become greater than 1 or equal to 1 which is which leads to non-stationarity and this is what not is not desirable so this is the property of non-stationarity which is a non-desirable property and to summarize we expect the roots of this characteristic equation this is the characteristic equation of an ARP process we expect the root of this equation to be outside unit circle to the process to become stationary that means if the process stationary then all the autocorrelation coefficients first they will be smaller and smaller and converge to zero as we go ahead past in time that means phi p will be in terms of magnitude will be very small and second none of none of these autocorrelation coefficients should be either equal to 1 or greater than 1 none of them should be like that as long as the process is stationary which means all the roots characteristic roots of this equation lie outside the unit circle now that we have understood the theory behind the stationarity of an AR process let us try to work out a simple example let us consider a simple AR process like y t equal to y t minus 1 plus mu t essentially this process has a mean of equal to 0 as we said in general we'll take the mean as 0 and also it is only affected by its immediate previous lag that means only the immediate previous term in the series y t y t minus 1 y t minus 2 so for example y t minus 1 will affect y t y t minus 2 will affect y t minus 1 but none of the second lag y t minus 3 will have no relation with y t minus 1 or y t minus 2 will not affect y t so for this kind of process recall we can write y t minus 1 as L of y t and we can put it in a more compact lag notation as 1 minus L y t equal to mu t now with volts theorem we have the characteristic equation as 1 minus L or its algebraic or polynomial notation as 1 minus Z equal to 0 solving for this equation we get Z equal to 1 which indicates that there is only one characteristic root which is equal to 1 or lies on the unit circle recall the implication of Z equal to 1 means that one of the autocorrelation there is only one autocorrelation which is equal to 1 indicating a process which is non-stationary or having what we call as unit root unit root because its root equal to 1 although even without doing this calculation you could have already seen that the autocorrelation here is directly 1 which is not desirable why because autocorrelation of 1 means that whatever shock that has arrived or was there in the series y t minus 1 it will never die away as process progresses it will always let's say there's a shock of magnitude m it will always have a multiple of m so in second series m square m cube so the shock will keep on multiplying every time further and it will not die away the shock will always persist with the power of 1 into 1 so the shock will never die away and will always persist now and therefore the process is non-stationary and its inverse is also not possible because the inverse would not be possible as per the Boltz theorem if you recall what we said as phi L inverse which essentially in this case is 1 upon 1 minus L and the root characteristic root being 1 this is not defined so and therefore this process is not a stationary process it has a unit root which is exactly that is Z equal to 1 or indicating then process where the shocks do not die away moreover the unconditional mean of an AR process is given by this expression below unconditional mean you can think of as a long-term mean if there is no shock to process and we keep on moving ahead in future without any shock in the long term the process mean will converge to this although in the case of financial economics this is not a very relevant or important result for us why because in general we tend to assume this mu or the mean of the process as zero so in that case whatever the shocks may be the unconditional mean in long term will always be zero by the simple property this mu is already considered as zero but still it is important to note that there exists some kind of unconditional mean in general if we go outside the financial economics region in general in economics an unconditional mean is something which is called long-term mean so if we keep on projecting the process and there are no new information incorporated in the long term the process converges the projections or forecast from the process will converge to its unconditional or long-term mean which is a desirable property however like I said in our case financial economics this mu is equal to zero the mean is set to be zero and therefore this long-term and conditional mean also becomes zero now in case of let's say this AR1 process let's say this while we are not putting this mu equal to zero and there is only one autocorrelation which is phi 1 please recall this expression will become mu upon 1 minus phi 1 by the Boltz theorem so recall the phi our expression is phi L would be 1 minus phi 1 and essentially when you take the inverse it this 1 minus phi 1 will phi inverse will be 1 upon 1 minus phi 1 now in this case as long as this phi 1 if you recall if this as long as this is less than 1 in terms of magnitude then this process stationary but as long as as soon as this phi 1 becomes 1 which means the autocorrelation is 1 this process becomes non-stationary as we saw in the previous example also if it is more than 1 then also this inverse is not possible the Boltz theorem will not result in this kind of inverse and again the process will not be stationary for this process to be stationary this phi 1 has to be less than 1 in terms of magnitude then only the characteristic root of this equation which is 1 minus phi 1 Z will be equal will be available that means they will not be 1 for example if phi 1 1 then characteristic root equal to 1 or if phi 1 is let's say more phi 1 is greater than 1 then Z will be less than 1 that means again the root of this characteristic equation will be either on the unit circle or inside it which is which leads to non-stationary process so to summarize in this video we discussed what and it's with a simple example what would be the implication of non-stationary non-stationarity of an AR process and how do we model an AR process and convert it into a more amenable form like this and this we also saw with a simple AR 1 process how to convert into through Boltz theorem convert into a polynomial and obtain its characteristic root characteristic equation and then once the characteristic equation is available through simple programs even with a calculator or a simple program you can obtain its characteristic root and as long as those characteristics root lie either lie outside the circle the process stationary however if the characteristic root lies on the circle or inside the circle the process is non-stationary what is the implication of characteristic root lying inside the circle or on the circle it means that either the autocorrelation is equal to 1 which means shocks to the series will not die away or if it is inside 1 then they not only not die away but they explode which is not a desirable or feasible property in finance and economics