Understanding Time Series- Part III

[Music] [Music] [Applause] [Music] [Applause] [Music] welcome to the course business analytics and data mining modeling using our part two so in previous few lecture we started our discussion on our last module that is time series forecasting we also started discussing the first topic in this particular model that is understanding time series so we have in previous few lectures we have been discussed a bit about time series so in the last in the last part of previous lecture we started our discussion on Time series components so let's start from there itself so we talked about the time series components and typically we said as mentioned in this slide as well that this is probably the first step that one has to perform before we can move ahead and apply more formal techniques so identifying time series components is typically the first step before applying forecasting methods so as we discussed there are four components for a time series so three components we have grouped together here the label trend and seasonality because these are the three components which actually characterized the underline series and the remaining part is the noise so noise is the four component so then we started discussing each of these four components one by one so let's again discuss the first component that is label so as we discussed in the previous lecture label is average values of the series across periods so this gives us an indication about the width on an average level overall average level how the values are being taken by the time series so that sense that that indication we get from the level of the time series if we look at the second component trend this is more important so this Furka trend tells us about how the how pattern you know how values are changing in series from one period to the next so pattern so capturing the pattern of that change how the values are changing from one period to next that is defined as trend so as you can see in this life so trend is pattern of change in the series from one period to next this is or important part of time series and as we have discussed in previous lecture two types of approaches where one is model driven the another one is data driven so there if we are looking for you know you know if we are specifically focusing on trend component how we saw what kind of technique what kind of methods we should be using for modelling so a lot of that is going to depend depend on this particular component trend component so we'll discuss this in more detail in this and coming lectures the third component which is also very important is seasonality so what is seasonality so as you can see in the slide that seasonality can be defined as a sore thumb cyclically havior objects several times in the given series so typically what you know swings that we see in a particular series so if those swings are in a short term you know they keep on reoccurring in in a short-term manner in a cyclic almoner several times within within the you know specified series within the overall time period of the series that is there so that is defined as seasonality so seasonality is a sort of cyclically behavior observed several times in the given series so trend and seasonality these are the two most important components of a time series so whenever we are looking to model a time series for forecasting purposes it is these two components trend and seasonality which we specifically focus on so this discussion will happen later in the coming lectures as well the last and last component of time series is in wise so as we have done in our previous course as well the noise is defined in the you know similar similar fashion so noise here is also random variation due to unaccounted causes so anything that we cannot understand it becomes part of the noise term so random variation due to unaccounted and causes we call it we we define it as a random variation because if there is any pattern if there is any structure left in the series then we are supposed to capture that so whatever is left over you know so that should not obviously have any trend or you know any structural form or anything so therefore the remaining part should be the random you know variation part which is why wise is defined in this fashion so noise is random variation due to unaccounted causes so always present in a series to some extent so there is always going to be you know some noise that is going to be a associate associated or present in a series so these are four components so first one is label the average values overall average values that are taken by the series then the second one is the trend which tells about the you know change rate of change a pattern of change in the series and the third one is about the seasonality if there are any cyclicals behavior that is present in the series then the fourth one is the noise the unaccounted part so let's move forward so once we understand understand that these are the four important components of a series and specifically an understanding or dissecting trend and seasonality components is more important for our mode modeling purposes for forecasting purposes so how do we go ahead and identify these time components so there are various ways to do this so first we look at some of the visual techniques you know visual inspection that could be used to identify these components and how you know these components how whatever we have learned by applying these visual techniques visual methods how they can be incorporated in our formal modeling process so we will see that later so let's start with how some of these you know time series components can be inspected visually so the most important you know graphic the technique visualization technique that can be applied is you know time plot so we should create a time plot and by examining the time plot of his particular series we can look for inspect for mainly observe what are the you know component and how they are there so first point is always going to be examining a time plot so what is a time plot so it is a line chart of series values over time with temporal labels on x axis so in previous course on business analytics and data mining modeling so here we have here we have had extensive discussion on visualization technique we have also taken you know I think the same data set the same example on bicycle ridership and we have you know analyzed looked at the time plots and other things right so same thing similar things we are going to repeat here in our you know discussion on time series forecasting as well so time plot is a line chart of series values over time with tempera labels on x-axis so on x axis we have time index and on y axis we have the series values so that line chart is essentially the time plot so this time plots we are supposed to dissect you know you know the visually this time plot to understand nature of series components so we look to identify average level of the series so if once we have a look at the time plot we'll be able to see the you know visually also we were able to identify which value on the y-axis is typically in a way defining the level of the series we can also see the say Bob train whether it is you know linear or exponential or polynomial so that kind of thing cannot we can also observe using the time plot of series then cycle period of the seasonality so if there are any cyclicals behavior that is present in the series that should also be clearly visible in the time plot we would be able to you know more often than not we would be able to identify the cycle period and if there are you know and what are the peak you know period what are the non peak features so in that sense some of these details we would be able to observe from time plot so what we are going to do now will you know discuss this example data set by cycle data set so this particular data set data is on the ridership ridership in the campus so this is about the bycicle ridership so the number of buy cycles that are being used in the campus for you know traveling within the campus so within or outside the campus so that ridership that data is has been recorded on a monthly basis so as you can see here next point series of monthly ridership so for each month number of riders that have been there so that is recorded and this per choir you know series number of riders per month and based on that the series has been created so we are going to analyze this series to find out the you know time series components so before we go ahead and do that in our our environment let's discuss a few more points about identifying these time series components so one thing that we talked about is time plot so visually inspecting visually analyzing time plot can help us in identifying time series components other ways are also listed here so we can examine time series visually by zooming in so sometimes the period that is involved in a time series could be quite long so in that sense when we create a time plot for the value can get mixed up and if there are you know any you know for example that there is any cyclical behavior shown by the series that would not be clearly visible so there are you know some components which would not be you know very clearly you know inspected using a time plot just because the specified period is quite long so zooming in is one way we can you know handle a long series so as you can see zooming in recommended for long series so it can uncover hidden patterns typically when we talk about hidden but patterns it could be understanding the psyche little behavior and therefore the seasonality component of the time series if we look at other ways so the second one is also there the changing is changing scale of y-axis so as we talked about that on y-axis we have the values related to time series variable for example the data the data set that we just discussed by cycle Leinart zip so on y-axis we are going to have the number of riders and on x-axis the time index is going to indicate the month one two and three four and so on so that would essentially be Jan Feb March so in that fashion so for each of those calendar year months what has been the you know ridership what has been the number of riders in the campus so that we can that we typically can observe in the time plot now changing a scale of y-axis can help us understand the shape of trend so as we talked about what is a trend so a trend essentially we try to capture the pattern of change in the series from one period to next so how changing is scale of I axis is going to help so let's look at the other point so typically a scale of y-axis is changed to logarithmic scale so what would happen if we do that so for example if linear trend is visible so if we create the you know if we change the y-axis scale to you know logarithmic scale the what is going to happen is if there is any you know linear trend that is visible in the log scale that would be in exponential trend in the original scale so sometimes in the original scale if the exponential trend is not quite clearly visible due to the nature of values taken by the series so if we transform the scale on y axis let's make it a logarithmic scale and if somewhat linear trend is visible then that can give us the confirmation that the original series in the original series the trend must have been near to exponential so in that fashion changing a scale of Y axis can give us understanding about the trend that pattern of change in the series from one period to next and let's move forward the next the next thing that we can do to visually inspect the time series is adding trend lines so which type of trend fits the series well so we can add in the time plot itself so we can add some of the trend line for example typically we expect the trends to be linear exponential or cubic so therefore we can add these trend lines on top of the time plot and we can visually analyze whether those trend lines are fitting the time series values are fitting the data or not so that would also give us a sense in terms of the kind of trend that is there in the series so adding trend lines can help us in identifying the trend of the series let's move forward so there are a few more points which can also be used to find out you know to to understand to identify different time series components so the first one here in this upper curve slide is changing a scale of x axis just like we talked about changing a scale of Y axis and how you know if in the original series exponential trend is not you know cannot be clearly confirmed and changing the you know scale of y axis to law and if we see that close about linear trend so therefore that conformation can be done similarly we can also change the scale of x-axis so typically this is done so that to suppress seasonality so seasonality can be suppressed by changing scale of x-axis and that would actually help us in understanding the trend component so suppressing seasonality to understand the trend if we change the scale of x-axis so typically time is scale for x-axis it changed to a cruder one by aggregating data so as we have talked about in the previous lecture also that when we are discussing the label component specifically that if we aggregate data to a coarser label then some of the sources of noid might cancel out might average out there for the forecasting part of the series might increase and all that so that was the discount similarly if we change the scale of x-axis so essentially that would mean aggregating data so if we change this scale to a cruder one then of course some of the noises will cancel out and typically when they no I just cancel out we are you know some of these cyclicals behavior so as you might have seen before that if you create a plot you know for a shorter time period you would essentially see you know some sort of cycles in those those plots in those time plots however if you take a longer horizon so those values average out and you see a clear trend so really essentially talking about that scenario so if we change the scale of x-axis for example if we take the India's GDP numbers and if we create a flawed for GDP numbers for a lost decade from 2005 to 2015 or two thousand seven to two thousand seven teen and that kind of you know Series B plot we would you know Z ups and downs in the GDP numbers however if we take a plot from for lost three records and we would see a linear trend there right where the GDP has been continuously growing so once we change the you know scale of the series once we increase the length the some of the things some of the these sources of noise cancel out so essentially that also suppresses the seasonality you know component of the series and that that would actually help us in understanding in visualizing the trend component of the series so this is one thing that can be done so next point is about moving average plots so this per clock point is actually does the same thing so if we create moving average plots so that would also suppress seasonality and help us in understanding the trend component so this perkupp point is more related to what we are going to cover later in this module that is about smoothing methods so the moving average plots they are essentially part of that particular technique smoothing methods so we'll discuss more of more on this point in in those lectures so what we are going to do is till now we have discussed a number of points about how to identify time series components so we started with time plot and how time plot can be examined to find out different components of the series we discussed this data set by cycle ladder set and then some other points which could be used for example zooming in especially for a long series then the next one that we discussed is changing scale of y-axis so that would also help us in terms of understanding the you know trend of the series then the another thing adding trend lines so that would also help us in understanding trend of the series so if we look at the point that we are discussing we are mainly focusing on the trend component so 10 component is the most important component in in modeling series or forecasting time series then few more points we discussed about changing scale of x-axis and how that can suppress seasonality and help us in understanding the planned component similarly moving average village plots will also enable us and doing the same thing so what we'll do is we'll open our studio and go through this data set that we talked about by cycle address it and apply some of some of these points and try to identify the time series components so let's open our studio so let's load this library X plus X so this is the data set by cycle added zip dot X plus X is the file that we would like to import into our environment so let's run this code so this is the file you would see that in the environment section the file has been loaded and we can see for D F is the data frame we have 159 observations of two variables let's run this part of code also to eliminate any deleted columns now will have let's have a look at the first six values so you can see the first variable is month here which is essentially capturing the time is specific information the second variable is the main variable the time series variable which is capturing the riders the number of riders and you know four different ones of here you can see in the first variable month here we can see the value variation is in different roses the month value is changing from one two three four onwards so these are first six observations and let's look at the structure of this particular time series data so now in this we can see data frame 159 observations two variables month year we can see it has been identified as a date you know date type of you know data structure in our environment we can see this number of riders we can see it's numerical now to create before we can go ahead and analyze this time series and use use it for modeling purpose we need to create it time series object so for this we typically use this TS function so if we are interested in more if you are interested in finding more about this program function so we can look at the help that help section D s so we can find out so you can see here time series object the function T S is used to create time series objects more detail about this function and different arguments that could be used you can find out on these uses and arguments detail so let's go back to our course so here we can see I'm calling TS function and in the first argument itself I am using this data frame that I have full DF and the variable riders now this time series variable that is the number of riders you know that is going to be used to create this time series object so in the second argument I am also specifying the start and end time of the series so and then I am also specifying the frequency so as you would see if if the you know frequency is being is specified then I typically don't need to specify the third say a third argument that is end and of the series however we can do you know you know we can you know use all three function because these are that these are the arguments that typically we would be using while creating these time series objects in our studio environment so start of the series is this we can see here 2004 January so that is the start and we have data up to 2017 March so let's create this object so we would see a time series vector has been created you can see here in the environment section time series 1 to 59 and so these are the this translates object has been created using the function now to create the time plot we need to call this function plot first argument is the time series object that we have just created TSV the time series vector then the labels you can see right now air has been mentioned for x-axis and riders has been mentioned for y-axis however the label for x-axis hood can also be changed to month year because the data that we have in the series is monthly data so monthly ridership data that we have and then Ali where Elias is also mentioned for the presentation purpose select clear this plot so you can see the time plot for this particular series here it has been created so you can see we have data starting around 2004 for up to around 70 as we saw in the code the end of the series so you can see how ridership of this particular you know time series has been changing month by month so we can see here how this has been changing so from this looking at this particular plot this particular time plot itself we should be able to identify few of the components of the same time series but before we go ahead and analyze this time plot we can do certain transformation in the plot itself to reflect more detailed information so for example we would like to change the labels on x axis so for that you know because if you want to specify the month as well because this is monthly data so if we are interested in is specifying the month on x axis labels so we will have to use this part of a piece of code here you can see first time creating a sequence so you can see changing you know the comment now here change x axis labels from year to month here so here first I need to create the sequence as dot date and the starting date I mentioned here then the ending in date of the series mentioned also and then by two years so because those are the tick marks that I might want here in the in the plot so you know by you know two years I would like to have this sequence and month name to be depicted on the plot so let's run this we'll have the sequence now once this has been created you can see at one this is a date object having seven values here as you can see so from that what we can do is we can format this particular data object and get the labels so from this data information we are interested in finding in the we are interesting in extracting the information of month and year so in that sense if you are if you are interested in understanding this function a bit more how we are doing it so I had two one as I said the sequence 2004 1 1 2 up to 2017 3 1 having two years interval has been good so more if you are if you are interested in looking at the these values we can run this and you can look at the values up at 1 you can see 2004 1 1 2006 1 1 2008 so this is the gap is 2 years and we can see this sequence say mean values are here now we are going to format this because we want to create XX levels now we can look at the if you are interested in more detailed information on this particular you know function you can look at here format so you you will see that format this program function we can look at this particular function allows us understanding more about more about the understanding more about how we are going to use how we are going to format these levels so format dot date is the specific function that we might be interested in so you can see it format X and other arguments so here we can find out different labels that could be used so here we will see that in this let's go back to format so we are I just want to show you this these formatting option that I have picked here I have picked found this function so I think the details would be here and down in this function so you can find out the detail of these formats so what these different formats represent actually so in this we will see that either in the format or on the format dot date you would see that these percentage B and percentage Y so B is representing month and the capital y this is representing here and in exactly what format we can also see that from the output so what I'll do I'll learn this part of code and we will see this so labels I have just created this product of variable you can see this is a character vector and you can see in this format will get so smaller B that actually represented the month and in this form and then we had Y which represented here so now we have the labels in month and year format now we can use this now to identify the tick marks also so if we go back to the plotting so we need to identify these tick marks where the labelings are to be plotted and the labels name so what we have just created in the labels one we have created the labels for us now we need we also need to compute these tick marks where these labels are to be plotted so I had to will actually allow us so that so let's run this now we will stop at this point and we'll continue our discussion on timeseriesforecasting and swiftly this part of the code where we are looking to analyze time series components in our studio environment in the next section thank you [Music] [Applause] [Music] [Applause] [Music] [Music]