Lecture 43

IIT Kanpur July 201
20 followers
2 Video Views·Oct 5, 2024
In this part 4, we are expected to conduct the gender and performance data analysis.
We are expected to analyze the data by performing the following tasks.
First, we are supposed to import and merge the HR and performance data files
and examine whether average performance ratings differ by gender and also visualize the data.
Lastly, we are supposed to analyze the gender and performance data across job levels
and make inferences and also visualize the data.
In this part, we will start with part 4 of the case study,
where we will talk about the consistency of performance evaluation.
We would like to see whether performance evaluation is consistent, for example, across gender
and so on.
Now, this is a very important aspect. First, we will
import the data sets and join them.
We are supposed to use two data, HR data and performance data.
Let's load them one by one. First is the HR data.
We will import this data set. For reference, I will keep
this command for future use.
Let's see what is inside this data.
Let's have a glimpse with the glimpse command.
It contains again 1470 observations. First is employee
ID, second is department. We have seen this variable.
Next is performance data.
We will load the performance data.
Again, for future reference, I will just copy this command.
Like we do, I will use the glimpse of this
performance data as well.
Let's see what is inside. In this, there are two variables. One is employee ID and their ratings.
So, along with the ID, the ratings. Now, because we are going to use
both of these data frames or data, we need to merge them.
The next task is to join these data along
the employee ID variable.
That is simple. Let's create a new join data
variable. You can study
more about joins, but here I am going to use left join.
This will combine the data along the employee ID lines without
creating any duplicates.
Even if I do not specify, let's say I do not specify this
and it will be joined by employee ID only or if you want to
specify, that also you can do. Sometimes it happens that you want to
join by some specific variable. You can specify
that employee ID. That also you can do. This is also an available
option. Both will result in the same data. So, we have joined the data now.
Now, the next set of analysis, this first set of analysis, we would like to see if there
is any differences across genders in terms of performance rating. So, we will
compare the performance rating across genders.
That is the first step of analysis. So, I will use the join data.
I have already created it and group it
by gender. That is obviously the natural course of action here.
So, join it by gender. You can see now my data is joined
with gender. All I need to do is just summarize the ratings along
the gender lines. So, now my job is simple. I will just summarize it,
create an average rating variable which will hold the average ratings, mean of
ratings, so to speak, and run it. Now, you can see here
the average male rating is 2.92 and average female rating is
2.75. Now, the rating seems to be different. There seems to be some kind of differences
in male and female rating, but if there is a systematic bias, that needs some more
exploration. Let us examine this in more detail. So, let us assume
that those with rating of 4 or more are high performers. So, let's create another variable
which is high performer. In fact, we will create new data out of it.
So, first, let me create this join data and
in this join data, we will add a new variable. For that, we need to use
this mutate function and let's call it high performer,
and in this, like we have been doing earlier, we will make use of
felse function, rating, if it is greater than or equal to 4, then 4, else
1, otherwise 0, and if I run it, I get
the values. So, let's create this new data, let's call it performance data,
and let's store it. So, now I have this performance data ready
for me for subsequent analysis. So, create
a performance data with
objects classified as high performer.
High performer with a rating of greater than 4 will be a high performer.
So, now we have created the data that we need to analyze. From next set of videos, we will
do a detailed statistical analysis and data visualization as well. Now that we have created
this performance object, which takes a value of 1 if it is high performer with a rating
of 4 or more, or 0 if it is a low performer, we are good to start with
our analysis. First, we will do the gender-wise analysis of
ratings and we will see whether high performers are more in male versus female category.
So, first let us do the simple t-test. So, t-test of
high performer. Now, this t-test we will do in a
different manner now. So, we have been already doing this t-test. So,
let's do this high performer. I will show you a different
regression approach to t-testing. So, t.test
and all I need to do is use this tilde and
gender because I need the differentiation. It is the same thing that we have been doing.
I am using a different regression approach to that and you can see the t-test results.
You can see that it is the same approach. We can see
the average rating for male group is higher than the female group and it is quite
statistically significant. That means there is a significant difference in the ratings.
So, t-test of means. Now, here we conducted the t-test
of means, but please recall our data is of one zero form. So, we can also conduct the
chi-square testing here. So, that is also very useful to see whether there is a different
chi-square test. So, that is also quite simple chi-square
test and all I need to do is just provide
performance data, performance dollar.
Now, I need to provide the gender variable and
the gender is the differentiating variable and then provide the
high performer variable that we have created. So, if I run this code
just see the output. So, you can see here also I find the result to be
very statistically significant. So, overall I can say there is a very statistical
significant differences in the ratings given to male
particularly male higher ratings are given female lower ratings are given, but we
need to do some more analysis. As a first step we will try to visualize the ratings of
male versus female on bar plot. So, first we will visualize
the rating. So, let's use our performance variable.
So, what we will do is we will provide ggplot. Now, here we need to provide our performance
data which we are going to use and we need to specify our aesthetic
object. So, here we are going to analyze the gender variable
and fill as high performer.
So, fill with high performer. Now, I can use the bar
probably bar plot would be a good option. So, let's see what kind of diagram
the bar plot shows us. So, as you can see
also it would be great if I because there are two categories
we get if I use this factor command
that would make things more suitable. So, I missed
one bracket which I will add. Now, that will be better because
now I have added this thing. So, you can
see the data visualized. You can see the composition clearly
one the difference in numbers. So, it gives me some
idea but because the numbers are absolute probably it would be better if I
use the position fill argument. So, let me show you. So, this is
the visualized performance. Now,
to improve this graph probably I'll use few arguments. Let's
try some of them. So, let me use a new
argument position fill. This position fill may
improve the diagram. Let's see dodge
there is one more argument you can use dodge that also gives you
so it creates a separate number. So, you can see. So, while
overall it looks like we have more high performance in the
male category. There is also another approach to visualize data
through ratings. So, what I can do is instead of using
the fill as high performer. I can use ratings also with rating. So,
I'll just change this to rating. Let's see.
So, we get a very nicely distributed graph. We can see lot of high
performer in the male category particularly rating
4 is given a lot while the females are given 3 is a lot.
So, extreme high performers 4 and 5 are more in the male category.
Also, sometime it is misleading because of the different numbers
to compare the composition of different ratings. So, one good way
is to use the column approach. For example, here
I will slightly tweak the data. So, I'll use column here.
So, now but then I have to specify the
Y values which is high performer and now if I run
it they are stacked one to one and you can see the composition
more clearly. So, now you can see in the male the composition of high performer
is higher the same thing I can do with the ratings bit also.
So, again instead of bar I can use the column
object and but I then I need to provide the
Y axis separately show you the composition. So, if
you look at in terms of this thing you can see the composition clearly
the rating 5s are higher in male the rating 4s are higher in male
while the lower ratings are higher in female. So, this gives us a very clear
idea that high performers there is a high performance in the male category
probably there is some evidence of systematic bias. Maybe higher ratings
are given to male colleagues. However, this kind of analysis
is often fraud with omitted variable bias. For example, in the previous
case we saw rather than it was the pay differences in new versus old it is the
mix of employee job mix was changing similarly here it may be possible that
the bias is more on the job level. So, the ratings are different at job
level rather than the gender, but we will carry out this analysis
in the next video. In the previous video we argued that there
is one possible explanation to our results was that there is a bias across male and
female candidates in rating assignment or there is one possible
reason could be the missing omitted variable bias. For example,
there would be possibly case that there is a rating differences
or bias across different job levels. For example, hourly
salary managers and one particular category may be
the low rated ones are higher in female workers. So, for that let us start
this exercise. So, we need to examine the
differences in job level rating distribution by gender. So, we need to
check the difference in job level distribution
across genders. So, for the same let us start with
the visualization exercise. Visualization of composition of male
and female across different job
levels. For that let us start with our good old ggplot.
We need to provide a performance object. A is x
x is equal to gender. Now, we will fill it with the job level to see
the composition. This would be a better way to visualize and we will use
bar plot for starters.
Let us keep. Let us run
this. So, let us this gives us some idea. In fact,
I could have used position equal to fill also.
So, like we see stack fill and so on.
So, if you fill it, it will scale the entire thing. So, you
can see in terms of composition you can see hourly is
higher hourly is higher in the female while the salary
and managers are higher in male. Now, this gives us some credence to that
intuition that probably it is the case that these
if these are low rated the lower levels like early are low rated by managers
and salaried are more rated then we have some reason to believe that probably
it's not the gender bias. So, for that let's
do one more step goes one step further and
visualize the performance of male and female
across different job levels.
So, what we'll do is we'll use this ggplot
command performance performance data. We have now
in aesthetics. We have X as gender variable and we'll
fill the plots with factor high performer.
This would be our plotting scheme. Now here we need to add some more
information first and foremost. We'll use the geom bar
geom bar object with position equal to fill to again
standardize it to that scale of one so that we can compare but
now we'll facet wrap and if you recall the facet wrap will help us
by segregating the plots and in this case we want to
segregate them across job level. So, let's do that and let's look
at the plot now. Now here the differences are not very stark
if you look at salaried the performance of female versus male very close
distribution for manager a little bit higher
in the managerial category a little bit higher in hourly also same so
not as stark as we saw earlier. So, probably it is not
the gender wise difference because gender wise there is a minor difference but not as
stark as what we saw earlier. So, probably it is the composition of hourly manager
and salaried which is causing this difference. So, in the next set of videos we'll
conduct a more robust statistical testing to see whether there are indeed very
significant differences across gender and across these different job
mix levels in this final concluding video. We'll conduct
the statistical t-test and chi-square
test to examine whether there are significant differences
in performance ratings across
the job mix that is manager salaried and
hourly levels. So, we'll start with the salaried and we'll reuse the
same code. I'll not rewrite the full code. So, let's start with the salaried
class and I have the variable which is performance
now inside this performance variable we'll filter it out
with job level equal to salaried
and I'll store this in the object
called performance filter. So, we have created this object
so now this object is created. Now, the first step is
t-testing. So, let's start with the t-test and
for t-test as you have learnt we can simply
do this high performer object. This high performer
object will be segregated across gender variable data
is performance filter and if I conduct this t-test
the difference is not statistically significant. You can see
the p-value is turning out to be insignificant. So, this is not a very useful
result or not useful as in it says that
there is not significant difference across male and female. We can also apply our chi-square
test. For this
chi-square test I need to use the performance filter object
dollar gender because across gender
we have to see and our object is high performance. So, let's see
yep no difference. So, we can see there is not significant difference as per this
test also. Now, I'm not going to write the full code again I'll just copy paste
we have done it for salary. So, let's do it for other
also. So, next is manager. So, let's do it
for managers. Now, for managers we'll
have the filtering variable as manager. Let's
do that. First, let us do the t-test not significantly
different. Chi-square test not significantly different. So, across
managers also there is not much difference but that the difference has to come
from somewhere. If you recall there was difference in the performance. So, there has to be
somewhere this results should lie. So, let's try
the last one which is hourly. I'm hoping given our overall
result there has to be some positive. So, with t-test
yes. So, you can see it is 2.45% p-value that means
at 95% this is significant and similar significance we are
obtaining here also. So, now this ties up our overall results. Let me
summarize it for you. So, first we found that across the female versus
male there was a performance evaluation difference which gave us an impression
that in the male category there were more high performance
probably there was some systematic bias in rating assignment. However,
when we analyzed the proportion of female versus male we found that
relatively the hourly workers are more in the female category while
manager and the salaried are in the male category and as
it turns out here there is a very systematic kind of difference
in performance who are in the hourly
wage. So, in hourly level there is a significant difference
in the across gender. So, it seems that while at managerial and salaried
level there is no performance evaluation differences in gender wise
but at hourly level there is a significant difference. So, this
is what we conclude in this part. In this part 5 we are expected to conduct
location and accident analysis. We are expected to analyze
the data by performing the following tasks. First we are supposed to import and merge the HR
and accident data files and examine whether the accident rates have increased from
2016 to 2017. Then we are supposed to visualize the data.
We are also supposed to examine whether excessive employee overtime has any relationship with
the accidents. Using various descriptive statistics, visualization
techniques and logit regression model we are supposed to examine whether employee engagement has
changed over the years and if it has any relation with the accident. And lastly we are supposed
to conclude our findings with sharp and brief inferences. In this part
5 we will examine the employee safety related data. So,
we will look at employee safety related
aspects. Now, this exercise requires us in delving
two data. One is HR underscore
two CSV this data. So, let us import this data.
Let's copy this. So, this is
first data. So, let's see what is inside this. So, let's use glimpse command
HR underscore two. So, again there are 2940
observations. There is a year column where we can see there are two years
employee ID, their location and their overtime hours. So, from
this we will get the overtime hours and we will also see if the overtime hours have some relationship with accident.
The second data file that we are going to examine is accident data.
So, we will import this data file. Let us copy
this command for future reference accident data.
So, we will use this. Let us again run the glimpse command
and see what is inside this accident data. Again, three columns
year, employee ID and accident type, mild and so on. So, accident type
is provided. Now, as we can see year and employee ID are the two keys
for merging the data. Let's see how to merge the data. Again, left join. Let's call
this data HR joint the resulting data. Again, we will use left join.
This is a very useful command when you want to merge two data frames
for their intersection. So, it's a very useful command as you can
see accident data. Now, I will provide the two keys to merge the data
by equal to C. Now, one is employee ID
the other is year. So, when I merge this
so, this is HR two actually. So, when I merge this
let us see the glimpse of the resulting data frame and it is
HR joint. So, now the resulting data again it has
2940 observations, five columns now. So, we have year
and employee ID intact but along with location we have overtime hours
and accident type. So, now we will start with
our data examination. Now, as you can see in accident.type
there are some NA observations and some mild and others. So,
what we will do is we will put all those who did not have accident which means NA as 0.
So, these guys did not have accident while others who had some mild or any kind of accident
we will put them as 1 which means they had accident. So, let's modify our data
little bit HR underscore joint. So, this is our original data
we would like to mutate, modify and create a new variable
had accident. So, this had accident will have values
if 1 that means the employee had accident
if it is 0 that means they didn't have accident. So, I will use this is.na
accident type. So, now accident type and if it is 0
0 in case it is NA, 1 if it is not. So, that's
the way. So, you can see here for each NA we have a 0 because now with
0 lot of x analysis regression everything is possible with NA you
may not do that. So, and as of now we have not assigned it to any data.
So, let's use this assignment operation to create this basically this will have the
effect of creating a new column in my original HR joint data. Let me
print it and let me show you had accident this column is there. Now
the first question that would like to answer is whether if there is any
increase in accidents from
2016 to 17 or decrease or anything like that.
So, let's analyze the trend in accidents and that is simple. We will use our
data that we have created already HR joint. Now inside this data we
need to group by year. So, that by now you would have understood that
makes our job very easy. We'll just group it by year and then
we'll use the summarize. Now all I need to do is create an accident rate
variable and assign it the mean summary measure. So, I'll
just add a simple descriptive which is mean and this would be it.
So, you can see here if I look at year wise accident rates
it's approximately 0.085 in the year. Now because our
variable has 01 kind of form. So, we can interpret in sort of fractions.
So, 0.085 rate in 2016 in terms of fractions
and 0.12 in 2017 which means there is a clear cut increase.
Now we would not be very satisfied at this stage probably in a convention
that we have been doing would like to do some kind of statistical test. So, we would like to
statistically test this difference
in accident across 2016 and
2017. So, let's start with the typical t-test
and this is very simple. We'll use this head underscore accident
data that we have created. Head accident variable
year. So, year is our segregating variable and
now HR joined is the name of the data. So, that's pretty simple.
We'll just run this t-test and you can see here the difference mean values we have already
calculated the same mean we are seeing here but more importantly the p value
is even less than 1% which means the difference is very significant
statistically significant. So, first important result we had for this
t-test approach. Now you would also like to have chi-square testing
like we were doing chi-square test to just check the robustness or sort of verify
get a second opinion of sort in our result. So, I'll use this chi-square
test again that we know. Now we can use that because our variable is 0
1 form it's not exactly a typical numeric it's 0 1 form so we can apply
if it was not we probably wouldn't have been able to but now we can do
so dollar here we can apply this HR
joined. Now you can see the p-value again
coming out to be very significant which means there is both these tests suggest
that there is a clear cut increase in the accidents in this year. So,
we in this video we loaded the data we examined the accident and we found there is
a very significant increase in number of accidents in year 2016
to 17. We'll do the further analysis of this data in next
set of videos. In the previous video we discussed the increase in accident rates
over the year 16 to 17. Let us also see in this video
let us examine this data in more detail and see identify
the job location year wise patterns
in accident. So, we'll examine the year wise and
location wise patterns. So, first let us go back to our data that we created
HR joined. Now the question we have
we would like to examine location and year wise pattern. So, we
need to group it back location as well as year.
So, you can see it has grouped the data location as well as year.
Now we can appropriately summarize our data. So, when we use
the summarize command now and I can create a variable
accident rate. Let's say I can create this variable and
summarize it with the mean command which has
had underscore accident. Now if I summarize
it I'll actually get my data but I can rather create new variable
for future reference. So, accident rate let's call it accident rates. Let's
create the data and let me print this data now.
So, let us examine the data that we have got in more detail. So, look at
East Valley. So, here from 2016 to 17 from accident rate
has increased from 11% to 14%. Northward 6%
to 12%. Quite sharp increase. Southfield 7 to 13.
7.6 to 13 and West River 8 to 11. So, we can see across all the locations
there has been a significant rise in the accident rates. So, it seems there is some kind of
pattern. Let us also try to visualize this data. Let's see if we can draw some insight
by visualizing it. So, in this question we'll try to
visualize the accident data. So,
typical scheme we'll use the RGG plot for this kind of thing.
So, we have GG plot. We'll provide accident rate data that's
for this very purpose we have created the data. We need to supply aesthetics. Now
in aesthetics notice I am using factor as here. So, I don't want to
R let treat here as numeric variable. I want it to
treat as factor two categories and then accident rate. So, accident
rate variable and here I should have used accident rate
actually small r. Now it will be fine. So, I prefer this
kind of spellings. Now the next thing I need to specify geometric object
if you recall. So, I am using geom call probably that geom call
is a reasonable object and aesthetics I need to provide some kind of color
filling. So, I want it to segregate with here. That's why I am using factor here.
So, filling is good. It would look good the graph. Next
facet wrap command. So, facet wrap will ensure that I want to
plot. So, location wise I want to plot it location wise. So, this will
increase the sort of aesthetics of my plot. So, let us
draw it. So, you can see a very nice plot across there has been created and
there is a very clear-cut pattern. If you look at all the four locations East Valley there is a rise
Southfield there is a rise Northwood West River for all the locations there is a
clear-cut rise in the accident from 16 to 17. So, this graph gives a very
nice picture and from which we can draw lot of insights. Now
one potential reason in the increase in accident rates could be
the increase in overtime hours. So, probably we would like to examine
overtime hours from location wise and see if there is a
significant difference and we would like to do this exercise location wise.
So, we will examine overtime hours location
wise. So, for example, the first location is Southfield
first we will start with Southfield and we will
use our HR joint data. Now I will filter out with
location. So, I need to specify the location
which is Southfield. Let's see if I have run it correctly. Let me
print this. So, I have just I have done it correctly Southfield
I have categorized. Now, let's see if
let's do a simple summary and see if Southfield has
increase in this measure. So, we will group it by the year
we would like to examine the overtime hours. So, by year
and then use this summarize average
let's call it average overtime and
take it to the next line. Use the space here. Use the mean
of overtime hours. So,
we need to do it group by and let's see. So, we can see
from 16 to 17 there is a very clear cut increase in overtime hours. Now, we would
also like to add some kind of statistical test for this result. So,
because the data is numeric not that one zero kind of thing. So, we have to rely
on t-test. So, let's do that. So, t.test and
we have to check whether overtime hours
and segregated by year and data equal to Southfield.
So, data equal to Southfield.
Let's run it. Let's interpret the result. You can see
the mean we already saw. So, obviously the mean is higher but is it statistically
significant? It's like a cut-to-cut case. You can see
p-value is close to 11%. So, it's not significant at 90% level
but there is a strong signal that the overtime hours
have increased. Let's do this exercise for other localities also.
So, for other localities, I'll not rewrite the code. I'll just reuse the same
code that I have already written. For example, Northwood. Next locality is Northwood.
So, I'll just copy paste the name of the location
and put it here.
So, let's run it. So, there is a difference
approximately 0.5. Let's see if it is statistically significant.
No. So, as we can see the p and t-values, the difference between
2016 and 2017 overtime hours is not significant. So, let's pick
another location. The next location we can take is
maybe East Valley. So, we can take
East Valley. I'll just copy paste the name. Now, this name has a
space in between. So, for variable naming, I can't use it. I need to
modify it a little bit. I'll put an underscore which is an acceptable name in
our convention. So, I use this East Valley to place here
also and here also. Let's see what we get. So,
again the difference is very minor. So, I don't expect it to be statistically significant.
You can see with t and p-values, the result is not very useful. Or rather, I should say
it does not indicate any significant differences across the years 2016 and
2017. The last one is West River.
Sorry, this was East Valley.
This was the third one. The last one is West River.
So, if you look at this, let me just copy paste it.
Again, there is a space in the
name of the data. So, I need to add that underscore
to make it acceptable to R. Now, I'll just copy
paste the same name here and here.
Let me run it. So, again the difference is not
very big. So, I don't expect it to be statistically significant. As you can see here,
the p-value is quite small. The t-value and p-value indicate that the difference is not
statistically significant. So, we can say there is only one location which was the first one.
Southfield, where there was a statistically significant difference and the
t-test also confirmed the same. So, that is our takeaway from this
exercise. In the previous video, we have examined the
overtime data and accident data. Let us now examine the relationship
between engagement and accidents. And for this exercise,
we would need to use another data, which is Survey 2.
So, first let us examine the relationship
between accidents and engagement. So, we'll analyze this
and for this we need two databases. First, we need to
add this Survey 2 data. So, we need to use this Survey 2 data. We'll import this.
Let's copy this for future reference in our code file.
So, this is a useful practice. Copy it for your future reference
so that you don't have to remember the file name and location. Now, let's see what is inside
this data.
So, this carries again the year, the employee ID and engagement.
The next step would be to first merge this Survey 2 data.
So, for that we need this left join command.
We have already created this HR join data
and we also have this Survey 2 data.
So, we'll merge them. We need to be just mindful
that we are merging them by two variables, which is one is year
and another is employee ID.
These are the two variables. So, let us create a new variable. Let's call it
safety variable. So, we'll use it.
Now, first thing, we will again
create that disengaged variable that we have created earlier also.
We recall that. So, that variable was basically if the engagement level is
less than 3, then we call them disengaged. So, this would require
mutate function and we'll create
this disengaged variable and similar
exercise. It's very similar exercise. If engagement level is
less than equal to 2, then it is 1. That means employee is
disengaged and 0, then engaged. Now, we'll assign
it again in the same variable. So, if I print
it, you would notice that there is one more
variable disengaged is created now, which will take a value of 1. If
engagement score is less than equal to 2, that means less than 3. Otherwise, it will take 0.
0 means he is engaged. Also, for reference,
I'll create one more. I'll modify my year variable. That is also
needed. So, what I'll do is I'll modify the year because I need
it in a factor form for analysis because every time I keep repeating this,
I should rather create this factor here.
So, this would be useful. Now, we are ready. Let us
visualize first the disengagement level. So, let us visualize
the data engagement across
location. So, for that, we'll use our ggplot command
ggplot command and
what we'll do is we'll use our safety data
safety data
in x-axis, I want x as
year. So, I'm specifying x as year
and fill as factor
disengaged. So, this is
first. Now, I have created the plot. Now, next thing I'll do is I'll
create zoom bar. So, I'll probably use zoom bar and
position equal to fill. This will create a plot
for me. But one more thing, I would like to segregate
this thing with facetrap. So, I'll use facetrap because I want a
more detailed plot. So, I'll use facetrap and according
to location. Let us use the full code. Let us run it.
So, let us examine these plots. As you can see, if we move
from 2016 to 2017, the proportion of disengaged to
engaged is almost same except Southfield. So, in
Southfield, we can see that disengagement score has
increased actually quite a bit from 16 to 70 and this is also the place where
accident have also increased.
So, probably there is some connection here between the
disengagement and accidents. So, we'll explore this aspect
in more detail. Probably, we would like to conduct some kind of
statistical test. So, let us conduct one statistical test and
because we are fixated only on one location. So, probably I'll
create a subset of data, safety underscore South.
So, I'll use the safety underscore Southfield and I'll filter out my data.
So, for that, that is simple. We have already done that. So, I'll use my
safety data, safety. I'll filter out by location
Southfield. So, I'll create this data
and let's see if it has worked. So, this is my Southfield safety data where
only location that is there is Southfield. Now, we are good to go. So, we can do the
statistical test. So, we'll start with statistical testing and because
we have created our variable in 1-0 form, the disengagement
variable. So, statistical testing in disengagement
variable. So, I can use both the t-test. So, let's start with the t-test.
We want to see whether disengagement has increased over
years in Southfield. So, let's start with
t-test and disengaged. Now, the variable name is disengaged
and we want to examine across here specifically for the Southfield
location because visually it seems that there is some increase in disengagement.
Name of my data is safety South actually. So, I should use that.
Let me run it. Let's analyze the data. So, in
2017, the disengagement has increased. Clearly, we can see it was 0.18.
Recall, 1 is the disengagement. Those who are disengaged. So, the
proportion has increased actually and it is statistically quite significant. In fact, it is less than
1%. So, the significance level is actually 1% which means
99% confidence. So, the result is very significant. Also, we can just
test the robustness of this result with chi-square test.
That also because it is 1-0 form. So, we can conduct our chi-square, typical
chi-square test also and here we need to have that
variable safety South. We have the data which
is disengagement variable and we would like to segregate
it across the years. So, let's do that.
Let's run it and again very similar result is obtained. So, chi-square and t-test
they are in agreement. P-value is significant for chi-square as well as for the
t-test. So, we can conclude that there is quite significant difference in
disengagement across 2016 and 2017 for this location Southfield
and before concluding analysis as a precautionary check, we will
for other locations also we will check that whether there is an increase or decrease
in disengagement. So, that we will do in the next video. In this video, we
conclude by saying that there is a quite clear increase not only in terms of accident but
also in terms of disengagement level. So, there seems to be some relationship between
disengagement and accident levels. We have more to say. We will examine this in more detail
shortly. So, before culminating this exercise with the regression
modeling, we would like to see whether in other locations also there
has been increase or decrease in disengagement across years. So,
let's create a variable for other locations
disengagement levels. So, what we can do is we can
just filter out the data. So, safety variable we have already created.
In the safety variable, we are interested in
filtering filter location.
Now, in this location, except
south speed, we are interested in other. So, in one go, I am filtering all.
So, let's call it other underscore
location. Now, two tests we are going to do.
First is the typical t-test. Let's start with the t-test of other locations.
For that, it's simple. We'll use this other
underscore. So, t.test
Now, we have this disengage variable disengaged
and it has to be examined across here and the data is
other locations. So, let's see. Almost
similar. So, I am not expecting any significant p-value. So, as
we can see, both of them are similar. So, there is no significant increase.
We can cross check it with the chi-square test.
We can cross check this result. All we need to do is other
location dollar disengagement and
we need to segregate them across here. So, dollar here.
Run it. So, you can see it's not significant.
So, again, our hypothesis seems to be there that there is no difference
in other locations. The main difference is happening
in the south field. So, now, our final focus of analysis will be
south field location where we'd like to see what kind of relationship
in accident and engagement and over time is there. So, first, let's
do a simple correlation test and we'd like to see the correlation
in south field with accident and
disengage. So, let's see this correlation. South field accident
and disengagement. So, let's see. Although disengagement is 1-0
kind of variable, but let's see. Okay. So, there's a very significant correlation of
23% and the correlation is quite significant. So, it means there is some relationship
with accident and engagement. The second thing, we'd
also like to see if there is some relationship with over time.
So, that also we can test here and no, it's not
significant. So, not a very useful. So, over time seems to not be a very useful variable
but we still because we are doing the analysis in the multiple
regression analysis that we are going to do next, multiple regression analysis, we'll
still use this data. So, let's call it
save our output in the model multiple
multiple. It's called model multiple. This is where we'll save our output. Linear
modeling. Now, first variable is had accident. So, we'll use this had
accident variable which we have created. Again,
here we have already in the factor form. So, not to worry.
Again, then we have disengaged variable
disengaged variable plus
over time r. So, over time underscore r
that also we have created and data is safety
south. Just check the data if my year variable
is in. We just do a glimpse of this.
Safety south. So, I just wanted to confirm
if my year variable is in factor form. So, you can see it is factor. So, we are
good to go. So, one way would be to analyze this and just run the summary command and
see what variables are significant. Model underscore
multiple. Let's do this. Now, let's interpret the
output. So, here we can see the disengagement variable is very significant.
So, we can see over time r is not a very significant variable. So, we are going to remove it.
But disengagement is very significant. This is causing the accident. It is
playing a major role. One way to improve this exercise is to run a second
version. Remove the less important variable which is over time
r and also interact here with disengagement. So,
if there is any changes in disengagement, that will be highlighted. So, that
we can do. Let's interpret the result now. So, let us interpret
these results. First and foremost, the reference category is 2016.
As we can see, this is reference year. First and foremost, we can see from
16 to 17, there is an increase. So, we can clearly see
16 to 17, there is a significant increase in accident which is there.
More important result is that disengagement variable has a very
significant relationship. So, this disengagement variable. Now, this is for disengagement
for 2016. There also it has a very significant loading for accident.
2017 is not significant. That means whatever was in
16, the same level, the same kind of relationship can be assumed in
17. So, disengagement level has a significant relationship
with accident. It remains significant. So, we can say that whatever the
relationship was there in 16, the same carries on 17. So, there is that.
And in general, apart from that disengagement variable also, there is a kind of increase
yearly trend also in increase. So, that is also captured here.
So, there may be other factors also apart from disengagement level which are causing
increase from 16 to 17. To summarize this lesson, first we examined
the recruitment source data. We found that those coming from such firms are performing poorly
on sales quota percentage achievement and also having higher attrition rates.
In contrast, those coming from NA sources, most likely these are
lateral hirees with sources like LinkedIn or online direct mail application
are doing well on their sales quota targets and also have lower attrition rates.
Then we examined the employee engagement data. We found that engagement levels of
sales department employees are quite low as compared to other departments.
One possible reason of this could be low vacation days as the number of
vacation days are significantly lower from this department as compared to other departments.
In the next part, we examined pay fairness across the job mix and across departments.
We found that the pay of new hires is more than old employees. When we analyzed
across job mix levels, for example, hourly, salaried and managerial
levels, we found that the new hirees are less likely to be hourly workers and more
likely to be salaried and managerial employees. So, the difference in pay is not
on account of old versus new bias, but simply because the composition of new
employees is tilted more towards salaries and managers who
have higher salaries and less in case of hourly wages
who have lower salaries. We also examined the pay differences across departments.
We found that the pay differences are only significantly different for new and old employees
in the engineering department, but the pays are not significantly and statistically
different across sales and finance department. For these inferences,
we employed linear regression analysis and
t-test of means. In part 4, we examined the performance ratings across gender and department.
We found that there is a significant difference in performance ratings across male
versus female employees. Initial evidence suggested that male employees have
higher ratings than female. Subsequently, we analyzed the male versus female performance
evaluation across different job levels, for example, hourly, salaried and managers.
We found that the systematic bias or difference is only found along the hourly wages.
In contrast, the salaried and managerial levels, there is no systematic
or statistically significant bias or difference in the results.
In part 5 of the case, we examined the relationship between employee safety,
disengagement and vacation days taken. First, we found that there is
a significant increase in accident rates in the Southfield locality and there is a
considerable increase from 2016 to 2017. We found that
vacation days taken and disengagement levels may have some role to play in
demotivating the employees and causing accidents. In general, there is an increase
in accidents from 2016 to 2017 which needs further examination.
