Employee Retention Prediction¶
Hiring and retaining employees are extremely complex tasks that require capital, time and skills. Companies spend 15%-20% of the employee's salary to recruit a new candidate.
In this project an AI model will be develop to reduce hiring and training costs of employees by predicting which employees might leave the company.
Importing the Libraries¶
import numpy as np
import pandas as pd
import seaborn as sns
import tensorflow as tf
from matplotlib import pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import OneHotEncoder
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.metrics import confusion_matrix, classification_report
from warnings import simplefilter
simplefilter(action='ignore', category=FutureWarning)
Loading the Datasets¶
df = pd.read_csv('Human_Resources.csv')
pd.set_option('display.max_columns', None)
df
Data Cleaning¶
df.info()
df.describe()
df.columns
Checking for Duplicates¶
df.duplicated().sum()
Checking for Null Values¶
df.isna().sum()
Changing categorical values to numerical¶
df['Attrition'] = df['Attrition'].apply(lambda x: 1 if x == 'Yes' else 0)
df['Over18'] = df['Over18'].apply(lambda x: 1 if x == 'Y' else 0)
df['OverTime'] = df['OverTime'].apply(lambda x: 1 if x == 'Yes' else 0)
df
Droppping unneccessary columns¶
df.hist(bins = 30, figsize = (20,20), color = 'blue')
df.drop(columns = ['EmployeeCount', 'StandardHours', 'Over18', 'EmployeeNumber'], axis = 1, inplace = True)
df
Comparision for people who left and stayed in the company¶
left_df = df[df['Attrition'] == 1]
stayed_df = df[df['Attrition'] == 0]
print('Total numbers of employees: ', len(df))
print('The number of employee who left: ', len(left_df))
print('The number of employee who stayed: ', len(stayed_df))
left_df.describe()
stayed_df.describe()
From the data, It is shown that the mean age of the employees who stayed is higher compare to the employees who left. It is also observed that the daily rate is higher for the employees who stayed. Employees who lives closer to home stays in the company. Employees who stayed are generally more satisfied with their jobs and environment.
Exploratory Data Analysis¶
correlation = df.corr()
f, ax = plt.subplots(figsize = (20,20))
a = sns.heatmap(correlation, annot = True, cmap = sns.cubehelix_palette(light=1, as_cmap=True))
bottom, top = a.get_ylim()
a.set_ylim(bottom + 0.5, top - 0.5)
The Job Level is strongly correlated with the Monthly Income and Total Working Years. The monthly income is strongly correlated with the total working years. The more employees stayed longer in the company the higher the income. The age is also correlated with the Monthly income.
plt.figure(figsize = (25,12))
sns.countplot(x = 'Age', hue = 'Attrition', data = df)
Around ages of 28-31 years old has a higher count of employees who leave the company.However as the age grows older less employees leave the company.
plt.figure(figsize = (20,20))
plt.subplot(4,1,1)
sns.countplot(x = 'JobRole', hue = 'Attrition', data = df)
plt.subplot(4,1,2)
sns.countplot(x = 'MaritalStatus', hue = 'Attrition', data = df)
plt.subplot(4,1,3)
sns.countplot(x = 'JobInvolvement', hue = 'Attrition', data = df)
plt.subplot(4,1,4)
sns.countplot(x = 'JobLevel', hue = 'Attrition', data = df)
Almost half of the employees in the Sales Representative left the company. However employees in the Research Director and Manager has a very low of percentage of employees who left the company.
People who has little involvement in the job has a higher tendecies to leave the company while employees with greater involvement tend not to. As you become more experienced it is highly unlikely to leave the company however if the employee is starting out or exploring, they have more tendecies to leave the company.
plt.figure(figsize = (12,7))
sns.kdeplot(left_df['DistanceFromHome'], label = 'Employees who left', shade = True, color = 'b')
sns.kdeplot(stayed_df['DistanceFromHome'], label = 'Employees who stayed', shade = True, color = 'r')
plt.xlabel('Distance From Home')
From the distance between 10-30 , as the distance increases, the number of employees who tend to leave is higher than the employees who stayed.
plt.figure(figsize = (12,7))
sns.kdeplot(left_df['YearsWithCurrManager'], label = 'Employees who left', shade = True, color = 'b')
sns.kdeplot(stayed_df['YearsWithCurrManager'], label = 'Employees who stayed', shade = True, color = 'r')
plt.xlabel('Years with Current Manager')
If you had smaller number of years with the current manager, employees tend to leave more. However as the years increases, starting from 3 years employees tend to stay in the company.
plt.figure(figsize = (12,7))
sns.kdeplot(left_df['TotalWorkingYears'], label = 'Employees who left', shade = True, color = 'b')
sns.kdeplot(stayed_df['TotalWorkingYears'], label = 'Employees who stayed', shade = True, color = 'r')
plt.xlabel('Total Working Years')
Same with total working years, employees tend to leave the company when they spent smaller number of years in the company. However, as the years increases starting from a little less than 10 years the employees tend to stay with the company.
sns.boxplot(x='MonthlyIncome', y = 'Gender', data = df)
The average of the monthly income tend to be pretty the same with females are actually little more than the males.
plt.figure(figsize = (15,10))
sns.boxplot(x='MonthlyIncome', y = 'JobRole', data = df)
The Manager and Research Director tend to be paid more. Where the Sales Representative tend to be paid less.
Data Preprocessing¶
df
categorical_col = ['BusinessTravel', 'Department', 'EducationField', 'Gender', 'JobRole', 'MaritalStatus']
cat_col = df[categorical_col]
cat_col
onehotencoder = OneHotEncoder()
cat_col = onehotencoder.fit_transform(cat_col).toarray()
cat_col
cat_col = pd.DataFrame(cat_col)
cat_col
numerical_col = ['Age', 'DailyRate','DistanceFromHome', 'Education', 'EnvironmentSatisfaction', 'HourlyRate','JobInvolvement', 'JobLevel', 'JobSatisfaction', 'MonthlyIncome', 'MonthlyRate', 'NumCompaniesWorked', 'OverTime', 'PercentSalaryHike', 'PerformanceRating',
'RelationshipSatisfaction', 'StockOptionLevel','TotalWorkingYears', 'TrainingTimesLastYear', 'WorkLifeBalance','YearsAtCompany', 'YearsInCurrentRole', 'YearsSinceLastPromotion','YearsWithCurrManager']
num_col = df[numerical_col]
num_col
data = pd.concat([cat_col, num_col], axis = 1)
data
scaler = MinMaxScaler()
data = scaler.fit_transform(data)
data
target = df['Attrition']
target
Model Training¶
X_train, X_test, y_train, y_test = train_test_split(data,target, test_size = 0.25)
X_train.shape
X_test.shape
Logistic_model = LogisticRegression()
Logistic_model.fit(X_train, y_train)
RF_model = RandomForestClassifier()
RF_model.fit(X_train, y_train)
Testing the Model¶
y_predict_L = Logistic_model.predict(X_test)
y_predict_RF = RF_model.predict(X_test)
print('Logistic Regression Accuracy: {} %'.format(100*accuracy_score(y_predict_L,y_test)))
print('Random Forest Classifier Accuracy: {} %'.format(100*accuracy_score(y_predict_RF,y_test)))
cm = confusion_matrix(y_predict_L,y_test)
ax = sns.heatmap(cm, annot = True, cmap = 'Blues')
bottom, top = ax.get_ylim()
ax.set_ylim(bottom + 0.5, top - 0.5)
print(classification_report(y_test, y_predict_L))
Random Forest Classifier¶
cm = confusion_matrix(y_predict_RF,y_test)
ax = sns.heatmap(cm, annot = True, cmap = 'PuRd')
bottom, top = ax.get_ylim()
ax.set_ylim(bottom + 0.5, top - 0.5)
print(classification_report(y_test, y_predict_RF))
Training and Testing a Deep Learning Model¶
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(units = 500, activation = 'relu', input_shape = (50,)))
model.add(tf.keras.layers.Dense(units = 500, activation = 'relu'))
model.add(tf.keras.layers.Dense(units = 500, activation = 'relu'))
model.add(tf.keras.layers.Dense(units = 1, activation = 'sigmoid'))
model.summary()
model.compile(optimizer = 'Adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
epochs_hist = model.fit(X_train, y_train, epochs = 100, batch_size = 50)
y_pred = model.predict(X_test)
y_pred = (y_pred > 0.5)
plt.plot(epochs_hist.history['loss'])
plt.title('Model Loss Progress During Training')
plt.xlabel('Epoch')
plt.ylabel('Training Loss')
plt.plot(epochs_hist.history['acc'])
plt.title('Model Accuracy Progress During Training')
plt.xlabel('Epoch')
plt.ylabel('Training Accuracy')
sn = confusion_matrix(y_test, y_pred)
ax = sns.heatmap(sn, annot = True, cmap = 'PuBu')
bottom, top = ax.get_ylim()
ax.set_ylim(bottom + 0.5, top - 0.5)
print(classification_report(y_test, y_pred))