HR ANALYTICS  ·  IBM SAMPLE DATASET  ·  1,470 EMPLOYEES
👥 IBM HR Analytics: Employee Attrition Prevention Strategy
Who leaves, why, how well we can predict it, and what retention measures would be worth
👤 About this notebook
I made this analysis as part of my data analytics portfolio. It takes an HR question from raw data to recommendations: data checks, segmentation, statistical tests with effect sizes, a predictive model evaluated against a baseline, and a business case whose assumptions are explicit.

Mihail Deligrozev · Data analyst · Python · SQL · Power BI
📌 Note
The IBM HR Analytics Employee Attrition & Performance dataset is a fictional dataset created by IBM data scientists, not real IBM records. It is a single snapshot: it shows who left, not when, so every rate here is a share of this workforce, not an annual rate.

⭐ Summary

📉 16.1% of employees left (237 of 1,470).
⏰ Overtime is the strongest single factor. 30.5% attrition with overtime vs 10.4% without (2.9x).
🎯 Risk is concentrated. Sales Representatives (39.8%), employees in their first 2 years (29.8%), under-25s (35.8%) and the lowest income band (28.6%).
💰 Leavers earn less. Median monthly income $3,202 vs $5,204 for those who stayed.
🤖 A simple model ranks risk well. Logistic regression reaches test AUC 0.82 (random forest 0.77); flagging the top 20% of risk scores finds 62% of leavers.
⚠️ Accuracy is the wrong yardstick. Predicting "everyone stays" is already 83.9% accurate, so the model is judged on AUC, precision and recall.

🧭 Contents

Section Question it answers
⚙️ 1. Setup Libraries, helpers, data path
📊 2. Data loading and quality Is the data complete and clean?
🧹 3. Preprocessing and segments Which age, income and tenure bands, and do they differ?
📈 4. Overall attrition How big is the problem?
🏢 5. Departments and job roles Where is attrition concentrated?
👥 6. Demographics, pay and satisfaction Who leaves?
🔑 7. Key factor tests Which factors matter, and how strongly?
📉 8. Correlations How do numeric features relate to leaving?
🤖 9. Predictive model Can we flag at-risk employees in advance?
💼 10. Business case What would retention measures be worth?
🏁 11. Findings, roadmap and limits What to do, and what not to over-claim

How to read it: 🧠 purple boxes explain why a step is done, 💡 blue boxes give the computed insight, ⚠️ orange boxes flag caveats. Orange in charts means left or above the average attrition rate; teal means stayed. Cells starting with # Parameters hold assumptions you can change.

⚙️ 1. Setup

🧠 Why this step
Helper functions (chart style, confidence intervals, effect sizes, note boxes) live in hr_utils.py, so the notebook shows the analysis. Files are found rather than hard-coded, so it runs on Kaggle and locally without edits.
Reading Attrition.csv

📊 2. Data loading and quality

🧠 Why this step
Before any rate is computed: are there gaps, duplicates or columns that carry no information? One compact profile replaces separate checks, so problems are visible at a glance.
value
Employees (rows) 1,470
Columns 35
Numeric / text columns 26 / 9
Missing values 0
Duplicate rows 0
Duplicate employee numbers 0
Constant columns EmployeeCount, Over18, StandardHours
Job roles / departments 9 / 3
Age Attrition BusinessTravel DailyRate Department DistanceFromHome Education EducationField EmployeeCount EmployeeNumber EnvironmentSatisfaction Gender HourlyRate JobInvolvement JobLevel JobRole JobSatisfaction MaritalStatus MonthlyIncome MonthlyRate NumCompaniesWorked Over18 OverTime PercentSalaryHike PerformanceRating RelationshipSatisfaction StandardHours StockOptionLevel TotalWorkingYears TrainingTimesLastYear WorkLifeBalance YearsAtCompany YearsInCurrentRole YearsSinceLastPromotion YearsWithCurrManager
0 41 Yes Travel_Rarely 1102 Sales 1 2 Life Sciences 1 1 2 Female 94 3 2 Sales Executive 4 Single 5993 19479 8 Y Yes 11 3 1 80 0 8 0 1 6 4 0 5
1 49 No Travel_Frequently 279 Research & Development 8 1 Life Sciences 1 2 3 Male 61 2 2 Research Scientist 2 Married 5130 24907 1 Y No 23 4 4 80 1 10 3 3 10 7 1 7
2 37 Yes Travel_Rarely 1373 Research & Development 2 2 Other 1 4 4 Male 92 2 1 Laboratory Technician 3 Single 2090 2396 6 Y Yes 15 3 2 80 0 7 3 3 0 0 0 0
3 33 No Travel_Frequently 1392 Research & Development 3 4 Life Sciences 1 5 4 Female 56 3 1 Research Scientist 3 Married 2909 23159 1 Y Yes 11 3 3 80 0 8 3 3 8 7 3 0
4 27 No Travel_Rarely 591 Research & Development 2 1 Medical 1 7 1 Male 40 3 1 Laboratory Technician 2 Married 3468 16632 9 Y No 12 3 4 80 1 6 3 3 2 2 2 2
💡 Insight
The data is complete: 0 missing values and 0 duplicates. 3 columns hold a single value (EmployeeCount, Over18, StandardHours) and carry no information, so they are dropped. EmployeeNumber is an ID and is kept out of the analysis.

🧹 3. Preprocessing and segments

Three bands turn continuous variables into groups HR can act on. Each band is checked with a chi-square test and Cramér's V, an effect size from 0 (no link) to 1, where about 0.1 is small, 0.3 medium and 0.5 large.

Band Groups Career-stage logic
Age 18-25, 26-35, 36-45, 46-55, 56-60 First jobs, early career, mid-career, late career, pre-retirement
Monthly income Low (<$3k), Medium ($3-6k), High ($6-10k), Very high ($10k+) Roughly the quartiles of the income distribution
Tenure 0-2, 3-5, 6-10, 10+ years Onboarding, establishing, established, long-serving
🧠 Why this step
A p-value alone only says a difference exists; with 1,470 people even small differences are "significant". Cramér's V says how large it is.
chi2 p cramers_v
Age group 59.439 < 0.001 0.201
Monthly income 64.287 < 0.001 0.209
Years at company 65.695 < 0.001 0.211
chart
💡 Insight
Attrition falls with age, pay and tenure. It is highest for 18-25 year olds (35.8%), the lowest income band (28.6%) and employees in their first 2 years (29.8%), against 16.1% overall. All three bands differ significantly (p < 0.001) with small-to-medium effect sizes. The 56-60 group is small, so its rate has a wide interval.

📈 4. Overall attrition

🧠 Why this step
The headline rate sets the scale. Translating it into money makes it concrete, but the cost per leaver is an assumption, so it lives in a parameter cell where it can be changed.
chart
💡 Insight
237 employees (16.1%) left. At an assumed $50,000 per leaver, that is about $11.8M in replacement costs for this workforce, and each leaver prevented is worth about $50,000.
⚠️ Read with care
The data has no dates, so this is not an annual rate and cannot be compared directly with published yearly turnover benchmarks. The cost per leaver is an assumption: published estimates range from a fraction of a year's salary to more than one.

🏢 5. Departments and job roles

🧠 Why this step
Retention budgets are allocated by team. The 95% intervals show which differences are reliable: a role with 50 people can swing several points by chance.
chart
employees left attrition %
JobRole
Sales Representative 83 33 39.800
Laboratory Technician 259 62 23.900
Human Resources 52 12 23.100
Sales Executive 326 57 17.500
Research Scientist 292 47 16.100
Manufacturing Director 145 10 6.900
Healthcare Representative 131 9 6.900
Manager 102 5 4.900
Research Director 80 2 2.500
💡 Insight
Sales has the highest department rate (20.6%, 1.28x the average). By role, Sales Representatives stand out at 39.8% (33 of 83), followed by Laboratory Technicians (23.9%) and Human Resources (23.1%). Senior roles are the most stable: Research Directors 2.5%, Managers 4.9%.
⚠️ Read with care
Human Resources has only 52 people in the HR role and 63 in the department, so its rate has a wide interval; treat it as a signal to watch rather than a firm finding.

👥 6. Demographics, pay and satisfaction

🧠 Why this step
Income is heavily skewed (a few high earners pull the mean up), so groups are compared with medians and the Mann-Whitney test, which does not assume a normal distribution. Charts use the same colours throughout: teal = stayed, orange = left.
median left median stayed mean left mean stayed p (Mann-Whitney) rank-biserial r
feature
Age (years) 32.000 36.000 33.610 37.560 < 0.001 0.270
Monthly income ($) 3,202.000 5,204.000 4,787.090 6,832.740 < 0.001 0.310
Years at company 3.000 6.000 5.130 7.370 < 0.001 0.300
Total working years 7.000 10.000 8.240 11.860 < 0.001 0.310
chart
💡 Insight
Leavers are younger (median 32 vs 36 years) and earn less: median monthly income $3,202 vs $5,204 (means $4,787 vs $6,833, a gap of $2,046). Both differences are significant (p < 0.001). Age, pay and tenure move together, so these are partly the same story: junior people earn less and have been here less long.
chart
💡 Insight
The lowest score on each scale goes with the highest attrition: job satisfaction 1 → 22.8%, environment satisfaction 1 → 25.4%, work-life balance 1 → 31.2%. Above the lowest level the differences are small, so the risk sits with the clearly unhappy group rather than rising steadily.

🔑 7. Key factor tests

🧠 Why this step
Testing every factor the same way, and ranking by effect size instead of p-value, shows which levers matter most. Charts show the attrition rate per level rather than raw counts, because counts hide rates when groups differ in size.
chart
chi2 p cramers_v significant
OverTime 87.564 < 0.001 0.244 yes
StockOptionLevel 60.598 < 0.001 0.203 yes
MaritalStatus 46.164 < 0.001 0.177 yes
JobInvolvement 28.492 < 0.001 0.139 yes
BusinessTravel 24.182 < 0.001 0.128 yes
EnvironmentSatisfaction 22.504 < 0.001 0.124 yes
JobSatisfaction 17.505 < 0.001 0.109 yes
WorkLifeBalance 16.325 < 0.001 0.105 yes
RelationshipSatisfaction 5.241 0.155 0.060 no
Gender 1.117 0.291 0.028 no
chart
💡 Insight
Overtime is the strongest single factor: 30.5% attrition with overtime vs 10.4% without (2.9x), and 28% of employees work overtime. Single employees (25.5%) and frequent travellers (24.9%) are also at higher risk. Not significant: RelationshipSatisfaction, Gender.

📉 8. Correlations

🧠 Why this step
Correlation with the 0/1 attrition flag gives one comparable number per numeric feature. It only captures straight-line links, so the band and factor charts above remain the better guide to shape.
chart
💡 Insight
Strongest protective features: TotalWorkingYears (-0.17), JobLevel (-0.17), YearsInCurrentRole (-0.16), MonthlyIncome (-0.16). Risk features: DistanceFromHome (+0.08), NumCompaniesWorked (+0.04). All correlations are weak (|r| < 0.2), and tenure, age, job level and income are strongly correlated with each other, so they should not be read as separate causes.

🤖 9. Predictive model

🧠 Why this step
Only 16% of employees left, so a model that predicts "everyone stays" is already 84% accurate and useless. Models are therefore compared on AUC (how well they rank leavers above stayers), with 5-fold cross-validation on the training set because the test set holds only 71 leavers. The decision threshold is chosen on cross-validated training predictions, never on the test set, and then checked once on the test set.
42 features after encoding · train 1,029 (166 leavers) · test 441 (71 leavers)
CV AUC (mean) CV AUC (sd) test AUC test avg precision test accuracy at 0.5
model
Baseline: always "stays" 0.500 0.000 0.500 0.161 0.839
Random forest 0.809 0.040 0.766 0.406 0.819
Logistic regression 0.824 0.053 0.820 0.599 0.751
chart
💡 Insight
Logistic regression has the best cross-validated AUC (0.82) and test AUC 0.82; the random forest reaches 0.77. Ranking employees by risk, the top 20% contain 62% of all leavers in the test set. Accuracy hides all of this: always predicting "stays" scores 83.9%.
threshold flagged tp fp fn precision recall accuracy
threshold 0.5 0.500 137.000 49.000 88.000 22.000 0.358 0.690 0.751
threshold 0.66 (chosen) 0.660 74.000 39.000 35.000 32.000 0.527 0.549 0.848
chart
💡 Insight
At the chosen threshold the model flags 74 of 441 test employees and catches 39 of 71 leavers (recall 55%); 53% of flagged people actually leave (precision), against 16% if picked at random. The default 0.5 threshold would catch 49 leavers but flag 137 people (precision 36%); the threshold is a trade-off between missing leavers and spending time on people who would have stayed. In practice the output is a short list for stay interviews, not an automatic decision about individuals.
chart
💡 Insight
The model mainly uses OverTime_Yes, BusinessTravel_Travel_Frequently, JobRole_Sales Representative, JobRole_Laboratory Technician, YearsSinceLastPromotion. These are associations the model exploits, not proven causes.
⚠️ Read with care
The random forest's built-in importance ranks DailyRate #7, although its correlation with leaving is only -0.06. Built-in tree importance favours features with many distinct values, which is why permutation importance on the test set is used here.

💼 10. Business case

🧠 Why this step
Recommendations are more convincing with numbers, but only if the assumptions are visible. Each scenario below states its inputs in the parameter cell; the prevented departures come from the data where possible.
logic prevented savings cost net ROI
Reduce overtime 40% of 416 overtime workers drop to the no-overtime rate (−20.1 pts) 33 $1,650,000 $400,000 $1,250,000 +312%
Targeted pay rises 35% of 30 targeted leavers stay 10 $500,000 $850,000 $-350,000 -41%
Early-career programme 25% of 162 leavers with 0-5 years tenure stay 40 $2,000,000 $380,000 $1,620,000 +426%
Risk score + stay interviews model catches 55% of leavers, 30% of those are kept 39 $1,950,000 $150,000 $1,800,000 +1200%
chart
💡 Insight
Under these assumptions Risk score + stay interviews ($+1.80M) and Early-career programme ($+1.62M) give the highest net value; 3 of 4 options pay back their cost. Targeted pay rises come out at -41% on replacement cost alone (10 departures prevented for $850,000); they may still be right for fairness or market reasons, but the retention case alone does not carry them.
⚠️ Read with care
The scenarios overlap (89 leavers are both early-career and on overtime), so their savings are not additive. Prevention rates and costs are assumptions, not estimates from this data, and the data has no time dimension, so "per year" figures are not possible.

🏁 11. Findings, roadmap and limits

📊 What the data says

  1. ⏰ Overtime is the biggest lever. 30.5% attrition with overtime vs 10.4% without (2.9x, the largest effect size of all factors).
  2. 🎯 Risk is concentrated in Sales Representatives (39.8%), Laboratory Technicians (23.9%), the first two years of tenure (29.8%), under-25s (35.8%) and the lowest pay band (28.6%).
  3. 💰 Pay, age and experience move together. Leavers have a median income of $3,202 vs $5,204, are younger and have shorter tenure; these are overlapping signals of being early in a career.
  4. 😟 Dissatisfaction matters at the bottom of the scale. The lowest job satisfaction, environment satisfaction and work-life balance scores have clearly higher attrition; above that, differences are small.
  5. 🤖 Risk can be ranked, not predicted with certainty. The logistic regression reaches test AUC 0.82; at a threshold aimed at catching 60% of leavers it catches 55% with a precision of 53%.

🗺️ Suggested roadmap

Phase Actions Measure of success
Months 1-3 Overtime dashboard per department; first risk-score list for HR business partners; stay interviews for the top-risk group Overtime share falls from 28.3%; every flagged employee has a conversation
Months 4-6 Mentoring and clear career steps for employees in their first 2 years; review pay for Sales Representatives and Laboratory Technicians First-2-year attrition falls from 29.8%
Months 7-12 Retrain the model on new data with dates; track results against a comparison group Overall attrition below its current 16.1%

📈 KPIs to track

Indicator Current value Type
Overtime share of employees 28.3% Leading
Share of employees with job satisfaction 1 19.7% Leading
Attrition, first 2 years of tenure 29.8% Lagging
Attrition, Sales Representatives 39.8% Lagging
Overall attrition 16.1% Lagging

⚠️ Limitations

🚀 Next steps

✅ Result
Saved output/attrition_clean.csv, output/test_risk_scores.csv, output/business_case.csv and 12 charts in figures (also zipped as output/figures.zip).