Skip to content
Tutorials Center
My learning

Topics

Lesson 67 of 76 ยท Projects

Project: Survey Analysis

Turn messy survey responses into findings: standardize free text, parse Likert answers, run grouped tests with effect sizes, draw charts, and write the report.

Survey data is where the clean tables of tutorials meet reality. People type their department four different ways, answer "how many days do you work remotely?" with "3", "three" or "3 days", skip questions, and sometimes submit the form twice. Yet decisions get made from surveys every day: whether to change an office policy, where to add support staff, which team needs attention. A careful analyst can turn the mess into a few findings people can trust.

In this project you will take an employee survey with 423 raw submissions through the whole path: inspect the mess, write a cleaning function that you can test, run the comparisons that answer the question, draw two honest charts, and write the findings. The data is synthetic and seeded, so everything is reproducible, and we know which differences were really built in. The question: who is less satisfied at work, and does remote work help?

flowchart LR
    A["Raw responses<br/>(423 rows)"] --> B["Inspect the mess"]
    B --> C["Cleaning function<br/>+ validation checks"]
    C --> D["Grouped comparisons<br/>(tests + effect sizes)"]
    D --> E["Charts"]
    E --> F["Findings report"]

Survey analysis in six steps. The cleaning function is separate from the analysis so you can rerun the whole thing on next quarter's data.

Step 1: see how bad it is

Never clean blindly. First list what each column contains. For categorical answers, unique() and value_counts() reveal every variant; for numeric-looking text columns, look at a sample. Printing the unique values as a Python list is useful because it shows hidden whitespace as part of the quoted text.

Python Inspect the raw survey

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output tells a clear story. The department column has 17 distinct spellings for four real departments: case differences (ENGINEERING), stray spaces (' Sales'), abbreviations (Eng., Mktg, Cust. Support) and a typo (Engeneering). The remote-days column mixes digits, words and n/a. Tenure comes as months, years or mo. Satisfaction appears in two capitalization styles, with blanks. And three submissions repeat an id.

If you counted departments without cleaning, the exact label "Engineering" would cover only 103 of its 170 respondents and every group comparison would be wrong. That is the real danger of messy categories: nothing fails, the numbers are just quietly incorrect.

Step 2: write the cleaning as one function

Put every fix into a function that takes the raw table and returns the clean one. Then the cleaning is documented, repeatable and testable. The rules:

  • Duplicates: keep the first submission per id.
  • Department: lowercase, remove punctuation, trim spaces, then look the result up in an explicit mapping dictionary. Anything not in the dictionary becomes missing, so new typos show up as a visible gap rather than silently spreading.
  • Remote days: lowercase, drop the word "days", map number words to digits, convert to numbers, and let n/a become missing.
  • Tenure: extract the number and the unit with one regular expression and convert everything to months.
  • Satisfaction: the standard five-point Likert labels become 1 to 5.
  • Recommend: any answer starting with "y" is 1, with "n" is 0.

Mapping with a dictionary instead of fuzzy matching is a deliberate choice: you can read every rule, and a surprising value fails loudly. See Cleaning Text and Categories for fuzzy techniques when the variants are too many to list.

After cleaning, validate. The assertions at the bottom state what must be true of clean data. If next quarter's file breaks one of them, you find out immediately.

Python Clean and validate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Three duplicate submissions were removed (423 to 420 rows), all four departments now have sensible counts (Engineering 170, Support 106, Sales 83, Marketing 61), and the only missing cells are the true blanks: 8 remote-days answers and 15 satisfaction answers. Notice that we did not fill the blanks. A missing satisfaction answer is information about the respondent; inventing a value would invent opinions. For the comparisons below we simply analyze the people who answered each question and report how many that was. This is acceptable when blanks are few and not obviously related to the answer; if the people who skip the question are the unhappy ones, the averages are biased upward, and you would say so in the report. See Handling Missing Values.

Step 3: ask the questions and test the answers

Satisfaction is an ordinal scale: 4 is more than 3, but we cannot say the gap between 3 and 4 equals the gap between 4 and 5. Means of Likert scores are widely used and fine for summaries, but for tests we prefer rank-based methods that make no assumption about equal spacing and cope with the many ties in five-point data.

The plan, written before looking at results:

  1. Do departments differ in satisfaction? Kruskal-Wallis is the rank-based version of a one-way comparison.
  2. Is Support, the department we suspect, lower than everyone else? One pre-planned comparison (Mann-Whitney) avoids running a dozen tests and hoping something is significant. We also report an effect size: the probability that a random Support answer is higher than a random answer from another department (0.50 would mean no difference).
  3. Do more remote days go with higher satisfaction? Spearman correlation, plus means in three bands of remote days with bootstrap confidence intervals.
  4. Would people recommend the company, by department? A chi-square test on the counts.
Python Grouped tests, intervals and two charts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Reading the results

Departments differ. Support averages 3.34, while Sales, Marketing and Engineering sit at 3.84 to 3.92. The Kruskal-Wallis test gives p = 0.000005. The planned Support-versus-others comparison gives p about 0.0000002, and a random Support answer is higher than a random answer from another department only 34 percent of the time. Medians tell the same story: 3 for Support, 4 elsewhere. A very small p-value alone would not make us care; the effect size does: this is a clear, moderate difference, not a rounding error. See Hypothesis Testing Done Right.

Remote work helps up to a point. The Spearman correlation of 0.21 says satisfaction rises with remote days, but the band means show the shape: 3.42 for people with 0 to 1 remote days, 3.90 for 2 to 3 days and 3.84 for 4 to 5 days. The bootstrap intervals for the last two bands overlap heavily (3.77 to 4.02 and 3.66 to 4.03), so we cannot claim that going fully remote beats a hybrid schedule; we can claim that people in the office almost every day are less satisfied. A single correlation number would have hidden this plateau, which is why we looked at the groups.

Recommendation follows the same pattern. Only 42 percent of Support staff would recommend the company, against 57 to 67 percent elsewhere (chi-square p = 0.0015).

Charts that tell the truth

The left chart is a diverging-color stacked bar of the five answer levels per department, ordered from least to most satisfied. Stacked bars of shares let you see the whole distribution instead of one mean: Support has far more red and orange. The right chart shows the means with their intervals, so the reader sees uncertainty, not false precision. Both follow the advice of Principles of Effective Visualization: sorted categories, direct labels, no decoration.

Correlation is not cause. The survey is observational. People who like their job may choose or be granted remote days, and Support roles may have fewer remote options for reasons that also affect satisfaction. The data says who is less satisfied and where, not what would fix it. Read Causal Thinking and Confounding before recommending a policy.

Step 4: the findings report

End with prose a manager can read in a minute: the question, the answer, the size of the effect, the caveats and a suggested next step.

Findings. Of 420 employees who submitted the survey (405 answered the satisfaction question), Support is clearly the least satisfied department: mean 3.3 out of 5 against 3.8 to 3.9 elsewhere, and only 42 percent would recommend the company, against 57 to 67 percent. Satisfaction is lowest for people who work remotely 0 to 1 days per week (3.4) and about 0.5 points higher for 2 to 5 days; we cannot separate 2 to 3 days from 4 to 5. Caveats: self-reported, one point in time, observational; 15 people skipped the satisfaction question. Suggested next step: ask Support staff what drives the gap with a short follow-up interview round before changing policy.

Notice what the report includes: the number of respondents, the effect sizes, the uncertainty and the limits. What it leaves out matters too: the list of every test you ran, the cleaning details (those go in the appendix or the code), and any claim the data cannot support. More templates are in Communicating Results to Non-Experts.

Ideas to extend the project

  • Add a free-text comment column and look for the most common words among the least satisfied respondents (Working with Text Data).
  • Test whether tenure matters, and whether Support's gap persists after accounting for tenure and remote days with a regression.
  • Check non-response: are respondents of one department over- or under-represented compared with the company headcount (Sampling and Collection Bias)?
  • Convert the cleaning function into tests with small fixtures (Testing Data Code).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does the cleaning function map department names with a dictionary and leave unknown values missing?

    Explicit rules are readable and fail loudly. A value you did not anticipate becomes missing, so the validation step catches it.
  2. 2. Why use Kruskal-Wallis and Mann-Whitney tests on five-point Likert answers?

    Likert scales are ordinal. Rank-based tests use the order of answers without pretending that the distances between levels are equal.
  3. 3. The bands 2-3 days and 4-5 days have overlapping bootstrap intervals. What can you conclude?

    Overlapping intervals mean the difference between those two bands could be zero. The clearly lower group is the one with 0 to 1 remote days.
  4. 4. Why not fill in the blank satisfaction answers with the average?

    Imputing a typical value for an opinion question adds artificial data. With few blanks, analyzing the answered cases and noting the count is more honest.
  5. 5. Which sentence is the most defensible conclusion from this observational survey?

    The survey shows an association and a difference between groups. It cannot show that a policy change would cause an improvement.

Summary

  • Inspect messy columns first: list unique values, sample the strings, count duplicates.
  • Put all cleaning in one function, map categories with explicit dictionaries, and validate the result with assertions.
  • Leave honest blanks blank; analyze who answered and say how many.
  • Use rank-based tests for ordinal scales, plan comparisons before looking, and always add an effect size and an interval.
  • Look at groups, not just a correlation: the remote-days effect has a plateau.
  • A report states the finding, its size, the caveats and the next step, and avoids causal claims from observational data.

Next: Practice Exercises with Solutions, to practice cleaning, EDA, statistics and modelling on your own.