Skip to content
Tutorials Center
My learning

Topics

Lesson 76 of 76 ยท Practice and Review

Data Science Cheat Sheet

A one-page data science cheat sheet: project checklist, metric chooser, statistical test chooser, model chooser and runnable pandas, scikit-learn and SciPy snippets.

This is the page to keep open while you work. It does not teach anything new; it condenses the course into things you look up: a checklist for a project, tables that help you choose a metric, a statistical test or a model, and snippets you can copy and run. Each section links to the lesson that explains the reasoning, because a cheat sheet is only safe to use when you remember why the entry is there.

Use it as a prompt for your own thinking. A table can tell you that a t-test fits two numeric groups, but only the data can tell you whether the groups are independent, whether the outcome is the right one and whether the effect matters.

The project checklist

Walk through this list at the start and again before you share anything. The full story is in The Data Science Workflow.

flowchart LR
    A["1 Question<br/>and metric"] --> B["2 Get and<br/>audit data"]
    B --> C["3 Clean and<br/>engineer features"]
    C --> D["4 Explore"]
    D --> E["5 Baseline<br/>then models"]
    E --> F["6 Validate<br/>and test once"]
    F --> G["7 Explain and<br/>communicate"]
    G --> H["8 Deploy,<br/>monitor, repeat"]
    D -. "new questions" .-> A
    F -. "problems found" .-> C

The workflow is a loop, not a line: exploring and validating regularly send you back.

Step Done when you can say
1. Question The decision, the target metric, the success threshold and the baseline are written down.
2. Data I know where each column comes from, its grain (one row = what?), its date range and who is missing.
3. Clean Every fix is in code, counts of changed rows are recorded, and raw data is untouched.
4. Explore I looked at every variable alone, in pairs and by group, and wrote down what surprised me.
5. Model A naive baseline exists; models are compared on identical folds.
6. Validate All fitted steps are inside the pipeline; the test set was used once.
7. Communicate One-sentence finding, effect size with uncertainty, caveats, next step.
8. Operate Inputs are checked, drift is monitored, a retraining and rollback plan exists.

Quick checks on any new table

Python A first look at a table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read those lines as the first questions you ask any table. What is a row? Are the types what they should be? Which columns have gaps? Are there duplicates? Is the key unique? See Auditing Data Quality and Loading Data from CSV, JSON and SQL.

Metric chooser

Start from the decision and the cost of each kind of error, then pick the metric that measures it. Always show a baseline next to the score.

Task Situation Metric Note
Regression All errors equally bad, outliers present MAE Same units as the target; robust.
Regression Large errors much worse RMSE Squares errors; sensitive to outliers.
Regression Targets differ in scale MAPE or error on log target Fails near zero targets.
Regression Share of variation explained R2 Compare against a baseline; can mislead for nonlinear models.
Classification Balanced classes, equal costs Accuracy Misleading under imbalance.
Classification False alarms are costly Precision Of the flagged, how many are real?
Classification Missing a case is costly Recall Of the real cases, how many caught?
Classification Need one number for both F1 (or F-beta) Beta above 1 favors recall.
Classification Ranking quality, any threshold ROC AUC Prefer PR AUC under heavy imbalance.
Classification You use predicted probabilities Log loss, calibration curve Check that 0.3 means about 30 percent.
Ranking and search Top results matter Precision at k, NDCG Depends on position.
Clustering No labels Silhouette plus business sense Profile clusters before trusting them.
Forecasting Time series MAE or MASE on a backtest Compare to the seasonal-naive forecast.

More in Evaluating Models with the Right Metrics.

Statistical test chooser

flowchart TD
    Q{"What is the outcome?"}
    Q -- "numeric" --> N{"How many groups?"}
    Q -- "categories (counts)" --> C["Chi-square test<br/>(Fisher if counts are small)"]
    N -- "one group vs a value" --> T1["One-sample t-test<br/>(Wilcoxon if skewed)"]
    N -- "two independent" --> T2["Welch t-test<br/>(Mann-Whitney if skewed)"]
    N -- "two paired" --> T3["Paired t-test<br/>(Wilcoxon signed-rank)"]
    N -- "three or more" --> T4["ANOVA<br/>(Kruskal-Wallis)"]
    Q -- "two numeric variables" --> R["Pearson correlation<br/>(Spearman if ranks or outliers)"]

Choose by the type of outcome and the number and relation of the groups. The nonparametric alternative is in parentheses.

Question Test Effect size to report
Two means differ? Welch t-test Mean difference, Cohen's d, interval
Distributions differ, skewed data Mann-Whitney U Probability one group is higher
Before/after on same units Paired t-test Mean change and interval
Proportions differ (2x2) Chi-square or two-proportion z-test Difference or ratio of rates
Several groups differ ANOVA or Kruskal-Wallis Eta-squared; then planned comparisons
Linear relationship Pearson r and interval
Monotonic relationship Spearman rho
Anything without a formula Permutation test or bootstrap Whatever statistic you chose

Rules of thumb that go with the table: decide the test and the metric before looking; report the interval and effect size, not only p; correct when you run many tests; and remember that "not significant" is not "no effect". See Hypothesis Testing Done Right.

Python Test and interval snippets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Model chooser

Situation Start with Why Watch out for
Any problem A baseline (mean, majority class, last value) Shows what no skill looks like Skipping it
Numeric target, need explanations Linear regression (maybe on log target) Interpretable, fast, stable Nonlinearity, outliers, multicollinearity
Binary target, need probabilities Logistic regression Interpretable, well calibrated Scale features; imbalance
Tabular data, nonlinear patterns Gradient boosting (HistGradientBoosting) Usually the strongest on tables Tuning, leakage, calibration
Tabular data, little tuning time Random forest Robust defaults Large models, extrapolation
Few rows (hundreds) Simple linear models Trees need more data Overfitting
No labels, find groups k-means or hierarchical clustering Fast, easy to profile Choose k, scale features
Many correlated features PCA, regularized regression Compress or shrink Interpretability of components
Time series Seasonal-naive, exponential smoothing, ARIMA Strong baselines Use time-ordered validation
Text TF-IDF plus logistic regression Surprisingly strong baseline Leakage from vocabulary fitted on all data

Pick the simplest model that meets the need, keep the pipeline honest, and only then make it fancier. See Models, Learning and the Modelling Loop and Decision Trees, Random Forests and Boosting.

A leak-proof modelling template

This is the skeleton of almost every supervised project: impute and scale numbers, encode categories, fit a model, and score it with stratified cross-validation, with all learned steps inside the pipeline.

Python Pipeline plus cross-validation template

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Swap in your own columns and model; leave the structure alone. Details are in Pipelines and Hyperparameter Tuning.

Snippet bank

These read-only lines cover the moves you use most. They assume df is a DataFrame and np, pd are imported.

Python
# Missing values, duplicates, types
df.isna().sum()                                  # missing per column
df = df.drop_duplicates(subset="id")             # keep the first row per id
df["date"] = pd.to_datetime(df["date"], errors="coerce")
df["price"] = pd.to_numeric(df["price"].str.replace(r"[$,]", "", regex=True))

# Text and categories
df["city"] = df["city"].str.strip().str.lower().map({"nyc": "new york"}).fillna(df["city"])

# Features
df["log_income"] = np.log1p(df["income"])
df["month"] = df["date"].dt.month
df["income_band"] = pd.qcut(df["income"], 4, labels=["Q1", "Q2", "Q3", "Q4"])
df["mean_by_group"] = df.groupby("group")["value"].transform("mean")

# Outlier fences (IQR rule)
q1, q3 = df["x"].quantile([0.25, 0.75])
mask = df["x"].between(q1 - 1.5 * (q3 - q1), q3 + 1.5 * (q3 - q1))

# Splits
from sklearn.model_selection import train_test_split, TimeSeriesSplit
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)

# Metrics
from sklearn.metrics import classification_report, roc_auc_score, mean_absolute_error
print(classification_report(y_te, model.predict(X_te)))

# Importance and interpretation
from sklearn.inspection import permutation_importance
imp = permutation_importance(model, X_te, y_te, n_repeats=10, random_state=0)

# Save a fitted pipeline (run on your machine)
import joblib
joblib.dump(model, "model.joblib")

Five habits that prevent most problems

  • Fix the seeds and print the versions of your libraries, so that results can be reproduced (Reproducible Analysis).
  • Look at the rows, not only the summaries: df.sample(10) finds things describe() cannot.
  • Change one thing at a time and compare on the same folds.
  • Write the finding in one sentence before building the slide (Communicating Results to Non-Experts).
  • Say what could be wrong: leakage, selection, noise, confounders (Common Data Science Mistakes).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. You compare average order value between two independent customer groups and the data is strongly skewed. Which test is a good fit?

    Two independent groups with a numeric, skewed outcome call for a rank-based test such as Mann-Whitney, or a permutation test on the statistic you care about.
  2. 2. Which metric combination is the most sensible for fraud detection where fraud is under 1 percent of transactions?

    Under heavy imbalance accuracy is almost always high. Precision and recall show the rare class, and the PR curve shows the trade-off.
  3. 3. Where must scalers, imputers and feature selectors be fitted?

    Fitting these steps on all data leaks information from validation or test rows into training.
  4. 4. What should always accompany a p-value in a report?

    The p-value says nothing about the size or practical importance of an effect. The effect size and its interval do.

Summary

  • A project is a loop: question, audit, clean, explore, baseline, validate, communicate, monitor.
  • Choose metrics from the cost of each kind of error, and always compare with a baseline.
  • Choose tests by outcome type and group structure; report effect sizes and intervals, not only p.
  • Start with simple models and put every learned step inside a pipeline.
  • Keep the snippets and habits here close, and return to the lessons for the reasoning.

You have reached the end of the track. Revisit the start with What Is Data Science?, or test yourself again with the practice exercises.