scikit-learn Tutorial
Learn scikit-learn step by step: 40 lessons in 9 chapters with examples, quizzes, diagrams and projects.
Start learning →
of lessons completed
Getting Started
- What Is scikit-learn? Meet scikit-learn, Python's standard library for classical machine learning: what it covers, what it leaves out, how to install it and how its parts fit together.
- Your First Model Build your first scikit-learn model on the iris dataset: split the data, fit a classifier, predict new flowers, score the result and inspect mistakes.
- Datasets and Data Formats Learn the three families of scikit-learn datasets (load, fetch, make), the Bunch object, how to use DataFrames, and what shape X and y must have.
- The Estimator API Understand the scikit-learn estimator, predictor and transformer interfaces, hyperparameters vs learned attributes, get_params, set_params, clone and repr.
- Train, Validation and Test Sets Split data correctly with train_test_split: choose test sizes, stratify classes, fix the seed, create a validation set and avoid the data leakage that inflates scores.
Data Preparation
- Scaling and Normalization Put features on comparable scales with StandardScaler, MinMaxScaler, RobustScaler, MaxAbsScaler and Normalizer, and learn which algorithms need scaling and which do not.
- Handling Missing Values Detect and fill missing data with SimpleImputer, KNNImputer and IterativeImputer, add missing-value indicators, and know which models accept NaN directly.
- Encoding Categorical Features Turn text categories into numbers with OneHotEncoder, OrdinalEncoder, LabelEncoder and TargetEncoder; handle unseen categories, rare values and high-cardinality columns.
- Binning, Power Transforms and Custom Features Reshape skewed or awkward features with KBinsDiscretizer, Binarizer, PowerTransformer, QuantileTransformer and FunctionTransformer, plus PolynomialFeatures.
- Outliers and Data Cleaning Find outliers with the IQR rule, IsolationForest, LocalOutlierFactor and EllipticEnvelope, decide whether to remove or clip them, and clean duplicates and constant columns.
Regression
- Linear Regression Fit LinearRegression in scikit-learn, read coef_ and intercept_, predict, measure error with MAE, RMSE and R-squared, and check residuals on a real dataset.
- Ridge, Lasso and Elastic Net Control overfitting in linear models with Ridge, Lasso and ElasticNet: how the alpha penalty works, why scaling matters, and how RidgeCV and LassoCV choose alpha.
- Polynomial Regression Fit curves with linear models: combine PolynomialFeatures and LinearRegression in a pipeline, choose the degree, see overfitting, and tame it with Ridge.
- Decision Tree and k-NN Regression Fit non-linear regression with DecisionTreeRegressor and KNeighborsRegressor: tune depth and neighbors, see step-shaped predictions, and learn their limits.
- Ensembles and Support Vector Regression Boost accuracy with RandomForestRegressor, GradientBoostingRegressor and HistGradientBoostingRegressor, fit non-linear data with SVR, and compare all regressors fairly.
Classification
- Logistic Regression Classify with LogisticRegression: how it turns a weighted sum into a probability, predict_proba, the C parameter, thresholds, multiclass problems and scaling.
- k-Nearest Neighbors Classifier Classify by majority vote of nearby samples with KNeighborsClassifier: choosing k, distance weighting, metrics, why scaling matters and the curse of dimensionality.
- Decision Tree Classifier Build and read a DecisionTreeClassifier: how Gini impurity picks splits, controlling depth and leaves, export_text, plot_tree, feature importances and pruning.
- Random Forests and Boosting Classify with RandomForestClassifier, ExtraTrees, AdaBoost, GradientBoostingClassifier and HistGradientBoostingClassifier; learn bagging, boosting, voting and key settings.
- Support Vector Machines Use SVC and LinearSVC for classification: maximum-margin idea, kernels (linear, rbf, poly), tuning C and gamma, scaling, probabilities and when SVMs shine.
- Naive Bayes Classify with Naive Bayes in scikit-learn: Bayes' rule and the naive assumption, GaussianNB, MultinomialNB for text counts, BernoulliNB, smoothing and a spam filter.
Model Evaluation and Selection
- Train/Test Split and Cross-Validation Estimate how a scikit-learn model will behave on new data with hold-out splits, k-fold, stratified and grouped cross-validation, cross_val_score and cross_validate.
- Evaluation Metrics for Classification and Regression Pick the right scikit-learn metric: confusion matrix, precision, recall, F1, ROC AUC and average precision for classifiers; MAE, RMSE and R-squared for regressors.
- Hyperparameter Tuning with Grid and Random Search Tune scikit-learn models with GridSearchCV, RandomizedSearchCV and successive halving; read cv_results_, search pipelines safely and avoid optimistic scores.
- Learning Curves, Validation Curves and Overfitting Diagnose underfitting and overfitting in scikit-learn with learning_curve and validation_curve, understand the bias-variance trade-off and know what to do next.
Unsupervised Learning
- K-Means Clustering Group unlabeled data with scikit-learn KMeans: how the algorithm works, choosing k with inertia and silhouette scores, scaling, k-means++ and MiniBatchKMeans.
- Hierarchical Clustering and DBSCAN Cluster non-round data with scikit-learn: agglomerative clustering and linkage, dendrograms, DBSCAN density clustering with eps and min_samples, plus Gaussian mixtures.
- Principal Component Analysis (PCA) Reduce dimensions with scikit-learn PCA: explained variance, choosing components, scaling, inverse_transform for compression, visualization, TruncatedSVD and KernelPCA.
- Anomaly Detection and Manifold Learning Find outliers with IsolationForest, LocalOutlierFactor and OneClassSVM, and explore nonlinear structure with t-SNE, Isomap and other scikit-learn manifold methods.
Pipelines and Feature Engineering
- Pipelines and ColumnTransformer Chain preprocessing and models with scikit-learn Pipeline, handle numeric and categorical columns with ColumnTransformer, avoid data leakage and tune the whole workflow.
- Writing Custom Transformers Create your own scikit-learn preprocessing steps with FunctionTransformer or by subclassing BaseEstimator and TransformerMixin, so custom logic works in pipelines and searches.
- Feature Engineering and Feature Selection Improve scikit-learn models with encoders, scalers, polynomial features, binning and imputation, then trim features with SelectKBest, RFE, L1 and permutation-based selection.
- Text Feature Extraction and Text Classification Turn text into numbers with scikit-learn CountVectorizer, TfidfVectorizer and HashingVectorizer, then build a text classification pipeline with Naive Bayes and logistic regression.
Saving and Using Models
- Saving and Loading Models with joblib Persist trained scikit-learn pipelines with joblib and pickle, avoid version and security pitfalls, store metadata, and learn when to use skops or ONNX instead.
- Inspecting and Explaining Models Understand what a scikit-learn model learned: linear coefficients, tree feature importances, permutation importance on held-out data, partial dependence and ICE curves.
- Probability Calibration and Decision Thresholds Make predicted probabilities trustworthy with CalibratedClassifierCV and calibration curves, then choose a decision threshold with precision-recall trade-offs and costs.
Projects and Practice
- Project: Predict House Prices Build an end-to-end scikit-learn regression project: create a realistic housing table, clean it, compare models, tune a gradient boosting pipeline and report honest test errors.
- Project: Predict Customer Churn Build an imbalanced-classification project in scikit-learn: prepare a churn table, compare models with the right metrics, tune a threshold and turn predictions into a ranked call list.
- Project: Customer Segmentation Segment customers with scikit-learn: build RFM features from transactions, scale and cluster with k-means, choose k, validate stability, name the segments and visualize them with PCA.
- scikit-learn Practice Exercises Practice scikit-learn with eight exercises on cross-validation, metrics, tuning, clustering, PCA, pipelines, leakage and saving models, each with a runnable solution you can edit.