Skip to content
Tutorials Center
My learning

Topics

Python Reference ยท Third-party Libraries

The scikit-learn library

Overview of scikit-learn: the estimator API with fit and predict, train_test_split, preprocessing, pipelines, cross-validation, metrics and common data-leak mistakes.

scikit-learn (imported as sklearn) is the standard Python library for classical machine learning: classification, regression, clustering, dimensionality reduction, preprocessing and model evaluation. Its great strength is a consistent interface. Almost every model is an estimator with the same few methods, so swapping one algorithm for another is usually a one-line change.

Install it with python -m pip install scikit-learn. It works on NumPy arrays and pandas tables. The examples run in the browser with small built-in datasets.

Python
from sklearn.linear_model import LinearRegression

model = LinearRegression().fit([[1], [2], [3]], [2, 4, 6])
print(model.predict([[4]]))

Try it

Python The fit / predict / score cycle

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python A pipeline with scaling and cross-validation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The estimator API

Method Meaning
fit(X, y) Learn from data. Returns the estimator itself, so calls can be chained.
predict(X) Produce predictions for new rows.
predict_proba(X) Class probabilities, for classifiers that support them.
score(X, y) Default quality score (accuracy for classifiers, R squared for regressors).
transform(X) For transformers such as scalers: apply what was learned.
fit_transform(X) Learn and transform in one step (training data only).
get_params(), set_params() Read or change hyperparameters.

X is a 2-D table of features (rows are samples, columns are features); y is a 1-D array of targets. Learned values are stored in attributes ending with an underscore, such as model.coef_.

Most used modules

Module Contents
sklearn.model_selection train_test_split, cross_val_score, GridSearchCV.
sklearn.preprocessing StandardScaler, OneHotEncoder, LabelEncoder.
sklearn.linear_model LinearRegression, LogisticRegression, Ridge.
sklearn.tree, sklearn.ensemble Decision trees, random forests, gradient boosting.
sklearn.neighbors, sklearn.svm k-nearest neighbors, support vector machines.
sklearn.cluster, sklearn.decomposition KMeans, PCA.
sklearn.metrics accuracy_score, confusion_matrix, mean_squared_error, r2_score.
sklearn.pipeline Pipeline, make_pipeline, ColumnTransformer.

Usage notes

  • Always hold out test data. Scoring a model on the rows it learned from tells you almost nothing. Use train_test_split or cross-validation.
  • Avoid data leakage. Fit scalers and encoders on the training data only. A Pipeline does this correctly, including inside cross-validation.
  • Set random_state wherever randomness is involved (train_test_split, forests, KMeans) to get repeatable results.
  • Features must be numeric. Convert categories with OneHotEncoder and handle missing values with SimpleImputer.
  • Scale when distances or gradients matter (k-nearest neighbors, SVMs, logistic regression). Tree-based models do not need it.
  • Accuracy can mislead on imbalanced classes; look at precision, recall and the confusion matrix.
  • fit on new data starts over. Fitting an estimator again replaces what it learned.
  • Shape errors: a single sample must be 2-D, for example [[5.1, 3.5, 1.4, 0.2]], not a flat list.
  • Not for deep learning. Neural networks at scale use PyTorch or TensorFlow; scikit-learn covers classical methods well.