Python Reference ยท Third-party Libraries
The scikit-learn library
Overview of scikit-learn: the estimator API with fit and predict, train_test_split, preprocessing, pipelines, cross-validation, metrics and common data-leak mistakes.
scikit-learn (imported as sklearn) is the standard Python library for classical machine learning: classification, regression, clustering, dimensionality reduction, preprocessing and model evaluation. Its great strength is a consistent interface. Almost every model is an estimator with the same few methods, so swapping one algorithm for another is usually a one-line change.
Install it with python -m pip install scikit-learn. It works on NumPy arrays and pandas tables. The examples run in the browser with small built-in datasets.
from sklearn.linear_model import LinearRegression model = LinearRegression().fit([[1], [2], [3]], [2, 4, 6]) print(model.predict([[4]]))
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
The estimator API
| Method | Meaning |
|---|---|
fit(X, y) |
Learn from data. Returns the estimator itself, so calls can be chained. |
predict(X) |
Produce predictions for new rows. |
predict_proba(X) |
Class probabilities, for classifiers that support them. |
score(X, y) |
Default quality score (accuracy for classifiers, R squared for regressors). |
transform(X) |
For transformers such as scalers: apply what was learned. |
fit_transform(X) |
Learn and transform in one step (training data only). |
get_params(), set_params() |
Read or change hyperparameters. |
X is a 2-D table of features (rows are samples, columns are features); y is a 1-D array of targets. Learned values are stored in attributes ending with an underscore, such as model.coef_.
Most used modules
| Module | Contents |
|---|---|
sklearn.model_selection |
train_test_split, cross_val_score, GridSearchCV. |
sklearn.preprocessing |
StandardScaler, OneHotEncoder, LabelEncoder. |
sklearn.linear_model |
LinearRegression, LogisticRegression, Ridge. |
sklearn.tree, sklearn.ensemble |
Decision trees, random forests, gradient boosting. |
sklearn.neighbors, sklearn.svm |
k-nearest neighbors, support vector machines. |
sklearn.cluster, sklearn.decomposition |
KMeans, PCA. |
sklearn.metrics |
accuracy_score, confusion_matrix, mean_squared_error, r2_score. |
sklearn.pipeline |
Pipeline, make_pipeline, ColumnTransformer. |
Usage notes
- Always hold out test data. Scoring a model on the rows it learned from tells you almost nothing. Use
train_test_splitor cross-validation. - Avoid data leakage. Fit scalers and encoders on the training data only. A
Pipelinedoes this correctly, including inside cross-validation. - Set
random_statewherever randomness is involved (train_test_split, forests,KMeans) to get repeatable results. - Features must be numeric. Convert categories with
OneHotEncoderand handle missing values withSimpleImputer. - Scale when distances or gradients matter (k-nearest neighbors, SVMs, logistic regression). Tree-based models do not need it.
- Accuracy can mislead on imbalanced classes; look at precision, recall and the confusion matrix.
fiton new data starts over. Fitting an estimator again replaces what it learned.- Shape errors: a single sample must be 2-D, for example
[[5.1, 3.5, 1.4, 0.2]], not a flat list. - Not for deep learning. Neural networks at scale use PyTorch or TensorFlow; scikit-learn covers classical methods well.
Related
- NumPy, pandas, Matplotlib,
statistics - Lessons: Machine learning basics, NumPy