Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 40 ยท Getting Started

Installing and Setting Up XGBoost and LightGBM

Install XGBoost, LightGBM and friends with pip or conda, check versions and CPU threads, set up reproducible environments and run a smoke test of both libraries.

Installing a gradient boosting library is usually one command, but the details decide whether your results are reproducible next month and whether training uses all your CPU cores. This lesson covers installation, version pinning, a quick health check and the few environment issues that trip people up.

On this site the libraries are already available in the in-browser runner, so every example in the course runs without installing anything. The commands below are for your own machine and are shown read-only.

Installing with pip

Create a virtual environment first, then install the packages you need. A typical data stack is:

Terminal
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install numpy pandas scikit-learn matplotlib
pip install xgboost lightgbm

Both libraries ship prebuilt wheels for Windows, macOS (Intel and Apple silicon) and Linux, so no compiler is needed. For CatBoost and SHAP, which we meet later, the commands are pip install catboost and pip install shap.

Installing with conda

Terminal
conda create -n boost python=3.12
conda activate boost
conda install -c conda-forge xgboost lightgbm scikit-learn pandas

Conda is convenient when you need a specific CUDA toolkit or the OpenMP runtime that LightGBM uses.

Pin versions in a requirements.txt or pyproject.toml (for example xgboost==2.1.4). Boosting libraries change defaults between major versions, and an unpinned upgrade can silently change your scores.

Check what you have

Every library exposes its version. Run this smoke test; it trains a tiny model with each library and prints the versions.

Python Smoke test and versions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

If both accuracies print, your environment can train models. A failure usually means a broken install (an old wheel, a missing OpenMP runtime on macOS, or a mismatch between NumPy and the library).

Threads and reproducibility

Both libraries train in parallel on CPU threads. Two parameters matter.

  • Threads: n_jobs in the scikit-learn APIs of both libraries (nthread or num_threads in the native APIs). Leave it at the default unless you share a machine, in which case set it explicitly. In containers, the number of cores the OS reports may exceed your quota; set the thread count yourself.
  • Random seed: random_state in the scikit-learn APIs (seed in the native API). Row and column sampling use it, so the same seed gives the same model on the same data and library version.

Multi-threaded floating point sums can differ in the last bits depending on thread count, so exact reproducibility also needs a fixed thread count. For most work, matching to several decimals is enough.

Python Same seed, same model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because we sample 70% of rows and columns per tree, a different seed changes which data each tree sees and so changes the predictions. Without sampling (subsample=1), the seed would hardly matter.

GPU builds

XGBoost supports GPU training with device="cuda", and LightGBM has a GPU/CUDA build, but both need compatible NVIDIA drivers and often a special build or wheel. The in-browser runner has no GPU, so this is read-only:

Python
import xgboost as xgb

model = xgb.XGBClassifier(device="cuda", tree_method="hist", n_estimators=500)
model.fit(X_train, y_train)      # needs an NVIDIA GPU

GPUs pay off for large data (millions of rows and many features). For small and medium tables a modern CPU is just as fast, because the work is dominated by setup rather than computation.

Project layout

A tidy start for any boosting project:

TEXT
project/
  data/            raw and processed files
  notebooks/       exploration
  src/             feature code and training script
  models/          saved models plus a JSON of their metrics
  requirements.txt pinned versions

Save the exact library versions with each trained model. A model saved by one XGBoost version usually loads in newer ones, but the safest habit is to keep the version in the model's metadata.

Common installation problems

  • libomp missing on macOS: install it with brew install libomp.
  • Import works but training is slow: check the thread count; a container may be limited to one core.
  • Version clash between scikit-learn and XGBoost: upgrade both together, since the scikit-learn wrapper tracks scikit-learn's estimator API.
  • Two copies of a library installed: pip list | grep -i xgboost shows duplicates from different environments.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why should you pin library versions for a boosting project?

    An unpinned upgrade can change defaults, so the same code may give different scores later.
  2. 2. Which setting makes row and column sampling repeatable?

    Sampling uses a random generator, and the seed fixes it.
  3. 3. When is GPU training most worthwhile?

    GPU overhead dominates on small data; big data benefits from the parallel histogram building.
  4. 4. What is a good habit when saving a trained model?

    Knowing the versions and the evaluation scores lets you reproduce and compare models later.

Summary

  • Install with pip install xgboost lightgbm (or conda-forge) inside a virtual environment.
  • Print __version__ and run a smoke test to confirm the environment works.
  • Set random_state for repeatable sampling and n_jobs when you share hardware.
  • GPUs help on large data; the browser runner is CPU only.
  • Pin versions and store them with every saved model.

Next: Tabular Data for Boosting