Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 111 ยท Trees and Ensembles

XGBoost, LightGBM and CatBoost

What XGBoost, LightGBM and CatBoost share: shrinkage, histogram bins, leaf-wise trees, early stopping and categorical handling, demonstrated with scikit-learn.

If you read job postings, Kaggle write-ups or production code for tabular data, you will meet three names over and over: XGBoost, LightGBM and CatBoost. All three are fast, heavily engineered implementations of gradient boosting: trees are added one at a time, each fixing the errors of the ensemble so far. They usually beat linear models and random forests on tables, and they are the default first choice for many data scientists.

Here is the honest situation for this course: none of the three libraries is installed in the in-browser runner, so their code is shown read-only. That is not a big loss, because what makes them good is a handful of shared ideas, and scikit-learn's HistGradientBoosting lets you run each of those ideas live. This lesson shows the ideas first, runs them, and then maps them onto the three libraries so you know which knob is called what.

The shared recipe

Boosted trees Many small trees, each fixing the last.
Shrinkage Small learning rate, many trees.
Histograms Bin features to find splits fast.
1 / 2

Idea 1: shrinkage, trees and early stopping

Each new tree is added with a weight, the learning rate (also called shrinkage, eta in XGBoost). A small rate means each tree contributes only a little, so you need more trees, but the final model is smoother and usually generalizes better than a few big steps. The pair works as a trade-off: halve the rate and you roughly double the trees needed.

You should not guess the number of trees. Early stopping holds out part of the training data, scores it after every tree, and stops when the score has not improved for a while. All three libraries support it (early_stopping_rounds), and so does scikit-learn's histogram model. The experiment below uses noisy data (10% of the labels are flipped on purpose) and three learning rates; each run is allowed up to 1,000 trees.

Python Learning rate and early stopping

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Early stopping picked a different number of trees for each rate: 26 at 0.5, 50 at 0.1 and 196 at 0.02. Test accuracy is similar (0.903, 0.914 and 0.908), so on noisy data like this the learning rate matters less than you might fear, but it clearly controls cost: the smallest rate needs about four times the trees of 0.1 and takes about three times as long to fit. A rate around 0.05 to 0.1 with early stopping is a sound default, and you can lower it at the very end for a small final gain.

Idea 2: histograms and leaf-wise growth

Finding the best split of a numeric feature normally means sorting every value. The histogram trick, used by LightGBM and then adopted by XGBoost (tree_method="hist") and CatBoost, bins each feature into a few hundred buckets once, then searches split points only between bucket edges. It is the main reason these libraries handle millions of rows.

Trees can also be grown in two orders. Depth-wise (the classic way, XGBoost's default) grows every branch to the same depth. Leaf-wise (LightGBM, and HistGradientBoosting) always splits the leaf that reduces the loss the most, so trees become lopsided and reach a lower loss with fewer leaves, but they can overfit small data unless you cap the number of leaves (num_leaves, max_leaf_nodes). The experiment below changes three things on a regression problem: how many leaves, how many bins, and the L2 penalty on leaf values.

Python Leaves, bins and regularization

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the train and test columns together. More leaves make the model fit the training set better (Rยฒ of 0.911, 0.984 and 1.000) while the test score peaks at 31 leaves and falls at 127: a perfect training fit is a sign of overfitting, and the 127-leaf model is also the slowest. A plain max_depth = 3 limit gave the best test score here (0.897) and the fastest fit, a reminder that simpler trees often win on small, noisy data. Bins show the other side: 8 bins is too coarse (test Rยฒ 0.871), 32 is nearly as good as 255 (0.888 versus 0.891). Stronger L2 regularization lowers the training score (0.984 to 0.960) and the gap to the test score closes a little. These differences are small and come from one train/test split, so treat them as directions to try, not as rules.

Idea 3: handling categories

Boosted trees need numbers, so categorical columns have to be turned into numbers somehow. The three libraries take different routes:

  • LightGBM can split directly on a category: a split sends a subset of categories left and the rest right (like scikit-learn's native categorical support).
  • XGBoost historically required one-hot encoding, and newer versions can do native categorical splits too (enable_categorical=True).
  • CatBoost replaces each category with a target statistic, the average label of rows in that category, computed in an ordered way to avoid leakage. That is its signature feature, and it matters most for columns with many levels (user IDs, cities, product codes).

The leakage problem is worth seeing. Suppose a column has 300 different codes that have no connection to the label. A naive target encoding, "replace each code by the mean label of the training rows with that code", memorizes the training labels, because each code appears in only about 10 rows. The model looks brilliant on the training set and no better than a coin flip on new data. Two cures follow: compute the statistic out of fold (each row is encoded using only other folds), or ordered, as CatBoost does (each row sees only the rows that come before it in a random order).

Python Target encoding leakage and its cures

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The naive encoding looks better than chance on the training rows (accuracy 0.65) even though the labels are pure coin flips, and that gain disappears on new rows (0.51). Out-of-fold and ordered encoding both give training accuracy near 0.53, which is the honest picture: there is nothing to learn. The leaked score is the dangerous one, because it would make a model look useful during development. Out-of-fold encoding is easy to put in a scikit-learn workflow; the ordered version is what CatBoost does internally, using several random orders.

Any feature built from the label (target encoding, mean-by-group) must be computed inside cross-validation folds, never on the full training set. This is the same leakage rule as in Nested CV and Data Leakage. Put the encoder inside a Pipeline so it is refit per fold.

Inside the speed-ups: sparsity, GOSS, EFB and DART

The three libraries share a core (boosted trees on gradients) and differ in tricks that make them fast on large or sparse data. You do not need these details to use them, but they explain the names of many settings and why the libraries behave as they do.

Sparsity-aware split finding (XGBoost). Real tables are full of zeros and missing values. For every split XGBoost learns a default direction: rows whose value is missing (or zero in a sparse matrix) are sent either left or right, whichever lowers the loss more. It only loops over the non-missing entries, so the cost depends on the number of non-zero values, not on the table size. Scikit-learn's HistGradientBoostingClassifier does the same for NaN.

Cache-aware access and block structure (XGBoost). Sorting a column for every split is slow. XGBoost sorts each feature once, stores the data in compressed column blocks, and fetches gradients through small buffers that stay in the CPU cache (cache-aware access), which matters because memory access, not arithmetic, is usually the bottleneck. Blocks can also be spread over several threads, or kept on disk for out-of-core training.

Gradient-based one-side sampling, GOSS (LightGBM). Rows with small gradients are already well fitted and contribute little to the split decision. GOSS keeps all rows with the largest gradients (the top fraction a), samples a fraction b of the remaining rows at random, and multiplies the sampled small-gradient rows by (1 - a) / b so that the total stays approximately unbiased. Training then looks at only a + b of the data per tree.

Exclusive feature bundling, EFB (LightGBM). In sparse data, many features are mutually exclusive: they are almost never non-zero in the same row (one-hot columns are the extreme case). EFB merges such features into one column by adding offsets to their value ranges, so that the histogram work depends on the number of bundles, not on the number of original features.

DART (dropouts meet Multiple Additive Regression Trees). Ordinary boosting lets late trees correct tiny errors that the first trees made, so the first trees dominate. DART randomly drops some existing trees when computing each new tree's target, as dropout does in neural networks, and rescales the result. This reduces over-specialization and can improve accuracy, at the price of slower training and no simple early stopping. It is available in XGBoost (booster="dart") and LightGBM (boosting_type="dart").

The two examples below show GOSS and EFB on small, hand-made data, so that you can see the idea without installing either library.

Python GOSS: estimate a statistic from a fraction of the rows

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With the same budget of 20% of the rows, keeping the largest gradients and sampling the rest estimates the total with an average error of 0.35%, against 0.85% for sampling 20% of all rows uniformly: the error is more than halved, because the rare, large gradients that dominate the sum are always included.

Python EFB: bundle mutually exclusive features

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Features in the same group never fire together (zero conflicts), while features from different groups sometimes do (43 rows here), so the greedy search packs the 20 columns into 4 bundles, each holding features from different groups. Histogram building then touches 4 columns instead of 20, and the original values can still be recovered from the offsets. LightGBM also tolerates a small number of conflicts to bundle even more, and it does all of this automatically.

Installing the libraries (read-only, on your own machine): pip install xgboost lightgbm catboost. CatBoost is also available for R, and XGBoost and LightGBM have R packages as well. In the browser runner of this site only scikit-learn's own histogram boosting is available.

The three libraries side by side (read-only)

The parameters you tuned above exist in all three libraries under different names. The snippet below is not runnable here, because the packages are not installed in the browser runner. It shows the same model as three constructors and the early-stopping call, as you would write it on your own machine after pip install xgboost lightgbm catboost.

Python
# Read-only: these packages are not available in the browser runner.
from xgboost import XGBClassifier
from lightgbm import LGBMClassifier, early_stopping
from catboost import CatBoostClassifier

xgb = XGBClassifier(
    n_estimators=2000, learning_rate=0.05, max_depth=6,       # depth-wise by default
    tree_method="hist", reg_lambda=1.0, subsample=0.8, colsample_bytree=0.8,
    early_stopping_rounds=50, eval_metric="logloss",
)
xgb.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=False)

lgbm = LGBMClassifier(
    n_estimators=2000, learning_rate=0.05, num_leaves=31,     # leaf-wise
    reg_lambda=1.0, subsample=0.8, subsample_freq=1, colsample_bytree=0.8,
)
lgbm.fit(X_train, y_train, eval_set=[(X_val, y_val)],
         callbacks=[early_stopping(50)], categorical_feature=["city"])

cat = CatBoostClassifier(
    iterations=2000, learning_rate=0.05, depth=6,             # symmetric trees
    l2_leaf_reg=3.0, early_stopping_rounds=50, verbose=False,
)
cat.fit(X_train, y_train, eval_set=(X_val, y_val), cat_features=["city"])
Idea scikit-learn HistGradientBoosting XGBoost LightGBM CatBoost
Learning rate learning_rate learning_rate / eta learning_rate learning_rate
Number of trees max_iter n_estimators n_estimators iterations
Tree size max_leaf_nodes, max_depth max_depth num_leaves depth
L2 penalty l2_regularization reg_lambda reg_lambda l2_leaf_reg
Bins max_bins max_bin max_bin border_count
Early stopping early_stopping, n_iter_no_change early_stopping_rounds early_stopping callback early_stopping_rounds
Categories categorical_features enable_categorical categorical_feature cat_features (ordered target statistics)

CatBoost builds symmetric (oblivious) trees, where every node at one depth uses the same split. They are weaker individually but very fast to evaluate and hard to overfit, which is why CatBoost often does well with little tuning.

Which one should you pick?

  • Start with HistGradientBoosting from scikit-learn. No extra install, same API, and it teaches every idea above.
  • LightGBM for large tables when training speed matters, or when you want leaf-wise trees.
  • XGBoost when you need the most mature ecosystem, GPU training, distributed training or broad deployment support.
  • CatBoost when you have many high-cardinality categorical columns or little time to tune.
  • For a final model, try two of them and compare with the same cross-validation folds. The differences are often smaller than the gain from good features or from hyperparameter search.
  • Boosting is not always the answer: on small data, on images or text, or when you need a very simple model, look elsewhere (see Random Forests and Logistic Regression).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. You halve the learning rate of a boosted model. What usually happens to the number of trees it needs?

    Each tree contributes less, so more trees are needed to fit the same signal. Early stopping finds the right number automatically.
  2. 2. What does leaf-wise tree growth do?

    LightGBM and HistGradientBoosting split the best leaf anywhere in the tree, so you control size with a leaf limit rather than only a depth limit.
  3. 3. Why does naive target encoding of a high-cardinality column leak?

    When a category has few rows, its mean label contains each row's own label, so the model can memorize training rows. Out-of-fold or ordered statistics keep the row's own label out.
  4. 4. Why are the XGBoost, LightGBM and CatBoost snippets in this lesson read-only?

    Only scikit-learn, NumPy, SciPy and similar packages are available in the browser, so the shared ideas are demonstrated with HistGradientBoosting.

Summary

  • XGBoost, LightGBM and CatBoost are fast, regularized gradient-boosting libraries for tabular data; none is available in the browser runner, so their code is read-only here.
  • The shared ideas are shrinkage with many trees, early stopping, histogram-based split finding and tree-size and L2 regularization.
  • LightGBM grows leaf-wise, XGBoost defaults to depth-wise, CatBoost uses symmetric trees; control complexity with leaves or depth.
  • High-cardinality categories need care: native splits (LightGBM), one-hot or native (XGBoost) or ordered target statistics (CatBoost).
  • Target encodings must be out-of-fold or ordered, never computed on the rows they are applied to.
  • Start with HistGradientBoosting, compare a second library with the same folds, and expect small differences.

Next, learn how to blend different models in Stacking and Blending Models.