Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 40 ยท XGBoost API

XGBoost Objectives and Evaluation Metrics

Pick the right XGBoost objective for regression, counts, binary, multiclass and ranking, understand how it differs from eval_metric, and add a custom metric to xgb.train.

Two settings decide what XGBoost optimizes and what it reports. The objective is the loss the trees are trained to reduce. The evaluation metric (eval_metric) is the number printed for each round and used by early stopping. They do not have to match, and choosing them separately is often the right move. This lesson maps common problems to objectives and shows how to evaluate with the right yardstick.

Objectives by task

Task Objective Prediction is Notes
Regression, average error reg:squarederror the number sensitive to outliers
Regression, robust reg:absoluteerror the number learns the median
Regression, robust and smooth reg:pseudohubererror the number quadratic for small errors, linear for big ones
Positive skewed target reg:squaredlogerror, reg:gamma, reg:tweedie the number targets must be positive (gamma, squaredlogerror)
Counts count:poisson expected count targets are non-negative integers
Binary classes binary:logistic probability binary:logitraw returns the raw score
Multiclass multi:softprob / multi:softmax probability matrix / class index native API needs num_class
Quantiles reg:quantileerror the chosen quantile(s) pass quantile_alpha
Ranking rank:ndcg, rank:pairwise, rank:map a score needs query groups
Survival survival:cox, survival:aft hazard or time specialized labels

The scikit-learn wrapper picks binary:logistic or multi:softprob for classification automatically and reg:squarederror for regression. You override it with objective="...".

Choosing a regression loss

Real targets often have outliers. Squared error chases them (large errors dominate the gradient) while absolute error ignores their size.

Python Squared error vs absolute error vs pseudo-Huber

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Squared error ends up with the worst median and mean absolute error (outliers drag the fit), while absolute error and pseudo-Huber achieve a clearly lower typical error. The RMSE is almost the same for all three, because the test set contains outliers that no model can predict, and RMSE is dominated by them. There is no universally best loss: pick the one that matches the business cost of mistakes, then evaluate with the metric you actually care about.

Counts and skewed positives

For counts (calls per hour, clicks, claims), the Poisson objective models the log of the expected value, which keeps predictions positive and handles the variance growing with the mean.

Python Poisson objective for count data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Poisson model always predicts positive values, while squared error has no such guarantee (its lowest prediction on this easy example stayed positive, but nothing enforces that). A lower Poisson deviance is the right way to compare the two on count data; on this simple synthetic problem the two are close, and the gap grows with skewed, low-count data.

eval_metric: what you watch

Common built-in metrics: rmse, mae, mape, logloss, error (misclassification rate), auc, aucpr, merror, mlogloss, poisson-nloglik, ndcg, map. Pass one or a list. The last metric drives early stopping, so list them in the order you want.

Prefer a metric that fits the decision, not just the loss. On imbalanced data auc or aucpr is more informative than error; with probabilities you intend to use directly, logloss rewards good calibration.

A custom metric in the native API

If no built-in metric matches your needs, supply a Python function. In XGBoost 2.x it is the custom_metric argument and receives the raw predictions and the DMatrix.

Python Multiclass training with a custom metric

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When a custom metric is supplied, XGBoost reports it as the last metric, so it also controls early stopping. If a higher value is better (such as an AUC variant), pass maximize=True to xgb.train.

flowchart LR
  O["objective<br/>(what the trees minimize)"] --> M["model"]
  M --> E["eval_metric<br/>(what you watch and stop on)"]
  E --> D["Decision: stop, tune, ship"]

They answer different questions: "how should the trees learn" and "how well is the model doing for my purposes".

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which objective suits a count target like the number of support tickets per day?

    Poisson models non-negative counts and keeps predictions positive.
  2. 2. What is the difference between objective and eval_metric?

    The gradients come from the objective. The metric just scores the predictions each round.
  3. 3. A training set has 5% wild outliers and you care about the typical error. A good first try is...

    Absolute and pseudo-Huber losses give outliers a bounded influence on the gradient.
  4. 4. What does multi:softprob return from predict in the native API?

    multi:softmax returns class indices, whereas multi:softprob returns the probability matrix.

Summary

  • The objective shapes the trees; the evaluation metric tells you how you are doing and drives early stopping.
  • Match the objective to the target: squared, absolute or pseudo-Huber for regression, Poisson for counts, logistic or softprob for classes.
  • The last metric in eval_metric decides when early stopping fires.
  • Use custom_metric in the native API for anything the built-ins do not cover.
  • Always judge the model with the metric that reflects the real cost of errors.

Next: Early Stopping, Cross-Validation and Saving Models