Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 40 ยท Imbalanced Data and Custom Losses

Objectives for Counts, Quantiles and Ranking

Choose the right built-in objective: Poisson and Tweedie for counts and claims, quantile and absolute error for intervals and outliers, and rank objectives for sorted result lists.

Before you write a custom loss, check whether a built-in objective already fits your target. Choosing the right one often matters more than any tuning, because the objective defines what "good" means. This lesson covers the most useful non-default objectives for regression-like problems: counts and positive skewed amounts, quantiles and prediction intervals, robust losses, and ranking.

An overview

Target type XGBoost objective LightGBM objective Notes
Continuous, symmetric noise reg:squarederror regression the default
Continuous with outliers reg:pseudohubererror, reg:absoluteerror huber, regression_l1 less sensitive to extreme values
Counts (0, 1, 2, ...) count:poisson poisson predicts a positive rate
Positive skewed amounts reg:gamma, reg:tweedie gamma, tweedie insurance claims, sales, durations
Quantiles reg:quantileerror quantile quantile_alpha / alpha sets the level
Binary labels binary:logistic binary probabilities
Multiclass multi:softprob multiclass one probability per class
Ranking rank:ndcg, rank:pairwise lambdarank needs query groups

Counts and skewed positive targets

Poisson, gamma and Tweedie objectives use a log link: the trees add up on the log scale and the prediction is the exponential of that sum, so predictions are always positive and the model captures multiplicative effects (doubling the exposure doubles the expected count). With squared error on a count target, predictions can be negative and the model overweights a few huge values.

Python Poisson objective on count data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In one run the squared error model had an RMSE of 0.419 against the true rate and a Poisson deviance of 1.149, the Poisson model 0.354 and 1.129: closer to the true rate and a lower deviance (the natural error measure for counts). Both stayed positive on this easy data (smallest predictions 0.11 and 0.31), but a squared error model is free to go negative for low-rate rows, and the Poisson one never can. For skewed positive amounts such as insurance claims, reg:tweedie with tweedie_variance_power between 1 and 2 mixes the two worlds: many zeros plus a heavy right tail.

Quantile regression and prediction intervals

Squared error predicts the mean. The quantile loss predicts a chosen percentile: with alpha 0.9 about 90% of the actual values should fall below the prediction. Training one model for alpha 0.1 and one for 0.9 gives an 80% prediction interval, with no assumption about the shape of the noise.

Python An 80 percent prediction interval from two quantile models

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In one run the coverage was 0.787 against the nominal 0.80, and the interval width grew from about 2.4 at x=1 to 8.4 at x=5 and 13.1 at x=9, because the noise is larger there. That adaptive width is the main benefit over a single fixed-width band around the mean. Common problems: quantile models for extreme alphas need more data, the two quantile models can cross (the lower prediction above the upper one for a few rows), and you should check coverage on held-out data every time. In XGBoost use objective="reg:quantileerror" with quantile_alpha=[0.1, 0.5, 0.9] to fit several quantiles in one model.

Ranking

Search results, recommendations and "which leads to call first" are ranking problems: only the order within a group (a query) matters. Ranking objectives learn from groups of rows with graded relevance labels. In XGBoost you pass qid (the group of each row, rows of one group must be adjacent), and in LightGBM a group array of group sizes.

Python XGBRanker: ordering items inside each query

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The ranker is evaluated only inside each query, with NDCG (normalized discounted cumulative gain) at the top three positions, which rewards putting highly relevant items first. A random ordering gives the baseline to beat (about 0.37 against about 0.93 for the ranker in one run). Two practical rules: keep rows of a query together and sorted by qid, and never split a query between train and validation.

Pairwise ranking objectives (rank:pairwise, LightGBM lambdarank) learn from pairs inside a group. For recommendations with implicit feedback (clicks), start with rank:ndcg or lambdarank, and use click labels as relevance grades.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which objective suits a target of non-negative integer counts?

    Poisson uses a log link, so predictions are positive rates and multiplicative effects are natural.
  2. 2. You fit quantile models at alpha 0.1 and 0.9. What interval do they give?

    About 10% of values fall below the lower model and 10% above the upper one, leaving 80% in between. It is a prediction interval for individual values, not for the mean.
  3. 3. What must be true of the rows when training XGBRanker with qid?

    Ranking learns the order within groups, so groups must be intact and contiguous.
  4. 4. Why can squared error be a poor choice for count data?

    Counts are non-negative and skewed; a log-link objective matches their structure better.

Summary

  • Match the objective to the target: Poisson or Tweedie for counts and skewed positive amounts, quantile for intervals, Huber or absolute error for outliers, rank objectives for ordered lists.
  • Log-link objectives give positive predictions and multiplicative effects.
  • Quantile pairs give adaptive prediction intervals; verify their coverage on held-out data.
  • Ranking needs query groups kept intact; evaluate with NDCG per query.

Next: Saving, Loading and Versioning Models