Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 40 ยท XGBoost API

DMatrix and the Native train API

Learn XGBoost's native interface: build a DMatrix with labels, weights and feature names, pass a params dict to xgb.train, watch evals, predict, inspect trees and set a base margin.

Under the scikit-learn wrapper sits XGBoost's own interface. It is built around two objects: a DMatrix, the library's internal data container, and xgb.train, a function that takes a parameter dictionary and returns a Booster. You should know it because it exposes features the wrapper hides (custom objectives, xgb.cv, base margins, direct control over rounds) and because most online documentation uses it.

DMatrix: data in XGBoost's format

A DMatrix holds your features plus optional extras. It stores data in a compact, column-oriented form that the training code is optimized for. Building it yourself means you convert once and reuse it for many training runs.

Python Building and inspecting a DMatrix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A DMatrix can be built from NumPy arrays, pandas DataFrames, SciPy sparse matrices and files. Useful arguments:

Argument Purpose
label targets (required for training and evaluation)
weight per-row weights, used by the loss and the metrics
base_margin starting raw score per row (offset), see below
missing value to treat as missing besides NaN
feature_names, feature_types names are taken from a DataFrame automatically; "c" types mark categoricals
enable_categorical read pandas category columns as categorical features

Training with xgb.train

You pass parameters as a dictionary, the training DMatrix, the number of boosting rounds and an optional list of (DMatrix, name) pairs to evaluate each round.

Python xgb.train with an evaluation list

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Points to notice:

  • Parameter names differ slightly from the wrapper: eta for learning_rate, lambda for reg_lambda, alpha for reg_alpha (both spellings are accepted for the common ones).
  • The number of trees is num_boost_round, an argument of train, not a key in params.
  • Early stopping uses the last entry in evals and the last metric in eval_metric. The metric list above ends with auc, so that is what decides when to stop; the result says best_score is an AUC. To stop on logloss, put it last. (In the run above, AUC peaks at round 3 on this easy dataset, so training stops after 19 rounds; stopping on logloss keeps the model longer.)
  • booster.predict(dvalid) returns probabilities for binary:logistic, a probability matrix for multi:softprob, and the raw number for regression. Add output_margin=True for raw scores.
  • After early stopping the booster remembers best_iteration, and predict uses it by default in recent versions. You can also pass iteration_range=(0, k) to predict with only the first k trees.

Inspecting the Booster

The Booster is an object you can look into:

Python Look inside a Booster

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

get_score supports weight (how many splits used the feature), gain (average improvement per split), cover (average rows affected), and the total_* variants. Features that were never used do not appear at all. The dump with with_stats=True prints each split's gain and cover next to its condition.

Base margin: starting from a known score

By default every row starts from the same base score (the mean for regression, a probability for classification). With base_margin you give each row its own starting raw score. It is the standard way to model an offset (an exposure variable) or to continue from another model's predictions.

Python Boosting on top of another model with base_margin

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Boosting learns the residual structure the linear model missed, so the combined error is lower. The rule to remember: if you trained with a base margin you must supply one when predicting, otherwise the predictions are shifted.

When to choose which interface

Need Use
Pipelines, grid search, cross_val_score, SHAP wrappers scikit-learn API
Custom objective or metric, xgb.cv, base margins, direct DMatrix reuse native API
Huge data that does not fit in memory native API with QuantileDMatrix or external-memory iterators
flowchart LR
  A["NumPy / pandas / sparse"] --> B["DMatrix<br/>label, weight, base_margin"]
  P["params dict"] --> T["xgb.train"]
  B --> T
  T --> C["Booster"]
  C --> D["predict(DMatrix)"]
  C --> E["get_score, get_dump, save_model"]

The wrapper builds the DMatrix and params for you; the native API lets you do it by hand.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Where does the number of boosting rounds go in xgb.train?

    xgb.train takes num_boost_round separately from the parameter dictionary.
  2. 2. With eval_metric=['logloss', 'auc'] and early stopping, which metric decides when to stop?

    XGBoost uses the last metric in the list for early stopping, so order the list on purpose.
  3. 3. What does base_margin do?

    It is a per-row offset in raw-score space, used for exposure terms or stacking on another model.
  4. 4. Which importance type counts how many times a feature is used in a split?

    weight is the split count; gain and cover average the improvement and rows affected per split.
  5. 5. Why build a DMatrix yourself?

    Conversion costs time; a reusable DMatrix saves it and carries extra information with the data.

Summary

  • DMatrix holds features, labels, weights, base margins and feature metadata.
  • xgb.train(params, dtrain, num_boost_round, evals=...) returns a Booster; early stopping uses the last evaluation set and last metric.
  • Native names differ a little (eta, lambda, alpha); predict returns probabilities or raw values depending on the objective.
  • get_score and get_dump expose importances and tree structure.
  • base_margin provides per-row offsets and must be supplied again when predicting.

Next: XGBoost Categorical and Missing Values