Skip to content
Tutorials Center
My learning

Topics

Lesson 27 of 36 ยท MLflow and ML

MLflow Tracking and Experiments

Track machine-learning work on Databricks with MLflow: experiments, runs, parameters, metrics, artifacts, autologging, comparing runs, plus a runnable run-tracker stand-in.

Machine learning is full of experiments: which features, which learning rate, which data version? Without records you end up with files called model_final_v2_REAL.pkl. MLflow is the open-source tool that Databricks builds in to record every training attempt, so you can compare them, reproduce the best one and hand it to production.

Databricks and MLflow's server are not available on this site. MLflow code below is read-only; runnable blocks are plain Python stand-ins (numpy only) that reproduce what tracking stores and how you query it.

The vocabulary

  • Experiment: a named group of related runs (for example "churn-model"). In Databricks it appears in the workspace and has its own page.
  • Run: one execution of training code. It stores everything below.
  • Parameters: inputs you chose (learning rate, tree depth). Logged once, as strings.
  • Metrics: numbers measuring results (accuracy, RMSE), optionally logged many times with a step (per epoch).
  • Tags: free labels (team=growth, data_version=v3).
  • Artifacts: files: the model, plots, a feature list, a confusion-matrix image.
  • Model: a packaged, loadable form of the trained model, in an MLmodel format.

Logging a run

Python
import mlflow
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error

mlflow.set_experiment("/Users/[email protected]/price-model")

with mlflow.start_run(run_name="rf-depth-8") as run:
    mlflow.log_param("n_estimators", 200)
    mlflow.log_param("max_depth", 8)

    model = RandomForestRegressor(n_estimators=200, max_depth=8, random_state=42)
    model.fit(X_train, y_train)

    rmse = mean_squared_error(y_test, model.predict(X_test)) ** 0.5
    mlflow.log_metric("rmse", rmse)
    mlflow.set_tag("data_version", "v3")

    mlflow.sklearn.log_model(model, "model", input_example=X_train.head(3))
    print("run id:", run.info.run_id)

The with block starts and ends the run. On Databricks, the experiment is created in your workspace and the run page shows parameters, metrics, charts and the model files; you do not run a tracking server yourself.

Autologging

For common libraries one line logs parameters, metrics, the model and plots without extra code:

Python
mlflow.autolog()          # scikit-learn, XGBoost, LightGBM, PyTorch Lightning, Keras, ...
model.fit(X_train, y_train)

You can still add your own metrics and tags next to autologged data. Autolog is the best way to start; add manual logging for metrics the library does not know.

Metrics over time

Training loops log a metric with a step:

Python
for epoch in range(20):
    loss = train_one_epoch(...)
    mlflow.log_metric("train_loss", loss, step=epoch)

The run page then draws a loss curve, and you can overlay curves from several runs.

Finding and comparing runs

Python
runs = mlflow.search_runs(
    experiment_names=["/Users/[email protected]/price-model"],
    filter_string="metrics.rmse < 30 and params.max_depth = '8'",
    order_by=["metrics.rmse ASC"],
)
best = runs.iloc[0]
print(best["run_id"], best["metrics.rmse"])

search_runs returns a pandas DataFrame. The UI has the same ideas as a table, parallel-coordinates plot and scatter plot, so you can see which parameter moved the metric.

A run recorder you can run here

This is not MLflow. It stores runs in a list, but its API and the questions you ask of it match tracking: log parameters and metrics, then find the best run. The model is a one-feature linear fit with numpy and a polynomial degree as the parameter.

Python Stand-in: tracking runs and picking the best

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Run it: degree 1 is too simple (large error), and 2 or 3 fit well; the exact winner can vary slightly with the random data but the pattern of a worse straight line is stable. That is the whole point of tracking: you chose parameters, saw metrics, and can reproduce the best.

Reproducibility: record the context

A run is only reproducible if you also store:

  • the code version (Databricks records the notebook or Git commit automatically),
  • the data version (a Delta table version number, see time travel in the Delta Lake chapter, or a tag),
  • the environment (MLflow logs requirements/conda files with the model),
  • the random seeds.
Python Stand-in: logging data and seed with a run

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Practical tips

  • One experiment per problem, not per person; use tags for who and why.
  • Name runs meaningfully; log the dataset version.
  • Log few, comparable metrics; keep the same name across runs.
  • Use mlflow.autolog() first, then add custom items.
  • Compare on a held-out test set, never on training data.
  • Clean up: delete failed or abandoned runs from the UI.

Once you have a good run, you promote its model to the registry: see The Model Registry.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is an MLflow run?

    A run is the unit that stores what happened in one attempt.
  2. 2. What does mlflow.autolog() do?

    It captures common information without manual log calls.
  3. 3. How do you log a metric once per epoch so you can plot a curve?

    The step argument orders a metric's values over time.
  4. 4. Which of these helps reproduce a run exactly?

    All inputs must be recorded, not just the output.

Summary

  • MLflow tracking records experiments, runs, parameters, metrics, tags and artifacts.
  • Databricks hosts it for you; autolog() is the quick start.
  • search_runs and the UI let you compare and choose the best run.
  • Record code, data, environment and seed to make runs reproducible.

Next: The Model Registry