Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 41 ยท Governance and Tooling

The MLOps Tooling Landscape

A map of MLOps tool categories (tracking, data, orchestration, serving, monitoring, platforms), a build-versus-buy method and a weighted scoring exercise in Python.

Search for "MLOps tools" and you will find hundreds. The names change every year, but the jobs they do stay the same. If you learn the jobs, any tool becomes easy to place, compare and replace. This lesson gives you that map, a method for choosing, and advice on avoiding lock-in.

Tool names below are examples, not endorsements; check each project's current documentation before adopting one.

The map: jobs, not brands

flowchart TB
  A["Data: versioning, validation, feature stores"] --> B["Experiments: tracking, tuning"]
  B --> C["Pipelines: orchestration, CI/CD"]
  C --> D["Registry: models, metadata, approvals"]
  D --> E["Serving: APIs, batch, edge"]
  E --> F["Monitoring: service, data, model"]
  F --> A

The lifecycle is a loop; every job has several tool options.

Job What it solves Example tools
Data versioning Which exact data trained this? DVC, lakeFS, Delta Lake and Iceberg table versions
Data validation Is today's data sane? Great Expectations, Pandera, TFX Data Validation
Feature stores Same features in training and serving Feast, Tecton, cloud-native stores
Experiment tracking Which run produced which metrics? MLflow, Weights and Biases, Neptune, Comet
Orchestration Run steps in order, retry, schedule Airflow, Dagster, Prefect, Kubeflow Pipelines, Argo Workflows
CI/CD Test and release automatically GitHub Actions, GitLab CI, Jenkins
Registry Versions, stages, approvals MLflow Model Registry, cloud registries
Serving Low-latency or batch inference BentoML, KServe, Seldon, TorchServe, Triton, cloud endpoints
Monitoring Drift, quality, service health Evidently, NannyML, WhyLogs, Prometheus and Grafana
End-to-end platforms Many jobs in one product SageMaker, Vertex AI, Azure ML, Databricks

Build, buy or assemble?

  • Build (your own scripts): maximum control, lowest cost at small scale, but you own every bug and migration.
  • Assemble open source: each job gets a focused tool; you own the glue and the upgrades.
  • Buy a platform: quickest to start and well integrated, but costs money and shapes how you work.

Most teams begin with simple scripts plus one or two tools for their biggest pain (usually experiment tracking and orchestration) and grow from there. The maturity levels from MLOps Maturity Levels are a good guide to what you need next, not what you could possibly use.

Score the options

Write down what matters and weigh it. A scoring sheet will not make the decision for you, but it makes the trade-offs visible and the argument short.

Python Weighted scoring of tool options

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The second half of the script is the important habit: test whether the winner changes when you change a weight. If a small change flips the result, the options are close and other factors (team skills, existing contracts) should decide.

Avoid lock-in with thin interfaces

Whatever you pick, keep your own code talking to a small interface you control, with the vendor tool behind it. Swapping the tool then means rewriting one adapter instead of the whole project.

Python A tracker interface with two interchangeable backends

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In real projects the adapter wraps MLflow, Weights and Biases or a cloud service. Model files should also use portable formats where possible (for example ONNX for many model types), and data should live in open formats such as Parquet.

Questions to ask about any tool

  1. Which job does it do, and is that my biggest problem right now?
  2. How does it fit with my orchestrator, registry and cloud?
  3. What does it store, where, and who can read it?
  4. How do I leave? Can I export all runs, models and metadata?
  5. Who maintains it? Release cadence, community, license.
  6. What is the total cost, including people who must learn and run it?

Common mistakes

  • Adopting a platform before having a repeatable manual process.
  • Choosing tools by popularity instead of by the job and the team.
  • Using five overlapping tools that each store a different copy of the truth.
  • Ignoring the cost of running and upgrading self-hosted tools.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why organize tools by job rather than by brand?

    Understanding the job lets you evaluate unfamiliar tools quickly.
  2. 2. What is the point of testing how sensitive a scoring result is to the weights?

    If a small weight change flips the winner, the decision should rest on other factors.
  3. 3. How does a thin interface reduce lock-in?

    Vendor-specific calls are isolated in one place.
  4. 4. What should most teams adopt first?

    Start with the process and the largest pain; add tools as maturity grows.

Summary

  • Learn the jobs (data, experiments, pipelines, registry, serving, monitoring); tools are interchangeable ways to do them.
  • Choose between building, assembling and buying based on team size, maturity and budget.
  • Use weighted scoring and sensitivity checks to make trade-offs visible.
  • Keep a thin interface of your own between your code and any vendor.
  • Ask how to leave before you join.

Next: Project: An End-to-End Mini MLOps Pipeline