MLflow and Experiment Tracking – Model Registry, Runs and Lifecycle Management

Most data scientists have experienced the reproducibility crisis at first hand: a model that performed well last quarter cannot be reproduced because no one recorded which preprocessing steps were applied, which hyperparameters were used, which version of the training data was active, or which random seed produced the results. Experiment tracking — systematically recording everything that characterises an ML experiment — is the engineering practice that prevents this crisis. MLflow is the most widely adopted open-source tool for experiment tracking and the ML model lifecycle, with native integrations across every major ML framework and production deployment target. This guide covers the full MLflow ecosystem: the Tracking API, the Autolog feature, experiment organisation best practices, the Model Registry and stage transitions, model serving, and integration with hyperparameter optimisation, model deployment, and cloud MLOps platforms.

MLflow Tracking — Logging Runs and Experiments

An MLflow experiment is a named collection of runs — all attempts to solve a particular modelling problem. A run is a single training attempt, capturing: parameters (the configuration values — hyperparameters, preprocessing choices, model architecture decisions), metrics (the performance numbers — training loss, validation AUC, RMSE — which can be logged at each epoch for learning curves), tags (free-form metadata — who ran the experiment, what Git commit, what Jira ticket), and artefacts (binary files — the serialised model, plots, confusion matrices, feature importance charts, the training dataset as a reference). Everything logged to a run is stored in the tracking server backend — a local filesystem, a remote database + object store, or a fully managed service like Databricks Managed MLflow.

MLflow Autolog is the most important feature for adoption: calling mlflow.autolog() before training automatically logs all relevant parameters, metrics, and model artefacts for the active framework — scikit-learn, XGBoost, LightGBM, PyTorch Lightning, or Hugging Face Trainer — with no additional code. For gradient boosting models, autolog records every tree’s training and validation loss, the final hyperparameters, and the model binary. For neural networks, it records the epoch-level training and validation metrics as a time series. The result is a complete record of the training run without any manual logging calls, reducing the friction that prevents practitioners from tracking experiments consistently.

For custom logging — metrics computed outside the framework’s native training loop, custom artefacts like SHAP plots or calibration curves — the tracking API is clean and explicit:

with mlflow.start_run(run_name="lgbm-v3-tuned") as run:
    mlflow.log_params({"n_estimators": 500, "learning_rate": 0.05, "max_depth": 6})
    mlflow.log_metric("val_auc", 0.923)
    mlflow.log_metric("val_pr_auc", 0.741)
    mlflow.sklearn.log_model(pipeline, "model")          # saves the sklearn Pipeline
    mlflow.log_artifact("shap_summary.png")              # saves any file
    mlflow.set_tag("dataset_version", "2026-10-01")

Run IDs are returned by mlflow.start_run() and are permanent, unique identifiers — they are the primary key for referencing a run in the Model Registry, in CI/CD promotion scripts, and in audit trails. Connecting run IDs to the Git commit that produced them (logged automatically by MLflow’s Git integration) creates a complete chain of custody from source code to deployed model, directly supporting the data lineage and audit requirements that regulated industries impose.

Model Registry — Versioning and Stage Transitions

The MLflow Model Registry is a centralised catalogue of all registered models, their versions, and their current deployment stage. A registered model has a name (e.g. “churn-prediction”) and one or more versions, each created from a specific training run. Versions progress through stages: None (newly registered, under evaluation), Staging (passing automated evaluation tests, ready for user acceptance testing), Production (currently serving live traffic), and Archived (replaced by a newer version, retained for rollback).

The stage transition workflow is the operationalisation of the model lifecycle: after a training run, the best run is registered via mlflow.register_model(run_uri, "churn-prediction"). A CI/CD pipeline (GitHub Actions, GitLab CI, or Airflow — covered in our ETL pipelines guide) then automatically evaluates the new version against a held-out evaluation set and a set of behavioural tests — checking that predictions are in the expected range, that the model’s fairness metrics across demographic groups meet the ethical AI thresholds set by the team, and that the model’s latency meets the SLA. If all tests pass, the pipeline transitions the version to Staging via client.transition_model_version_stage("churn-prediction", version, "Staging"). A human reviewer approves the Staging-to-Production transition (or the pipeline does it automatically for lower-stakes models). The previously Production version is automatically Archived, available for instant rollback if the new version degrades in production.

Model aliases (introduced in MLflow 2.0) provide stable reference names — @champion for the current production model, @challenger for the staged candidate — that deployment code references without needing to know the specific version number. This decouples deployment code from version management: the serving infrastructure always loads the model tagged @champion, and the Registry handles which version that maps to. The connection to our model deployment guide is direct: MLflow’s model serving command launches a Flask REST API that loads the @champion model and serves predictions, and the containerised deployment workflow uses MLflow’s Docker image generation to package the model with its dependencies.

Integration with Hyperparameter Optimisation and AutoML

MLflow’s nested runs API integrates cleanly with Optuna, Ray Tune, and Hyperopt for systematic hyperparameter search with full experiment tracking. Each Optuna trial is a child run under a parent run representing the full hyperparameter search study. The parent run logs the best parameters and best metric; each child run logs its own parameters and validation metric. This structure makes the full search history queryable and visualisable in the MLflow UI — you can plot validation AUC against each hyperparameter, identify interactions between parameters, and compare the pareto front of accuracy vs. latency tradeoffs.

The MLflow tracking integration with Optuna requires fewer than 10 lines of callback code:

import optuna, mlflow

def objective(trial):
    with mlflow.start_run(nested=True):
        params = {"n_estimators": trial.suggest_int("n_estimators", 100, 1000),
                  "max_depth":    trial.suggest_int("max_depth", 3, 10)}
        mlflow.log_params(params)
        score = train_and_evaluate(params)
        mlflow.log_metric("val_auc", score)
    return score

with mlflow.start_run(run_name="optuna-search"):
    study = optuna.create_study(direction="maximize")
    study.optimize(objective, n_trials=100)
    mlflow.log_params(study.best_params)
    mlflow.log_metric("best_val_auc", study.best_value)

The resulting MLflow experiment contains 101 runs (1 parent + 100 trials), fully queryable. The best run can be registered directly from the parent run via the MLflow UI or API, entering the stage-transition workflow described above. This end-to-end pipeline — data preparation, feature engineering, hyperparameter search, experiment tracking, model registration, and deployment — is the core workflow described in our MLOps interview guide and is a standard expectation for senior data science roles as surveyed in our salary guide.

✦ SUMMARIZE THIS ARTICLE WITH AI

ChatGPT Perplexity Claude Grok

Hyperparameter optimisation with Optuna that integrates with MLflow tracking is in our AutoML guide. Model deployment via REST API and Docker — the downstream step after MLflow model registration — is in our Model Deployment guide. MLOps interview questions covering MLflow, model registries, and CI/CD pipelines are in our MLOps Interview guide. Data quality and lineage tracking that complement MLflow experiment tracking are in our Data Quality guide. The modern data stack that MLflow sits within — alongside dbt, Airflow, and feature stores — is in our Modern Data Architecture guide.