Thursday, November 19, 2026
HomeData ScienceScore-Based Generative Models โ€“ Denoising Score Matching, SDEs, Flow Matching and Guidance

Score-Based Generative Models โ€“ Denoising Score Matching, SDEs, Flow Matching and Guidance

Table of Content

📋 KEY INSIGHTS

  • Score-based generative models (SBGMs) define a generative process via the score function โ€” the gradient of the log data density with respect to the data: s(x) = โˆ‡_x log p(x). Instead of learning the data density p(x) directly (intractable for high-dimensional data), SBGMs learn the score function and use it to navigate toward high-density regions via Langevin dynamics: x_{t+1} = x_t + ฮตยทs(x_t) + โˆš(2ฮต)ยทz, where z ~ N(0,I) and ฮต is the step size. This iterative update drifts x toward high-probability regions (via the score term) while exploring the distribution (via the noise term), ultimately generating samples from the learned distribution. Score-based models and diffusion models are mathematically equivalent โ€” denoising diffusion probabilistic models (DDPMs) can be derived from the score-based framework as a specific discretisation of a stochastic differential equation.
  • Denoising Score Matching (DSM, Vincent 2011; Song & Ermon 2019) is the practical training objective for score-based models. The challenge: estimating the score function โˆ‡_x log p(x) directly from data requires computing the partition function, which is intractable. DSM sidesteps this by training a neural network s_ฮธ(x, ฯƒ) to predict the score of a noisy data distribution: given a clean sample x_0 and noise level ฯƒ, perturb x to x_tilde = x_0 + ฯƒฮต (where ฮต ~ N(0,I)), and train the network to predict the score of the noisy distribution: L_DSM = E[||s_ฮธ(x_tilde, ฯƒ) + ฮต/ฯƒ||^2]. The key insight: the score of the noisy distribution โˆ‡_{x_tilde} log p_ฯƒ(x_tilde) = โˆ’(x_tilde โˆ’ x_0)/ฯƒ^2 = โˆ’ฮต/ฯƒ, so DSM reduces to training the network to predict the noise ฮต โ€” exactly the denoising objective used in DDPMs. This establishes the fundamental equivalence between score matching and denoising.
  • The SDE (Stochastic Differential Equation) framework (Song et al., NeurIPS 2021) unifies all score-based and diffusion models under a continuous-time perspective. The forward SDE gradually adds noise: dx = f(x,t)dt + g(t)dw (where f is the drift coefficient, g is the diffusion coefficient, and w is Brownian motion). The reverse SDE generates samples by running the time-reversed process: dx = [f(x,t) โˆ’ g(t)^2 โˆ‡_x log p_t(x)]dt + g(t)dw_bar. Any choice of forward SDE (VP-SDE matching DDPM, VE-SDE matching NCSN, sub-VP-SDE) gives a valid generative model once the score function โˆ‡_x log p_t(x) is learned. The SDE framework enables probability flow ODEs: the reverse ODE dx = [f(x,t) โˆ’ (1/2)g(t)^2 โˆ‡_x log p_t(x)]dt has the same marginal densities as the reverse SDE but is deterministic โ€” enabling faster sampling with ODE solvers and exact likelihood computation.
  • Noise Conditional Score Networks (NCSN, Song & Ermon, NeurIPS 2019) train a single score network conditioned on the noise level ฯƒ across a geometric sequence ฯƒ_1 < ฯƒ_2 < ... < ฯƒ_L. At large ฯƒ (high noise), the score pushes samples toward the modes of the data distribution from far away; at small ฯƒ (low noise), the score refines samples to lie on the data manifold. Annealed Langevin dynamics uses this multi-scale score: start with large ฯƒ_L, run T Langevin steps, decrease ฯƒ, repeat โ€” an annealing schedule that reliably converges to high-quality samples. NCSN++ and NCSNv3 scale this to 256ร—256 image generation with FID scores competitive with GANs, without adversarial training instabilities.
  • Flow Matching (Lipman et al., 2022; Liu et al., 2022) is a recently developed training paradigm that learns a continuous normalising flow by regressing on simple target vector fields rather than optimising a likelihood lower bound or score matching objective. The conditional flow matching loss regresses on the vector field that transports a source distribution (Gaussian noise) to the data distribution via straight-line paths: L_CFM = E[||v_ฮธ(x_t, t) โˆ’ (x_1 โˆ’ x_0)||^2], where x_t = (1โˆ’t)x_0 + tยทx_1 is the linear interpolant between noise x_0 and data x_1. Flow matching avoids the complex curved trajectories of diffusion models (which follow the reverse SDE), enabling faster sampling (fewer NFEs โ€” Number of Function Evaluations) with the same or better sample quality. Stable Diffusion 3 and FLUX use flow matching as the generative backbone, achieving state-of-the-art text-to-image generation quality with 10-20 NFEs compared to 50-1000 for DDPM-based models.
  • Guidance mechanisms for score-based and diffusion models enable conditional generation without retraining the unconditional model. Classifier guidance (Dhariwal & Nichol, 2021) adds the gradient of a classifier log p(y|x_t) to the score: s_guided(x_t, y) = s_ฮธ(x_t) + wยทโˆ‡_{x_t} log p_ฯ†(y|x_t). Classifier-free guidance (Ho & Salimans, 2022) is now standard: train the score network jointly on conditional and unconditional inputs (drop the condition with probability 10-20%), then at sampling time use a linear combination: s_CFG = (1+w)ยทs_ฮธ(x_t, y) โˆ’ wยทs_ฮธ(x_t). Higher guidance weight w increases sample fidelity to the condition at the cost of diversity. The connection to our diffusion models guide is direct: classifier-free guidance is used in DALL-E 2, Stable Diffusion, and all production text-to-image models โ€” it is the bridge between the theoretical score-based framework and the practical conditional generation capabilities described in that guide.

Diffusion models have become the dominant generative modelling paradigm for images, audio, video, and molecular structures โ€” but their theoretical foundations in score-based generative modelling are less widely understood than the DDPM training objective and U-Net architecture. Score-based models offer a unifying mathematical perspective on diffusion: the denoising objective, the reverse sampling process, the probability flow ODE, and the guidance mechanisms all follow from the score function framework in a principled way. Understanding score-based generative modelling is essential for researchers pushing the boundaries of generative AI and for practitioners who want to understand why design choices (noise schedule, sampler, guidance weight) affect sample quality. This guide covers the full score-based generative modelling landscape: score functions and Langevin dynamics, denoising score matching, NCSN and annealed Langevin sampling, the SDE framework unifying all diffusion models, probability flow ODEs for fast sampling and likelihood computation, flow matching as the next-generation training objective, and guidance mechanisms for conditional generation.

Score Functions, Denoising Score Matching and Langevin Dynamics

The score function โˆ‡_x log p(x) is the gradient of the log-probability density with respect to the data point โ€” it points in the direction of steepest increase in log-probability, i.e. toward the nearest mode of the data distribution. If we know the score function, we can generate samples by starting from random noise and following the score direction: Langevin dynamics x_{t+1} = x_t + ฮตยทโˆ‡_x log p(x_t) + โˆš(2ฮต)ยทz_t converges to samples from p(x) under mild conditions. The score function is attractive because it does not require computing the partition function Z (the intractable normalising constant of p(x) = exp(E(x))/Z): the score โˆ‡_x log p(x) = โˆ‡_x E(x) โˆ’ โˆ‡_x log Z = โˆ‡_x E(x) (since Z does not depend on x, its gradient is zero). This makes score-based models a principled alternative to explicit density models (which require tractable normalising constants) and implicit models (GANs, which avoid density estimation entirely). Denoising Score Matching provides a tractable training objective: perturb data with noise, train a network to predict the noise, and the trained network implicitly estimates the score of the noisy distribution. The DSM objective L_DSM(ฮธ; ฯƒ) = (1/2)E[||s_ฮธ(x + ฯƒฮต) โˆ’ (โˆ’ฮต/ฯƒ)||^2] trains the network to predict โˆ’ฮต/ฯƒ given a noisy observation x + ฯƒฮต, which equals the score of the Gaussian-smoothed data distribution: โˆ‡_x log p_ฯƒ(x) = โˆ’(x โˆ’ x_0)/ฯƒ^2 when p_ฯƒ is Gaussian noise centred at x_0. The connection to our diffusion models guide is direct: the DDPM training objective E[||ฮต โˆ’ ฮต_ฮธ(x_t, t)||^2] is identical to DSM with the reparameterisation ฮต_ฮธ = โˆ’ฯƒ_tยทs_ฮธ โ€” DDPMs are score-based models trained with DSM, expressed in terms of predicted noise rather than predicted score. The connection to our Bayesian deep learning guide is direct: Langevin dynamics is a Markov chain Monte Carlo method with deep connections to Bayesian posterior sampling โ€” Stochastic Gradient Langevin Dynamics (SGLD) uses the same update rule with a gradient of the log-posterior rather than the log-data-density, connecting generative modelling and Bayesian inference at a fundamental level. The U-Net architecture used in NCSN and DDPM score networks is chosen specifically for its ability to process images at multiple spatial resolutions and merge global structure with local detail โ€” the same multi-scale processing used in semantic segmentation. The noise level ฯƒ (or equivalently the diffusion time t) is provided to the score network as a conditioning signal via sinusoidal time embeddings (identical to the positional embeddings used in Transformers, described in our Transformers guide), allowing a single network to represent the score function at all noise levels.

Model FamilyTraining ObjectiveSampling MethodNFEs (Steps)Key Papers
NCSN / NCSNv3Denoising Score MatchingAnnealed Langevin1000+Song & Ermon 2019, 2020
DDPMNoise prediction (= DSM)Reverse diffusion1000Ho et al. 2020
DDIMSame as DDPMDeterministic ODE solver50โ€“250Song et al. 2021
Score SDE (VP/VE)Continuous-time DSMReverse SDE / prob. flow ODEVariableSong et al. NeurIPS 2021
Flow MatchingConditional flow matchingODE integration (straight paths)10โ€“20Lipman et al. 2022; SD3, FLUX

SDE Framework, Flow Matching and Guidance Mechanisms

a robot with a light saber
Photo by Growtika on Unsplash

The SDE unification (Song et al., 2021) reveals that DDPM and NCSN are special cases of a general continuous-time forward-reverse SDE framework differing only in the choice of forward SDE (VP-SDE for DDPM, VE-SDE for NCSN). This unification enables several practical advances: (1) the probability flow ODE provides a deterministic sampling path with the same marginals as the stochastic sampler, enabling likelihood computation via the instantaneous change of variables formula; (2) any fast ODE solver (DPM-Solver, DEIS, UniPC) can be applied to the probability flow ODE for accelerated sampling with 20-50 NFEs rather than 1000; and (3) the continuous-time view enables learned noise schedules that optimise the signal-to-noise ratio trajectory. DPM-Solver++ (Lu et al., 2022) is the most widely used fast ODE solver for diffusion models in production: it achieves near-perfect sample quality in 10-20 NFEs by using a Taylor expansion of the probability flow ODE solution around the previous step, dramatically reducing the sampling cost of Stable Diffusion and other large-scale models. Flow Matching (Lipman et al., 2022; Liu et al., 2022) replaces the complex curved SDE trajectories with straight-line optimal transport paths between noise and data: the vector field v*(x, t) = x_1 โˆ’ x_0 (pointing from noise to data) is constant along each path, making regression more stable and enabling accurate generation with very few ODE integration steps. The conditional flow matching training objective L_CFM = E[||v_ฮธ((1โˆ’t)x_0 + tยทx_1, t) โˆ’ (x_1 โˆ’ x_0)||^2] regresses the neural network onto this constant target vector field, conditioned on individual data-noise pairs rather than the full marginal flow. This conditioning avoids the high-variance regression problem of standard flow matching while recovering the same optimal transport velocity field in expectation. Stable Diffusion 3 (Esser et al., 2024) and FLUX (Black Forest Labs, 2024) adopt flow matching over Gaussian noise with a Multi-Modal Diffusion Transformer (MMDiT) architecture combining text and image tokens in a single stream, achieving state-of-the-art text-to-image quality. The connection to our diffusion models guide is direct: flow matching is the theoretical successor to DDPM and DDIM โ€” understanding flow matching requires the DDPM foundations from that guide, and the classifier-free guidance, VAE latent space, and text conditioning mechanisms are shared between diffusion and flow matching models. Consistency models (Song et al., 2023) take the ODE perspective further: train a model f_ฮธ(x_t, t) that directly maps any noisy observation x_t to the clean data x_0 (the endpoint of the probability flow ODE trajectory), satisfying the self-consistency condition f_ฮธ(x_t, t) = f_ฮธ(x_t’, t’) for t, t’ on the same ODE trajectory. This enables 1-step generation (from pure noise to data in a single function evaluation) while maintaining sample quality close to multi-step diffusion โ€” the fastest generative modelling approach at parity with diffusion quality.

✦ SUMMARIZE THIS ARTICLE WITH AI

Diffusion models (DDPM, DDIM, Stable Diffusion, classifier-free guidance) are in our Diffusion Models guide. Generative AI (practical applications of diffusion and flow matching) is in our Generative AI guide. Bayesian deep learning (SGLD connecting Langevin dynamics to Bayesian inference) is in our Bayesian Deep Learning guide. Transformers (time embeddings; MMDiT for SD3 and FLUX) are in our Transformers guide. ML for drug discovery (score-based models for molecular generation in REINVENT and DiffSBDD) is in our Drug Discovery guide. Neural ODEs (probability flow ODE as a continuous normalising flow) are in our Neural ODEs guide. Self-supervised learning (score matching as an unsupervised representation learning objective) is in our Self-Supervised Learning guide.

Leave feedback about this

  • Rating

Durgesh Kekare
Durgesh Kekarehttps://www.dataexpertise.in
Durgesh Kekare is a data science educator and founder of DataExpertise.in. With expertise in Python, machine learning, and analytics, he helps 10,000+ learners break into data careers.

Latest Posts

List of Categories