Glossary
Diffusion models explained: what a diffusion model is
In short
A diffusion model is a neural network that creates an image, a video or a sound by removing noise from a random picture, step by step.
Images, video, audio · diffusion model
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine a window so fogged up that you cannot see anything through it. You wipe it a little at a time, and a house, a tree and a cat on a fence appear through the spots. Such a neural network draws in a similar way: it starts with a picture made only of dots and carefully “wipes” it many times until a drawing appears.
In plain words
Training works the other way around. Noise is added to good photos little by little until only gray static is left, and the model learns to guess which noise was added at each step. Once it has learned this, the process runs in reverse. The model gets pure static and removes a little noise from it again and again. If you also give it a text, such as “a ginger cat on a windowsill”, it removes the noise so that exactly such a cat appears. That is why the same model draws different things from different descriptions.
For business
Image, video and music generation services run on such models, and marketing, design and sales use them for banners, product cards, storyboards and photo backgrounds. You can run open versions on your own server with a graphics card, so images and descriptions do not go to an outside service. The cost depends on the resolution, the number of generation steps and the amount of video memory; video costs much more than an image. The risks: rights to the training images differ between models, licenses limit commercial use, and a model can copy someone else’s style or logo. Before launch, a lawyer checks the license, and a designer decides where generation saves hours and where manual work is still needed. Hardware needs for such a task are calculated by the amount of video memory.
For a specialist
Diffusion models define a forward process that adds Gaussian noise to data over T steps and train a reverse process that restores the data from noise. Sohl-Dickstein et al. (2015) proposed the idea, and denoising diffusion probabilistic models (DDPM, Ho, Jain, Abbeel, 2020) showed high image quality: a network, usually a U-Net, predicts the added noise, and training is linked to denoising score matching. Latent diffusion (Rombach et al., 2021) moves the process into the compressed space of a pretrained autoencoder, which cuts computation sharply, and feeds the text condition through cross-attention. Sampling is sped up by deterministic samplers such as DDIM and by distillation into a few steps, and classifier-free guidance sets how closely the output follows the prompt. Newer architectures replace the U-Net with a transformer (diffusion transformer).
Example
A furniture factory generates backgrounds for sofa photos: a Scandinavian kitchen, a living room with a fireplace, a loft. The model runs on the company’s server, and in one hour a designer builds a series of cards for a marketplace instead of an on-site photo shoot.
Related terms
Questions and answers
How does a diffusion model differ from a GAN?
Sources
Not sure whether your company needs this?
AI solutions map