Glossary
What is a variational autoencoder?
In short
A variational autoencoder (VAE) is a neural network that compresses data into a random code with a set distribution and can create new, similar examples from that code space.
Machine learning · variational autoencoder
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine a box of modeling clay. You make a bunny, squash it into a lump, and then make it again, but a little different: with longer ears or a fluffier tail. The lump remembers it was a bunny, but not every detail. This program does the same with pictures: it squashes them into a small lump of numbers and shapes new, similar ones from it.
In plain words
An ordinary autoencoder compresses a picture into a short code and restores it. A variational one goes further: instead of one point, it describes each picture as a small cloud of possible codes and trains all the clouds to lie close together, in one neat area. So you can take a random point from this area and get a new image that was not in the training data. You can also move smoothly from one point to another: a smile on a face appears step by step. Modern image generators compress pictures in a similar way.
For business
This method is rarely sold as a product on its own, but it works inside many systems. In text-to-image generators, the variational encoder compresses the picture so the main model works with a small representation, which cuts the cost of computing. Other uses are anomaly detection in equipment readings and transactions, where a poorly restored example counts as suspicious, synthetic table data and signal compression. Pictures from a plain VAE come out softer and blurrier than those of diffusion models, so it is not enough on its own for advertising graphics. What to check: what data the model was trained on and how often it misses real anomalies. A pilot on your incident history is a good way to run this check.
For a specialist
Kingma and Welling described the variational autoencoder (VAE) in the paper Auto-Encoding Variational Bayes (2013). It is a latent generative model p(x|z)p(z) trained by variational inference: the encoder, or recognition model, sets an approximate posterior distribution q(z|x), usually Gaussian with diagonal covariance, and the decoder restores x from z. Training maximizes the evidence lower bound (ELBO): the expected reconstruction quality minus the KL divergence between q(z|x) and the prior p(z). The reparameterization trick writes z as a deterministic function of the parameters and noise, so the estimate can be optimized by stochastic gradient descent. Rezende, Mohamed and Wierstra (2014) published a close approach with stochastic backpropagation. Variants include beta-VAE for disentangling factors, the conditional VAE and VQ-VAE with a discrete code. In latent diffusion models (Rombach et al., 2021), diffusion runs in the latent space of a pretrained autoencoder.
Example
A plant records the vibration of its pumps. A VAE was trained only on records of normal operation, and when a new record is restored much worse than usual, the system warns the mechanic about a possible bearing failure.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map