Variantional Bayesian (VB) methods are a popular family of techniques (along with Generative Advesarial Networks (GANs) [1] and Fully visible belief networks (FVBNs) [2]) in statistical machine learning, in particular, for training generative models. VB methods allow us to cast the statistical inference problems into optimization problems which can be solved by latest optimization algorithms. The optimization objective known as Variational Lower Bound - also called Evidence Lower Bound (ELBO) - was proposed in Varionational Autoencoders (VAE) [3], which is one of the simplest form of VB methods. This goal of this post is to learn and take notes about this Variational Lower Bound.

Variational Lower Bound

References

[1] Ian Goodfellow et al, NIPS 2014.
[2] (Frey et al., 1996; Frey, 1998).
[3] Varionational Autoencoders.