I'm aware of the topic of variational inference (VI) however I'm not really sure what Black box VI is?
In particular I am watching a video by David Blei titled Black box variational inference and on this slide it mentions "black box criteria".
I would really appreciate an example as well.
Best Answer
So he’s referring the technique introduced in this paper: https://arxiv.org/abs/1401.0118
The idea behind black box VI is that normally in VI it takes a significant amount of work to decide on a variational posterior and derive the ELBO and it’s gradients. As such there was room for a more general algorithm that can be easily implemented and doesn’t require the practitioner to derive these forms every time. The “black box” part of the name is just from the fact it’s a general algorithm that works and you don’t need to think about what’s going on inside.
Essentially black box VI is a method that yields an estimator for the gradient of the ELBO with respect to the variational parameters with very little constraint on the form of the posterior or the variational distribution. These constraints (the black box criteria you mention) are only that you can evaluate the first derivatives of the log of the variational distribution with respect to its parameters (which you should be able to as you would normally pick a relatively simple distribution for the variational) and that you can evaluate the log of the joint of the data and latent variables (again very standard in the setting of probabilistic modelling).