As far as I know, both Gaussian mixtures as well as Gaussian processes can be used for regression. My question is: what is better and why?
The answers might contain theoretic insights, practical experience or reference to further resources.
EDIT
I would like to know which one is better for approximation of nonlinear regression problems
$$Y_i=g(x_i)+V_i$$
where $V_i\sim\mathcal{N}(0,\sigma^2)$ for all considered $i$. The function $g(\cdot)$ is assumed to be continuous. For the simplicity sake, we can admit that $x_i\in\mathbb{R}$. The criterion is the one-leave out mean square error.
The second criterion, perhaps more important. Since I assume that both (GM and GP) have the universal approaximation property, I would be curious which one is less complex (in terms of time and/or memory) in order to obtain given level of error.
Finally, I assume that the new data points $(x_i,y_i)$ can be generated on demand with a constant computational compexity.
Best Answer
To answer your last question, Gaussian process is a discriminative model as opposed to generative. Therefore, you will not be able to model $p(x, y)$ using Gaussian process. Gaussian process models $p(y | x)$ instead. To generate samples $(x_i, y_i)$ you need to work with a generative model such as Gaussian mixture model.