I have modeled a distribution, $f$, over a r.v. $x \in \mathbb{R}^3$. At inference a set of measuring points, $X$, of the r.v. variables show up. I want to form a distribution over this sample set so that I can sample from it so that more probable points will show up more often.
My idea was to evaluate $f$ for each point in the set and then normalize it to form a discrete distribution over the points. Since $f \approx 0$ I want to use the log likelihood. However, the $\log$ is a nonlinear transform and normalizing it means calculating:
$\log \sum_i f(x_i)$,
which is bound to give numerical errors due to the fact that $f(x_i)\approx 0$. So the best I can do is a log sum exp calculation.
My question is: is there a better way of doing this? And is this a completely wrong approach in terms of modeling probability distributions?
Best Answer
I'm not sure I understand things correctly. Here are my thoughts:
Does that help?