I'm familiar with expectation of functions of some discrete random variable x and it is given by
, where
p(x) is probability distribution function of x.
And if the random variable is continues valued then the expectation is given by
Recently I was reading some paper and it contains equation for expected loss. Part of the equation contains expectation of some function of some other random function. Here is some approximate equation just to show you the expectation part.
Here the function f it self is random function and x is random variable.
My question is how can I calculate the expectation of the above equation, where the function it self is random?
Please be noted that I've only high school level mathematics with some basics on statistics, calculus and linear algebra.
Best Answer
The short answer is you take the answer for each individual choice for the function $f$, then average over them all. How does the long answer flesh that out? Well, it depends on which $f$ are options. (It might help if you specified which paper you read.)
For example, if you know $f$ up to a finite number of real-valued parameters, $E_x[f]$ becomes a function of such parameters, which can be averaged over their distribution. That's well within the purview of what you've learned (unless you haven't yet learned multiple integrals such as $\int_{\mathbb{R}^2}g(u,\,v)dudv$, but even then you could think through the $1$-parameter case).
If on the other hand $f$ can roam through a family of functions with infinitely many degrees of freedom, you need something called functional integration. As a branch of mathematics, it's actually still not very well-understood except for some (admittedly very useful) special cases. However, if you try learning it you'll find most of the relevant intuitions have counterparts in the "normal" calculus you've already covered. For example, you can think of a function as an infinite-dimensional vector, with one component per value of its argument(s). Once you learn how to integrate $\exp -ax^2+Jx$ on $\mathbb{R}$, and $\exp -x^TAx+J^Tx$ on $\mathbb{R}^n$ with a matrix $A$ and vector $J$, the functional equivalent promotes $x,\,A,\,J$ to functions with $1,\,2,\,1$ arguments respectively, where the "dot product" is an integral.