I am studing the exponential family – and trying to understand the flow of formulas per below. In my book it is defined that $A(\theta)=\log \int \left(h(x) \exp \theta^T \phi(x) \right)$
In the formulas per below – i don't understand how the denominator between 9.28 and 9.29 goes from $\int \left(\exp \theta^T \phi(x) h(x) \right)$ to $exp(A(\theta))$ , as this would in my mind imply that $A(\theta)= \int \left(h(x) \exp \theta^T \phi(x) \right)$ ?
i am probably making an elementary mistake – any quick tip much appreciated
(please let me know if it is norm on this forum to write out all formulas in these cases in Mathjax)
Best Answer
We have the definition of $A$ given as
$$A(\theta)\equiv\log\left(\int h(x)e^{\theta\phi(x)}dx\right)\tag 1$$
Let $z(\theta)\equiv\int h(x)e^{\theta\phi(x)}dx$ so that $A(\theta)=\log z(\theta)$. Then, inasmuch as $e^{\log z}=z$, we have
$$\begin{align} e^{A(\theta)}&=e^{\log z(\theta)}\\\\ &=\int h(x)e^{\theta\phi(x)}dx\tag2 \end{align}$$
From the text, Equation $(9.28)$ is
$$\frac{\frac{d}{d\theta}\int h(x)e^{\theta\phi(x)}dx }{\int h(x)e^{\theta\phi(x)}dx}\tag{9.28}$$
Note that the denominator in $(9.28)$ is $z(\theta)$, which is from $(2)$ equal to $e^{A(\theta)}$.