I am working my way through LDA and I think I got they main idea of it. Please correct me if I am wrong. Given the Plate notation:
The variables $\alpha$ and $\beta$ are Dirichlet distribution parameters. The variable $Z_{d,n}$ assigns observed word $W_{d,n}$ to topic $\phi_k$, which is a distribution over words. Variable $\theta_d$ is the document-specific topic distribution. Both distributions $\theta_d$ and $\phi_k$ are drawn from Dirichlet distributions.
Now, only $W_{d,n}$ is observed and can be "directly" calculated. My question:
What exactly is inferred/calculated with e.g. Gibbs sampling, variational Inference and so on?
For instance: For a Gaussian Naive Bayes classifier one assumes that the likelihood of each feature is Gaussian. In other words each feature has a Gaussian distribution:
$$
P(x) = \frac{1}{{\sigma \sqrt {2\pi } }} e^{ \frac{ – ( {x – \mu } )^2} {2\sigma ^2 } }
$$
To find this distribution $\sigma$ and $\mu$ have to be determined which is pretty straight forward.
However, plainly said: What Numbers do I determine for LDA?
Best Answer
The mean and variance of a Gaussian are the unknown parameters that specify that distribution in that case. Likewise, in topic modeling, you attempt to learn the unknown parameters of $K$ topics, where each topic is a multinomial distribution over words in the vocabulary. Thus, it is the parameters of each multinomial distribution (each topic) that you seek to infer.
Note that your learning algorithm will also output another set of multinomial paramaters that represent the distribution of topics for each document.