Imagine that we have mixture of two normal distributions with mixture parameter $\theta$:
$$p(y_i|\theta) = \theta\phi(y_i;\mu_1, \sigma_1^2) + (1 – \theta)\phi(y_i; \mu_2, \sigma_2^2)$$
Assume that the ONLY unknown parameter is $\theta$. I want to get the MLE estimate of $\theta$. Here is what I do:
$$L(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) = \prod_{i = 1}^{n} p(y_i|\theta) = \prod_{i = 1}^{n} \Big(\theta\phi(y_i;\mu_1, \sigma_1^2) + (1 – \theta)\phi(y_i; \mu_2, \sigma_2^2)\Big) \Rightarrow$$
$$L(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) \propto \prod_{i = 1}^{n}\Big[ \theta \Big(\phi_{i1} – \phi_{i2}\Big)\Big] = \theta^n \prod_{i = 1}^{n}\Big(\phi_{i1} – \phi_{i2}\Big) = \theta^nG(Y, param)$$
where $\phi_{i1}$ refers to the first normal density with $y_i$ plugged in and $\phi_{i2}$ refers to the second normal density. Note that there is no $\theta$ involved in G(Y, param).
Now, in order to get the MLE, I take log and then derivative:
$$l(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) = log(L(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2)) \propto n log(\theta) + log(G) \Rightarrow $$
$$l'(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) = \frac{n}{\theta}$$
If my calculations above is right, then it's a very weird derivative and I don't know how to get the MLE estimate.
Best Answer
The problem is in the factorisation after "$\Rightarrow$" since
$$\theta \phi_{i1} +(1-\theta)\phi_{i2} = \theta(\phi_{i1}-\phi_{i2})+\phi_{i2}.$$
Then, the term $\phi_{i2}$ cannot be eliminated using an argument of proportionality and it has to be considered in the product. This produces a different likelihood and the corresponding estimator can also be found using the log-likelihood.