Why in "Method of Moments", we equate sample moments to population moments for finding point estimator?
Where is the logic behind this?
Method of Moments – Understanding the Logic Behind Method of Moments
intuitionmethod of moments
Related Solutions
Least squares estimator in the classical linear regression model is a Method of Moments estimator.
The model is
$$\mathbf y = \mathbf X\beta + \mathbf u$$
Instead of minimizing the sum of squared residuals, we can obtain the OLS estimator by noting that under the assumptions of the specific model, it holds that ("orhtogonality condition")
$$E(\mathbf X' \mathbf u)= \mathbf 0$$
$$\implies E(\mathbf X'( \mathbf y - \mathbf X\beta))=\mathbf 0 \implies E(\mathbf X'\mathbf y)=E(\mathbf X'\mathbf X)\beta$$
$$\implies \beta = \left[E(\mathbf X'\mathbf X)\right]^{-1}E(\mathbf X'\mathbf y)$$
So if we knew the true expected values (and our assumptions were correct), we could calculate the true value of the unknown coefficient.
With non-experimental data, we don't. But we know that if our sample is ergodic-stationary (and i.i.d. samples are), then expected values are consistently estimated by their sample analogues, the corresponding sample means. Hence we have an "acceptable" estimator in
$$\hat \beta = \left[((1/n)\mathbf X'\mathbf X)\right]^{-1}((1/n)\mathbf X'\mathbf y) = (\mathbf X'\mathbf X)^{-1}\mathbf X'\mathbf y $$
which is the same estimator we will obtain if we minimize the sum of squared residuals.
If you reverse the calculations, and noting that the residuals are a function of $\hat \beta$, $\mathbf {\hat u} = \mathbf {\hat u(\hat \beta)}$ you will find that $\mathbf X' \mathbf {\hat u} (\hat \beta) = \mathbf 0$. Divide by $n$ for this to look like a sample mean.
So "we choose those estimates that make the sample obey what we assumed the population obeys". And we do that because we accept that the sample is representative of the population, so it should "behave like" the population (as we assumed the latter to behave).
As for $GMM$, each orthogonality condition is an equation. If you have $m$ equations and $k<m$ unknown coefficients, then the system of equations is "over-identified" and no exact solution exists, there is nothing more to it.
Hansen contrasted this with the original use of "Method of Moments": if a distribution is characterized by, say, three unknown parameters, the MoM tactic is to estimate using the sample the first three moments of the distribution (which appear in equations involving these parameters), and obtain an exactly-identified system of equations. See how this works in this answer, as an example.
The least squares estimator is the solution to the estimating equation:
$$ 0 = \mathbf{X}^T \left( Y - \mathbf{X}\beta \right)$$
Where $\mathbf{X} = [1, x_1, x_2, \ldots, x_p]$ is a $n \times p$ model matrix of covariate(s).
This is a trivial result, but a more general discussion on estimating equations can be found in Wakefield "Bayesian and Frequentist Regression Methods". Estimating Equations are also called M-estimators. Another reference would be Boos, Stefanski "Essential Statistical Inference" ch. 7.
Best Answer
A sample consisting of $n$ realizations from identically and independently distributed random variables is ergodic. In a such a case, "sample moments" are consistent estimators of theoretical moments of the common distribution, if the theoretical moments exist and are finite.
This means that
$$\hat \mu_k(n) = \mu_k(\theta) + e_k(n), \;\;\; e_k(n) \xrightarrow{p} 0 \tag{1}$$
So by equating the theoretical moment with the corresponding sample moment we have
$$\hat \mu_k(n) = \mu_k(\theta) \Rightarrow \hat \theta(n) = \mu_k^{-1}(\hat \mu_k(n)) = \mu_k^{-1}[\mu_k(\theta) + e_k(n)]$$
So ($\mu_k$ does not depend on $n$)
$$\text{plim} \hat \theta(n) = \text{plim}\big[\mu_k^{-1}(\mu_k(\theta) + e_k)\big] = \mu_k^{-1}\big(\mu_k(\theta) + \text{plim}e_k(n)\big)$$
$$=\mu_k^{-1}\big(\mu_k(\theta) + 0\big) = \mu_k^{-1}\mu_k(\theta) = \theta$$
So we do that because we obtain consistent estimators for the unknown parameters.