I've been trying to teach myself General Reletivity recently, I've came across two problems whisled doing so; I don't really understand why $R=g^{\mu\nu}R_{\mu\nu}$, why couldn't I contract the indices with a different two index tensor eg. $R=\gamma^{\mu\nu}R_{\mu\nu}$, and I don't understand how the Einstein Field Equations are derived. I've searched online and most people derive it by using the Einstein-Hilbert action, but my understanding on Calculus of Variations is a bit shakey and I don't understand what Einstein and Hilbert were trying to minimise. So if you can explain an easy way of deriving the Einstein Field Equations or explain the Einstein-Hilbert action and deriving the equations through it that will be much appreciated.
[Physics] How does one derive the Einstein Field Equations
general-relativity
Related Solutions
Your questions essentially amounts to ask
How do we quantize GR?
which is the starting point of quantum gravity (QG). GR is a non-renormalizable theory, at least from the traditional perspective of perturbation theory in QFT. So the path integral with the (exponentiated) Einstein-Hilbert action as weight factor cannot easily be used to make meaningful physical predictions. New approaches to QG are needed, such as e.g. string theory (ST).
In some sense, yes it is, and in others it is certainly not.
The fact that in the Einstein field equations: $G_{\mu \nu} = 8 \pi T_{\mu \nu}$, you have complete 'freedom' to define $T_{\mu \nu}$ however you would like (within some kind of exceptions), means you can allow for highly non-trivial curvature dependent terms to be coupled to the stress-energy. See this really interesting paper by Capozziello et al (http://arxiv.org/abs/grqc/0703067) that shows how you can 'bunch' the additional curvature terms for an $f(R)$ gravity into the 'effective' stress-energy tensor. In this way, the uniqueness of what you may call the 'Einstein' field equations does not hold.
However, once one moves to the vacuum the answer is suddently a definitive yes. The Einstein equations are generated by having an action that contains a geometric invariant quantity. If it did not then the principles of general covariance would no longer hold since arbitrary coordinate transformations would necessarily destroy the gauge symmetries. Now, that being said, one can consider the most general possible geometric invariant Lagrangian in this sense, it would look something like.. $\mathcal{L} = f(g_{\mu \nu} R^{\mu \nu}, R^{\mu \nu} R_{\mu \nu}, R^{\alpha \beta \gamma \delta} R_{\alpha \beta \gamma \delta}, C^{\alpha \beta \gamma \delta} C_{\alpha \beta \gamma \delta}, g^{\mu \nu} R^{\alpha \beta} R_{\alpha \beta \mu \nu}...)$ and the list of possibilities goes on. Since you want to replicate the Einstein equations in themselves you can immediatly throw away all but the linear terms in the above. Any of the other variants when you perform the calculus of variations gives you higher than second order derivatives.
Now, keeping only the linear terms you see that the action will necessarily be the Einstein-Hilbert one plus (maybe) some other curvature invariants. Since these will not vanish once you perform the variation, unless they either vanish identically or vanish on the boundary as a divergence term, your field equations will necessarily still be the Einstein-Hilbert ones provided you only keep $\mathcal{L} = R^{\sigma}_{\sigma}$.
A rigorous proof of this would be interesting. I am unaware of any in the literature.
Best Answer
Once the geometric context of gravity is clear (which is really kind of independent of the specific field equations), one can reason as follows.
The field equation must reduce to the field equation of Newtonian gravity, which is $$ \nabla^2\phi=\kappa\rho, $$ where $\rho$ is the density of the source ($\kappa$ is an irrelevant constant).
We know from special relativity, that mass/energy density is by itself not a scalar quantity, but (in $c=1$ units) is the $00$ component of the stress-energy tensor, $\rho=T^{00}$.
If we want a geometric description and the equivalence principle, then particles moving under the effects of solely gravity must obey the geodesic equation, which is $$ \frac{d^2\gamma^\mu}{d\tau^2}=-\Gamma^\mu_{\kappa\lambda}\frac{d\gamma^\kappa}{d\tau}\frac{d\gamma^\lambda}{d\tau}. $$
In the nearly flat spacetime ($g_{\mu\nu}=\eta_{\mu\nu}+h_{\mu\nu}$) and slow particle ($d\gamma^0/d\tau$ dominates over the spatial components) we get $$ \frac{d^2\gamma^\mu}{d\tau^2}=-\Gamma^\mu_{00}\left(\frac{d\gamma^0}{d\tau}\right)^2, $$ where in this limit the connection is $\Gamma^\mu_{00}=\frac{1}{2}\eta^{\mu\rho}(2\partial_0h_{0\rho}-\partial_\rho h_{00})$. We further assume that the metric doesn't change in time (Newtonian gravitation has no time evolution), then we get $$ \frac{d^2\gamma^\mu}{d\tau^2}=\frac{1}{2}\left(\frac{d\gamma^0}{d\tau}\right)^2\partial^\mu h_{00}. $$
Due to staticness, the 0 component of this equation is just $d^2\gamma^0/d\tau^2=0$, which means that $d\gamma^0/d\tau$ is constant, so we can divide by its square (and use that $\gamma^0$ is basically the $t$ coordinate time), we get $$ \frac{d^2\gamma^i}{dt^2}=\frac{1}{2}\partial_i h_{00}, $$ which is the equation of motion in a Newtonian gravitational field if $h_{00}=-2\phi$.
So going back to the Poisson equation $\nabla^2\phi=\kappa\rho$, we know that $\rho=T^{00}$, and $\phi$ is related to $h_{00}$ and as such to $g_{00}$, and second derivatives of $\phi$ appear, so the Newtonian equation has the form $$ K^{00}=\kappa T^{00}, $$ but we want a tensorial equation, so the full gravitational equation should be $$ K^{\mu\nu}=\kappa T^{\mu\nu}, $$ where $K^{\mu\nu}$ is some kind of tensor field constructed out of the metric's second derivatives.
But we know from differential geometry, that all tensor fields depending on the metric's second derivatives are derived from $ R^\kappa_{\ \lambda\mu\nu} $, the curvature tensor.
There are not many second-rank tensors that can be made of this, one of them is $R^{\mu\nu}$ the Ricci tensor. Here we could also look at the Ricci tensor's Newtonian limit to see how it is related to $h_{00}$ but I don't want to do that here.
Unfortunately $K^{\mu\nu}$ cannot be proportional to $R^{\mu\nu}$ because the stress energy tensor must satisfy $\nabla_\nu T^{\mu\nu}=0$, but we have $$ \nabla_\nu R^{\mu\nu}=\frac{1}{2}\nabla^\mu R $$ from the Bianchi identity, so by rearranging we get $$ \nabla_\nu R^{\mu\nu}=\frac{1}{2}g^{\mu\nu}\nabla_\nu R \\ \nabla_\nu\left(R^{\mu\nu}-\frac{1}{2}g^{\mu\nu}R\right)=\nabla_\nu G^{\mu\nu}=0, $$ eg. the Einstein tensor $G^{\mu\nu}$ is automatically divergenceless. So we should have $$ G^{\mu\nu}\sim T^{\mu\nu}, $$ which is the Einstein field equation.