General Relativity – How Much Choice Did Einstein Have in Choosing His GR Equations?

Question

General relativity was summarised by Wheeler as "Spacetime tells matter how to move; Matter tells spacetime how to curve". I have a fairly good mental picture of how the first part works.

However, I don't have much of an understanding of the second part. In particular, what I'd like to know is whether the equations describing how matter tells spacetime "could have been different." In other words, did Einstein choose the form of his equations to fit empirical observations, or are they the sort of thing that are worked out entirely from first principles and can't be changed at all without breaking the whole theory?

Evidently there was some choice in choosing the equations, namely the cosmological constant – Einstein included it originally, then took it out, and now it seems like it might be there after all. But is adding a cosmological constant the only possible way in which Einstein's equations can be modified?

Another way to ask this question is to ask what assumptions are required to derive general relativity. I know that deriving special relativity really only requires the principle that the laws of physics (including Maxwell's equations) are the same in all intertial reference frames, and I know that the first part of Wheeler's quote comes from the principle of equivalence between gravity and acceleration. But what, if any, additional assumptions are required in order to determine the way in which matter curves spacetime?

Answer 1

I think Hawking, Ellis "The Large Scale Structure of Space-Time" 3.4 would be an interesting read for you.

From this book:

With Ricciscalar $R$, cosmological constant $\lambda$ and matter Lagrangian $L_m$ $$ I=\int_M (A (R - 2 \lambda ) + L_m) $$

One might ask whether varying an action derived from some other scalar combination of the metric and curvature tensors might not give a reasonable alternative set of equations. However the curvature scalar is the only such scalar linear in second derivatives of the metric tensor; so only in this case can one transform away a surface integral and be left with an equation involving only second derivatives of the metric.

If one tried any other scalar such as $R_{ab}R^{ab}$ or $R_{abcd}R^{abcd}$ one would obtain an equation involving fourth derivatives of the metric tensor. This would seem objectionable, as all other equations of physics are first or second order. If the field equations were fourth order, it would be necessary to specify not only the initial values of the metric and its first derivatives, but also the second and third derivatives, in order to determine the evolution of the metric.

Another nice read is Carroll "Spacetime and Geometry: An Introduction to General Relativity" 4.8. Alternative Theories. There is an shorter version online for free in Chapter 4 here