It is well known and often argued that the contracted Bianchi identity (and vanishing divergence of the stress-energy tensor) of General Relativity can be seen as a consequence of the theory's (and any other reasonable physical theory's) invariance under what physicists will call diffeomorphisms.
As very well discussed here on physics.stackexchange, this is an abuse of terminology and:
Indeed, "diffeomorphism invariance" of GR in physics in this context means in proper mathematical parlance that isometric (pseudo-)Riemannian manifolds are physically equivalent.
going on…
Under such a change, the metric and all fields transform naturally via pushforward, and we essentially define the diffeomorphism to be an isometry so that the fields on the target with the new coordinates are equivalent to the fields on the source.
Thus we get a definition like this for physicts'"diffeomorphisms:" two pseudo-Riemannian manifolds $R'′=(M',g')$ and $R=(M,g)$, where $M'$,$M$ are smooth manifolds and $g'$,$g$ are metric tensors, are physically equivalent iff there exists a diffeomorphism $ϕ:M'→M$ between them such that $g'=ϕ^∗g$ and matter fields, if any, transform as $ψ'=ϕ^*ψ$. In other words, if there is an isometry between them.
Geodesics will be mapped to geodesics and the Einstein-Hilbert action will be left the same, giving us the same theory.
One more note: The "diffeomorphism" often implemented by physicists is actually an auto-diffeomorphism (and hence autoisometry) which can be realized as a passive or active transformation on the spacetime. Let's take the passive perspective in which the diffeomorphism is simply a change of coordinates – an autodiffeomorphism from the manifold to itself, where the source carries one choice of coordinate charts and the target another.
My Questions: I see often physics texts claiming the change in the components of the metric (or its inverse) under such a transformation (which they implement as $x'^μ(x)=x^μ+αk^μ(x)$ using a small parameter $\alpha$ and components of a vector field $k$ on $M$) goes like
$$g'_{μν}(x)=g_{μν}(x)−α(g_{μρ}∂_{ν}k^{ρ}+g_{νρ}∂_{μ}k^{ρ}+k^{ρ}∂_{ρ}g_{μν})+O(α^2)$$
and so when varying the metric under auto-isometry we can make statements like $\delta g^{\mu \nu} = \mathcal{L}_{k} g^{\mu \nu}$ following from which standard arguments of differential geometry (roughly done here) applied to the Einstein-Hilbert action (perhaps with Gibbons-Hawking boundary term) yields contracted Bianchi.
How can we make this change of coordinates put on more solid differential geometric footing, without the use of infinitesimals. In general, in this perspective of auto-isometry, how does one correctly formulate the variational principle of the Einstein-Hilbert action (as in what, mathematically, really does $\delta g^{\mu \nu}$ mean?
Best Answer
Do you know the "flow" definition of the Lie derivative? For any vector field $X$, the flow of $X$ defines a 1-parameter family of (mathematical) diffeomorphisms, (which you can interpret as a family of coordinate-change maps). Then the Lie derivative $\mathcal L_X T$ of any tensor $T$ is defined as the "derivative along this flow" (formally using pullbacks/pushforwards of the diffeomorphisms).
Formally the change of coordinates is $p\mapsto \varphi_t(p)$, where $\varphi_t$ is the flow of $X$. But for small values of $t$, you can probably say that $\varphi_t(p) \approx p + tX_p$.
Then by definition the change in $g$ as you change coordinates will be $\mathcal L_X g$.
You can go into coordinates and prove that $L_X T$ agrees with the coordinate definition of lie derivatives.
This might be a case of trying to connect math language to physics language though, so I hope I actually answered what you were really asking.