the Lagrangian density is $L=T-U$, the difference between the kinetic and potential energy density, as we're used to in all of mechanics.
The kinetic energy density is $T = \rho v(x,y,z)^2/2$ where one has to calculate the density $\rho$ properly. The potential energy is more general and complicated,
$$U = \frac{1}{2} C_{ijkl}u_{ij}u_{kl}$$
where the tensor $C$ with four indices is called the elastic. The tensors $u$ are obtained from the displacement vectors as
$$ u_{ij} = \frac{1}{2} (\partial_i u_j + \partial_j u_i) $$
Note that the density $\rho$ in the kinetic energy also depends on the displacement vectors, if you want to re-express it via the mass density at rest (without displacement).
For isotropic materials,
$$ C_{ijkl} = \lambda \delta_{ij} \delta_{kl} + \mu (\delta_{ik} \delta_{jl} + \delta_{il} \delta_{jk} ) $$
where $\lambda$ and $\mu$ are known as Lamé constants. For more general (but linear) crystals, however, $C$ is a general tensor symmetric under the $ij$ exchange, $kl$ exchange, and the exchange of $ij$ and $kl$ as pairs.
(1) In general, what is meant by non-linear system in classical mechanics?
A linear system is described by a set of differential equations that are a linear combination of the dependent variable and its derivatives. Some examples of linear systems in classical mechanics:
- A damped harmonic oscillator, $$m \frac{d^2 x(t)}{dt^2} + c \frac{d x(t)}{dt} + k x(t) = 0$$
- The heat equation, $$\frac{\partial u(\vec x, t)}{\partial t} -\alpha \nabla^2 u(\vec x, t) = 0$$
- The wave equation, $$\frac{\partial^2 u(\vec x, t)}{\partial t^2} -c \nabla^2 u(\vec x, t) = 0$$
Non-linear systems cannot be described by a linear set of differential equations. Some examples of non-linear systems in classical mechanics:
- Aerodynamic drag, where the drag force is proportional to the square of velocity, $$F_d = \frac 1 2 \rho v^2 C_D A$$
- The Navier-Stokes equations, which are notoriously non-linear,
$$\rho \left( \frac{\partial v}{\partial t} + \vec v \cdot \vec \nabla v \right) = -\vec \nabla p + \vec \nabla T + \vec f$$
- Gravitational systems, where the force is inversely proportional to the square of distance between objects,
$$\vec F = -\frac {GMm}{||\vec r||^3}\vec r$$
(2) Furthermore, why is it that most non-linear systems are considered non-integrable?
That term, "non-integrable" has two very distinct meanings. One sense is that the integral cannot be expressed as a finite combination of elementary functions. The elementary functions are polynomials, rational roots, the exponential function, the logarithmic function, and the trigonometric functions. This is a rather arbitrary division. For example, the integrals $\operatorname{li}(x) = \int_0^x 1/\ln(t)\,dt$ and $\operatorname{Si}(x) = \int_0^x \sin(t)/t\,dt$ cannot be expressed in the elementary functions. These are the logarithmic and sine integrals. These "special functions" appear so often that algorithms have been devised to estimate their values. Categorizing functions as elementary versus non-elementary is a bit arbitrary.
Just because the solution to a problem can't be expressed in elementary functions doesn't mean the problem is unsolvable. It just mean it's not solvable in the elementary functions. For example, people oftentimes say the three body problem is not "solvable". That's nonsense (ignoring collision cases). In the sense of solvability in the elementary functions, even the two body problem is not "integrable". Kepler's equation, $M = E - e\sin E$, gets in the way. Just because the two body problem cannot be expressed in terms of a finite combination of elementary functions does not mean we can't solve the two body problem.
There's another sense of "integrability", which is "does the integral exist?" Going back to the n body problem, a problem exists with collisions. These collisions introduce singularities, so that one could say that the n body problem is not integrable in the case of collisions. Collisions represent one kind of singularity. Painlevé conjectured that the n body problem has collisionless singularities in when $n\ge 4$. This has been proven to be true when $n \ge 5$. Newtonian mechanics allows some configurations of gravitating point masses to be sent to infinity in finite time. This truly is an example of non-integrability.
Proving integrability (or lack thereof) in this sense is a much tougher problem than showing that a problem is (or is not) solvable in the elementary functions. There's a million dollar prize for the first person who can either prove that the Navier-Stokes equations have globally-defined, smooth solutions, or come up with a counterexample that shows that that the Navier-Stokes equations are not "integrable."
Best Answer
The shell theorem applies to spherically-symmetric mass distributions. If you have a compressible fluid so that the density varies with depth, you can still apply the shell theorem (unless there is something else going on, for example net angular momentum that breaks the spherical symmetry). Specifically, the gravitational field is the same as that of a point mass at the center of the distribution.
Here is a demonstration:
Newton's law for the gravitational field of a point mass is
$$F_g/m = GM/r^2$$
with $F_g/m$ the acceleration of a test particle and M the mass of the gravitating body. This is linear in M, meaning that we can take two mass distributions and superimpose them, then find the gravitational field of this new mass distribution by superimposing the two associated individual gravitational fields.
Assume the shell theorem holds for constant density spheres. Take two spheres, one slightly larger than the other, so their radii are $r$ and $r+dr$. Give the larger sphere density $\rho$ and the smaller one density $-\rho$. Superimpose them. You get a thin shell of radius $r$ and thickness $dr$. The shell is empty in the middle because the positive and negative density distributions are canceling there. (It doesn't matter that there is no such thing as negative mass - the shell theorem is purely mathematical and so holds for negative numbers, and the final distribution we're dealing with has no negative mass; it was just a calculation trick.)
By superimposing the two gravitational fields of the positive and negative density distributions, we see that the gravitational field of this shell is the same as that of a point mass at the center of the shell with the mass $4\pi r^2 dr \rho$.
The shell theorem thus holds for these thin shells. A distribution with varying density at different radii is simply a lot of such shells, so the shell theorem holds for arbitrary $\rho(r)$ as long as $\rho$ is a function of $r$ alone.
As for the change in density as a function of radius, we need to balance the hydrostatic pressure with the gravitational attraction.
Take a little piece of the mass distribution with volume $dV$. It is pulled down by a gravitational force
$$F_g = \frac{GM(r) dm}{r^2}$$
$dm$ is $\rho dV$ and $M(r)$ is the mass located closer to the center of the distribution than the point in question, so
$$M(r) = \int_0^r 4\pi x^2 \rho(x) dx$$
This is balanced by the force from pressure
$$F_p = \nabla P dV$$
We assume there is a relationship between pressure and density so
$$\nabla P = \frac{\partial P}{\partial r} = \frac{d P}{d\rho}\frac{d \rho}{dr}$$
Equating the gravitational and pressure forces gives, after some algebra
$$4\pi G r^2\rho = \frac{d}{dr} \left( \frac{r^2}{\rho} \frac{dP}{d\rho} \frac{d\rho}{dr} \right)$$
The solution depends on $\rho(P)$, and is subject to the boundary condition $P(R) = 0$ with $R$ the radius of the distribution, as well as the condition $M(R) = M$, the total mass. We will probably find $R = \infty$.
Thus, once we specify a relationship between $\rho$ and $P$ based on the thermodynamics of the system involved, we can integrate the differential equation to find $\rho(r)$.