This is a reference resources question, masquerading as an answer, given the constraints of the site. The question hardly belongs here, and has been duplicated in the overflow cousin site . It might well be deleted.
There have been schools and proceedings on the subject,
Integrability: From Statistical Systems to Gauge TheoryLecture Notes of the Les Houches Summer School: Volume 106, June 2016, Volume 106,
Patrick Dorey, Gregory Korchemsky, Nikita Nekrasov, Volker Schomerus, Didina Serban, and Leticia Cugliandolo. Print publication date: 2019, ISBN-13: 9780198828150, Published to Oxford Scholarship Online: September 2019.
DOI: 10.1093/oso/9780198828150.001.0001
including, specifically,
Integrability in 2D fields theory/sigma-models, Sergei L Lukyanov &
Alexander B Zamolodchikov.
DOI:10.1093/oso/9780198828150.003.0006
Integrability in sigma-models, K. Zarembo.
DOI:10.1093/oso/9780198828150.003.0005
https://arxiv.org/abs/1712.07725
I am particular to
Integrable 2d sigma models: Quantum corrections to geometry from RG flow, Ben Hoare, Nat Levine, Arkady Tseytlin, Nucl Phys
B949 (2019) 114798 , but that's only by dint of personal connectivity...
The key point here is that, any dynamical system that is not completely integrable will exhibit chaotic regimes1. In other words not all orbits will lie on an invariant torus (Liouville's torus is the topological structure of a fully integrable system), in principle a chaotic system can even have closed stable periodic orbits (typical for regular/integrable systems) for some initial conditions, the set of such conditions has measure zero (meaning the states on that orbit are only reachable from other states of the same orbit).
In order to get yourself acquainted with such concepts, I suggest looking into 2D dynamic billiards. These models are of great interest because their dynamics are solely defined by the shape of the boundary, circular, ellipsoid, stadium etc. Now an interesting example to showcase here would be the oval shaped boundary (note circular and ellipsoid billiards are regular because of their symmetries):
In the above image (by Tureci, Hakan, et al. 2002), on left you see the poincaré map2 of the 2D oval billiard (with specular reflection), and on the right you see 3 examples of different regimes of the system. This is a perfect example showcasing a system that admits only locally integrable regions. Case a) corresponds to a quasi-periodic orbit, only marginally stable. Case b) shows a stable periodic orbit surrounded by a stable island and finally case c) corresponding to the entirety of densely dotted regions of the map, is indicative of chaotic motion. For further reading, I suggest looking into some of the articles on scholarpedia, and of course not to miss this fantastic review by A. Douglas Stone.
1For example all non-linear systems that are not Liouville integrable (as explained in comments). Note that linear systems can always be solved by exponentiation. But that said one must be wary of distinctions between solvability and integrability.
2These maps are obtained by choosing a poincaré section, and finding the intersection of trajectories in phase space with this section. Such maps allow for a representation of the evolution of any dynamical system, regardless of the dynamics involved. For more intuition, see here.
Best Answer
(1) In general, what is meant by non-linear system in classical mechanics?
A linear system is described by a set of differential equations that are a linear combination of the dependent variable and its derivatives. Some examples of linear systems in classical mechanics:
Non-linear systems cannot be described by a linear set of differential equations. Some examples of non-linear systems in classical mechanics:
(2) Furthermore, why is it that most non-linear systems are considered non-integrable?
That term, "non-integrable" has two very distinct meanings. One sense is that the integral cannot be expressed as a finite combination of elementary functions. The elementary functions are polynomials, rational roots, the exponential function, the logarithmic function, and the trigonometric functions. This is a rather arbitrary division. For example, the integrals $\operatorname{li}(x) = \int_0^x 1/\ln(t)\,dt$ and $\operatorname{Si}(x) = \int_0^x \sin(t)/t\,dt$ cannot be expressed in the elementary functions. These are the logarithmic and sine integrals. These "special functions" appear so often that algorithms have been devised to estimate their values. Categorizing functions as elementary versus non-elementary is a bit arbitrary.
Just because the solution to a problem can't be expressed in elementary functions doesn't mean the problem is unsolvable. It just mean it's not solvable in the elementary functions. For example, people oftentimes say the three body problem is not "solvable". That's nonsense (ignoring collision cases). In the sense of solvability in the elementary functions, even the two body problem is not "integrable". Kepler's equation, $M = E - e\sin E$, gets in the way. Just because the two body problem cannot be expressed in terms of a finite combination of elementary functions does not mean we can't solve the two body problem.
There's another sense of "integrability", which is "does the integral exist?" Going back to the n body problem, a problem exists with collisions. These collisions introduce singularities, so that one could say that the n body problem is not integrable in the case of collisions. Collisions represent one kind of singularity. Painlevé conjectured that the n body problem has collisionless singularities in when $n\ge 4$. This has been proven to be true when $n \ge 5$. Newtonian mechanics allows some configurations of gravitating point masses to be sent to infinity in finite time. This truly is an example of non-integrability.
Proving integrability (or lack thereof) in this sense is a much tougher problem than showing that a problem is (or is not) solvable in the elementary functions. There's a million dollar prize for the first person who can either prove that the Navier-Stokes equations have globally-defined, smooth solutions, or come up with a counterexample that shows that that the Navier-Stokes equations are not "integrable."