Recently, I got obsessed with working out this story, to the detriment of my mathematical work. Here is a quick crib sheet for relating properties of continued fractions to properties of real quadratic fields. Warning: I haven't checked this against a standard reference, so there may be errors.
Preliminary notation: Continued fractions
It is convenient to convert a continued fraction to a sequence of binary symbols; I've been using red and blue dots, which I'll represent here as $r$ and $b$. For example, $\sqrt{13}=[3,1,1,1,1,6,1,1,1,1,6,\cdots]$; I'll write this as
$$r\ r\ r\ b\ r\ b\ r\ b\ b\ b\ b\ b\ b\ r\ b\ r\ b\ r\ r\ r\ r\ r\ r\ \cdots$$
This makes the continued fraction purely periodic instead of having that peculiar $3$ at the beginning; the period starts again in the middle of that last block of six $r$'s.
Note also that the period of the fraction now appears to be twice what it was; when we get to the old period, we have switched colors. Finally, a sequence which starts with $b$'s should be thought of as a continued fraction for a number in $(0,1)$; the sequence starts with zero copies of $r$.
Preliminary notation: Real quadratic fields
Let $K$ be a quadratic field and let $\Lambda$ be a rank $2$ sublattice of $K$, with $\mathbb{Q} \Lambda =K$. Let $\mathrm{End}(\Lambda)$ be the ring of $\theta \in K$ such that $\theta \Lambda \subseteq \Lambda$. This is an order in $\mathcal{O}_K$.
Let $K$ be a real quadratic field, with fixed embedding $K \to \mathbb{R}$, and write $z \mapsto \overline{z}$ for the Galois symetry of $K$. We'll say that two lattices $\Lambda_1$ and $\Lambda_2$ with CM are "strictly equivalent" if there is an element $\alpha \in K$ such that $\Lambda_1 = \alpha \Lambda_2$ with $\alpha$ and $\overline{\alpha}$ both positive.
Summary of results:
$\bullet$ Take any periodic sequence $a_i$ of $r$'s and $b$'s and turn it into a continued fraction. Let $z$ be the value of that continued fraction. Then $z$ is a quadratic irrational, with $z>0$ and $\overline{z}<0$. We have $a_0=r$ if $z>1$ and $a_0=b$ if $z<1$. Extending the periodicity to negative indices, $a_{-1}=r$ if $\overline{z} < -1$ and $a_{-1} =b$ if $\overline{z} > -1$.
$\bullet$ Switching the colors changes $z$ to $1/z$. Reversing the sequence changes $z$ to $-\overline{z}$.
Strict ideal classes
$\bullet$ Let $K = \mathbb{Q}(z)$ and let $\Lambda = \langle 1, z \rangle$. Shifting the periodic sequence does not change the strict equivalence class of $\Lambda$.
$\bullet$ The above is a bijection between periodic sequences of $r$'s and $b$'s, up to shift, and strict equivalence classes of lattices in real quadratic fields.
$\bullet$ Switching the colors corresponds to multiplying by our lattice by an element of
negative norm. So producing a lattice which is equivalent, but not strictly equivalent.
$\bullet$ Reversing the sequence sends $\Lambda$ to $\overline{\Lambda}$. If $\Lambda$ is a lattice in $K$, and $R = \mathrm{End}(\Lambda)$, then $\Lambda \overline{\Lambda}$ is a strictly principal fractional ideal for $R$. (This is a special property of quadratic fields, which I know of no generalization of in higher degree number fields.) So, with the understanding that we treat a fractional ideal as a fractional ideal for its full endomorphism ring, reversing the sequence sends $\Lambda$ to $\Lambda^{-1}$.
Units
$\bullet$ Let $R$ be the endomorphism ring of $\Lambda$. Let $p/q$ be the convergent obtained by truncating the continued fraction just before the first repetition of the block which contains $a_0$. Let $u=p-qz$. Then $u$ is a unit of $R$ with norm $1$, and is the fundamental generator of the group of such units.
For example, $\langle 1, \sqrt{13} \rangle$ has endomorphism ring $\mathbb{Z}[\sqrt{13}]$. We truncate the above sequence to
$$r\ r\ r\ b\ r\ b\ r\ b\ b\ b\ b\ b\ b\ r\ b\ r\ b$$
or
$$[3,1,1,1,1,6,1,1,1,1] = \frac{649}{180}$$
and $649-180 \sqrt{13}$ is the fundamental positive unit of $\mathbb{Z}[\sqrt{13}]$.
$\bullet$ The color reversal of our sequence is a shift of itself if and only if $R$ has units of norm $-1$. We can recover them by the same recipe, truncating before the color reversed copy of $a_0$.
For example,
$$[3,1,1,1,1] = \frac{18}{5}$$
and $18-5 \sqrt{13}$ is the fundamental unit of norm $-1$ in $\mathbb{Z}[\sqrt{13}]$.
Note, by the way, that we have not yet seen the fundamental unit of $\mathbb{Q}(\sqrt{13})$, which is $(3-\sqrt{13})/2$. That's because $\langle 1, \sqrt{13} \rangle$ doesn't have CM by this unit.
We can obtain the same unit from two quite different looking continued fractions. For example
$$\sqrt{10} = r\ r\ r\ b\ b\ b\ b\ b\ b\ r\ r\ r \cdots \quad \sqrt{10}/2 = r\ b\ r\ b\ b\ r\ b\ r \cdots$$
where I have given a full period for each fraction. Truncating to before the middle block gives
$$r\ r\ r = \frac{3}{1} \quad r\ b\ r\ = \frac{3}{2}.$$
Both of these give the unit $3-\sqrt{10} = 3-2 \frac{\sqrt{10}}{2}$.
Fixing the endomorphism ring; working with triples $(a,b,c)$
$\bullet$ Let $R = \mathbb{Z}[\sqrt{D}]$, for $D>0$ and not square. The continued fractions which give rise to rings containing $R$ correspond to ordered triples $(a,b,c)$ of integers with $D=b^2+ac$ and $a$, $c>0$, by the recipe $(a,b,c) \mapsto (b+\sqrt{D})/a$.
The corresponding ring is exactly $\mathbb{Z}[\sqrt{D}]$ if and only if $(a,2b,c)$ have no common factor.
$\bullet$ Let $R = \mathbb{Z}[(1+\sqrt{D})/2]$ with $D \equiv 1 \mod 4$, positive and not square. The continued fractions which give rise to rings containing $R$ correspond to ordered triples $(a,b,c)$ with $D=b^2+ac$, $a$ and $c>0$, and the additional condition that $b$ is odd and $a$ and $c$ are even.
Example: If we want to get the ring $\mathbb{Z}[(1+\sqrt{13})/2]$, we need to pick $z$ so that $\langle 1, z \rangle$ has CM by this ring. An obvious choice is $z=(1+\sqrt{13})/2$, with $(a,b,c) = (2,1,6)$. The continued fraction is
$$r\ r\ b\ b\ b\ r\ r\ r\ b\ b\ b\ r\ r\ r\ b\ b\ b\ $$
or $[2, 3,3,3,3,\ldots]$ in conventional notation.
$\bullet$ The corresponding ring is exactly $\mathbb{Z}[(1+\sqrt{D})/2]$ if and only if $(a,b,c)$ have no common factor.
$\bullet$ There are only finitely many $(a,b,c)$ for any $R$.
$\bullet$ Adding an $r$ at the beginning of the sequence changes $(a,b,c)$ to $(a,a+b,c-a-2b)$. Adding a $b$ at the beginning changes $(a,b,c)$ to $(a-c-2b,b+c,c)$. Reversing the sequence changes $(a,b,c)$ to $(a,-b,c)$; color switching the sequence sends $(a,b,c)$ to $(c,-b,a)$.
Continued fractions with special symmetry
$\bullet$ A continued fraction is a shift of its color switch if and only if $R$ contains a unit with norm $-1$; we have already described how to find this unit.
$\bullet$ A continued fraction is a shift of its reversal if and only if it is a $2$-torsion class in the strict ideal class group.
Consider continued fractions which equal their reversal, so the periodic sequence starts at the middle of an even block of $r$'s or $b$'s, like the sequence for $\sqrt{13}$ above. These correspond to $z = \sqrt{D}/a$ for some divisor $a$ of $D$.
Consider continued fractions which are off from one by a shift of their reversal, so the periodic sequence starts in the middle of an odd block of $r$'s or $b$'s. These correspond to $z = (b+\sqrt{D})/(2b)$. If $D$ is odd, then we can take $b$ to be any divisor of $D$. If $D$ is $2 \mod 4$, then there are no solutions to $b^2+2bc=D$. If $D$ is $0 \mod 4$, then we can take $b$ of the form $2 b'$, where $b'$ is a divisor of $D/4$.
$\bullet$ Let $(-)$ denote the strict ideal class of principal $\Lambda$ ideals generated by elements of negative norm. A continued fraction is a shift of its color switched reversal if and only if $\Lambda^2 = (-)$ in the strict ideal class group.
A continued fraction actually equals its color switched reversal if and only if $a=c$. In other words, such continued fractions for $R = \mathbb{Z}[\sqrt{D}]$ are in bijection with solutions to $a^2+b^2=D$ with $a$ and $b>0$, and $GCD(a,2b)=1$. Such continued fractions for $R=\mathbb{Z}[(1+\sqrt{D})/2]$ are in bijection with solutions to $a^2+b^2=D$ with $a$, $b>0$, such that $a$ even and $b$ odd.
Example: We have $34=3^2+5^2$. So take $z=(3+\sqrt{34})/5$. Take $\Lambda$ to be the lattice $\langle 1, (3+\sqrt{34})/5 \rangle$, which is strictly equivalent to the ideal $I=\langle 5, 3+\sqrt{34} \rangle$ in $\mathbb{Z}[\sqrt{34}]$. This is a non-principal prime ideal dividing $5$. We have $I^2 = \langle 3+\sqrt{34} \rangle$, which is principal, but not strictly principal.
The corresponding continued fraction is
$$r\ b\ r\ r\ r\ b\ b\ b\ r\ b\ r\ b\ r\ r\ r\ b\ b\ b\ r\ b\ r\ b\ \cdots$$
or $[1,1,3,3,1,1,1,1,3,3,1,1,1,1,\cdots]$ in conventional notation. This sequence is its own color switched reversal, reflecting that $I^2=(-)$. However, it is not a shift of its own reversal, reflecting that $I^2$ is not strictly principal, and it is not a shift of its color switch, reflecting that $\mathbb{Z}[\sqrt{34}]$ does not have a unit of norm $-1$.
Best Answer
Well here is one answer: The study of the Gauss-Kuzmin problem eventually led to a fruitful connection between continued fractions and functional analysis. Namely, the distribution function of the Gauss measure is the leading eigenfunction for the transfer operator associated to the Gauss transformation. It turns out that this is the hidden explanation for the theorem you quoted in your question.
Understanding this connection recently led to a detailed study of transfer operators of the Gauss map and related transformations, and their associated Dirichlet series, which paved the way for major breakthroughs by Baladi, Vallee, and others, in understanding the statistics of the Euclidean algorithm and its various analogues. It would be difficult to give more details than this without writing a very long post, but if you are interested in finding out more then here are two references:
1) "Euclidean algorithms are Gaussian", Baladi and Vallee, available on Viviane Baladi's webpage.
2) "Continued fractions", Doug Hensley, Chapter 9.