Well, one trivial connection is that if you look at $1\times 1$ matrices (which have only a single complex entry), then you'll find that it is real iff it is Hermitian, its complex conjugate is its conjugate transpose, and its polar decomposition is the polar form.
Also, just like a complex number can be uniquely decomposed into a real and an imaginary part ($z = a+\mathrm ib$ with real $a,b$), a complex matrix can be uniquely decomposed into a Hermitian and an "anti-Hermitian" part, i.e,. $M =A + \mathrm iB$ with $A$ and $B$ Hermitian. And just like $\Re(z)=\frac12(z+\bar z)$ and $\Im(z)=\frac1{2\mathrm i}(z-\bar z)$, the Hermitian part of a matrix is $\frac12(M+M^*)$ and the "anti-Hermitian" part is $\frac1{2\mathrm i}(M-M^*)$.
Moreover, just like $\bar zz$ is a non-negative real number, $M^*M$ is a positive semidefinite matrix.
Another point: Hermitian matrices have real eigenvalues, and unitary matrices have eigenvalues of the form $\mathrm e^{\mathrm i\phi}$.
About the usefulness of the analogy:
In classical physics, observables should be real. In quantum physics, observables are represented by Hermitian matrices. Also, the quantum analogue to probability densities, which are non-negative functions with integral $1$, are density operators, which are positive semidefinite matrices with trace $1$. So there's indeed some connection.
I give here the real version of your equality. Notice that the matrix $A^tA$ is symmetric positive semi-definite then it's diagonalizable and its eigenvalues are non negative:
$$A^tA=QDQ^{-1}$$
where $D=\operatorname{diag}(\lambda_1,\ldots,\lambda_n)$ and we can define the square root of $A^tA$ denoted by $\sqrt{A^t A}$ by
$$\sqrt {A^tA}=Q\sqrt D Q^{-1}$$
where
$$D=\operatorname{diag}(\sqrt\lambda_1,\ldots,\sqrt\lambda_n)$$
so let $P=\sqrt {A^tA}$ and $O=AP^{-1}$ then it's clear that $P$ is symmetric positive semi-definite and we have
$$O^tO=(P^{-1})^tA^tAP^{-1}=(P^{-1})^tP^2P^{-1}=I$$
so $O$ is orthogonal.
Best Answer
The "dot product" in $\Bbb R^n$ has the property that $$ d(x) = \sqrt{x \cdot x} = \sqrt{x^t x} $$ is a metric. In particular, $x \cdot x = \sum_i x_i^2$ is always a nonnegative real, so we can take a square root. You can fancy this up and look at $$ \sqrt{x^t M x} $$ for some matrix $M$, or (generalizing a little) look at a product defined by $$ (x, y) \mapsto x^t M y. $$
If you want that to be symmetric (which is a nice thing for generalized inner products), then $M$ has to be symmetric.
If you try to do the same thing for a complex vector $z$, with complex number entries $z_i$, and define $$ z \cdot z = \sum_j z_j^2, $$ then the resulting sum is usually not real. (Example: if each $z_j$ is $\sqrt{-1}$, then...)
But if you say that $$ z \cdot z = \sum_j z_j \overline{z}_j, $$ then you DO get something that's a nonnegative real, and can mimic all the stuff you did for $\Bbb R^n$. But when it comes to the matrix $M$, you need (to get symmetric of your generalized inner product) not that $M^t = M$, but that $\overline{M}^t = M$...so that's where the generalization comes from.