Say if $A$ is an $n \times n$ matrix, why is it that if $A^{T}A$ is positive definite, the matrix $A$ is then invertible? All I know is $A^{T}A$ gives a symmetric matrix but what does $A^{T}A$ is positive definite tell or imply or hint about the matrix $A$ itself that leads to the fact that it will be invertible?
Linear Algebra – Positive Definite $A^{T}A$ Implies A is Invertible
linear algebramatrices
Related Solutions
I think it's fairly straight-forward actually, naturally depending on how much you want to show. Do you take the following as given?
If $\mathbf{A}$ is non-invertible, then there exists a non-zero vector $\mathbf{x}$ such that $\mathbf{Ax}=\boldsymbol{0}$.
If you do, then it shouldn't be too problematic.
If $\mathbf{A}$ is not invertible, then that means there is a vector such that $\mathbf{Ax}=\boldsymbol{0}$. Thus, $\mathbf{x}^\prime \mathbf{Ax}=\mathbf{x}^\prime \boldsymbol{0}=0$. But, if $\mathbf{A}$ is positive definite, then $\mathbf{x}^\prime \mathbf{Ax}>0$. So $\mathbf{A}$ cannot be both non-invertible and positive definite. Hence, it must be invertible if it is positive definite.
Edit: This only addresses the second part.
The positive definiteness (as you already pointed out) is a property of quadratic forms. However, there is a "natural" one-to-one correspondence between symmetric matrices and quadratic forms, so I really cannot see any reason why not to "decorate" symmetric matrices with positive definiteness (and other similar adjectives) just because it is in "reality" the form they define which actually has this property. I can see this one-to-one correspondence as one of the reasons why the symmetry should be implicitly assumed when talking about positive definite matrices.
One can of course devise a different name for this property, but why? In addition, positive definite matrix is a pretty standard term so if you continue reading on matrices I'm sure you will find it more and more often.
Some authors (not only on Math.SE) allow positive definite matrices to be nonsymmetric by saying that $M$ is such that $x^TMx>0$ for all nonzero $x$. In my opinion this adds more confusion than good (not only on Math.SE). Also note that (with a properly "fixed" inner product) such a definition would not even make sense in the complex case if the matrix was allowed to be non-Hermitian ($x^*Mx$ is real for all $x$ if and only if...).
Anyway, for real matrices, it of course makes sense to study nonsymmetric matrices giving a positive definite quadratic form through $x^TMx$ (which effectively means that the symmetric part is positive definite). However, I find denoting them as positive definite quite unlucky.
Best Answer
Actually $\ker(A^T A) = ker(A)$ so one is invertible if and only if the other is invertible. Proof:
If $v \in \ker(A)$ then $Av=0$ so $A^T A v = 0$, hence $v \in \ker(A^T A)$. On the other hand, if $v \in \ker (A^T A)$ then $\left\langle w,A^{T}Av\right\rangle =0$ for all $w \in \mathbb{R}^n$, so $\left\langle Aw,Av\right\rangle =0$ for all $w$, and in particular for $w=v$ you get $Av=0$.
A positive definite matrix is invertible (for instance, because it has positive eigenvalues) so you're done.