Let $X$ be a matrix of dimension $n\times k$ where $n>k$, $\text{rk}(X)=k$ so $X$ is of full column rank. Then how do I prove $X^tX$ is always positive definite, where $X^t$ is transpose of $X$? This is given sortta like a lemma in our lecture slides without proof but would like to have some reasoning behind this. Thank you for your help!
[Math] Prove $X^tX$, where $X$ is a matrix of full column rank, is positive definite
linear algebramatricespositive definite
Related Question
- [Math] Is $X’X$ positive definite a necessary condition for $X’X$ to have full rank
- [Math] positive definite matrix plus positive semi matrix equals positive definite
- [Math] $A$ has full column rank if and only if the symmetric matrix $B=A^TA$ is positive definite.
- [Math] Showing Orthogonal Projection Matrix Multiplied by Full-Rank Matrices is Positive-Definite
- [Math] Is not full rank matrix invertible
Best Answer
If $X$ has full column rank, then its columns are linearly independent. The product $Xv$ with a vector $v$ is a linear combination of the columns of $X$. Therefore, if $v\neq 0$ ($v$ is not the zero vector) then $y := Xv \neq0$ ($Xv$ is a non-zero vector, since it is a non-trivial linear combination). Therefore, given any $v\neq 0$ we get $$ v^tX^t X v = (Xv)^t Xv = y^t y = \sum_i y_i^2 > 0 $$ which is the implication of positive definite.