[Math] Singular covariance matrix

determinantlinear algebrastochastic-processes

I am looking into the process $\{X_t, t\in\mathbb{Z}\}$, $X_t=A\cos(\lambda t)+B\sin(\lambda t)$, here $\lambda\in(0,\pi)$ is fixed, $A$ and $B$ are uncorrelated random variables with $EA=EB=0$, $EA^2=EB^2=\sigma^2$.

I have found the covariance function $r(k)=\sigma^2\cos(\lambda k)$ and now I want to show that process' covariance matrix

$$\sigma^2
\begin{pmatrix}
1 & \cos(\lambda) & \cos(2\lambda) & \cdots & \cos(n\lambda) \\
\cos(\lambda) & 1 & \cos(\lambda) & \cdots & \cos((n-1)\lambda) \\
\cdots & \cdots & \cdots & \cdots & \cdots \\
\cos(n\lambda) & \cos((n-1)\lambda) & \cos((n-2)\lambda) & \cdots & 1
\end{pmatrix}
$$

is singular when $n\geq 2$. Also the relationship $X_{n+1}=2X_n\cos\lambda-X_{n-1}$, $n\geq 2$ holds, from which I get that $r(k)=\frac{r(k-1)+r(k+1)}{2\cos\lambda}$. But I can not find a fast way to show that the covariance matrix is singular neither using latter relationship nor the matrix above.

Best Answer

Let $Z = (X_0 , X_1 , \dots , X_n )^T$,

and let $$U = \left( \begin{matrix}1 & 0 \\ \cos \lambda & \sin \lambda \\ \dots & \dots \\ \cos \lambda n &\sin \lambda n \end{matrix} \right).$$

Then $$ Z = U \left( \begin{matrix} A \\ B \end{matrix} \right)$$ where $$\text{Cov} \left( \begin{matrix} A \\ B \end{matrix} \right ) = \left( \begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix} \right).$$ $EZ = 0$ and $$ \text{Cov} Z = EZZ^T = U \text{Cov} \left( \begin{matrix} A \\ B \end{matrix} \right) U^T = U\left( \begin{matrix} 1 & 0 \\ 0 & 1\\ \end{matrix} \right) U^T = UU^T.$$

Clearly, the covariance matrix of $Z$ is singular for $ n \geq 2 $ as rank of $U$ is atmost 2 as it is a $ n \times 2 $ matrix and rank of $ \text{Cov}(Z) $ is less or equal to the rank of $U$.

Related Solutions

Linear Algebra – Minimum Eigenvalue and Singular Value of a Square Matrix

On one hand, if $v$ is a unit eigenvector of $A$ corresponding to the eigenvalue $\lambda_\min$, then $v^\ast A^\ast Av=\|Av\|^2=|\lambda_\min|^2$. On the other hand, $A^\ast A$ is positive semidefinite. Hence it can be unitarily diagonalised as $U^\ast DU$, where $D$ is a nonnegative diagonal matrix. The minimum diagonal entry of $D$, i.e. the minimum eigenvalue of $A^\ast A$, is equal to $\sigma_\min^2$. Therefore $$ |\lambda_\min|^2 =v^\ast A^\ast Av \ge\min_{\|x\|=1}x^\ast A^\ast Ax =\min_{\|x\|=1}(Ux)^\ast D (Ux) =\min_{\|y\|=1}y^\ast D y =\sigma_\min^2. $$

[Math] Find a singular value decomposition of the matrix

If $A$ is a matrix and $A \in \mathbb{R}^{ m \times n}$ then it has an SVD. i.e

$$A_{m \times n} = U_{m \times m} \Sigma_{m \times n} V_{n \times n}^{T} \tag{1}$$

now if the rank of $A$ is $r$ then the reduced or truncated SVD is given by an SVD like the following

$$ A_{m \times n} = U_{m \times r} \Sigma_{r \times r} V_{r \times n}^{T} \tag{2} $$

that means your reduced SVD is

$$ A_{m \times n} = \begin{bmatrix} \frac{2}{3} \\ \frac{1}{3} \\ \frac{-2}{3} \end{bmatrix} \begin{bmatrix} \sqrt{18} \end{bmatrix} \begin{bmatrix} \frac{-1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\end{bmatrix} \tag{3}$$

checking this in Python

import numpy as np

A1 = np.matrix([[-2, 2], [-1, 1], [2,-2]])



a1 = np.matrix([[2/3] , [1/3] ,[-2/3] ])
a2 = np.matrix([np.sqrt(18)])
a3 = np.matrix([-1/np.sqrt(2),1/np.sqrt(2)])

A2 = a1*a2*a3

error = np.linalg.norm(A1-A2)
error
Out[13]: 9.42055475210265e-16

Best Answer

Related Solutions

Linear Algebra – Minimum Eigenvalue and Singular Value of a Square Matrix

[Math] Find a singular value decomposition of the matrix

Related Question