Eigenvalues of a Covariance Matrix with Noise

correlationcovarianceeigenvalues-eigenvectorssvdsymmetric matrices

Imagine to have a covariance matrix $2\times 2$ called $\Sigma^*$.

\begin{bmatrix}1+\sigma^2&\rho_{12}\\\rho_{21}&1+\sigma^2\end{bmatrix}

I know that $\rho_{12} = \rho_{21}$ because it's symmetric.

I proceed in calculating the eigenvalues from

$$\det(\Sigma^*-\lambda I) = \lambda^2 – 2\lambda(1+\sigma^2) + (1+\sigma^4+2\sigma^2-\rho_{12}^2).$$

and finally the eigenvalues are:

$\lambda_1 = 1+\sigma^2 + \rho_{12}$

$\lambda_2 = 1+\sigma^2 – \rho_{12}.$

Now I've tried to do the same with a $3 \times 3$ matrix

$$\Sigma^* = \begin{bmatrix}1+\sigma^2&\rho_{12}&\rho_{13}\\\rho_{21}&1+\sigma^2&\rho_{23}\\\rho_{31}&\rho_{32}&1+\sigma^2\end{bmatrix}.$$

I know that $\rho_{12} = \rho_{21}, \rho_{13} = \rho_{31}, \rho_{23} = \rho_{32}$ because it's symmetric.

Now I'm kinda stuck. I'll leave below what I did:

$$\det(Σ^*-\lambda I) = (1+\sigma^2 -\lambda)^3 – (1+\sigma^2 -\lambda)(\rho_{12}^2 + \rho_{13}^2 + \rho_{23}^2) + 2 \rho_{12}\rho_{13}\rho_{23}$$

and if $z = 1+\sigma^2 -\lambda$

$$\det(Σ^*-\lambda I) = z^3 – z(\rho_{12}^2 + \rho_{13}^2 + \rho_{23}^2) + 2 \rho_{12}\rho_{13}\rho_{23}.$$

How can I go further? Is it better to either find the values of z or to use another approach like SVD if applicable?

My main goal is to find those eigenvalues.

****** UPDATE ******

Following the answer below from @GNUSupporter 8964民主女神 地下教會, I see that since the discriminant is less or equal to zero, I have that

$u^3, v^3 = -\dfrac{q}{2} ± i\sqrt{-\Delta}$

their module is $R = \sqrt{(-\frac{q}{2})^2+(\sqrt{-\Delta})^2}$

Do I need $R$? If yes, I need to calculate $\theta = arctg (-\frac{\sqrt{-\Delta}}{-\frac{q}{2}})$

If not, is it possible to just say that

$$x = u + v = \sqrt[3]{-\dfrac{q}{2} + i\sqrt{-\Delta}} + \sqrt[3]{-\dfrac{q}{2} – i\sqrt{-\Delta}}?$$

Best Answer

My main goal is to find those eigenvalues.

To find $\Sigma^*$'s eigenvalues, apply Cardano's method with $p = -(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2)$ and $q = 2 \rho_{12}\rho_{23}\rho_{31}$ to your last equality

$$\det(Σ^*-\lambda I) = z^3 - z(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2) + 2 \rho_{12}\rho_{23}\rho_{31}. \tag1 \label1$$

We introduce variables $u$ and $v$ so that $z = u + v$

\begin{align} u^{3} &= -{q \over 2}+{\sqrt {{q^{2} \over 4}+{p^{3} \over 27}}} = -\rho_{12}\rho_{23}\rho_{31} + \sqrt{(\rho_{12}\rho_{23}\rho_{31})^2 - \frac{(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2)^3}{27}} \tag2 \label2 \\ v^{3} &= -{q \over 2}-{\sqrt {{q^{2} \over 4}+{p^{3} \over 27}}} = -\rho_{12}\rho_{23}\rho_{31} - \sqrt{(\rho_{12}\rho_{23}\rho_{31})^2 - \frac{(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2)^3}{27}}. \tag3 \label3 \end{align}

Note that by the ,

$$\sqrt[3]{\rho_{12}^2 \rho_{23}^2 \rho_{31}^2} \le \frac{\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2}{3},$$

so the discriminant $$\Delta = {q^{2} \over 4}+{p^{3} \over 27} \le 0.$$

$\Delta = 0$ iff equality holds in the AM-GM inequality iff $\rho_{12}^2 = \rho_{23}^2 = \rho_{31}^2$.

Observe from \eqref{2} and \eqref{3} that $u^3 = \overline{v^3}$. Denote $\omega$ the third root of unity not equal to $1$. (i.e. $\omega^3 = 1$ but $\omega \ne 1$.)

Since all eigenvalues of a real symmetric matrix are real, you just take $u + \overline{u}$, $\omega u + \overline{\omega u}$ and $\omega^2 u + \overline{\omega^2 u}$ as roots for \eqref{1}, where $u$ is fixed as any one of the three roots of \eqref{2}.

It doesn't matter which root of \eqref{2} is chosen since $\omega$ permutes the three roots, so eventually, all three roots of \eqref{2} are covered.

To finish this question, just do the substitution $\lambda = 1+\sigma^2 -z$.


Edit in response to OP's comment

Despite OP's edit to include some trigonometric function, in this section, I shall make no use of them since this is essentially an algebra problem for finding of , and that trigonometric functions are transcendental. I'll keep my existing working algebraic at the first place. By algebra, I don't mean to be arithmetic, that is, I avoid as much calculations as possible.

Each of equations \eqref{2} and \eqref{3} have three distinct roots, so there are at most $3 \times 3 = 9$ candidates for $z = u + v$.

I say "at most" since I haven't (and won't directly) checked some of them repeat themselves.

The reason for \eqref{2} and \eqref{3} to have three distinct roots is that the cubic equation $\omega^3 = 1$ has three distinct roots: $1$ and the two conjugate complex roots $\omega$ and $\bar{\omega} = \omega ^2$, so that $\omega^2 + \omega + 1 = 0$. If you fix a root $u$, then $\omega u$ and $\omega^2 u$ are also roots to \eqref{2} since $(\omega u)^3 = \omega^3 u^3 = u^3$ and $(\omega^2 u)^3 = (\omega^3)^2 u^3 = u^3$. Since we want to represent all three roots of \eqref{2} as $u, \omega u, \omega^2 u$, we don't want $\omega = 1$.

To visualize $u$, you may consider it as a nonzero arrow on the complex plane, and $\omega$ as a rotation of 120°.


Third roots of unity
Image source: Wikimedia commons

Given that \eqref{2} is solved and that $u^3 = \overline{v^3}$, the easiest root for \eqref{3} would be $v = \bar{u}$. Apply the same argument above to see that roots of \eqref{3} are $\bar{u}, \omega \bar{u}, \omega^2 \bar{u}$.

The remaining work is too select the three real values from $\omega^i u + \omega^j \bar{u}$, $i,j \in \{0,1,2\}$. The easiest choice is the case when $i = j = 0$, so that $z = u + \bar{u} \in \Bbb{R}$. It should be quite obvious that we should choose $\omega u + \overline{\omega u}$ and $\omega^2 u + \overline{\omega^2 u}$. These three roots are distinct since rotation $\omega \ne 1$. Since \eqref{1} is at most three roots, we're done.

To conclude, to solve for $z$, fix any root $u$ in \eqref{2}, then the first root is $2\mathrm{Re}(u)$. Rotate $u$ by $120°$ (anti-)clockwisely to get another root $2\mathrm{Re}(\omega u)$, repeat this to get the last root $2\mathrm{Re}(\omega^2 u)$.

Remark: I used "third root of unity" in the first version of this answer since it's easier to type in words than its explicit expression.


Edit in response to updated question body

Do I need $R$?

As shown by the previous section, we can solve the problem without introducing the modulus $$R = \sqrt{(-\frac{q}{2})^2+(\sqrt{-\Delta})^2}$$ and $$\theta = \arctan\frac{-\sqrt{-\Delta}}{-q/2}$$ of $u^3$ (or $v^3$).

Nonetheless, $R$ and $\theta$ provide a more compact expression for $u^3$ and $v^3$: $v^3 = Re^{i\theta}$ and $u^3 = Re^{-i\theta}$. Therefore, $u = \sqrt[3]{R} e^{-i(\theta + 2k\pi)/3}$ and $v = \sqrt[3]{R} e^{i(\theta + 2m\pi)/3}$, $k,m \in \{0,1,2\}$. The checking for $u + v \in \Bbb{R}$ is similar to the previous section, and I left this as exercise.

To litterally respond to your question, it depends on your personal choice. Nobody can answer that for you.

  • If you don't wish to overload some transcendental stuff like $\arctan$, $e$, Euler's identity, then the answer is "no".
  • Otherwise, it's "yes".

If not, is it possible to just say that $$x = u + v = \sqrt[3]{-\dfrac{q}{2} + i\sqrt{-\Delta}} + \sqrt[3]{-\dfrac{q}{2} - i\sqrt{-\Delta}}?$$

First, I suppose you mean $z = u + v$, since $x$ has never been used before. Second, this notation for cubic root $\sqrt[3]{\cdots}$ is ambiguous unless $\Delta = 0$. The notation $\sqrt[3]{\cdots}$ can mean any one of the tree cubic root unless the number inside the radical sign is real.

To see this ambiguity, consider $\sqrt[3]{i}$. One graphical way to interpret this is to trisect a right angle: $3 \times \frac\pi6 = \frac\pi2$. However, there are two more possible visualisations: $3 \times \frac{5\pi}{6} = \frac{5\pi}{2}$ and $3 \times \frac{3\pi}{2} = \frac{9\pi}{2}$. Therefore, the equation $z^3 = i$ has three distinct roots $e^{\pi/6}$, $e^{5\pi/6}$ and $e^{3\pi/2}$. One way to decide which root $\sqrt[3]{i}$ should take is the choose the least nonnegative argument. This is called the principal root. Wolfram Alpha's calculation for $\sqrt[3]{i}$ adopts this definition, but that's not universally accepted according to Wikipedia's page on $n$-th root (search for principal root on the page to see this). This can be explained by the observation that conjugation is not preserved by the action of taking principal root: $i$ and $-i$ are conjugate to each other, but their principal roots ($e^{\pi/6}$ and $e^{\pi/2} = i$) aren't.

This ambiguity would give rise to at most $3 \times 3 = 9$ possible candidates of $z$, but a cubic equation admits exactly three roots in $\Bbb{C}$. Therefore, an additional condition has to be added.

$uv = -\dfrac{p}{3}$

This condition comes from Cardano's trick $z = u + v$, which transforms the depressed cubic equation $z^3 + pz + q = 0$ into $$\begin{aligned} (u+v)^3 + p(u+v) + q &= u^3 + v^3 + 3uv(u+v) + p(u+v) + q \\ &= u^3 + v^3 + (3uv + p)(u+v) + q = 0 \end{aligned}$$

Set $3uv + p = 0$. Since a new variable is introduced, we can add one more constraint. This is where the $uv = -\frac{p}{3}$ from. Then the above equation becomes $u^3 + v^3 = -q$. To solve for $u$ and $v$, we make use of the well-known quadratic formula on $u^3$ and $v^3$ to get \eqref{2} and \eqref{3}.

Related Question