I am trying to minimize the mean square error. More precisely, I am trying to minimize the following optimization problem
$$\arg \min _{\bf{w_1},\bf{w_2}}\mathbb{E} \,\,[\|{\bf s} – {\bf Wy}\|^2 ]$$
$${\bf W} = \begin{bmatrix}
{\bf w_1} &{\bf 0 } \\
{\bf 0 } & {\bf w_2 }
\end{bmatrix}$$
where ${\bf W}$ is a $2\times N$ matrix and where ${\bf w_i}$ is a $1\times N/2$ for $i\in[1:2]$ and ${\bf 0}$ is $1\times N/2$ vector while ${\bf s}$ is 2 $\times $ 1 and ${\bf y}$ is $N \times 1$ vectors.
Any hints or ideas on finding the minimizing vectors of this problem?
It is also given that
$${\bf y = A F s + z}$$ where ${\bf A}$ is $N\times N$ matrix while
$${\bf F} = \begin{bmatrix}
{\bf f_1} &{\bf 0} \\
{\bf 0} & {\bf f_2}
\end{bmatrix}$$
where ${\bf f_i}$ are $N/2\times 1$ vector and ${\bf Z}$ is $N\times 1$ vector
I am thinking of starting to start the solution as following
$$\arg\min _{\bf{w_1},\bf{w_2}}\mathbb{E} \,\,[[{\bf s} – {\bf Wy}]^H[{\bf s} – {\bf Wy}]]$$
But I assume given the specific block diagonal structure of matrix ${\bf W}$ and ${\bf F}$ it should be easier to solve …
Thanks
Best Answer
I assume you wish to find the non random matrix $W$ and hence its elements are not boldface in my solution. $\mathbf s$ and $\mathbf y$ are random vectors, hence they are boldface. First let me write the matrix ${W} = \begin{bmatrix} {w_{1}^*} &{ 0 } \\ { 0 } & { w_{2}^* } \end{bmatrix} $
where $*$ denotes the hermitian operation. Now all lower case vectors are column vectors.
This problem can be restated as
$\arg \min _{{w_1},{w_2}}\mathbb{E} \,\,\left[\left\Vert\begin{bmatrix} \bf s_1^* \\ \bf s_{2}^* \end{bmatrix} - \begin{bmatrix} {\bf y_{1}^*} &{\bf 0 } \\ {\bf 0 } & {\bf y_{2}^* } \end{bmatrix}\begin{bmatrix} w_1 \\ w_{2} \end{bmatrix}\right\Vert^2\right] $.
After this, the problem decouples to solving for $w_1$ and $w_2$. For eg. for $w_1$, we need to minimize $J =\mathbb E[ ||\mathbf s_1^* - \mathbf y_1^* w_1||^2]$.
Just expand the inside argument and differentiate w.r.t. $w_1^*$ and put the gradient to $0$. We get the solution for $w_1$ as
$\mathbb E [\mathbf y_1 \mathbf y_1^*] w_1 = \mathbb E [\mathbf y_1^* \mathbf s_1^*]$.
Now, assuming you can find the correlation matrix of $\mathbf y_1$ and it is invertible, and the cross correlation between $\mathbf y_1$ and $\mathbf s_1$, you can find $w_1$. Similarly, you can solve for $w_2$.