$\textbf{Question:}$
Prove that for every $A\in \mathbb{R}^{m\times n}$, $\exists$ a sequence of full rank matrices $A_i \in \mathbb{R}^{m\times n}$ such that $A_i \rightarrow A$ in $\|\cdot\|_2$.
$\textbf{Answer Given:}$
If $A$ is of full rank, then $A_i = A$ $\forall i$.
If A is not of full rank, then $A$ has the form
$$A= U
\begin{bmatrix}
\sigma_1 & 0 & \dots & \dots& 0\\
0 & \sigma_2 & 0 &\dots & 0 \\
\vdots & & \ddots & & \vdots \\
0 & \dots & 0 & \sigma_r & 0 \\
0 & \dots & \dots & \dots & \mathcal{O}
\end{bmatrix}
V^T
$$
where $\mathcal{O}$ is a $(m-r)\times (n-r)$ matrix of zeros.
Then for $m=n$,
$$A_i= U
\begin{bmatrix}
\sigma_1 & 0 & \dots & \dots& 0 & \dots & 0\\
0 & \sigma_2 & 0 &\dots & 0 & & \vdots\\
\vdots & & \ddots & & \vdots & &\vdots\\
0 & \dots & 0 & \sigma_r & 0 & & \vdots\\
0 & \dots & \dots & \dots & \frac{1}{i} & & \vdots\\
\vdots & & & & & \ddots & \vdots\\
0 & \dots & \dots & \dots & \dots & \dots & \frac{1}{i}
\end{bmatrix}
V^T
$$
and for $m > n$,
$$A_i= U
\begin{bmatrix}
\sigma_1 & 0 & \dots & \dots& 0 & \dots & 0\\
0 & \sigma_2 & 0 &\dots & 0 & & \vdots\\
\vdots & & \ddots & & \vdots & &\vdots\\
0 & \dots & 0 & \sigma_r & 0 & & \vdots\\
0 & \dots & \dots & \dots & \frac{1}{i} & & \vdots\\
\vdots & & & & & \ddots & \vdots\\
0 & \dots & \dots & \dots & \dots & \dots & \frac{1}{i} \\
0 & \dots & \dots & \dots & \dots & \dots & 0\\
\vdots & & & & & & \vdots\\
0 & \dots &\dots & \dots & \dots & \dots & 0
\end{bmatrix}
V^T
$$
and for $m<n$,
$$A_i= U
\begin{bmatrix}
\sigma_1 & 0 & \dots & \dots& 0 & \dots & 0 & 0 & \dots & 0\\
0 & \sigma_2 & 0 &\dots & 0 & & \vdots & \vdots & &\vdots\\
\vdots & & \ddots & & \vdots & &\vdots & \vdots & &\vdots\\
0 & \dots & 0 & \sigma_r & 0 & & \vdots & \vdots & &\vdots\\
0 & \dots & \dots & \dots & \frac{1}{i} & & \vdots & \vdots & &\vdots\\
\vdots & & & & & \ddots & \vdots & \vdots & &\vdots\\
0 & \dots & \dots & \dots & \dots & \dots & \frac{1}{i} & 0 & \dots & 0
\end{bmatrix}
V^T
$$
Thus $\operatorname{rank}(A_i) = \min\{m,n\}$ and each $A_i$ is of full rank, which gives us
$$ \|A-A_i \|_2 =
\begin{vmatrix} \begin{vmatrix} \begin{bmatrix}
0 & \dots & \dots & \dots & \dots & 0\\
\vdots & \ddots & & & & \vdots\\
0 & & 0 \\
0 & & 0 & \frac{1}{i} & \dots & 0 \\
\vdots & & & & \ddots & \vdots \\
0 &\dots &\dots &\dots & 0 & \frac{1}{i}
\end{bmatrix}\end{vmatrix}\end{vmatrix}_2
= \sigma_{i+1} = \frac{1}{i} \rightarrow 0 \text{ as } i \rightarrow \infty.
$$
by the best rank-$k$ approximation theorem with respect to $\| \cdot \|_2$.
$\textbf{My Question (the only part that I do not understand):}$
How were the $\frac{1}{i}$ diagonals derived?
Best Answer
They were by choice, with the goal of obtain $\|A-A_i\|_2=\frac1{i}$, in fact, you can replace $\frac1{i}$ by another positive function of $i$ that goes to $0$ as $i \to \infty$.