I believe the largest singular value is the value of the 2-norm, but how can we relate this to the operator norm?
The largest singular value is the operator norm:
$$\sigma_1=\max_{\|x\|=1}\|Ax\|=\|A\| \tag0$$
Here and below I use the Euclidean norm on vectors, and the corresponding operator norm on matrices.
The smallest singular value of $A$, denoted by $\sigma_n$ below, has a description similar to (0):
$$\sigma_n=\min_{\|x\|=1}\|Ax\|=\min\{\|A-B\|: \operatorname{rank}B<n \} \tag1$$
Indeed, from the SVD decomposition you see that
$$\min_{\|x\|=1}\|Ax\|^2 = \min_{\|y\|=1} \sum_k \sigma_k^2 y_k^2 = \sigma_n^2+ \min_{\|y\|=1} \sum_k (\sigma_k^2-\sigma_n^2) y_k^2$$
where the last sum is minimized by letting $y_n$ be the only nonzero component. I used the fact that the unitaries preserve the vector norm.
By the way, you can prove (0) by modifying the proof above.
As for the second part of (1), the inequality $$\min_{\|x\|=1}\|Ax\|\le \min\{\|A-B\|: \operatorname{rank}B<n \}\tag2$$ follows from the fact that $Bx=0$ for some unit vector $x$. The reverse inequality follows by taking $B=A\circ P$ where $P$ is the projection onto the orthogonal complement of vector $x$ that attains the minimum on the left side of (2). Indeed, $A-A\circ P=A\circ Q$ where $Q$ is the projection onto the span of $x$, hence $\|A\circ Q\|=\|Ax\|$.
In particular, (1) implies that $\sigma_n(A)$ is a $1$-Lipschitz function of $A$ in the operator norm, namely $$|\sigma_n(A)-\sigma_n(B)|\le \|A-B\| \tag3$$ Where the right hand side is $\sigma_1(A-B)$, by (0).
In fact, (3) holds for all singular values, because $\sigma_k$ is the distance from $A$ to the set of operators of rank less than $k$. See Wikipedia: Singular value.
In order to get the SVD of the matrix you need to get the eigenvalues of the covariance matrix
$$ A =\begin{bmatrix} 2& -6\\ 1 & -3 \\ 0 & 0 \end{bmatrix} \tag{1}$$
$$ A^{T}A = \begin{bmatrix} 5& -15\\ -15& 45 \end{bmatrix}\tag{2}$$
$$ \det(A^{T}A - \lambda I) = \begin{vmatrix} 5- \lambda& -15\\ -15& 45-\lambda \end{vmatrix} = (5-\lambda)(45-\lambda) -225 \tag{3}$$
$$ \det(A^{T}A - \lambda I) = \lambda(\lambda-50) \implies \lambda_{1} = 50 \lambda_{2} = 0 \tag{4}$$
now $A^{T}A = V \Lambda V^{T} $
we find the right singular vectors
$$ A^{T}A -50I = \begin{bmatrix} 5-50& -15\\ -15& 45 -50 \end{bmatrix} \begin{bmatrix}x_{1} \\ x_{2} \end{bmatrix} \begin{bmatrix} 0 \\ 0 \end{bmatrix} \tag{5}$$
$$ A^{T}A-50I = \begin{bmatrix} -45 &-15\\ -15& -5 \end{bmatrix} \begin{bmatrix}x_{1} \\ x_{2} \end{bmatrix} \begin{bmatrix} 0 \\ 0 \end{bmatrix} \tag{6}$$
$$ -45x_{1} -15x_{2} = 0 \\ -15x_{1} - 5x_{2} = 0 $$
then $x_{2} = \frac{x_{1}}{3}$
$$ x = \begin{bmatrix} 1 \\ 3\end{bmatrix} \tag{7} $$
now normalize
$$ v_{1} = \begin{bmatrix} \frac{1}{\sqrt{3^{2}+1}}\\ \frac{3}{\sqrt{3^{2}+1}} \end{bmatrix} \tag{8} $$
$$ v_{1} = \begin{bmatrix} \frac{1}{\sqrt{10}}\\ \frac{3}{\sqrt{10}} \end{bmatrix} \tag{9} $$
for the next
$$ A^{T}A = \begin{bmatrix} 5& -15\\ -15& 45 \end{bmatrix} \begin{bmatrix}x_{1} \\ x_{2} \end{bmatrix} \begin{bmatrix} 0 \\ 0 \end{bmatrix} \tag{10}$$
you get the opposite
$$ v_{2} = \begin{bmatrix} \frac{-3}{\sqrt{10}}\\ \frac{1}{\sqrt{10}} \end{bmatrix} \tag{11} $$
the singular values are the square roots of the eigenvalues so..you have $\sigma_{1} = \sqrt{50} , \sigma_{2} =0$
if you do it by python ..
import numpy as np
A = np.matrix([[2,-6],[1,-3],[0,0]])
u,s,vt = np.linalg.svd(A)
these aren't unique.. you should note..
matrix([[-0.31622777, 0.9486833 ],
[ 0.9486833 , 0.31622777]])
if you do the $AA^{T}$
$$AA^{T} = \begin{bmatrix} 40& 20 & 0 \\ 20 & 10 & 0 \\ 0 & 0 & 0 \end{bmatrix} \tag{12}$$
then take
$$ \det(AA^{T} - \lambda I) = \begin{vmatrix} 40-\lambda& 20 & 0 \\ 20 & 10-\lambda & 0 \\ 0 & 0 & 0 \end{vmatrix} \tag{13}$$
in the end if $U \Sigma V^{T} = A $ you'll be fine..
Best Answer
We are given:
$$A = \begin{bmatrix}1 & 1\\1 & 0 \end{bmatrix}$$
We have:
$$W = A^T A = \begin{bmatrix}2 & 1\\1 & 1 \end{bmatrix}$$
The characteristic polynomial and eigenvalues of W are:
$$\lambda^2 +3 \lambda -1 = 0 \implies \lambda_1 = \frac{1}{2} \left(3-\sqrt{5}\right), ~ \lambda_2 = \frac{1}{2} \left(3+\sqrt{5}\right)$$
This gives us the singular values:
$$\sigma_1= \sqrt{\frac{1}{2} \left(3-\sqrt{5}\right)}, ~ \sigma_2= \sqrt{\frac{1}{2} \left(3+\sqrt{5}\right)}$$
The eigenvectors of $W$ are:
$$v_1 = \begin{bmatrix} \frac{1}{2} \left(1-\sqrt{5}\right) \\ 1 \end{bmatrix}, ~ v_2 = \begin{bmatrix} \frac{1}{2} \left(1+\sqrt{5}\right) \\ 1 \end{bmatrix}$$
Normalizing these eigenvectors, we have:
$$v_1 = \begin{bmatrix} \frac{1+\sqrt{5}}{2 \sqrt{\frac{1}{4} \left(1+\sqrt{5}\right)^2+1}} \\ \frac{1}{\sqrt{\frac{1}{4} \left(1+\sqrt{5}\right)^2+1}} \end{bmatrix}, ~ v_2 = \begin{bmatrix} \frac{1-\sqrt{5}}{2 \sqrt{\frac{1}{4} \left(1+\sqrt{5}\right)^2+1}} \\ \frac{1}{\sqrt{\frac{1}{4} \left(1-\sqrt{5}\right)^2+1}} \end{bmatrix}$$
Now, we can write $W = U \Sigma V^T$ as:
$$U = \left( \begin{array}{cc} \frac{\frac{1}{2} \left(1+\sqrt{5}\right)+1}{\sqrt{\frac{1}{4} \left(1+\sqrt{5}\right)^2+\left(\frac{1}{2} \left(1+\sqrt{5}\right)+1\right)^2}} & \frac{\frac{1}{2} \left(1-\sqrt{5}\right)+1}{\sqrt{\frac{1}{4} \left(1-\sqrt{5}\right)^2+\left(\frac{1}{2} \left(1-\sqrt{5}\right)+1\right)^2}} \\ \frac{1+\sqrt{5}}{2 \sqrt{\frac{1}{4} \left(1+\sqrt{5}\right)^2+\left(\frac{1}{2} \left(1+\sqrt{5}\right)+1\right)^2}} & \frac{1-\sqrt{5}}{2 \sqrt{\frac{1}{4} \left(1-\sqrt{5}\right)^2+\left(\frac{1}{2} \left(1-\sqrt{5}\right)+1\right)^2}} \\ \end{array} \right)$$
$$\Sigma = \left( \begin{array}{cc} \sqrt{\frac{1}{2} \left(3+\sqrt{5}\right)} & 0 \\ 0 & \sqrt{\frac{1}{2} \left(3-\sqrt{5}\right)} \\ \end{array} \right)$$
$$V = \left( \begin{array}{cc} \frac{1+\sqrt{5}}{2 \sqrt{\frac{1}{4} \left(1+\sqrt{5}\right)^2+1}} & \frac{1-\sqrt{5}}{2 \sqrt{\frac{1}{4} \left(1-\sqrt{5}\right)^2+1}} \\ \frac{1}{\sqrt{\frac{1}{4} \left(1+\sqrt{5}\right)^2+1}} & \frac{1}{\sqrt{\frac{1}{4} \left(1-\sqrt{5}\right)^2+1}} \\ \end{array} \right)$$
Notes:
$$u_1 = \dfrac{1}{\sigma_1} A v_1, ~ u_2 = \dfrac{1}{\sigma_2} A v_2$$
Care needs to be taken when one or both of the eigenvalues are zero!
This SVD can also be written as:
$$\sigma_1~u_1~v_1^T + \sigma_2~u_2~v_2^T$$