I'm pretty new to linear algebra, and am learning about norms. I'm having trouble understanding how to apply the Hilbert-Schmidt norm.
How would you do it for a simple example like $\begin{bmatrix}0 & 1\\1 & 0\end{bmatrix}$?
Example of application of hilbert-schmidt norm
linear algebramatrix-norms
Related Solutions
Hint: The Frobenius norm of an $m \times n$ matrix $A$ is defined as the square root of the sum of the absolute squares of its elements.
Example: Consider matrix
$$A = \left( \begin{array}{ccc} ~~~~1 & -2 & ~~~~~~3 \\ -4 & ~~5 & ~-6 \\ ~~~7 & -8 & ~~~~~~9 \\ \end{array} \right)$$
$\lVert A \rVert _{F} = \sqrt (|1|^2+|-2|^2+|3|^2+|-4|^2+|5|^2+|-6|^2+|7|^2+|-9|^2 + |-8|^2) = \sqrt(285) = 16.8819$
For convergence of Jacobi iteration method you need to find iteration matrix $P = -D^{-1}(L+U)$. Note that the Jacobi iterative scheme will converge if $\lVert P\rVert_{F} $ is strictly less than $1$, where $F$ stands for the Frobenius norm. There may be some other matrix norm (such as the $1$-norm ) that is strictly less than $1$, in which case convergence is still guaranteed. In any case, however, the condition $\lVert P\rVert <1 $ is only a sufficient condition for convergence, not a necessary one.
For the given example
$D = \left( \begin{array}{ccc} ~~1 & ~0 & ~0 \\ ~~0 & ~5 & ~0 \\ ~~0 & ~0 & ~9 \\ \end{array} \right)$ $L = \left( \begin{array}{ccc} ~~~0 & ~~0 & ~0 \\ -4 & ~~0 & ~0 \\ ~~~~7 & -8 & ~0 \\ \end{array} \right)$ $U = \left( \begin{array}{ccc} ~~0 & -2 &~~3 \\ ~~0 & ~~~0 & -6 \\ ~~0 & ~~~0 & ~~~0 \\ \end{array} \right)$
Added: Consider to solve $3\times 3$ size system of linear equation $Ax = b$, where coefficient matrix $A = \left( \begin{array}{ccc} a_{11} & a_{12} & a_{13} \\ a_{21}& a_{22} & a_{23} \\ a_{31}& a_{32} & a_{32} \\ \end{array} \right)$
Assume coefficient matrix $A$ has no zeros on its main diagonal i.e. $a_{11}$, $a_{22}$, $a_{23}$ are non zeros, then $D = \left( \begin{array}{ccc} ~~\frac{1}{a_{11}} & 0 & 0 \\ 0 & \frac{1}{a_{22}} & ~0 \\ ~~0 & 0 & \frac{1}{a_{33}}\\ \end{array} \right)$
$L = \left( \begin{array}{ccc} 0 & 0 & 0 \\ a_{21} & 0 & ~0 \\ a_{31}& a_{32} & 0\\ \end{array} \right)$ $U = \left( \begin{array}{ccc} 0 & a_{12} & a_{13} \\ 0 & 0 & a_{23} \\ 0 & 0 & 0\\ \end{array} \right)$
Formally, Gram-Schmidt is an algorithm when working in finite dimensions. In infinite dimensions it lacks an important property of algorithms: Termination after finitely many steps. However, do you see how we can handwave that restriction away with a limit process for the case of a Hilbert space and a countable Hilbert basis?
After your clarification of the question: For a family $\{\,w_j\mid j\in J\,\}\subseteq V$ we let $\overline{Sp}(\{\,w_j\mid j\in J\,\})$ be the space of all $\sum_{j\in J}c_jw_j$ with at most countably many $c_j\ne 0$ and $\sum \lvert c_jw_j\rvert^2<\infty$. Define a "partial Gram-Schmidt" as a subset $J\subseteq I$ together with a total order $\le$ on $J$ and orthonormal $\{\,b_j\mid j\in J\,\}$ such that $b_j\in\overline{Sp}(\{v_j\}\cup\{\,b_k\mid k\in J, k<j\,\})$ for all $j\in J$. Then the set of partial Gram-Schmidts is inductively ordered in an obvious manner, hence ny Zorn's lemma there is a maximal one among them. I claim that $J=I$ holds for this. Assume otherwise, i.e. there exists $i\in I\setminus J$. For $j\in J$ let $c_j=\langle b_j,v_i\rangle$. If $S$ is any finite subset of $J$ verify that $\sum_{j\in S} c_jb_j$ is the unique vector in $\operatorname{span}(\{\,b_j\mid j\in S\,\})$ that is closest to $v_i$ and that it is shorter than $v_i$. Conclude that at most countably many $c_j$ are nonzero (otherwise one could find finite $S$ that would produce an $\sum_{j\in S} c_jb_j$ longer than $\lVert v_i\rVert$, for example) and that $\sum c_j^2\le \lVert v_i\rVert^2$. Subtract $\sum c_jb_j$ from $v_i$ and normalize to obtain $b_i$ and by letting $i>j$ for all $j\in J$ thus a partial Gram-Schmidt extended to $J\cup \{i\}$, contradicting maximiality of $J$.
Thus for "Gram-Schmidt with series allowed" the answer to your question is "yes" for arbitrary cardinalities.
Best Answer
For matrices the Hilbert-Schmidt norm is just $$\|(a_{j,k})\|_{HS} = \sqrt{\sum_{j,k=1}^n |a_{j,k}|^2}.$$ In your example this results in $\sqrt{2}$.