I am currently following the Machine Learning Crash Course on Tensorflow and came across this formula:
$$L_2\text{ regularization term} = \|\boldsymbol w\|_2^2 = {w_1^2 + w_2^2 + \cdots + w_n^2}$$
I am having trouble understanding its notations.
I understand that the double bar, $\|\boldsymbol w\|$, denote some kind of norm, but what kind of norm is this? I am only familiar with the Euclidean norm.
Secondly, I don't understand why there's a subscript and superscript, $_2^2$, attached to the term. I'd think the subscript should be $n$ instead of $2$?
Best Answer
The genereal notation for $p$-norm for $p \in [1,+\infty)$ of vector $v \in \mathbb{R}^n$ is this:
$$ \| v \|_p = \sqrt[p]{\sum^n_{i=1} |v_i|^p}. $$
It is easy to see that $\| v\|_2$ is indeed an Euclidean norm (let $p=2$ in the formula above) That is, Euclidean norm is 2-norm.
Then squaring produces
$$ \| v\|_2^2 = (\|v\|_2)^2 =\sum^n_{i=1} v_i^2 = v_1^2 + v_2^2 \ldots + v_n^2 $$
which is what you have specified.