I'm currently taking an online course where the professor has a habit of writing norms like this:
$||a^{[l](C)} – a^{[l](G)} ||^2$
Since I don't have a great amount of experience in math or the concepts of deep learning, I was often confused whether the 2 simply meant, in conjunction with the double bars, "apply the L2 norm to the terms within, i.e. square each of them and then sum the result" or if the 2 was, itself, a squaring of whatever the double brackets meant on their own.
So I googled for confirmation of the notation and it seems that the 2 is normally written as a subscript, not a superscript. For example you can see the notation in Wikipedia is written this way: https://en.wikipedia.org/wiki/Norm_(mathematics)#Notation
So is the superscript notation wrong? Or is this just one of those unfortunate cases where there is no standard?
Best Answer
I have never seen an author disambiguate the norm delimiters $\lVert\quad\rVert$ through the use of a superscript. In analysis, such notation would be incredibly confusing, since we frequently need to establish inequalities among norms of vectors raised to some power.
Also, an $L^2$ norm of a vector is the square root of the sum of the absolute squares of its components: $$\lVert x\rVert_2=\sqrt{\sum_{i=1}^n\lvert x_i\rvert^2}\text{;}$$ consequently, $$\lVert x\rVert^2_2=\sum_{i=1}^n\lvert x_i\rvert^2\text{.}$$